ai-sdk-rate-limiter

Appearance

ai-sdk-rate-limiterProduction-grade rate limiting for AI APIs

Smart queuing, cost tracking, and budget enforcement for Vercel AI SDK and raw OpenAI/Anthropic clients. Zero required dependencies.

Get Started
View on GitHub
🚦

Zero-config start

Works out of the box with built-in limits for OpenAI, Anthropic, Google, Groq, Mistral, and Cohere models. No setup required.

📊

Cost tracking & forecasting

Tracks spend per model, per scope, and per period. Set hard budget caps with throw, queue, or fallback behavior. Forecast end-of-day costs from current rate.

âš¡

Smart queuing

Priority queue with high/normal/low lanes. Per-request timeouts. Exponential backoff with jitter. Honors Retry-After headers from APIs.

🔀

Multi-tenant scopes

Isolate rate limits per user, org, or tenant with wildcard patterns. Each scope gets its own independent sliding window.

🔌

Redis for multi-instance

Drop-in RedisStore to share rate limit state and cost history across multiple server instances. Survives restarts.

🧰

Observability built in

Structured events, Prometheus metrics, StatsD support, and OpenTelemetry spans. Debug mode logs every decision to the console.

Installation ​

bash
npm install ai-sdk-rate-limiter
bash
pnpm add ai-sdk-rate-limiter
bash
yarn add ai-sdk-rate-limiter
bash
# Deno
deno add jsr:@piyushgupta344/ai-sdk-rate-limiter

# Node.js via npx
npx jsr add @piyushgupta344/ai-sdk-rate-limiter

Quickstart ​

typescript
import { createRateLimiter } from 'ai-sdk-rate-limiter'
import { openai } from '@ai-sdk/openai'
import { generateText } from 'ai'

const limiter = createRateLimiter()
const model = limiter.wrap(openai('gpt-4o'))

const { text } = await generateText({ model, prompt: 'Hello!' })

That's it. The limiter automatically applies the built-in rate limits for gpt-4o, queues requests when the limit is reached, retries on 429s, and tracks cost.

→ Full guide

Released under the MIT License.