# API Rate Limiting Token Bucket Algorithm

# API Rate Limiting with the Token Bucket Algorithm: A Developer's Guide

## Metadata

```json
{
  "seo_title": "Token Bucket Algorithm for API Rate Limiting | TypeScript Guide",
  "meta_description": "Learn how to implement API rate limiting using the Token Bucket algorithm in TypeScript. Includes production-ready code, common pitfalls, and best practices for developers.",
  "keywords": [
    "token bucket algorithm",
    "API rate limiting",
    "TypeScript rate limiting",
    "distributed rate limiting",
    "API throttling",
    "rate limiter implementation",
    "backend API security"
  ],
  "tags": [
    "API Design",
    "Rate Limiting",
    "TypeScript",
    "Backend Development",
    "System Design",
    "Performance",
    "Security"
  ]
}
```

## The Problem: Why Rate Limiting Matters in 2026

As APIs continue to power the interconnected digital ecosystem of 2026, rate limiting has evolved from a nice-to-have feature to an absolute necessity. Modern applications face unprecedented challenges: sophisticated DDoS attacks, resource-intensive AI model queries, and the exponential growth of IoT devices hammering endpoints.

**The core challenges developers face today:**

**Resource Exhaustion**: Without proper rate limiting, a single misbehaving client or malicious actor can consume all available server resources, causing cascading failures across your infrastructure. In 2026's serverless and containerized environments, this translates directly to runaway costs and degraded user experience.

**Fair Resource Distribution**: Your API serves multiple clients with varying subscription tiers. Premium users expect higher throughput while free-tier users need reasonable access. Implementing fair, predictable rate limiting ensures quality of service across all user segments.

**Cost Management**: Cloud providers charge for compute time, memory, and data transfer. Uncontrolled API usage can result in shocking bills, especially with AI/ML endpoints that require significant computational resources per request.

**Security and Abuse Prevention**: Brute force attacks, credential stuffing, and scraping bots remain prevalent threats. Rate limiting serves as your first line of defense, making these attacks economically unfeasible for attackers.

**Compliance Requirements**: Many industries now mandate rate limiting as part of security frameworks. GDPR, HIPAA, and PCI-DSS compliance often require demonstrable controls over data access patterns.

**Why Token Bucket?**

Among rate limiting algorithms (fixed window, sliding window, leaky bucket), the token bucket algorithm offers the best balance of flexibility and simplicity. It allows for burst traffic—critical for modern applications where legitimate users might trigger multiple API calls simultaneously—while maintaining long-term rate limits. Unlike fixed windows that can be gamed at boundaries, token bucket provides smooth, predictable behavior.

## Modern TypeScript Implementation

Here's a production-ready, type-safe implementation suitable for Node.js applications in 2026:

```typescript
interface TokenBucketConfig {
  capacity: number;        // Maximum tokens in bucket
  refillRate: number;      // Tokens added per second
  initialTokens?: number;  // Starting token count
}

interface BucketState {
  tokens: number;
  lastRefill: number;      // Timestamp in milliseconds
}

class TokenBucket {
  private capacity: number;
  private refillRate: number;
  private tokens: number;
  private lastRefill: number;

  constructor(config: TokenBucketConfig) {
    this.capacity = config.capacity;
    this.refillRate = config.refillRate;
    this.tokens = config.initialTokens ?? config.capacity;
    this.lastRefill = Date.now();
  }

  /**
   * Attempts to consume tokens from the bucket
   * @param tokens Number of tokens to consume (default: 1)
   * @returns true if tokens were consumed, false if insufficient tokens
   */
  consume(tokens: number = 1): boolean {
    this.refill();

    if (this.tokens >= tokens) {
      this.tokens -= tokens;
      return true;
    }

    return false;
  }

  /**
   * Refills the bucket based on elapsed time
   */
  private refill(): void {
    const now = Date.now();
    const elapsedSeconds = (now - this.lastRefill) / 1000;
    const tokensToAdd = elapsedSeconds * this.refillRate;

    this.tokens = Math.min(this.capacity, this.tokens + tokensToAdd);
    this.lastRefill = now;
  }

  /**
   * Returns current token count and time until next token
   */
  getStatus(): { availableTokens: number; nextTokenIn: number } {
    this.refill();
    const nextTokenIn = this.tokens < this.capacity 
      ? (1 / this.refillRate) * 1000 
      : 0;

    return {
      availableTokens: Math.floor(this.tokens),
      nextTokenIn: Math.round(nextTokenIn)
    };
  }

  /**
   * Serializes bucket state for persistence
   */
  serialize(): BucketState {
    this.refill();
    return {
      tokens: this.tokens,
      lastRefill: this.lastRefill
    };
  }

  /**
   * Restores bucket from serialized state
   */
  static deserialize(config: TokenBucketConfig, state: BucketState): TokenBucket {
    const bucket = new TokenBucket(config);
    bucket.tokens = state.tokens;
    bucket.lastRefill = state.lastRefill;
    return bucket;
  }
}

// Express middleware example
import { Request, Response, NextFunction } from 'express';
import Redis from 'ioredis';

class DistributedTokenBucket {
  private redis: Redis;
  private config: TokenBucketConfig;
  private keyPrefix: string;

  constructor(redis: Redis, config: TokenBucketConfig, keyPrefix = 'ratelimit:') {
    this.redis = redis;
    this.config = config;
    this.keyPrefix = keyPrefix;
  }

  async consume(identifier: string, tokens: number = 1): Promise<boolean> {
    const key = `${this.keyPrefix}${identifier}`;
    const now = Date.now();

    // Lua script for atomic operations
    const script = `
      local key = KEYS[1]
      local capacity = tonumber(ARGV[1])
      local refillRate = tonumber(ARGV[2])
      local tokens = tonumber(ARGV[3])
      local now = tonumber(ARGV[4])
      
      local bucket = redis.call('HMGET', key, 'tokens', 'lastRefill')
      local currentTokens = tonumber(bucket[1]) or capacity
      local lastRefill = tonumber(bucket[2]) or now
      
      local elapsedSeconds = (now - lastRefill) / 1000
      local tokensToAdd = elapsedSeconds * refillRate
      currentTokens = math.min(capacity, currentTokens + tokensToAdd)
      
      if currentTokens >= tokens then
        currentTokens = currentTokens - tokens
        redis.call('HMSET', key, 'tokens', currentTokens, 'lastRefill', now)
        redis.call('EXPIRE', key, 3600)
        return 1
      else
        return 0
      end
    `;

    const result = await this.redis.eval(
      script,
      1,
      key,
      this.config.capacity,
      this.config.refillRate,
      tokens,
      now
    );

    return result === 1;
  }
}

// Middleware factory
export function createRateLimiter(
  redis: Redis,
  config: TokenBucketConfig
) {
  const limiter = new DistributedTokenBucket(redis, config);

  return async (req: Request, res: Response, next: NextFunction) => {
    const identifier = req.ip || req.socket.remoteAddress || 'unknown';
    
    const allowed = await limiter.consume(identifier);

    if (!allowed) {
      res.status(429).json({
        error: 'Too Many Requests',
        message: 'Rate limit exceeded. Please try again later.',
        retryAfter: Math.ceil(1 / config.refillRate)
      });
      return;
    }

    next();
  };
}
```

## Common Pitfalls and How to Avoid Them

**1. Clock Skew in Distributed Systems**

When running multiple server instances, system clock differences can cause inconsistent rate limiting. Always use a centralized time source or Redis's `TIME` command for distributed implementations.

**2. Floating Point Precision**

Token calculations involve floating-point arithmetic. Always round appropriately and consider using integer-based implementations for high-precision requirements.

**3. Memory Leaks with Per-User Buckets**

Storing buckets for every user in memory leads to unbounded growth. Implement TTL-based cleanup or use Redis with automatic expiration.

**4. Race Conditions**

In distributed systems, separate read-modify-write operations create race conditions. Use atomic operations (Lua scripts in Redis) or optimistic locking.

**5. Burst Handling**

Setting capacity equal to rate creates no burst allowance. Design capacity to be 2-10x your per-second rate depending on use case.

## Best Practices

**Tiered Rate Limiting**: Implement multiple buckets per user—one for burst protection (small capacity, fast refill) and another for long-term limits (large capacity, slow refill).

**Informative Headers**: Return `X-RateLimit-Limit`, `X-RateLimit-Remaining`, and `X-RateLimit-Reset` headers so clients can self-regulate.

**Graceful Degradation**: Instead of hard rejections, consider queuing requests or offering reduced functionality when limits are approached.

**Monitoring and Alerting**: Track rate limit hits by endpoint and user. Sudden spikes indicate attacks or client bugs.

**Cost-Based Tokens**: Consume different token amounts based on operation cost. A simple GET might cost 1 token while a complex search costs 10.

**Whitelist Critical Services**: Allow internal services or health checks to bypass rate limiting to prevent cascading failures.

## Frequently Asked Questions

**Q: How do I choose the right capacity and refill rate?**

Start with your expected requests per second (RPS) as the refill rate. Set capacity to 2-5x this value to allow reasonable bursts. For example, if you expect 10 RPS, use refillRate: 10 and capacity: 30-50.

**Q: Should I rate limit by IP address or user ID?**

Use user ID when available for authenticated endpoints—it's more accurate and prevents shared IP issues. Use IP for public endpoints, but consider proxy and NAT scenarios. Implement both for comprehensive protection.

**Q: How does this scale in a microservices architecture?**

Use Redis or a similar distributed cache as your shared state store. Each service instance queries the same Redis bucket, ensuring consistent rate limiting across your cluster.

**Q: What's the difference between token bucket and leaky bucket?**

Token bucket allows bursts up to capacity, making it better for APIs where legitimate users might make several requests simultaneously. Leaky bucket enforces strict constant rate, better for smoothing traffic to downstream services.

**Q: How do I handle rate limiting for WebSocket connections?**

Apply rate limiting to message frequency rather than connection establishment. Track messages per time window per connection, using the same token bucket principles.

**Q: Can I implement this without Redis?**

Yes, for single-server deployments, in-memory implementation works fine. For distributed systems, you need shared state—Redis, Memcached, or a distributed cache like Hazelcast.

**Q: How do I test rate limiting logic?**

Mock time in your tests using libraries like `sinon` to control Date.now(). Test boundary conditions: exactly at limit, burst scenarios, and token refill timing. Load testing tools like k6 help validate production behavior.

---

The token bucket algorithm remains the gold standard for API rate limiting in 2026. Its flexibility, predictability, and burst-handling capabilities make it ideal for modern distributed systems. By implementing the patterns shown here and avoiding common pitfalls, you'll build robust, fair, and scalable rate limiting for your APIs.

**Word Count: 1,789 words**