# Request Deduplication: Prevent Duplicate Calls

# Request Deduplication: Prevent Duplicate Calls with Caching

## Problem

When multiple requests for the same resource are made simultaneously or in quick succession, your application may:
- Execute the same expensive operation multiple times
- Hit rate limits on external APIs
- Waste database queries and computational resources
- Create race conditions and inconsistent state
- Degrade performance and increase latency

**Example:** Three users request user profile data simultaneously → three identical database queries execute instead of one.

## Solution

Implement request deduplication by caching pending requests. When a request is already in-flight, subsequent identical requests wait for the same result rather than triggering new operations.

**Key Principles:**
1. **Identify duplicate requests** using a cache key (URL, parameters, etc.)
2. **Store pending promises** instead of results
3. **Share the same promise** across duplicate requests
4. **Resolve once** when the operation completes
5. **Clean up** after resolution to prevent memory leaks

## Code

### Basic Implementation

```javascript
class RequestDeduplicator {
  constructor() {
    this.pendingRequests = new Map();
  }

  async execute(key, requestFn) {
    // Return existing pending request if available
    if (this.pendingRequests.has(key)) {
      return this.pendingRequests.get(key);
    }

    // Create new promise for this request
    const promise = requestFn()
      .then(result => {
        this.pendingRequests.delete(key);
        return result;
      })
      .catch(error => {
        this.pendingRequests.delete(key);
        throw error;
      });

    // Store pending promise
    this.pendingRequests.set(key, promise);
    return promise;
  }
}

// Usage
const deduplicator = new RequestDeduplicator();

async function fetchUserProfile(userId) {
  return deduplicator.execute(`user:${userId}`, async () => {
    const response = await fetch(`/api/users/${userId}`);
    return response.json();
  });
}

// All three calls share the same request
Promise.all([
  fetchUserProfile(1),
  fetchUserProfile(1),
  fetchUserProfile(1)
]);
```

### Advanced Implementation with TTL

```javascript
class SmartRequestDeduplicator {
  constructor(options = {}) {
    this.pendingRequests = new Map();
    this.cache = new Map();
    this.ttl = options.ttl || 0; // milliseconds, 0 = no caching
    this.maxRetries = options.maxRetries || 0;
  }

  async execute(key, requestFn, options = {}) {
    const cacheKey = key;
    const now = Date.now();

    // Check if result is cached and still valid
    if (this.cache.has(cacheKey)) {
      const { result, timestamp } = this.cache.get(cacheKey);
      if (now - timestamp < this.ttl) {
        return result;
      }
      this.cache.delete(cacheKey);
    }

    // Return pending request if exists
    if (this.pendingRequests.has(cacheKey)) {
      return this.pendingRequests.get(cacheKey);
    }

    // Execute with retry logic
    const promise = this._executeWithRetry(
      requestFn,
      options.retries ?? this.maxRetries
    )
      .then(result => {
        this.pendingRequests.delete(cacheKey);
        
        // Cache result if TTL is set
        if (this.ttl > 0) {
          this.cache.set(cacheKey, { result, timestamp: Date.now() });
        }
        
        return result;
      })
      .catch(error => {
        this.pendingRequests.delete(cacheKey);
        throw error;
      });

    this.pendingRequests.set(cacheKey, promise);
    return promise;
  }

  async _executeWithRetry(requestFn, retries) {
    let lastError;
    
    for (let attempt = 0; attempt <= retries; attempt++) {
      try {
        return await requestFn();
      } catch (error) {
        lastError = error;
        if (attempt < retries) {
          // Exponential backoff
          await new Promise(resolve => 
            setTimeout(resolve, Math.pow(2, attempt) * 100)
          );
        }
      }
    }
    
    throw lastError;
  }

  invalidate(key) {
    this.cache.delete(key);
  }

  clear() {
    this.cache.clear();
    this.pendingRequests.clear();
  }

  getStats() {
    return {
      pendingRequests: this.pendingRequests.size,
      cachedResults: this.cache.size
    };
  }
}

// Usage
const deduplicator = new SmartRequestDeduplicator({
  ttl: 5000, // Cache for 5 seconds
  maxRetries: 2
});

async function fetchData(id) {
  return deduplicator.execute(`data:${id}`, async () => {
    console.log(`Fetching data for ${id}`);
    const response = await fetch(`/api/data/${id}`);
    if (!response.ok) throw new Error('Failed to fetch');
    return response.json();
  });
}

// Concurrent requests - only one actual fetch
await Promise.all([
  fetchData(1),
  fetchData(1),
  fetchData(1)
]); // Logs "Fetching data for 1" once

// Subsequent request within TTL uses cache
await fetchData(1); // No log - uses cache

// After TTL expires
setTimeout(() => fetchData(1), 6000); // Logs again
```

### React Hook Implementation

```javascript
function useDeduplicatedFetch(url, options = {}) {
  const [data, setData] = useState(null);
  const [loading, setLoading] = useState(false);
  const [error, setError] = useState(null);
  const deduplicatorRef = useRef(new SmartRequestDeduplicator({
    ttl: options.cacheTtl || 0
  }));

  useEffect(() => {
    setLoading(true);
    
    deduplicatorRef.current
      .execute(url, async () => {
        const response = await fetch(url);
        if (!response.ok) throw new Error('Fetch failed');
        return response.json();
      })
      .then(result => {
        setData(result);
        setError(null);
      })
      .catch(err => {
        setError(err);
        setData(null);
      })
      .finally(() => setLoading(false));
  }, [url]);

  return { data, loading, error };
}

// Usage
function UserProfile({ userId }) {
  const { data: user, loading, error } = useDeduplicatedFetch(
    `/api/users/${userId}`,
    { cacheTtl: 5000 }
  );

  if (loading) return <div>Loading...</div>;
  if (error) return <div>Error: {error.message}</div>;
  return <div>{user.name}</div>;
}
```

### Database Query Deduplication

```javascript
class DatabaseQueryDeduplicator {
  constructor(db) {
    this.db = db;
    this.pendingQueries = new Map();
  }

  async query(sql, params = []) {
    const key = this._generateKey(sql, params);

    if (this.pendingQueries.has(key)) {
      return this.pendingQueries.get(key);
    }

    const promise = this.db.query(sql, params)
      .then(result => {
        this.pendingQueries.delete(key);
        return result;
      })
      .catch(error => {
        this.pendingQueries.delete(key);
        throw error;
      });

    this.pendingQueries.set(key, promise);
    return promise;
  }

  _generateKey(sql, params) {
    return `${sql}:${JSON.stringify(params)}`;
  }
}

// Usage
const dbDedup = new DatabaseQueryDeduplicator(database);

// Multiple concurrent requests
const results = await Promise.all([
  dbDedup.query('SELECT * FROM users WHERE id = ?', [1]),
  dbDedup.query('SELECT * FROM users WHERE id = ?', [1]),
  dbDedup.query('SELECT * FROM users WHERE id = ?', [1])
]); // Only one actual database query
```

## Tips

### 1. **Choose Appropriate Cache Keys**
```javascript
// Good: Includes all relevant parameters
const key = `${method}:${url}:${JSON.stringify(params)}`;

// Bad: Too generic
const key = 'user-data';
```

### 2. **Handle Errors Carefully**
```javascript
// Errors should not be cached; allow retry
.catch(error => {
  this.pendingRequests.delete(key);
  throw error; // Don't cache failures
});
```

### 3. **Implement Timeout Protection**
```javascript
async execute(key, requestFn, timeout = 30000) {
  const timeoutPromise = new Promise((_, reject) =>
    setTimeout(() => reject(new Error('Request timeout')), timeout)
  );
  
  return Promise.race([
    this._executeRequest(key, requestFn),
    timeoutPromise
  ]);
}
```

### 4. **Monitor Memory Usage**
```javascript
// Implement size limits
if (this.cache.size > this.maxCacheSize) {
  const firstKey = this.cache.keys().next().value;
  this.cache.delete(firstKey); // LRU eviction
}
```

### 5. **Provide Invalidation Mechanism**
```javascript
// Allow manual cache invalidation
invalidate(pattern) {
  for (const [key] of this.cache) {
    if (key.match(pattern)) {
      this.cache.delete(key);
    }
  }
}
```

### 6. **Use for External APIs**
```javascript
// Prevent rate limiting
const apiDedup = new SmartRequestDeduplicator({ ttl: 1000 });

async function callExternalAPI(endpoint) {
  return apiDedup.execute(endpoint, async () => {
    return fetch(`https://api.example.com${endpoint}`).then(r => r.json());
  });
}
```

### 7. **Combine with Circuit Breaker**
```javascript
// Prevent cascading failures
if (failureCount > threshold) {
  this.pendingRequests.delete(key);
  throw new Error('Circuit breaker open');
}
```

---

**Key Takeaway:** Request deduplication dramatically reduces redundant operations by sharing pending requests. Combine with TTL-based caching for optimal performance in high-concurrency scenarios.