The API I Wish I Never Built: Design Regrets

The 3 AM Wake-Up Call

My phone buzzed at 3:17 AM. Then again at 3:19. By 3:23, I had twelve Slack messages, all saying the same thing: "The API is down. Again."

I rolled out of bed, opened my laptop, and stared at the monitoring dashboard. 47,000 requests queued. Database connections maxed out. Redis screaming. And there, in the error logs, was the endpoint I'd built eighteen months ago—the one I'd been so proud of.

POST /api/v1/users/batch-update

That single endpoint was bringing down our entire platform. And it was entirely my fault.

The Mistake That Kept On Giving

Let me take you back to where it all went wrong. We needed a way for clients to update multiple user records at once. Simple enough, right? I was under pressure, the deadline was tight, and I made what seemed like pragmatic decisions.

Here's what I built:

// The API endpoint that haunted my dreams
app.post('/api/v1/users/batch-update', async (req, res) => {
  const { users } = req.body; // Array of user objects

  // Mistake #1: No size limit
  for (const user of users) {
    // Mistake #2: Sequential processing
    await db.users.update(user.id, {
      name: user.name,
      email: user.email,
      preferences: user.preferences, // Mistake #3: Nested object without validation
      metadata: user.metadata // Mistake #4: Arbitrary data accepted
    });

    // Mistake #5: Synchronous cache invalidation
    await cache.delete(`user:${user.id}`);

    // Mistake #6: Webhook calls in the request cycle
    await notifyWebhooks(user.id, 'user.updated');
  }

  res.json({ success: true, updated: users.length });
});

It worked beautifully in testing. It worked fine in the first few months of production. Then our biggest client decided to sync their entire user database—250,000 records—in a single request.

The Seven Deadly Sins of API Design

Sin #1: Unbounded Input

I never set a limit on the array size. Why would I? "Let the clients decide what they need," I thought.

The Reality: One client sent 250,000 users. The request timeout was 30 seconds. The math didn't math.

What I Should Have Done:

app.post('/api/v1/users/batch-update', async (req, res) => {
  const { users } = req.body;

  // Set clear boundaries
  if (!Array.isArray(users) || users.length === 0) {
    return res.status(400).json({ 
      error: 'users must be a non-empty array' 
    });
  }

  if (users.length > 100) {
    return res.status(400).json({ 
      error: 'Maximum 100 users per request. Use pagination for larger batches.',
      max_allowed: 100,
      received: users.length
    });
  }

  // ... rest of the logic
});

Sin #2: Synchronous Processing

I processed everything sequentially in the request-response cycle. Each database write, each cache invalidation, each webhook—all blocking the next operation.

What I Should Have Done:

app.post('/api/v1/users/batch-update', async (req, res) => {
  const { users } = req.body;

  // Validate and create a job
  const jobId = uuidv4();

  await jobQueue.add('batch-user-update', {
    jobId,
    users,
    requestedBy: req.user.id,
    requestedAt: new Date()
  });

  // Return immediately
  res.status(202).json({
    jobId,
    status: 'processing',
    statusUrl: `/api/v1/jobs/${jobId}`,
    message: 'Batch update queued. Check status at the provided URL.'
  });
});

// Separate worker process
async function processBatchUpdate(job) {
  const { jobId, users } = job.data;

  for (const user of users) {
    try {
      await db.users.update(user.id, user);
      await updateJobProgress(jobId, 'success', user.id);
    } catch (error) {
      await updateJobProgress(jobId, 'failed', user.id, error);
    }
  }
}

Sin #3: No Input Validation

I accepted nested objects without validation. preferences could be anything. metadata could be a 10MB JSON blob. I trusted the clients way too much.

What I Should Have Done:

const Joi = require('joi');

const userUpdateSchema = Joi.object({
  id: Joi.string().uuid().required(),
  name: Joi.string().max(100).optional(),
  email: Joi.string().email().optional(),
  preferences: Joi.object({
    theme: Joi.string().valid('light', 'dark'),
    notifications: Joi.boolean(),
    language: Joi.string().length(2)
  }).optional(),
  metadata: Joi.object().max(10).optional() // Max 10 keys
});

const batchSchema = Joi.object({
  users: Joi.array().items(userUpdateSchema).min(1).max(100).required()
});

app.post('/api/v1/users/batch-update', async (req, res) => {
  const { error, value } = batchSchema.validate(req.body);

  if (error) {
    return res.status(400).json({
      error: 'Validation failed',
      details: error.details.map(d => d.message)
    });
  }

  // Process validated data
});

Sin #4: Mixing Concerns

I mixed immediate operations (database writes) with side effects (webhooks, notifications) in the same transaction. When webhooks were slow, everything was slow.

What I Should Have Done:

// Separate immediate operations from side effects
async function updateUsers(users) {
  // Immediate: Database updates only
  const results = await db.transaction(async (trx) => {
    return Promise.all(
      users.map(user => 
        trx.users.update(user.id, user)
      )
    );
  });

  // Deferred: Side effects via event bus
  for (const user of users) {
    await eventBus.publish('user.updated', {
      userId: user.id,
      timestamp: Date.now()
    });
  }

  return results;
}

// Separate service handles webhooks
eventBus.subscribe('user.updated', async (event) => {
  await notifyWebhooks(event.userId, 'user.updated');
});

Sin #5: No Idempotency

If a request failed halfway through, retrying it would create duplicate updates or inconsistent state. No idempotency keys, no deduplication.

What I Should Have Done:

app.post('/api/v1/users/batch-update', async (req, res) => {
  const idempotencyKey = req.headers['idempotency-key'];

  if (!idempotencyKey) {
    return res.status(400).json({
      error: 'Idempotency-Key header required'
    });
  }

  // Check if we've seen this request before
  const cached = await cache.get(`idempotency:${idempotencyKey}`);
  if (cached) {
    return res.status(200).json(cached);
  }

  // Process the request
  const result = await processBatchUpdate(req.body.users);

  // Cache the result for 24 hours
  await cache.set(
    `idempotency:${idempotencyKey}`, 
    result, 
    { ttl: 86400 }
  );

  res.status(200).json(result);
});

Sin #6: Poor Error Handling

When something failed, the entire batch failed. No partial success reporting, no way to know which records succeeded and which didn't.

What I Should Have Done:

async function processBatchUpdate(users) {
  const results = {
    successful: [],
    failed: [],
    total: users.length
  };

  for (const user of users) {
    try {
      await db.users.update(user.id, user);
      results.successful.push({
        id: user.id,
        status: 'updated'
      });
    } catch (error) {
      results.failed.push({
        id: user.id,
        error: error.message,
        code: error.code
      });
    }
  }

  return {
    ...results,
    successRate: results.successful.length / results.total
  };
}

Sin #7: No Rate Limiting

Any client could hammer this endpoint as much as they wanted. No rate limits, no throttling, no backpressure.

What I Should Have Done:

const rateLimit = require('express-rate-limit');

const batchUpdateLimiter = rateLimit({
  windowMs: 15 * 60 * 1000, // 15 minutes
  max: 10, // 10 requests per window
  message: {
    error: 'Too many batch update requests',
    retryAfter: '15 minutes'
  },
  standardHeaders: true,
  legacyHeaders: false,
});

app.post('/api/v1/users/batch-update', 
  batchUpdateLimiter,
  async (req, res) => {
    // ... handler
  }
);

The Redesign

After that 3 AM incident, I spent two weeks redesigning the API. Here's what the new version looks like:

// v2: The API I wish I'd built the first time
app.post('/api/v2/users/batch-update', 
  authenticate,
  rateLimit({ windowMs: 900000, max: 10 }),
  async (req, res) => {
    // 1. Validate input
    const { error, value } = batchSchema.validate(req.body);
    if (error) {
      return res.status(400).json({
        error: 'Validation failed',
        details: error.details
      });
    }

    // 2. Check idempotency
    const idempotencyKey = req.headers['idempotency-key'];
    if (!idempotencyKey) {
      return res.status(400).json({
        error: 'Idempotency-Key header required'
      });
    }

    const cached = await cache.get(`idempotency:${idempotencyKey}`);
    if (cached) {
      return res.status(200).json(cached);
    }

    // 3. Create async job
    const jobId = uuidv4();
    await jobQueue.add('batch-user-update', {
      jobId,
      users: value.users,
      requestedBy: req.user.id,
      idempotencyKey
    }, {
      attempts: 3,
      backoff: { type: 'exponential', delay: 2000 }
    });

    // 4. Return immediately
    const response = {
      jobId,
      status: 'queued',
      statusUrl: `/api/v2/jobs/${jobId}`,
      estimatedCompletion: new Date(Date.now() + value.users.length * 100)
    };

    await cache.set(`idempotency:${idempotencyKey}`, response, { ttl: 86400 });

    res.status(202).json(response);
  }
);

// Status check endpoint
app.get('/api/v2/jobs/:jobId', authenticate, async (req, res) => {
  const job = await jobQueue.getJob(req.params.jobId);

  if (!job) {
    return res.status(404).json({ error: 'Job not found' });
  }

  const state = await job.getState();
  const progress = job.progress();

  res.json({
    jobId: job.id,
    status: state,
    progress: progress,
    result: state === 'completed' ? job.returnvalue : null,
    error: state === 'failed' ? job.failedReason : null
  });
});

The Lessons That Stuck

1. Async is your friend for batch operations. Never process large batches synchronously. Return a job ID immediately and let workers handle the heavy lifting.

2. Boundaries are not optional. Set limits on everything: array sizes, string lengths, nesting depth, request rates. Your future self will thank you.

3. Validate everything. Trust no one, not even your own frontend. Schema validation isn't paranoia—it's professionalism.

4. Idempotency is not a nice-to-have. In distributed systems, requests will be retried. Make sure retries are safe.

5. Separate concerns ruthlessly. Database writes, cache invalidation, webhooks, notifications—these should never block each other.

6. Partial success is success. Batch operations should report granular results. All-or-nothing is rarely the right choice.

7. Rate limiting is self-defense. Protect your API from both malicious actors and well-meaning clients who don't know better.

The Takeaway

That 3 AM wake-up call cost us three hours of downtime, angry customers, and a week of emergency fixes. But it taught me something invaluable: the best time to think about API design is before you write the first line of code. The second best time is right now.

If you're building an API today, especially one that handles batch operations, learn from my mistakes. Add those validations. Implement that rate limiting. Make it async. Your 3 AM self will thank you.

And if you've already built that problematic API? It's never too late to version it properly. v2 exists for a reason.

What's the API design decision you regret most? I'd love to hear your war stories in the comments.