Skip to main content

Command Palette

Search for a command to run...

The Bug That Taught Me About Race Conditions

Learn: The Bug That Taught Me About Race Conditions

Updated
7 min readView as Markdown
T

Welcome to TopperBlog! 👋

I'm a tech content creator passionate about helping developers level up their careers and master cutting-edge technologies.

🎯 What I Write About: • AI/ML Engineering & LLMs • Web3 & Blockchain Development
• System Design & Architecture • Interview Preparation (FAANG) • Freelancing & Remote Work • Modern Tech Stacks (Next.js, React, Rust, TypeScript) • Performance Optimization & Best Practices

💼 Mission: Sharing practical, actionable insights that accelerate your tech career and maximize your earning potential.

📚 15+ In-Depth Guides covering everything from earning $10k/month as a freelancer to cracking FAANG interviews.

🌐 Let's connect and grow together in this amazing tech journey!

#TechBlogger #SoftwareEngineering #CareerGrowth #WebDevelopment #AIEngineering

The Bug That Taught Me About Race Conditions

When 47 users bought the same concert ticket, and I learned concurrency the hard way

I'll never forget the Monday morning I walked into the office to find 47 angry support tickets, three Slack channels on fire, and my product manager standing by my desk with what I can only describe as "the look." You know the one—equal parts confusion, disappointment, and "we need to talk."

"Did you know," she said, her voice unnaturally calm, "that we just sold the same VIP concert ticket to 47 different people?"

I did not know that. But I was about to get a crash course in race conditions that no computer science textbook could have prepared me for.

The Setup: Everything Was Fine (Until It Wasn't)

Let me take you back a few weeks. Our startup had built this slick ticket-selling platform. Think Ticketmaster, but smaller and with way more optimism than funding. The app was straightforward: users browse events, click "Buy Now," and boom—they've got tickets.

For months, everything worked beautifully. We'd processed thousands of transactions without a hitch. I was feeling pretty good about my code. Maybe even a little cocky.

Then we landed our first big client: a popular indie band with a genuinely rabid fanbase. They were releasing exactly 100 VIP meet-and-greet tickets at noon on a Friday. We're talking the kind of tickets that sell out in seconds. The kind where fans have browser tabs open, fingers hovering over refresh buttons, ready to pounce.

I remember thinking, "This is going to be great for our metrics!"

Narrator voice: It was not great for their metrics.

The Nightmare Unfolds

At 12:00:01 PM, our servers got absolutely hammered. Thousands of requests per second. Our monitoring dashboard looked like a heart rate monitor during a panic attack. But hey, the site stayed up! I high-fived my coworker. We were handling the load!

By 12:00:15 PM, all 100 tickets showed as "Sold Out" on the website. Perfect. Exactly as planned.

By Monday morning, we'd somehow confirmed 147 ticket purchases for those same 100 tickets.

Record scratch. Freeze frame. Yep, that's me. You're probably wondering how I got into this situation.

What Went Wrong: The Race Condition From Hell

Here's what my code looked like. Innocent, right?

def purchase_ticket(event_id, user_id):
    # Check if tickets are available
    available_tickets = db.query(
        "SELECT remaining_tickets FROM events WHERE id = ?", 
        event_id
    )

    if available_tickets > 0:
        # Great! Let's sell one
        db.execute(
            "UPDATE events SET remaining_tickets = remaining_tickets - 1 WHERE id = ?",
            event_id
        )

        db.execute(
            "INSERT INTO purchases (event_id, user_id) VALUES (?, ?)",
            event_id, user_id
        )

        return {"success": True, "message": "Ticket purchased!"}
    else:
        return {"success": False, "message": "Sold out!"}

Looks reasonable, doesn't it? Check inventory, decrease inventory, record the sale. What could go wrong?

Everything. Everything could go wrong.

Here's what actually happened in those 15 seconds of chaos:

  1. User A's request arrives: "Are there tickets?" → "Yes! 100 available!"
  2. User B's request arrives: "Are there tickets?" → "Yes! 100 available!"
  3. User C's request arrives: "Are there tickets?" → "Yes! 100 available!"
  4. Users D through Z's requests arrive...

You see the problem? Between checking the inventory and updating it, there was a gap. A tiny, microscopic gap in time. But when you've got thousands of concurrent requests, that gap becomes a six-lane highway for bugs to drive through.

This is a race condition—multiple processes racing to access and modify the same data, with the outcome depending on the precise timing of their execution. It's like if you and your roommate both saw one slice of pizza left, both reached for it, and somehow you both ended up with a slice. Except in our case, 47 people ended up with the same slice.

The Solution: Atomic Operations and Pessimistic Locking

After a weekend of stress-eating and reading database documentation, I learned about atomic operations and pessimistic locking. These became my new best friends.

Here's the fixed version:

def purchase_ticket(event_id, user_id):
    # Start a database transaction
    with db.transaction():
        # Lock the row for update - no one else can read it until we're done
        event = db.query(
            "SELECT remaining_tickets FROM events WHERE id = ? FOR UPDATE",
            event_id
        )

        if event.remaining_tickets > 0:
            # Update and insert in the same transaction
            db.execute(
                "UPDATE events SET remaining_tickets = remaining_tickets - 1 WHERE id = ?",
                event_id
            )

            db.execute(
                "INSERT INTO purchases (event_id, user_id) VALUES (?, ?)",
                event_id, user_id
            )

            return {"success": True, "message": "Ticket purchased!"}
        else:
            return {"success": False, "message": "Sold out!"}

The magic is in that FOR UPDATE clause. It's like putting a "Do Not Disturb" sign on that database row. When User A's request locks the row, Users B through Z have to wait their turn. No more checking the same inventory count simultaneously.

But I didn't stop there. I also implemented an optimistic locking approach as a backup:

def purchase_ticket_optimistic(event_id, user_id):
    max_retries = 3

    for attempt in range(max_retries):
        # Read the current version
        event = db.query(
            "SELECT remaining_tickets, version FROM events WHERE id = ?",
            event_id
        )

        if event.remaining_tickets > 0:
            # Try to update only if version hasn't changed
            rows_affected = db.execute(
                """UPDATE events 
                   SET remaining_tickets = remaining_tickets - 1,
                       version = version + 1
                   WHERE id = ? AND version = ?""",
                event_id, event.version
            )

            if rows_affected > 0:
                # Success! The version matched, so no one else modified it
                db.execute(
                    "INSERT INTO purchases (event_id, user_id) VALUES (?, ?)",
                    event_id, user_id
                )
                return {"success": True, "message": "Ticket purchased!"}
            else:
                # Someone else modified it, retry
                continue
        else:
            return {"success": False, "message": "Sold out!"}

    return {"success": False, "message": "Too much traffic, please try again"}

This approach is more forgiving under high load. Instead of making everyone wait in line, it lets them try, and if there's a conflict, they retry. It's like when you're editing a Google Doc and someone else makes a change—you get a notification and can try again.

The Aftermath: Refunds, Apologies, and Redemption

We had to refund 47 people. Well, technically 46—we let one lucky person keep their ticket and gave the other 46 a full refund plus credit for future purchases. Our client was surprisingly understanding, probably because we were honest about what happened and fixed it immediately.

The next big sale we handled? Flawless. Zero oversells. I watched the monitoring dashboard like a hawk, but the new code held up beautifully.

Lessons I'll Never Forget

1. Concurrency is not optional thinking

If more than one user can access your system at the same time (spoiler: they can), you need to think about concurrency. It doesn't matter if you're a tiny startup. It only takes two simultaneous requests to create a race condition.

2. "It works on my machine" means nothing

I'd tested the ticket purchasing flow dozens of times. It worked perfectly! But I was testing sequentially, one request at a time. Real-world traffic doesn't work that way. Load testing isn't just about volume—it's about concurrency.

3. Databases are smarter than you think

I used to think database features like transactions, locks, and isolation levels were advanced topics I'd learn "someday." Turns out, they're fundamental. FOR UPDATE, SERIALIZABLE isolation, optimistic locking—these aren't fancy extras. They're essential tools.

4. There's no shame in not knowing

I didn't know about race conditions in a practical sense until they bit me. That's okay. What matters is that I learned, fixed it, and made sure it wouldn't happen again. Every senior developer you admire has a story like this.

5. Always have a rollback plan

We were lucky we could identify and refund the oversold tickets. Now, every feature I ship has a "what if this goes horribly wrong" plan. It's not pessimism—it's professionalism.

The Takeaway

Race conditions are sneaky. They hide in code that looks perfectly fine. They don't show up in development. They wait until you have real traffic, real users, and real stakes. Then they strike.

But here's the thing: once you've been burned by a race condition, you start seeing them everywhere. You develop a sixth sense. You look at code and think, "What if two users do this at the exact same time?"

That concert ticket disaster was embarrassing and stressful, but it made me a better developer. Now, whenever I write code that modifies shared state, I pause and ask myself: "What's racing here?"

And I always, always use proper locking.

Your turn: Have you been bitten by a race condition? Drop a comment. Let's commiserate together. Because if there's one thing I've learned, it's that we're all just one concurrent request away from our next humbling experience.

P.S. - That indie band? They're still our client. And yes, I get nervous every time they release tickets. Some scars never fully heal.