Flashsale System
A backend for flash sales thousands of buyers, limited stock,one second. Built twice: synchronously, then queue-based.
⚡ Engineering Challenge: Shared State Under Pressure
Flash sales fail for one reason: shared-state contention. When thousands of people hit "Buy" in the same second, a naive "check stock → write order" flow turns the database into a queue with extra steps. The real challenge wasn't the endpoint — it was building a control layer that stays correct when everything underneath it is under load.
Reservation vs. Confirmation No Oversell
Two counters, two different questions. Redis answers "can I accept another order?" via
atomic DECR/INCR. Postgres answers
"what actually sold." Confirming an order must never touch the reservation counter — the one bug
that broke this rule oversold ~146 units in testing.
Lock Duration, Not Architecture Root Cause
The synchronous path's "collapse" (73% failures) wasn't blocking I/O or a bad architecture — it was a row lock held across three network round-trips instead of one atomic statement. Fixing that one line took it from 184/1,000 units sold to 1,000/1,000.
🐛 The Bug Hunt — 21 Defects, Audited Against the Code
Every claim below comes from a k6 run or a SQL query against the database after that run — not from memory of what was built. Six of the sharper findings:
Deleted the cached response on success and cached the result on failure — the exact opposite of what an idempotency layer is for. A retry after success consumed a second unit of stock; a retry after a transient error was locked out for the rest of the 5-minute sale.
Writing Postgres's confirmed count back over Redis's reservation count made every pending reservation evaporate — and let the API accept ~146 more orders against stock that no longer existed. The worst bug in the project.
The report claimed the worker's stock re-check was removed. It was moved — from Redis to a
SELECT … FOR UPDATE in Postgres — which serialised every payment worker
behind one row lock, and was quietly the only thing preventing real overselling.
SELECT … FOR UPDATE held a lock across a read → compute → write →
commit cycle. Swapping it for an atomic
UPDATE … WHERE stock > 0 RETURNING cut the lock window from tens of
milliseconds to microseconds — same guarantee, no architectural change required.
Between reserving a unit in Redis and successfully handing it to the queue, two operations could throw. If either did, the unit was reserved forever and released to no one. Fixed with an explicit "do I still own this reservation?" flag around the release path.
sequelize.sync({ alter: true }) ran in all four clustered API workers
simultaneously — the first dropped a constraint, the next found it missing and crashed mid-benchmark,
invalidating earlier runs and corrupting the schema on every restart.
🧠 Approach 1 vs. Approach 2
Its purchase_latency is what the customer waits — the response is the
confirmation. Median 1,090 ms, p99 11.3 s, worst case 24.5 s: a real person watching a spinner, plus
62 server errors and 3 units of inventory nobody could account for at the end of the run.
It doesn't hold a database transaction or a socket to a third party while the customer waits. It costs real seconds of fulfillment latency and, under a closed-load model, less raw throughput. What it buys: a 6 ms answer instead of a multi-second one, zero server errors across 108,844 requests, and exact reconciliation between reserved, confirmed, and stranded stock.
🏗️ High-level Architecture
📸 Architecture Evolution (Before → After)
Real diagrams from the project: synchronous request path vs. distributed execution engine.
The pivot was simple: stop doing critical decisions inside the DB request path, and move them into a control layer with a measured, atomic reservation — then let workers do the slow, external part without anyone holding a connection open for it.