Dealing with Contention
One of the most common deep-dive topics in system design interviews — and, at its core, simpler than most engineers expect.
Contents
Contention is what happens when multiple processes compete for the same resource at the same time — two people racing for the last concert ticket, shoppers fighting over the last flash-sale item, bidders on an auction. If more than one party wants a resource at once, you have a contention issue. Left unhandled, it produces race conditions, double-bookings, and inconsistent state.
01The race condition
One seat is left for The Weeknd. Alice and Bob both hit “Buy Now” in the same instant. The obvious implementation reads the seat count, checks it’s above zero, then decrements and sells:
-- Read the current count SELECT available_seats FROM concerts WHERE concert_id = 'weeknd_tour'; -- The app checks available_seats > 0, then writes the new value back UPDATE concerts SET available_seats = available_seats - 1 WHERE concert_id = 'weeknd_tour';
For a single buyer this is exactly right. With two, it breaks: Alice reads 1 seat. A fraction of a millisecond later, before Alice writes anything back, Bob reads the same count and also sees 1. Both conclude there’s a seat to sell. Alice’s update commits and the count drops to 0. Bob’s commits right after and the count slides to −1. Two cards charged $500, two confirmation emails for the same seat, two people in Row 5 Seat 12.
The window is tiny — microseconds in memory, milliseconds if there’s a network call. But with tens of thousands of concurrent requests all hammering the same row, that tiny window gets hit constantly. The decision about whether to sell is happening in your application server, not atomically inside the database — that’s the whole problem.
02The shared root cause — and the two moves
This failure is a lost update: two requests read the same value, both act, both write back, and one silently overwrites the other. That three-step shape — read → modify → write — is where contention lives. Every fix in this pattern is just a different way of making the gap between the read and the write safe.
There are two fundamental moves, and each relies on the same compare-and-set idea (make the write conditional on the value not having changed):
Move 1 · Collapse
- Fold the read and write into a single step that only lands if nothing changed since you last looked.
- That’s a compare-and-set — the cheapest thing you can do.
Move 2 · Hold still
- Hold the data still while you make a decision, so the gap is safe to stand in.
- Lock everyone else out while you do what you need to do.
SQL is the canonical choice for a datastore that supports this level of contention prevention through isolation and ACID. But the same two moves live in most serious datastores — the video notes Redis achieves the single-writer guarantee because it’s single-threaded, and DynamoDB does it with an optimistic check. We add coordination only as each approach hits its limit.
03Conditional writes — the simplest fix
A lot of safety rules boil down to a simple if: decrement the count if a seat is left; mark the order shipped if it hasn’t been cancelled. When the rule is an if about the current data, the database can check it and make the change in one statement — no locks, no version numbers. Move the check out of application code and into a WHERE clause:
UPDATE concerts SET available_seats = available_seats - 1 WHERE concert_id = 'weeknd_tour' AND available_seats > 0;
The database won’t let two writes change the same row at the same time. When a write touches a row it takes an exclusive lock automatically and holds it until the transaction commits, so any other write to that row waits its turn. Alice’s update goes first, 1 → 0. Bob’s waits, then re-checks available_seats > 0 against the row Alice left behind (not the row Bob first read), finds it false, and matches zero rows. The check and the decrement are a single atomic step — no gap. This row-level locking is how Postgres/MySQL deliver the “I” (Isolation) in ACID.
Purchases need a transaction — and a subtle guard
A real purchase also creates a ticket record, so two writes must stick together. You wrap them in BEGIN…COMMIT. But there’s a catch:
When the conditional UPDATE matches nothing, it succeeds anyway — it just reports “0 rows changed.” So the transaction doesn’t roll back, and an unconditional INSERT after it still runs — handing Bob a ticket for a seat he never got. Make the insert depend on the update actually changing a row.
BEGIN TRANSACTION; WITH reservation AS ( UPDATE concerts SET available_seats = available_seats - 1 WHERE concert_id = 'weeknd_tour' AND available_seats > 0 RETURNING concert_id -- only returns a row if one changed ) INSERT INTO tickets (user_id, concert_id, seat_number, purchase_time) SELECT 'user123', concert_id, 'A15', NOW() FROM reservation; -- writes nothing when update matched nothing COMMIT;
The INSERT … SELECT draws its rows from the update’s result. Alternatively, check the affected-row count in application code and roll back when it’s zero.
Guard the resource actually being contended
This guard only protects the count, not a specific seat. Picture 20 seats left: Alice and Bob both pass available_seats > 0 (20 and 19 both > 0) and both insert a ticket for seat A15. The count is perfectly accurate and two people still own A15. Give every seat its own row and aim the same conditional write at the ticket:
UPDATE tickets SET status = 'sold', user_id = 'user123' WHERE concert_id = 'weeknd_tour' AND seat_number = 'A15' AND status = 'available';
The guard must protect the resource being contended. A counter can answer “is there a ticket left.” Only the ticket row can answer “is this ticket left.”
This handles a huge share of real contention — decrementing inventory, flipping a status from available to sold, claiming an unassigned row, accepting a bid higher than the current one. As long as the database can evaluate your guard as part of the write, you’re done, and you never touched an explicit lock. The WHERE clause is just how SQL spells compare-and-set — the same move as DynamoDB’s ConditionExpression, Redis SET NX, Cassandra’s lightweight transactions, or an HTTP If-Match header.
Junior candidates reach for extra complexity to show off; senior/staff candidates recognize the simple solution is enough. Say it out loud: “This check is simple enough to live in the write itself, so I’ll use a conditional update instead of reaching for an explicit lock.” Postgres or MySQL handle most contention on their own.
04Pessimistic locking — when the decision needs application logic
Now a group of four wants to sit together. Your app must read the seat map, find four open seats in a row, and only then claim them. “Four adjacent open seats” isn’t a predicate you can drop into a WHERE clause, so the read and write can’t collapse into one statement. Two groups both read the same map, both land on A15–A18, and whoever writes second overwrites the other. Back to a lost update. This is where you need an explicit row lock.
Pessimistic locking prevents conflicts by acquiring locks up front — “pessimistic” because it assumes conflicts will happen and blocks them before they can. In SQL you use SELECT … FOR UPDATE, which locks every row the select returns until you commit.
BEGIN TRANSACTION; -- Lock the open seats in this section while we pick a block SELECT seat_number FROM seats WHERE concert_id = 'weeknd_tour' AND section = 'floor' AND status = 'available' FOR UPDATE; -- App scans the result, finds A15-A18 open and adjacent, then claims them UPDATE seats SET status = 'sold', user_id = 'user123' WHERE concert_id = 'weeknd_tour' AND seat_number IN ('A15', 'A16', 'A17', 'A18'); COMMIT;
While you pick a block, no other transaction can claim those seats — a second group running the same query waits until you commit, by which point A15–A18 are gone and drop out of its results. One block each, no seat sold twice. The same shape covers any read-decide-write where the decision isn’t a predicate: checking a balance against a daily limit, reading stock across items before claiming a bundle, routing a task to the least-busy agent.
1. Locking too much, for too long. A lock only buys safety for the rows it covers and the time it’s held — keep both small. Locking a whole table serializes every writer; holding for seconds instead of milliseconds creates a bottleneck. The worst offender is slow I/O: never make a payment API call inside the locked transaction — do that work before taking the lock or after releasing it. Scope the lock to one section, release it the instant the block is claimed.
2. Deadlocks from inconsistent ordering. Two transactions grabbing the same rows in opposite orders deadlock forever. Always acquire locks in a consistent order (e.g. sorted by ID). Full walkthrough in the deep dives.
The main downside: every transaction pays the locking cost, even the overwhelming majority that never collide. If your logic ever simplifies back to something a single WHERE clause can express, drop the lock and return to the conditional write. Don’t pay for a lock you don’t need.
05Optimistic concurrency control — the opposite bet
Optimistic concurrency control (OCC) assumes conflicts are rare and detects them after the fact instead of blocking to prevent them. Everyone proceeds without locking; you catch the collision at write time. Under low contention this skips locking overhead entirely.
OCC needs one value on the row that changes on every write — usually a version number (a dedicated integer you bump by one). Read the row and note the version; when you write, apply the change only if the version is still what you read, and bump it in the same statement. If someone slipped in ahead of you, they already increased the version, so your WHERE matches nothing — your signal to retry or fail.
-- Both Alice and Bob read: 1 seat, version 42 -- Alice writes first: UPDATE concerts SET available_seats = available_seats - 1, version = version + 1 WHERE concert_id = 'weeknd_tour' AND version = 42; -- matches → seats=0, version=43 -- Bob writes against the version he read: UPDATE concerts SET available_seats = available_seats - 1, version = version + 1 WHERE concert_id = 'weeknd_tour' AND version = 42; -- stale! row is on 43 → 0 rows. Roll back.
The same trap applies: a stale version updates zero rows rather than erroring, so your app must check the affected count and roll back (otherwise the INSERT runs and you get a ticket with no seat). Outside SQL, this exact idea is HTTP ETags / If-Match (server returns 412 when the resource moved), etcd revisions, or DynamoDB’s version attribute.
What can serve as the “version”
- Dedicated counter (increments by one every write) — the easiest and safest default.
- Last-updated timestamp — works only if every write touches it and your clock resolution is fine enough that two quick writes can’t land on the same value.
- A business value already on the row (e.g. an auction’s high bid) — safe only if it moves in one direction.
If you reuse a business value, it can change from A → B → A between your read and write, and an equality check can’t see the round trip. E.g. a balance goes $100 → $50 → $100; your “is it still 100?” check passes even though the account changed meaningfully underneath you. Reuse a business value only when it’s monotonic (like an ever-increasing bid); when in doubt, use a dedicated ever-incrementing version.
How often do your writes collide? Common collisions → lock up front (pessimistic): with OCC the loser must re-read and redo work from scratch, and under heavy contention those retries pile up. Rare collisions (most e-commerce) → OCC: happily eat the occasional retry and skip the blocking.
06Isolation levels & write skew — the conflict with no shared row
Conditional writes, locks, and versions all guard a row both transactions touch. But some conflicts hide in what the transactions read, not what they write — so single-row guards sail straight past them.
Write skew: two transactions read an overlapping set of rows, each makes a decision that’s valid on its own, and only together do they break a rule — while writing to different rows.
Invariant: at least one engineer must always be on call. Alice and Bob are both on call and both get the flu. Alice’s transaction reads the schedule, sees Bob still on call, decides it’s safe to remove herself. Bob’s reads the same schedule, sees Alice still on, decides the same. Both commit → nobody is on call. Neither did anything wrong alone. They write to different rows (their own), so there’s no shared row to lock, version, or guard — Alice’s write invalidated Bob’s read and vice versa.
Catching this atomically means having the database track those read-write dependencies — which is what isolation levels control. They set how much one transaction can see of another’s in-flight work. The SQL standard defines four (options, not a ladder you climb):
| Level | Guarantee | Note |
|---|---|---|
| READ UNCOMMITTED | Can see other txns’ uncommitted changes | Dangerous; rarely used — “the one to forget.” |
| READ COMMITTED | Only sees committed data | Default in PostgreSQL. |
| REPEATABLE READ | Data you’ve read stays stable for the txn’s life | Default in MySQL. |
| SERIALIZABLE | Outcome equals some one-at-a-time order | The only level that catches write skew. |
REPEATABLE READ does NOT catch write skew, despite the reassuring name. It only catches two writes landing on the same row; here Alice and Bob write to different ones. Only SERIALIZABLE is guaranteed to catch it — it forces an order and aborts one transaction with a serialization error your app retries.
SERIALIZABLE is the most expensive option — the database must track everything every transaction reads and writes, and every abort throws away work. Whenever you can, restructure so the conflict lands on a single row: keep an on-call count on the team row and decrement it, or take a FOR UPDATE lock on the team row before anyone steps down. Both turn a cross-row invariant back into same-row contention the cheaper tools handle. (Isolation levels are also relational-only — most NoSQL/distributed stores don’t give true SERIALIZABLE, so folding onto one cell is your only option there anyway.)
07Distributed locks — when the hold outlives a transaction
Every tool so far lives inside a single database transaction, and a DB lock only lasts as long as that transaction. But real Ticketmaster holds seat A15 for ~10 minutes while you dig out your credit card. A FOR UPDATE lock can’t do this — it would pin a connection for the full 10 minutes, stall every other transaction needing the row, and it isn’t visible to your other stateless web servers.
A distributed lock holds the lock as data — a record of who holds the resource and when their hold expires — rather than as a transaction. Any server can read it, and it outlives any single transaction. It’s the same pessimistic-locking idea, just held as a lease with its own lifetime. The design question is where the lease lives.
SET with NX (set only if not exists) plus a TTL — atomically creates a lock Redis auto-clears when the TTL passes. Fast and simple; any web server checks the same key.-- Database-column lease: acquire seat A15 for 10 minutes UPDATE seats SET reserved_by = 'user123', reserved_until = NOW() + INTERVAL '10 minutes' WHERE seat_id = 'A15' AND (reserved_until IS NULL OR reserved_until < NOW()); -- 1 row updated = you got the hold; 0 = someone else has it
You aren’t guaranteed the TTL releases at exactly the specified time — a long GC pause or slow network can stall the holder past its TTL. Redis then hands the lock to the next caller, and for a moment two clients think they hold it. For a soft seat hold that’s rare and harmless — because a conditional write at the actual purchase still guards against a double sale underneath. But when a double-grant would truly corrupt data, a plain TTL lock isn’t enough; add a fencing token — a number that climbs with every grant and rides along with the write, so the resource rejects anyone holding a stale one. (Redis is also a single point of failure to plan around.)
It’s slower than Redis, and that lock row becomes a hotspot under heavy load — if 10k–100k people compete for the same lock, the DB can’t take the write throughput. That’s when you pull it into in-memory Redis. Reach for ZooKeeper/etcd only when you need something that can never double-grant and the DB isn’t viable.
The same moves outside SQL
| Technique | In SQL | The same move elsewhere |
|---|---|---|
| Conditional write | WHERE predicate on the write | DynamoDB ConditionExpression, Redis SET NX, Cassandra LWT, HTTP If-Match |
| Optimistic concurrency | version column + WHERE version = … | HTTP ETags / If-Match, etcd revision, DynamoDB version attribute |
| Pessimistic locking | SELECT … FOR UPDATE | a mutex or a distributed lock held while you decide |
| Serializable isolation | ISOLATION LEVEL SERIALIZABLE | mostly relational only → fold the invariant onto one cell |
| Distributed lock | reservation row with a TTL | Redis SET NX EX, ZooKeeper or etcd lease |
Keep the contended resource in one authoritative store — every technique depends on it. Two situations break that and leave contention entirely: (1) one operation must atomically span multiple services/shards (a transfer between sharded accounts) → that’s a distributed transaction (sagas, 2PC, outbox — the Multi-step Processes pattern); (2) the same record is writable in multiple places at once (multi-leader/leaderless replication) → that’s conflict resolution (last-write-wins, vector clocks, CRDTs).
08Choosing the right approach
These choices step up in complexity. The common mistake is hearing “concurrency” and reaching straight for something heavy — a pessimistic lock or a Redis lock — when a conditional write was the simple correct choice. Every component you add is a new failure mode and something else to explain. A single modern Postgres instance absorbs far more contention at a single source of truth than most people assume.
09Using it in an interview
Don’t wait to be asked — flag contention when you see multiple processes competing for a shared resource, typically once you’ve decided the system needs strong consistency in your non-functional requirements. Recognition signals: limited resources (tickets, auctions, flash-sale inventory, driver matching), preventing double-booking/double-charging, consistency under high concurrency (balances, inventory), and order-sensitive operations across servers.
| Scenario | Go-to approach |
|---|---|
| Online auction | OCC using the current high bid as the version (bids only go up → no ABA); accept a bid only if higher than expected current. |
| Ticketmaster / booking | Seat selection looks like pessimistic locking, but the bigger win is a 10-min reservation via a distributed lock — avoids losing a seat after filling out payment. |
| Banking / payments | Within one DB → pessimistic locking or OCC (lives here). Across services/shards → distributed transaction (Multi-step Processes). |
| Ride-sharing dispatch | Set driver status to pending_request with a TTL so simultaneous requests can’t double-book a driver; stale request reads as free. |
| Flash sale / inventory | OCC with a dedicated version column + cart “holds” (distributed lock with TTL) to cut checkout contention. |
| Yelp / reviews | OCC with a dedicated version column when updating a business’s average rating from concurrent reviews. |
Don’t reach for a distributed lock (Redis, etc.) when a DB transaction with row locking or OCC suffices. Low-contention edits (admin-only product descriptions), single-user operations (private docs, todo lists, preferences), and read-heavy workloads need little or no coordination — basic OCC with retry handles the rare conflict.
10Common deep dives
Preventing deadlocks with pessimistic locking
Alice transfers $100 to Bob while Bob simultaneously transfers $50 to Alice. Transaction A locks Alice then reaches for Bob; Transaction B locks Bob then reaches for Alice. Each holds the lock the other needs — they wait forever. That’s a deadlock, caused by acquiring locks in different orders.
Always acquire locks in a consistent, deterministic order — sort the resources by a global key (e.g. user ID) before locking any. If Alice (456) transfers to Bob (123), lock Bob first because 123 < 456; if Bob transfers to Alice, still lock Bob first. Sort by the key, not “lock the initiator first.” This makes the circular wait impossible.
A lock_wait timeout alone just repeats the problem — the retry can grab locks in the same order and deadlock again. Better: every major database runs automatic deadlock detection, killing one transaction with a retryable error. So use ordered locking to prevent cycles, and lean on the detector to clean up the rest.
Handling the ABA problem with OCC
If you reuse a business value as the version (e.g. a restaurant’s review_count), it can go 100 → 99 (a review deleted) → 100 (a new one lands) between your read and write. Your “is it still 100?” check passes and you silently clobber the update in between.
-- Fix: a dedicated version column that increments on every write UPDATE restaurants SET avg_rating = 4.1, review_count = review_count + 1, version = version + 1 WHERE restaurant_id = 'pizza_palace' AND version = 42;
If you can’t add a version column, the fallback is to put every field you read into the WHERE clause so the write only applies if the whole row still matches. Some databases also expose a built-in row version (e.g. Postgres’s system xmin).
Performance when everyone wants the same resource (the hot partition / celebrity problem)
A surprise Taylor Swift show, a rare eBay drop — everyone hammers one row. Normal scaling breaks down: sharding splits across rows, but everyone wants the same row, so there’s nothing to split. Load balancing just queues requests on that same row. Read replicas offload reads, but the fight is over writes.
First, question the problem itself. Can you split one hot resource into many? Ten identical collectibles → ten rows → contention divided by ten. Does a celebrity follower count need to be accurate to the second, or can it be eventually consistent and batched?
Put a queue in front of the database — all requests line up, and a single worker pulls off and applies updates one by one. Contention disappears entirely. The trade-off is throughput: you’ve capped the resource to how fast one worker goes (and that worker is a SPOF you’d back with a standby). You can have accurate contention handling or high throughput on a single hot row — not both.
11The one idea underneath all of it
Every contended resource has a single source of truth — the one place that owns the real value — and that’s where correctness gets enforced. Conditional writes, pessimistic locking, isolation levels, and OCC are just different ways of coordinating access at that source of truth, escalating as the gap you must protect grows; a distributed lock takes over when the gap outlives a transaction. Your first job, always, is to make the contested thing exist as a single addressable cell — a row, a key, or an item — that the store can check as it writes.
A modern database like PostgreSQL absorbs far more contention at a single source of truth than people assume. Reach for external locks and reservations when traffic or UX demands it, not by default. And keep the boundary clear: the moment an operation spans multiple sources of truth, you’ve left contention and entered distributed-transaction territory.
12Active recall
What is the root cause shared by every contention bug, and what are the two ways to fix it?
A lost update in a read-modify-write: a second request slips into the gap between the read and the write and acts on stale data. Two fixes, both forms of compare-and-set: collapse the read and write into one atomic step (conditional write), or hold the data still while you decide (a lock).
Why is UPDATE … WHERE available_seats > 0 already concurrency-safe?
The database won’t let two writes change the same row at once — it takes an automatic exclusive lock until commit. The second writer waits, then re-checks the condition against the row the first writer left behind, not the row it originally read. Check + decrement are one atomic step, so there’s no gap. This is row-level locking delivering the “I” in ACID.
Why can wrapping the decrement + insert in a transaction still oversell — and how do you fix it?
A conditional UPDATE that matches zero rows succeeds (0 rows changed) rather than erroring, so it doesn’t roll back the transaction, and an unconditional INSERT after it still runs. Fix: make the insert depend on the update actually changing a row — WITH … UPDATE … RETURNING feeding an INSERT … SELECT, or check the affected-row count and roll back when it’s zero.
When must you use pessimistic locking instead of a conditional write?
When the decision runs in application code between the read and the write and can’t be expressed as a WHERE predicate — e.g. finding four adjacent open seats. Use SELECT … FOR UPDATE to lock the rows you read, decide while they’re held, and write before releasing. If the logic ever simplifies to a single predicate, drop the lock and go back to the conditional write.
Pessimistic vs optimistic concurrency — what decides it, and what can serve as the version?
Collision frequency. Frequent collisions → pessimistic (OCC retries pile up under heavy contention). Rare collisions (most e-commerce) → OCC. The version can be a dedicated ever-incrementing counter (safest), a last-updated timestamp (if resolution is fine enough), or a monotonic business value like an ever-increasing bid — never a business value that can move both ways (ABA problem).
What is write skew, and why do row locks / versions miss it?
Two transactions read overlapping rows, each makes a locally valid decision, and only together break an invariant — while writing to different rows (e.g. both on-call engineers stepping down). There’s no shared row to guard, so conditional writes, versions, and row locks all pass. Only SERIALIZABLE catches it (REPEATABLE READ does not). Cheaper: materialize the invariant onto a single row (a count or a FOR UPDATE on the team row).
Why can’t FOR UPDATE hold a 10-minute seat reservation, and what replaces it?
It would pin a connection for 10 minutes, stall every other transaction needing the row, and isn’t visible to other stateless servers. Replace it with a distributed lock: hold the lock as data (a lease: who holds it + expiry) that any server can read and that outlives the transaction — Redis SET NX + TTL, DB columns, or ZooKeeper/etcd.
What’s the risk with a Redis TTL lock, and when does it matter?
The TTL may not release exactly on time (GC pause, slow network), so Redis can hand the lock to the next caller while the original holder still thinks it holds it — a brief double-grant. Harmless for a soft seat hold (a conditional write at purchase still guards the sale). When a double-grant would corrupt data, add a fencing token (an ever-climbing number carried with the write) so the resource rejects stale holders.
Everyone wants the same row. Why doesn’t sharding help, and what does?
Sharding splits load across rows, but everyone wants the same row — nothing to split; replicas only offload reads, not the write fight. First, change the problem (split one resource into many rows, or accept eventual consistency). If you truly need strong consistency on one hot row, serialize with a queue + single worker — contention vanishes, but throughput is capped to that worker.
13Quick revision
The frame
Contention = a read-modify-write gap. Make it safe by either collapsing (compare-and-set) or holding still (lock). The store can only protect a conflict it can see → make the contested thing a single addressable cell.
The escalation ladder
| Tool | Use when | Watch out |
|---|---|---|
| Conditional write (WHERE) | Check is a predicate on the row you write | 0-row UPDATE isn’t an error; guard the contended resource, not just a counter |
| Pessimistic lock (FOR UPDATE) | Read-decide-write in app code; high contention | Lock small/short; no slow I/O inside; order locks to avoid deadlock |
| Optimistic (version) | Read-decide-write; rare collisions | Retries pile up under contention; ABA with business values |
| SERIALIZABLE | Write skew / cross-row invariant | Expensive; REPEATABLE READ won’t catch it; prefer materializing onto one row |
| Distributed lock (lease + TTL) | Hold spans a wait / external call / steps | TTL double-grant → fencing token when corruption matters |
Must remember
- Guard the contended resource: a counter answers “is a ticket left”; only the seat row answers “is this seat left.”
- Zero-row UPDATE succeeds — gate follow-up writes on the affected-row count or use
RETURNING. - Deadlocks: always acquire locks in a globally consistent order (sort by key); lean on the DB’s detector.
- ABA: reuse a business value only if it moves one way; else a dedicated incrementing version.
- Write skew ≠ same-row conflict — only SERIALIZABLE catches it; better, fold onto one row.
- Hot row: can’t shard; question the problem, else serialize with a queue + single worker (caps throughput).
- Defaults: Postgres = READ COMMITTED, MySQL = REPEATABLE READ.
- Start simple. A single Postgres instance absorbs more contention than people assume.
Where the pattern ends
One operation spanning multiple services/shards → distributed transactions / sagas. Same record writable in many places → conflict resolution (LWW, vector clocks, CRDTs). Both are outside contention.