Consistency and Distributed Transactions
While all data sits in one database, correctness is free — the DB gives you an ACID transaction, and either every change applies or none does. The moment a service is split into several processes and databases, that guarantee disappears at the boundary of the first network call. You can no longer debit money in one service and reserve stock in another within a single DB transaction — there is a network between them that drops packets, stalls, and breaks. Correctness of distributed data has to be designed explicitly.
The central trap of this topic is carrying the intuition of a single-database transaction into the distributed world. The CAP theorem says that during a network partition you must choose between consistency and availability — you cannot promise both at once. Strong consistency costs latency and availability; eventual consistency buys them back at the price of replicas diverging for a while. Two-phase commit gives atomicity but blocks participants and hangs when the coordinator dies. The application-level patterns — TCC and SAGA — give up the global lock in favor of reservations and compensations, but demand idempotency at every step. This topic walks that spectrum from CAP theory to the concrete distributed-transaction protocols.
Topic Map
- The CAP theorem — during a network partition the system chooses between consistency and availability (CP or AP), not "two of three" in steady state.
- Consistency models — the spectrum from strong (linearizable) to eventual, with read-your-writes and monotonic reads in between, trading latency for availability.
- Two-phase commit — a coordinator gathers votes via PREPARE and commits only on a unanimous "yes," but blocks participants and hangs when it dies.
- Three-phase commit — adds a PRE-COMMIT phase so participants can decide without the coordinator, at the cost of an extra round trip and an assumption of a synchronous network.
- The TCC pattern — an application-level try-confirm-cancel distributed transaction: reserve the resource, then confirm or cancel across all services, with idempotency.
- The SAGA pattern — a long transaction as a chain of local steps, each with a compensating action to undo it, with no global locks and eventual consistency.
Common Mistakes and Traps
| Mistake | Consequence |
|---|---|
| Promising consistency and availability at once | CAP forbids it during a partition — the C-vs-A choice is unmade, behavior under failure is unpredictable |
| Reading CAP as "pick two of three properties" in normal mode | The point is lost: the choice arises only during a partition, not in steady state |
| Treating strong consistency as free | A linearizable read pays in latency and availability on every operation |
| Dragging 2PC across the network between services | Participants hold locks and hang if the coordinator dies after PREPARE |
| Measuring 2PC latency by the average node | The commit's tail latency is the slowest participant, not the average |
| Implementing TCC without idempotent try/confirm/cancel | A retry re-reserves or double-charges the same resource |
| Not persisting reservation state in TCC and SAGA | After a restart it's unclear what to confirm and what to compensate |
| Assuming SAGA gives atomicity like a DB transaction | SAGA gives only eventual consistency via compensations, with no isolation |
| Writing a non-idempotent SAGA compensation | A repeated rollback corrupts data or undoes someone else's step |
Interview Relevance
Consistency and distributed transactions are a mandatory part of system design at the senior level of a Go interview, and the question is not memorizing the letters of CAP but whether you understand which trade-off you choose deliberately and what you pay for it. The interviewer checks whether you distinguish behavior under a network partition from the calm state, whether you see the cost of strong consistency, and whether you know why 2PC across the network is dangerous while application patterns like TCC and SAGA survive on compensations and idempotency.
What interviewers usually check:
- What CAP actually claims — the choice of C or A only during a partition, not "two of three" in steady state.
- How strong consistency differs from eventual and which models sit between them (read-your-writes, monotonic reads).
- How 2PC works and why it is blocking — what happens to participants if the coordinator dies after PREPARE.
- Why 3PC adds a PRE-COMMIT phase and why it is almost never used in practice.
- How TCC (try-confirm-cancel) is structured and why every step must be idempotent.
- What a SAGA is, how orchestration differs from choreography, and how compensations work on a failure mid-chain.
A typical wrong answer: "we'll use a distributed transaction and it'll all be atomic like an ordinary database." This triggers a discussion that atomicity across the network is never free: 2PC blocks and hangs, while TCC and SAGA give only eventual consistency at the price of compensations that must themselves be idempotent, or else a retry under unreliable delivery (see outbox and delivery guarantees) re-applies an already-executed step.