Scaling
While a service runs in one process, a lot is forgiven — a counter in memory, a synchronous call to a neighbour, a cache without protection. Scaling begins where there are many instances: in-memory state is no longer shared, load arrives in bursts, and the client wants updates in real time rather than polling the server. Here what matters is not the algorithm on one node but how that algorithm behaves when there are a hundred nodes.
The central trap of this topic is designing for one process and deploying onto a fleet. A local rate-limiter counter scales with the instance count — that is not a rounding error but a multiplied limit. A naive cache whose key expires for everyone at once sends all traffic to the database simultaneously — a cache stampede. A synchronous call to a slow neighbour turns its latency into yours. And the choice of realtime transport (polling, long polling, SSE, WebSocket) decides whether you pay in wasted traffic or hold a persistent connection. This topic breaks scaling into layers — from a global limit down to a full-duplex channel.
Topic Map
- Distributed rate limiting — shared fleet-wide state and an atomic check so the limit is global, not per instance.
- Rate-limiting algorithms — fixed/sliding window, token bucket, and leaky bucket and their behaviour at interval boundaries.
- Caching strategies — a value under an
RWMutexwith a background refresh, and protection against a cache stampede. - Short key generation —
base62of a monotonic id yields unique short keys with no collision check. - Inter-service communication — synchronous RPC vs a message queue vs long-polling and their trade-offs.
- Long polling and realtime transport — polling, long polling, SSE, and WebSocket by direction of exchange and use case.
- WebSocket — a persistent, full-duplex connection opened by upgrading the initial HTTP request.
Common Mistakes and Traps
| Mistake | Consequence |
|---|---|
| Keeping the rate-limiter counter in process memory | The limit multiplies by the instance count — a multiplied overshoot |
| Treating drift between local counters as a "small error" | On a fleet it is a multiplied limit, not rounding |
| Ignoring an algorithm's behaviour at the window boundary | Fixed window lets a double burst through at the seam of two intervals |
| Not protecting the cache from a key expiring everywhere at once | A cache stampede — all traffic hits the database at once and takes it down |
| Checking for collisions when generating a key from a unique id | Wasted work — base62 of a unique id is already unique |
| Keeping a single id generator for the whole cluster | A contention bottleneck — a distributed monotonic id is needed |
| Calling a slow neighbour synchronously on the hot path | Its latency becomes your latency and your failure |
Returning an empty 200 instead of a 404 for a missing key | The client cannot tell "no data" from "all good, empty" |
| Polling the server more often to lower realtime latency | You pay twice — wasted traffic and still latency up to the interval |
Treating Sec-WebSocket-Key as authentication | It is a protocol check, not authorization — a security-model confusion |
Interview Relevance
Scaling is a mandatory topic at the senior level of a Go interview, and the question is not an algorithm on one node but its behaviour on a fleet. The interviewer checks whether you keep in mind that in-process state stops being shared once there are many instances.
What interviewers usually check:
- Why a rate limiter must store state centrally and check the limit atomically so it is global.
- How fixed window, sliding window, token bucket, and leaky bucket differ — especially at interval boundaries.
- What a cache stampede is and how a background refresh under an
RWMutexprevents it. - Why
base62of a monotonic id gives a short unique key with no collision check, and where the generator bottleneck is. - How to choose between synchronous RPC, a queue, and long-polling — by coupling, latency, and failure resilience.
- How polling, long polling, SSE, and WebSocket differ by direction of exchange and when each fits.
A typical wrong answer: "put a request counter in each service's memory — that's enough." This triggers a discussion that a local counter scales with the instance count and multiplies the overall limit, while a global limit requires shared state (e.g. in Redis) with an atomic check-and-increment.