Scale Up Before You Scale Out

Because the cloud is inherently distributed, distribution becomes the default reflex: shard the database, spread the writes, reach for eventual consistency — often on day one, for traffic a single node would barely notice. Each of those choices buys a permanent bill in complexity. This post is about paying that bill only when the numbers force you to, and a small repo that produces the numbers.

Runnable companion: scale-up-before-out on GitHub. make up && make seed && make loadtest && make headroom reproduces everything below against one Postgres node.

The reflex worth resisting

A distributed data system is a genuinely harder thing than a single node:

• Consistency stops being free — replication lag, quorums, or consensus to reason about.
• Transactions across nodes get expensive or disappear.
• Operations multiply — re-sharding, cross-node backups, partial failures.
• Latency gains a network hop on paths that were local memory.

You take on all of it the moment you distribute. The only honest question is whether the workload actually requires it — and that's a measurement, not an opinion.

What one node actually does

make loadtest ramps concurrency against a single Postgres node running an ordinary OLTP mix (indexed point reads plus a fraction of single-row updates), reporting throughput and tail latency at each level:

Concurrency       QPS   p50 ms   p95 ms   p99 ms
------------------------------------------------
          1     2,172     0.37     0.77     1.71
          2     4,186     0.42     0.76     1.29
          4     4,948     0.57     1.99     4.23
          8     9,373     0.73     1.48     2.57
         16    10,161     1.33     3.09     5.61
         32     9,247     2.90     7.54    12.32
         64     9,271     5.60    15.89    26.19

This is a laptop, Postgres in a Docker container — not tuned production hardware. And still: ~10,000 queries/sec with a p99 under 6 ms at the knee. Notice the shape — throughput climbs to a knee (here, concurrency 16), then plateaus while latency keeps rising. Past the knee, more concurrency buys latency, not throughput. That knee is the node's ceiling for this workload. Real hardware with tuned Postgres and pooling pushes it much higher.

From ceiling to a decision

A ceiling is only useful next to your real traffic. make headroom does the division:

  Single-node ceiling :   10,161 qps   (measured by make loadtest)
  Your average load   :      200 qps   (2.0% of ceiling)
  Your peak load      :      800 qps   (7.9% of ceiling)
  Headroom at peak    :     12.7x

Verdict:
  STAY SINGLE-NODE. Peak traffic uses 7.9% of one node — you have
  ~12.7x headroom. Distributing now buys complexity, not capacity.

Plug in your own numbers. Most services peak at a small fraction of what a single laptop-class node serves — and production boxes do far more. The gap between the ceiling and reality is usually an order of magnitude.

The cheap escalation ladder

When you do need more, climb one rung at a time — each only when the previous is genuinely exhausted:

1. Scale up — a bigger box, plus the boring wins (indexes, connection pooling, fixing the slow query). Cheap, and it buys a lot.
2. Scale out reads — read replicas absorb read-bound load with no sharding and no data-model change; you accept bounded replication lag on reads.
3. Shard / distribute writes — only when writes (not reads) exceed one node, and rungs 1–2 are spent.

Rungs 1 and 2 keep a single-writer model, which means none of the distributed-transaction or cross-shard-consistency tax.

When scaling out is actually justified

"Scale up first" must have a real trigger, or it rots into "never distribute until it's on fire". Move to distribution when — measured, not anticipated:

• Write throughput approaches the single-node ceiling (reads scale with replicas; writes don't).
• The dataset outgrows one node — storage, working set, or backup windows.
• Independent failure domains are a hard requirement — data residency, or an availability target one node plus failover can't meet.

What does not justify it: "it's best practice", "we might grow 100x someday", or "reads are slow" (that's caching and replicas first).

What I'd say in an interview

• "The cloud is distributed" is not a reason to distribute your database.
• Measure the single-node ceiling, compare it to real traffic, and let the ratio decide.
• The cheap ladder is up → replicas → shard, each rung earned by numbers.
• Distribution is a bill in consistency, operations and latency. Pay it when the workload forces you to — not on reflex.