Back-of-Envelope Capacity Estimation

What is Capacity Estimation?

In system design interviews and real world engineering, you must quickly prove whether your proposed design can scale to handle real world data volumes.

Back of the Envelope Capacity Estimation uses simple mental math to calculate required Queries Per Second (QPS), Network Bandwidth, RAM Storage, and GPU Compute.

Daily Active Users ──► Queries Per Second (QPS) ──► Network Bandwidth & RAM Memory

Essential Constants to Remember

• Seconds in a Day: $86,400$ seconds (round to $100,000$ for fast mental math!).
• Datatypes:
  • $1$ Byte = $8$ bits.
  • Float32 = $4$ Bytes per number.
  • Float16 = $2$ Bytes per number.
  • Int8 = $1$ Byte per number.
• Data Units:
  • $1,000$ Bytes = $1$ Kilobyte (KB).
  • $1,000,000$ Bytes = $1$ Megabyte (MB).
  • $1,000,000,000$ Bytes = $1$ Gigabyte (GB).

Worked Example: Vector Search Memory

Suppose you need to store and search 10 Million Product Vector Embeddings. Each vector has 512 dimensions stored in standard Float32 format.

Memory per Vector = 512 dimensions * 4 Bytes per Float32 = 2,048 Bytes ≈ 2 Kilobytes

Total Memory = 10,000,000 vectors * 2 Kilobytes = 20,000,000 Kilobytes = 20 Gigabytes

Conclusion: The entire 10 million vector index easily fits inside the RAM of a single cloud server!

Worked Example: Traffic QPS and Bandwidth

Suppose your application has 100 Million Daily Active Users (DAU). Each user makes 10 search requests per day.

Total Daily Requests = 100,000,000 users * 10 requests = 1 Billion Requests per day

Average QPS = 1,000,000,000 requests / 86,400 seconds ≈ 11,500 QPS

Peak QPS (2x Average) = 11,500 * 2 ≈ 23,000 QPS

If each request response payload returns 10 KB of JSON data:

$$\text{Peak Bandwidth} = 23,000 \text{ QPS} \times 10 \text{ KB} = 230 \text{ Megabytes per second (MB/s)}$$

This tells you that your network load requires load balancing across multiple web gateway servers.

Say this out loud

Back of the envelope capacity estimation uses simple numbers to calculate QPS, bandwidth, and memory requirements. Rounding seconds in a day to 100,000 makes mental math fast. Knowing data sizes like 4 bytes per float32 vector dimension lets you quickly estimate if a vector index fits in RAM or requires distributed memory clusters.

Followups to expect

1. How does quantization impact memory estimates? Quantizing float32 vectors to int8 reduces memory footprint by 4 times, allowing 80 Gigabytes of vector data to compress down to 20 Gigabytes.
2. Why is peak QPS more important than average QPS? Systems must be provisioned for peak traffic spikes (often 2 to 5 times average QPS) to prevent server crashes during high traffic hours.