Interview Prep
Back-of-the-Envelope Estimation
Back-of-the-envelope estimation is the practice of using simplified mathematical models to determine the feasibility of a system architecture before deep-diving into implementation. It matters because it allows engineers to identify critical bottlenecks in throughput, storage, and latency by focusing on orders of magnitude rather than precise values. You reach for this tool during the initial phase of a system design interview to justify your component choices and hardware requirements against the projected scale of user demand.
Understanding Powers of Two and Scale
The foundation of any estimation involves translating high-level requirements into binary-friendly units. In computing, we typically work with powers of two because memory addressing and data storage architectures align with these units. Instead of using arbitrary decimal figures, we define thresholds like one kilobyte as 2^10 bytes, one megabyte as 2^20, one gigabyte as 2^30, and one terabyte as 2^40. This allows us to perform rapid calculations in our heads without needing a calculator. When an interviewer asks about the volume of data generated by a social media platform, we think in terms of these exponential growths. By approximating daily active users and their average interaction frequency, we can project annual storage needs. If you know that a user generates one megabyte of data daily, and you have ten million users, you can instantly estimate the total growth rate. This mental shorthand is vital for maintaining the flow of an interview, demonstrating that you understand the underlying physical limits of hardware infrastructure before you attempt to design complex software patterns.
# Calculate total annual storage requirement for 10 million users
# Each user generates 1MB of content per day.
users = 10_000_000
data_per_user_mb = 1
days_in_year = 365
# Total MB = 10^7 * 1 * 365 = 3.65 billion MB
# To get TB: divide by 1024^2
total_tb = (users * data_per_user_mb * days_in_year) / (1024 * 1024)
print(f"Annual storage requirement: {total_tb:.2f} TB")Calculating Queries Per Second (QPS)
Estimating Queries Per Second is crucial for determining the capacity of your load balancers, web servers, and database clusters. QPS is derived by looking at the total volume of traffic during peak periods, as a system must be designed for the maximum load rather than the average. We start by estimating the number of daily active users and the average number of requests each user makes per day. By dividing this total request count by the number of seconds in a day, which is approximately 86,400, we arrive at the average QPS. However, real-world systems experience surges. It is professional practice to apply a multiplier, usually between two and five, to account for peak hours or traffic spikes. If your average load is 500 requests per second, you should design your infrastructure to comfortably handle at least 1,500 to 2,000 requests per second. This reasoning ensures that your system does not crash during typical operational volatility, demonstrating an awareness of reliability and availability requirements that a naive estimation would ignore entirely.
# Estimating required QPS for a system with 1 million daily users
users = 1_000_000
requests_per_user_per_day = 50
seconds_in_day = 86400
# Average QPS
avg_qps = (users * requests_per_user_per_day) / seconds_in_day
# Peak QPS with a 3x surge factor
peak_qps = avg_qps * 3
print(f"Design for Peak QPS: {peak_qps:.0f}")Latency and Throughput Constraints
Latency and throughput are the two axes upon which all system performance is measured. Latency is the time taken for a single request to travel through the system, while throughput is the total volume of work processed over a given period. When performing estimations, you must consider the latency of disk reads versus memory reads. A typical RAM read takes roughly 100 nanoseconds, while a standard solid-state disk read takes roughly 100 microseconds. This massive difference is why we design caching layers. In an interview, you must estimate the total latency of a request chain by adding the latency of each hop: the client to the load balancer, the load balancer to the application server, the application server to the database, and the final round-trip. By estimating the time for these serial dependencies, you can justify why specific components like asynchronous task queues or read replicas are necessary to improve the overall user experience and reduce the cumulative response time to within acceptable limits.
# Calculate total latency of a request chain in milliseconds
# Network round trip + DB query + Internal processing
network_latency_ms = 50
db_query_ms = 20
processing_ms = 5
# Total latency for one sequential request
total_latency = network_latency_ms + db_query_ms + processing_ms
print(f"Expected end-to-end latency: {total_latency} ms")Memory and Caching Requirements
Caching is the most effective way to optimize read-heavy systems, but it introduces costs and complexity. To estimate memory needs, consider the '80/20 rule,' which suggests that 80% of system traffic is usually directed at 20% of the data. If you have 10 terabytes of total system data, you do not necessarily need to store 10 terabytes in cache to achieve significant performance gains. Instead, you focus on caching the most 'hot' objects, which might only account for 2 terabytes. By calculating the cache hit ratio and the latency savings per hit, you can justify the cost of high-memory instances in your design. During the interview, you should explain that your cache size estimation is based on the working set size, not the total database size. This demonstrates a sophisticated understanding of resource management and shows that you can balance the trade-off between performance requirements and operational budgets effectively, preventing over-provisioning of expensive hardware resources.
# Determine cache capacity needed for hot data
total_data_tb = 10
percent_of_hot_data = 0.20
# Estimating cache size for 20% of the total dataset
cache_size_tb = total_data_tb * percent_of_hot_data
print(f"Recommended cache size: {cache_size_tb} TB")Bandwidth and Network Capacity
Network bandwidth is a finite resource that is often overlooked in preliminary designs until it becomes a catastrophic bottleneck. To estimate bandwidth, multiply the size of an average request or response by the number of requests per second. If an average user request is 50 kilobytes and you have 2,000 requests per second, you are looking at 100 megabytes per second of throughput, which is equivalent to 800 megabits per second. This calculation is essential when you decide whether your services need a dedicated internal network or if you need to optimize payload sizes using binary serialization formats. Furthermore, when dealing with distributed systems, you must account for internal traffic between microservices and data replication across geographical regions. Failing to estimate these background network flows can lead to congestion. Always provide a buffer for overhead and control plane traffic, ensuring your architectural plan remains robust even when network conditions are sub-optimal during peak periods of data synchronization.
# Calculate bandwidth usage in megabits per second (Mbps)
requests_per_second = 2000
size_per_request_kb = 50
# Convert KB to Megabits (1 KB = 8 Kilobits)
# Total Mbps = (requests * size * 8) / 1024
bandwidth_mbps = (requests_per_second * size_per_request_kb * 8) / 1024
print(f"Required network bandwidth: {bandwidth_mbps:.2f} Mbps")Key points
- Always convert all data and traffic metrics into common units before performing calculations.
- Binary powers of two are preferred over decimal approximations for storage and memory sizing.
- QPS should be calculated for peak times by applying a surge multiplier to average daily traffic.
- Latency estimates must account for the additive nature of serial network and database hops.
- Caching requirements should focus on the working set size rather than the entire dataset size.
- Bandwidth estimation is critical to avoid network bottlenecks between microservices.
- Back-of-the-envelope math demonstrates feasibility before committing to complex architectures.
- Always state your assumptions clearly when providing numerical estimates in an interview.
Common mistakes
- Mistake: Ignoring power-of-two versus decimal conversion. Why it's wrong: Engineers often confuse 1 KB as 1000 bytes instead of 1024 bytes, leading to cumulative errors in large-scale storage sizing. Fix: Use 10^3 for round numbers to simplify, but acknowledge the 1024 approximation when high precision is required.
- Mistake: Over-focusing on extreme edge cases. Why it's wrong: System design estimation is for scaling capacity, not optimizing for rare events that don't impact throughput significantly. Fix: Focus on the 80/20 rule, prioritizing high-traffic paths.
- Mistake: Calculating without considering network and disk latency. Why it's wrong: Ignoring I/O constraints leads to unrealistic throughput assumptions for database operations. Fix: Use standard latency benchmarks (e.g., L1 cache vs. RAM vs. SSD vs. Network) as a baseline for all estimates.
- Mistake: Calculating total daily storage without considering data retention policies. Why it's wrong: Calculating unbounded growth makes a system look infeasible when in reality data is tiered or deleted. Fix: Define the TTL (Time-To-Live) and retention strategy before finalizing storage requirements.
- Mistake: Failing to account for peak load. Why it's wrong: Designing for average traffic results in system crashes during surges. Fix: Apply a peak factor (usually 3x-5x) to average requests-per-second (RPS) to ensure the system survives bursts.
Interview questions
What is the primary purpose of Back-of-the-Envelope estimation in a System Design interview?
Back-of-the-Envelope estimation is used to ground your architectural decisions in reality by establishing the scale of the system. Its primary purpose is to determine the feasibility of an approach, identify potential bottlenecks, and help select the right technology stack. By calculating traffic patterns, storage needs, and bandwidth requirements early on, you ensure that the design can handle the projected load without wasting resources on over-engineering or creating a system that fails under expected traffic.
How do you calculate the storage requirement for a photo-sharing application that gets 10 million daily active users?
To calculate storage, we use a simple formula: (Total Users) * (Percentage of active users) * (Average items per user) * (Average size per item). If 10 million users upload one photo per day, and each photo averages 2MB, we calculate 10,000,000 * 1 * 2MB, resulting in 20TB of new storage per day. Over five years, this becomes 20TB * 365 * 5 = 36.5 Petabytes. We must also account for metadata size, which is smaller but crucial for database selection and indexing strategy.
When designing a system, how do you handle peak load versus average load in your capacity planning?
Average load tells us the baseline, but peak load determines whether the system crashes during traffic spikes. If a system averages 1,000 requests per second (RPS) but hits 10,000 RPS during peak events, we must design for the peak. We calculate this using: Peak RPS = (Average Daily Requests / 86,400 seconds) * Peak Multiplier. We then use auto-scaling groups or load balancers to distribute this traffic, ensuring the infrastructure can spin up resources to match that specific multiplier efficiently.
Compare the approach of designing for vertical scaling versus horizontal scaling when estimating capacity.
Vertical scaling involves adding more power to a single server, which is simple but limited by hardware ceilings and introduces a single point of failure. Horizontal scaling, or adding more nodes, is more complex due to data partitioning and load balancing, but it offers infinite theoretical growth. When estimating, horizontal scaling requires calculating the throughput per node: Nodes = (Total Required Throughput) / (Throughput per Node). Choosing horizontal scaling is usually better for distributed systems to ensure high availability and fault tolerance.
How does network bandwidth impact your estimation process for a high-traffic video streaming service?
Bandwidth is often the hidden bottleneck. For video streaming, we calculate it by: (Average Video Bitrate) * (Number of Concurrent Users). If 100,000 users stream at 5Mbps, we need 500Gbps of throughput. We express this in total data transferred over time: Total Bandwidth = (Data per request) * (Requests per second). We must ensure our load balancers and internal networks can support this sustained rate, or we will face latency issues regardless of how much compute power we provision.
Explain how you estimate the latency of a distributed system involving multiple microservices and a cache.
Latency estimation follows the critical path. We sum the round-trip times (RTT) for each hop: Total Latency = (Time_Cache) + (Time_ServiceA) + (Time_Database) + (Network_Overhead). If the cache hit rate is 80%, we use weighted averages: (0.8 * Cache_Latency) + (0.2 * (Cache_Miss + DB_Latency)). As a rule of thumb, we assume 1ms for internal network calls and 10ms for disk access. We then compare this to the maximum allowable response time to decide if we need asynchronous processing or pre-fetching.
Check yourself
1. When estimating QPS for a read-heavy service, which factor is most important to adjust for to ensure system stability?
- A.The total number of global users
- B.The peak-to-average traffic ratio
- C.The average size of a single request
- D.The geographical location of the servers
Show answer
B. The peak-to-average traffic ratio
The peak-to-average ratio is critical because systems must handle bursts; options like user count or request size are inputs, but don't define the burst threshold. Geography relates to latency, not QPS capacity directly.
2. If a system generates 100 terabytes of logs daily, but the policy deletes logs after 30 days, what is the required storage capacity?
- A.100 terabytes
- B.1.5 petabytes
- C.3 petabytes
- D.3.3 petabytes
Show answer
C. 3 petabytes
100 TB * 30 days equals 3000 TB, which is exactly 3 petabytes. Option 1 is daily, 1.5 petabytes is half the required capacity, and 3.3 petabytes overestimates due to unnecessary padding.
3. Which of the following describes the correct approach to estimating the number of database servers needed for a read-heavy application?
- A.Total Daily Requests / 86400
- B.Total QPS / Expected QPS per database node
- C.Total Storage / Disk size per node
- D.Number of concurrent users / Network bandwidth
Show answer
B. Total QPS / Expected QPS per database node
To size DB nodes, you divide throughput demand by the throughput capability of a single node. Daily requests ignore peak, storage doesn't account for CPU/IO pressure, and concurrent users is a poor proxy for throughput.
4. Why is it important to use 'power of 2' approximations (e.g., 1 TB = 10^12 bytes) when performing napkin math?
- A.To ensure code compatibility with binary-based architectures
- B.To simplify calculations while maintaining an acceptable margin of error
- C.Because storage hardware is strictly marketed in binary units
- D.To avoid overflow errors in the calculation tool
Show answer
B. To simplify calculations while maintaining an acceptable margin of error
Napkin math is for order-of-magnitude correctness; powers of 10 simplify mental math effectively. Other options are incorrect as they confuse architectural requirements with estimation methodology.
5. When calculating network bandwidth for a distributed system, what is the most critical variable to include?
- A.The cost of the network hardware
- B.The packet header overhead versus payload size
- C.The average size of the request and response in bytes
- D.The number of switches between client and server
Show answer
C. The average size of the request and response in bytes
Bandwidth is calculated as (Avg Request Size + Avg Response Size) * RPS. Hardware cost and switches are physical infrastructure details, and header overhead is secondary to the primary payload volume.