API Gateway Pattern

The Core Concept of Centralized Routing

At the simplest level, an API Gateway acts as a traffic cop for your distributed system. In a microservices architecture, a single client request might require data from three different services, such as a user profile service, an order history service, and an inventory service. Instead of forcing the client to know the IP addresses and endpoints of every internal service, which creates tight coupling and security risks, the gateway provides a single entry point. By routing requests based on path prefixes or headers, the gateway hides the internal topology of your network. This isolation allows you to refactor your internal services—splitting one service into two or renaming endpoints—without breaking the client application, as the client only interacts with the stable gateway contract, preserving system integrity through abstraction.

def route_request(path):
    # Mapping incoming paths to specific service endpoints
    routes = {
        "/users": "http://user-service:8080",
        "/orders": "http://order-service:8080"
    }
    service = routes.get(path.split('/')[1], "http://default-service")
    return f"Forwarding request to {service}"

print(route_request("/users/123"))

Offloading Cross-Cutting Concerns

Building security into every microservice is a maintenance nightmare. If you implement authentication, request logging, and rate limiting in every individual service, you guarantee inconsistency and high operational overhead. The API Gateway pattern provides a dedicated layer to handle these cross-cutting concerns before a request even reaches your business logic. By centralizing authentication checks, such as validating a signed security token, the gateway ensures that downstream services can focus solely on processing legitimate, pre-authenticated traffic. This architectural design creates a perimeter defense strategy. Furthermore, if you need to rotate keys or update your encryption standards, you only need to update the gateway configuration rather than redeploying dozens of disparate services, which significantly reduces the risk of human error and configuration drift across your production environment.

def authenticate(request):
    # Extracting token from header to validate access
    token = request.get("Authorization")
    if token == "secret_token_123":
        return True # Access granted
    return False # Access denied

request = {"Authorization": "secret_token_123"}
print(f"Access Status: {authenticate(request)}")

Request Aggregation and Transformation

One of the most powerful features of an API Gateway is its ability to reduce network latency for mobile clients. In a naive implementation, a client might need to make multiple sequential HTTP calls to assemble a dashboard, leading to slow performance over high-latency mobile networks. The gateway can perform request aggregation, where it receives one call from the client and triggers multiple parallel requests to internal services, combining the results into a single payload. Additionally, the gateway can handle protocol or data transformation. For example, if your internal services communicate using structured data formats optimized for internal parsing, but your mobile application requires a simplified, specific JSON shape, the gateway serves as the transformation engine. This minimizes the payload size sent over the wire, optimizing the client experience while maintaining efficient internal communications.

import json

def aggregate_responses(user_data, order_data):
    # Combining multiple responses into a single client-friendly object
    return json.dumps({
        "user": user_data,
        "orders": order_data,
        "status": "success"
    })

print(aggregate_responses({"id": 1}, ["order1", "order2"]))

Implementing Rate Limiting and Throttling

Protecting your internal infrastructure from being overwhelmed by spikes in traffic is critical for system stability. If a single aggressive client starts sending thousands of requests per second, it could potentially exhaust the connection pools of your microservices, leading to a cascading failure. The API Gateway acts as a traffic shaper by enforcing rate limits at the entry point. By using token bucket or leaky bucket algorithms, the gateway can throttle traffic per client, per API key, or per IP address. This ensures that even if one tenant or client malfunctions, the total system remains resilient and available for other users. This layer also provides a natural location to implement circuit breaking, where the gateway stops sending requests to a service that is known to be failing, preventing the entire system from hanging while waiting for timeouts.

class RateLimiter:
    def __init__(self, limit):
        self.limit = limit
        self.count = 0
    
    def is_allowed(self):
        # Simple counter to throttle excessive traffic
        if self.count < self.limit:
            self.count += 1
            return True
        return False

limiter = RateLimiter(limit=2)
print(limiter.is_allowed()) # True
print(limiter.is_allowed()) # True
print(limiter.is_allowed()) # False

Observability and Centralized Logging

Observability in a distributed system is challenging because requests span multiple services, making it hard to track the lifecycle of a single transaction. The API Gateway is the ideal location to initiate request tracing. By injecting a unique correlation ID into every incoming request header, the gateway allows you to track that specific request as it flows through your internal infrastructure. This centralized point provides a consistent view of request duration, error rates, and throughput metrics across the entire platform. Without the gateway, you would have to stitch together logs from various services, which is error-prone and labor-intensive. With the gateway, you gain a unified dashboard that highlights exactly where in the system a request failed or slowed down, which is essential for debugging and performance tuning at scale.

import uuid

def trace_request(original_headers):
    # Injecting a correlation ID for distributed tracing
    headers = original_headers.copy()
    headers["X-Correlation-ID"] = str(uuid.uuid4())
    return headers

print(trace_request({"Content-Type": "application/json"}))

Interview questions

All System Design interview questions →

Check yourself

Take the full System Design quiz →