Design Notification System

Core Architecture and Decoupling

The fundamental goal of any notification system is to decouple the business logic that triggers an event from the mechanisms required to deliver that event. In a monolithic approach, sending an email or push notification directly from a service blocks the request flow, causing latency and making the application fragile if the delivery provider experiences downtime. By introducing an asynchronous message queue, the producer service merely places a notification payload onto a queue and continues its execution. This architectural separation allows the system to absorb traffic spikes without overwhelming the target notification providers. We use a queue to buffer requests, ensuring that even if the downstream delivery services are slow or temporarily unreachable, messages remain persisted until they can be successfully dispatched by dedicated consumer workers. This pattern ensures that the system is resilient and capable of scaling independently based on the volume of incoming events.

# Simple producer-consumer structure using a message queue concept
class NotificationQueue:
    def __init__(self):
        self.queue = []  # Represents an in-memory buffer for messages

    def produce(self, message):
        self.queue.append(message)  # Business logic offloads task here
        return True

    def consume(self):
        return self.queue.pop(0) if self.queue else None  # Background workers pull tasks

Handling Channel-Specific Routing

Not all notifications are equal; they require specific routing logic based on the user's preferences, platform capabilities, and the urgency of the message. A robust notification system must maintain a template management service that stores the message structure, allowing developers to reuse formats across different channels without re-engineering the delivery pipeline. Furthermore, we must implement a routing layer that queries a user settings database to determine the preferred communication method for each recipient. If a user has disabled email notifications but enabled mobile push, the router must filter the request accordingly. By normalizing the input through an internal representation, we can easily extend the system to support new providers, like Slack or WhatsApp, by simply adding a new consumer service that translates our standardized internal format into the specific API requirements of the external service. This extensibility is vital for long-term platform maintenance.

def route_notification(user_prefs, message_template):
    # Routes messages based on channel priority and user configuration
    channels = ['push', 'email', 'sms']
    for channel in channels:
        if user_prefs.get(f'enable_{channel}'):
            # Send to specific worker pipeline for the channel
            print(f"Dispatching to {channel} pipeline for template: {message_template}")

Managing Rate Limiting and Backpressure

External notification providers often enforce strict rate limits to prevent abuse and ensure service quality. If we flood a third-party email provider with thousands of requests simultaneously, our account could be throttled or suspended. To mitigate this, our system must implement a rate-limiting mechanism at the worker level. By using a token bucket algorithm, we can control the flow of requests and distribute them over time. This approach allows us to handle bursts of events gracefully without breaching API limits. Furthermore, when an external service responds with a retryable error (such as a 429 Too Many Requests), our system must implement an exponential backoff strategy. Instead of immediately retrying and compounding the problem, we move the failed message back into the queue with a delayed visibility timeout, ensuring we do not overwhelm the external provider while maintaining the reliability of the notification delivery.

import time

def throttled_send(message, rate_limiter):
    # Uses token bucket to ensure we stay within API limits
    if rate_limiter.consume():
        return "Sent successfully"
    else:
        time.sleep(1) # Backpressure: wait before retry
        return "Rate limited, retrying"

Tracking and Delivery Guarantees

Reliability is defined by the ability to ensure that a notification is sent at least once. We achieve this by utilizing a database-backed storage layer to track the state of every message throughout its lifecycle, marking them as 'pending', 'sent', or 'failed'. When a worker retrieves a job from the queue, it performs a two-phase update: first to mark the status as 'processing' and eventually to 'sent' upon success. If a worker crashes before completion, the message will eventually reappear in the queue due to a timeout, allowing another worker to pick it up. This persistence layer is also essential for auditing and observability, providing insights into delivery success rates, latencies, and failed delivery patterns. By analyzing these logs, engineers can identify common failure points, such as invalid push tokens or server-side configuration issues, effectively improving the overall health of the notification system over time.

class MessageStore:
    def update_status(self, msg_id, status):
        # Simulates updating the status in a persistent database
        db = {'pending': [], 'sent': [], 'failed': []}
        db[status].append(msg_id)
        print(f"Message {msg_id} marked as {status}")

Security and Payload Privacy

Notification systems often carry sensitive user information, necessitating robust security measures throughout the delivery chain. Every request entering the system must be authenticated and validated to ensure that only authorized services can trigger notifications. Furthermore, the payload should be encrypted at rest and in transit. When constructing messages, we should avoid embedding PII (Personally Identifiable Information) directly into the push notification or email subject if it is not necessary. Instead, we can provide a secure deep link that redirects the user to the application, where they can authenticate and view the detailed content safely. By implementing strict validation logic on incoming event payloads, we protect our downstream workers from malformed data that could trigger vulnerabilities or system crashes. Treating every incoming notification request as potentially untrusted ensures the security of the broader ecosystem and protects against data leaks.

def sanitize_payload(payload):
    # Removes sensitive fields before queuing to prevent data leakage
    safe_data = {k: v for k, v in payload.items() if k != 'internal_user_id'}
    return safe_data # Only transmit sanitized data to downstream providers

Interview questions

All System Design interview questions →

Check yourself

Take the full System Design quiz →