DynamoDB — NoSQL on AWS

Core Data Model: Tables and Primary Keys

At its heart, DynamoDB stores data in tables as a collection of items, where each item contains one or more attributes. Unlike traditional relational databases, you do not pre-define a rigid schema for every column, providing the flexibility to evolve your data structure over time. The Primary Key is the most critical design decision because it uniquely identifies an item and determines how data is physically partitioned across the underlying storage nodes. A partition key approach allows the system to hash your input and distribute data uniformly across different servers. By choosing a high-cardinality key, such as a UserID or OrderID, you prevent 'hot partitions'—a scenario where one node experiences disproportionate load, leading to throttling. Understanding that the Primary Key dictates the performance and distribution of your data is the fundamental step in mastering DynamoDB architecture.

# Define a table schema focusing on the Partition Key
import boto3
ddb = boto3.client('dynamodb')

# Creating a table where 'UserID' is the unique Partition Key
ddb.create_table(
    TableName='UsersTable',
    AttributeDefinitions=[{'AttributeName': 'UserID', 'AttributeType': 'S'}],
    KeySchema=[{'AttributeName': 'UserID', 'KeyType': 'HASH'}],
    ProvisionedThroughput={'ReadCapacityUnits': 5, 'WriteCapacityUnits': 5}
)

Sorting and Composite Keys

While simple partition keys allow for direct lookups, many applications require retrieving a range of items associated with a single key. To solve this, DynamoDB offers a Composite Primary Key, consisting of a Partition Key and a Sort Key. The Partition Key routes the data to a specific physical bucket, while the Sort Key acts as an internal index within that partition. This enables efficient queries such as 'find all messages for this user where the timestamp is after yesterday.' Because items within a partition are stored in physical order based on the Sort Key, prefix queries and range operations become incredibly fast. Designing with this in mind avoids expensive 'scan' operations, which read every item in a table, significantly reducing costs and increasing responsiveness. Always align your Sort Key structure with the most frequent query patterns of your application logic.

# Using a Partition Key (UserID) and a Sort Key (Timestamp)
table = boto3.resource('dynamodb').Table('ActivityLogs')

# Querying for activity within a specific range
response = table.query(
    KeyConditionExpression='UserID = :u AND #ts > :t',
    ExpressionAttributeNames={'#ts': 'Timestamp'},
    ExpressionAttributeValues={':u': 'user123', ':t': '2023-01-01'}
)

Local and Global Secondary Indexes

Secondary Indexes allow you to query data using non-primary attributes as if they were keys. A Local Secondary Index (LSI) uses the same partition key as the base table but provides a different sort key, allowing you to re-sort existing data for specific views. A Global Secondary Index (GSI) is more powerful; it uses an entirely different partition key, effectively creating a new way to access your data set. When you write to the table, DynamoDB automatically replicates that data to your indexes in the background. Because GSIs are asynchronous, they add flexibility but introduce slight eventual consistency considerations. Use indexes to avoid full table scans, but remember that every index you create consumes additional read/write capacity and storage, so they must be planned carefully based on your secondary access requirements to keep your infrastructure costs optimized and manageable.

# Creating a Global Secondary Index on an existing table
ddb.update_table(
    TableName='Orders',
    AttributeDefinitions=[{'AttributeName': 'Status', 'AttributeType': 'S'}],
    GlobalSecondaryIndexUpdates=[{
        'Create': {
            'IndexName': 'StatusIndex',
            'KeySchema': [{'AttributeName': 'Status', 'KeyType': 'HASH'}],
            'Projection': {'ProjectionType': 'ALL'},
            'ProvisionedThroughput': {'ReadCapacityUnits': 5, 'WriteCapacityUnits': 5}
        }
    }]
)

Consistency Models: Strong vs. Eventual

DynamoDB provides two read consistency models to balance speed and accuracy. By default, reads are 'eventually consistent,' meaning if you write data, a read occurring immediately after might return stale data because the update has not propagated to all internal storage nodes yet. This mode is faster and consumes half the read capacity units compared to strongly consistent reads. Strongly consistent reads, however, return the latest version of data by ensuring the read receives a successful acknowledgement from all replicas before responding. This is essential for financial transactions or inventory systems where read-after-write accuracy is non-negotiable. Developers must analyze their workload; if a user is updating their profile, eventual consistency is likely fine, but when decrementing an account balance, you must explicitly request strong consistency to prevent race conditions that could lead to data corruption or incorrect application state.

# Requesting a strongly consistent read
item = table.get_item(
    Key={'UserID': 'user123'},
    ConsistentRead=True  # Force read from latest state
)

Performance and Throttling

Performance management is synonymous with understanding Throughput Capacity. When you configure your table, you specify the number of reads and writes per second. If your application exceeds these limits, DynamoDB issues a 'ProvisionedThroughputExceededException,' effectively throttling your requests to protect the system's stability. While you can manually adjust these values, the 'On-Demand' mode is often better for unpredictable traffic, as it scales capacity up or down automatically. To further optimize, use 'Exponential Backoff' in your code, which implements a retry logic with increasing delays when throttling occurs. Furthermore, consider 'DAX' (DynamoDB Accelerator) for read-heavy workloads; it sits in front of the table as an in-memory cache, providing microsecond responses and reducing the load on your underlying table. Balancing these mechanisms is how you achieve high availability while maintaining strict control over your operational budget.

# Implementing simple retry logic for throttled requests
import time
from botocore.exceptions import ClientError

def put_item_with_retry(item):
    try:
        table.put_item(Item=item)
    except ClientError as e:
        if e.response['Error']['Code'] == 'ProvisionedThroughputExceededException':
            time.sleep(1) # Simple backoff
            table.put_item(Item=item)

Interview questions

All AWS interview questions →

Check yourself

Take the full AWS quiz →