Docker Compose
docker-compose.yml Syntax
The docker-compose.yml file serves as the declarative configuration manifest for defining multi-container applications by mapping service definitions to infrastructure requirements. It matters because it abstracts complex command-line arguments into a version-controlled, repeatable blueprint that ensures environmental consistency across development and production pipelines. You should reach for this tool whenever your architecture necessitates the orchestration of interdependent services that require networking, persistent storage, and specific startup orchestration.
Service Definition and Image Specification
At its most fundamental level, the compose file functions as a directory of services, where each service represents a distinct component of your distributed system. By defining a service name, you establish a hostname within the internal network that other containers can use to communicate, effectively abstracting the underlying IP address mechanics. The 'image' directive is the primary instruction for the container engine, telling it which snapshot to pull from the registry to instantiate the environment. By utilizing specific tags rather than the default 'latest' tag, you ensure deterministic behavior, preventing silent updates from breaking your application. This declarative approach allows developers to represent the entire application stack as code, making it trivial to spin up an identical, isolated environment on any machine with the runtime installed, thereby eliminating the classic 'it works on my machine' syndrome through strict structural standardization.
version: '3.8'
services:
web-app:
# Use a specific version tag for immutable builds
image: nginx:1.25.3-alpine
container_name: production-web-serverPort Mapping and Network Isolation
Network isolation is a core benefit of using this syntax, as it allows services to communicate over a private bridge network without exposing every component to the host machine. The 'ports' directive acts as a firewall rule that explicitly maps a host-machine port to the container’s internal port, controlling exactly which services are accessible from the outside world. By only mapping the necessary entry points, you significantly reduce the attack surface of your application. When multiple containers are defined, they are automatically placed on a default network where they can discover each other using their service names as hostnames. This provides a clean mechanism for service discovery that does not rely on hardcoded IP addresses, which are inherently ephemeral and subject to change whenever a container is recreated. Mastering this mapping logic is essential for designing secure, multi-tier architectures that remain modular and manageable as the system scales.
services:
api:
image: custom-api:v1
# Mapping host port 8080 to container port 3000
ports:
- "8080:3000"
# Services are joined to a default network automaticallyPersistent Volumes and Data Management
Containers are designed to be ephemeral, meaning that any data generated within the writable layer is destroyed once the container is removed. To ensure data persistence across container restarts or upgrades, we utilize the 'volumes' directive to map host directories or named volumes into the container's file system. This decoupling of data from the compute layer is critical for stateful services like databases. By defining a volume, you establish a durable storage location that survives the container lifecycle. Furthermore, volumes are highly performant and allow for host-container synchronization, which is particularly useful during development when you want your local code changes to reflect immediately inside the running environment. Proper volume management ensures that your application state is protected, facilitating robust backup strategies and allowing for seamless migration of data between different deployment instances without manual intervention or risky file copying processes.
services:
db:
image: postgres:15
volumes:
# Persist database files to a named volume
- postgres_data:/var/lib/postgresql/data
volumes:
postgres_data: {}Environment Variables and Sensitive Configuration
Hardcoding configuration details such as API keys, database credentials, or environment-specific URLs is a significant security risk and architectural anti-pattern. The compose syntax provides multiple ways to inject environment variables, allowing the same image to be reused across development, testing, and production environments by simply swapping the configuration manifest. By leveraging an 'env_file' or 'environment' block, you can safely pass sensitive data into the container at runtime without baking those credentials into the image itself. This approach promotes the 'Build once, deploy anywhere' philosophy, as the image remains generic while the runtime configuration remains context-aware. This separation of concerns is vital for maintaining security compliance and allows infrastructure teams to manage secrets independently of the application logic. Properly configuring these variables ensures that your services are correctly wired to their specific dependencies without manual intervention during the startup sequence.
services:
app:
image: my-service:latest
environment:
- DB_HOST=db
- LOG_LEVEL=debug
# Optional: Load from a secure file
env_file:
- .env.productionService Dependency and Startup Ordering
In a distributed architecture, services often have implicit startup dependencies; for instance, an application server might fail if the database it relies on is not yet fully initialized. The 'depends_on' directive allows you to orchestrate the startup sequence by signaling to the orchestrator which containers should be initiated first. While 'depends_on' ensures that containers start in a specific order, it is important to note that it does not verify whether the dependency is 'ready' to accept connections. For scenarios requiring actual readiness checks, we combine these dependencies with custom health checks. These health checks run inside the container, periodically executing a command to determine if the service is truly functional. If the check fails, the orchestrator recognizes the state change, providing a robust mechanism to manage transient errors and ensure that the entire stack enters a 'ready' state predictably and automatically.
services:
app:
depends_on:
- db
healthcheck:
test: ["CMD", "curl", "-f", "http://localhost:3000/health"]
interval: 30s
retries: 3Key points
- Docker Compose uses declarative YAML syntax to define multi-container application stacks.
- The 'image' directive should always use specific version tags to ensure consistent, immutable deployments.
- Port mapping allows explicit control over which service ports are exposed to the external host.
- Volumes are essential for decoupling persistent application data from the ephemeral container lifecycle.
- Environment variables enable configuration parity across different deployment environments without modifying the underlying image.
- Service discovery occurs automatically through internal DNS based on the service names defined in the manifest.
- The 'depends_on' directive dictates the startup order of services to prevent race conditions during initialization.
- Health checks provide a mechanism for the orchestrator to monitor the operational status of a container internally.
Common mistakes
- Mistake: Indenting with tabs instead of spaces. Why it's wrong: YAML specification strictly prohibits tabs for indentation. Fix: Always use two spaces for each indentation level.
- Mistake: Misunderstanding the difference between 'build' and 'image'. Why it's wrong: 'image' pulls a pre-built artifact, while 'build' triggers a Dockerfile execution. Fix: Use 'image' for external registries and 'build' for local project source code.
- Mistake: Using version numbers like '3.9' unnecessarily. Why it's wrong: Version 3.x is deprecated and often limits feature sets in modern Docker Compose. Fix: Omit the version field entirely to use the latest specification automatically.
- Mistake: Improperly mapping volumes with relative paths. Why it's wrong: Relative paths in compose files are resolved relative to the compose file location, not the shell's current working directory. Fix: Use absolute paths or environment variables to ensure consistent mounting.
- Mistake: Confusing 'depends_on' with service readiness. Why it's wrong: 'depends_on' only controls startup order, not application-level readiness. Fix: Use healthchecks and the 'service_healthy' condition for robust service orchestration.
Interview questions
What is the primary purpose of the 'version' field in a docker-compose.yml file, and is it still required?
The 'version' field in a docker-compose.yml file was historically used to define the specific schema version of the configuration file. This allowed Docker to understand which features, such as specific networking drivers or build arguments, were supported in that particular file format. However, in modern Docker Compose (V2), the version field is no longer required and is actually ignored by the engine. The tool now defaults to the latest specification, making the file cleaner and preventing issues where users would inadvertently use outdated syntax while trying to access new orchestration capabilities.
How do you define environment variables for a service, and why would you prefer using an .env file over hardcoding them?
In a docker-compose.yml file, you define environment variables using the 'environment' key, where you map key-value pairs directly. For example, 'DB_PASSWORD: password123'. It is significantly better practice to use an '.env' file instead of hardcoding sensitive data. Hardcoding leads to security risks if the file is committed to version control. Using an '.env' file allows you to keep sensitive configuration separate, inject different values per environment without altering the container's logic, and ensures that sensitive production secrets remain managed outside of the base application deployment structure.
Explain the difference between the 'depends_on' key and the 'healthcheck' directive when managing service startup order.
The 'depends_on' key in Docker Compose merely expresses a dependency relationship, starting services in the correct order based on the defined graph. However, it does not wait for the application inside the container to be fully initialized and ready. This is where 'healthcheck' becomes critical. By adding a healthcheck command—like checking if a database port is listening—you can configure 'depends_on' with a 'condition: service_healthy' constraint. This ensures your application container doesn't crash on startup by attempting to connect to a service that is still in the process of booting up.
How can you leverage the 'build' context and 'dockerfile' keys to customize container images during development?
The 'build' context allows you to specify a directory—usually represented as '.'—which tells Docker to look for a Dockerfile within that path. By using the 'dockerfile' key, you can point to a specific file, such as 'Dockerfile.dev' or 'Dockerfile.prod'. This is essential because it allows you to separate build stages for local development versus production releases. It enables you to easily swap configurations or install extra debugging tools in local environments while keeping production images lightweight, secure, and optimized for performance during deployment in a Kubernetes cluster.
Compare the 'ports' key to the 'expose' key in Docker Compose: when would you use one over the other?
The 'ports' key is used for mapping a port from the host machine to the container, effectively making the service accessible from outside the Docker network. Conversely, the 'expose' key defines ports that are only accessible to other services within the internal Docker network. You should use 'ports' when you need to provide public or external access to an application. You should use 'expose' when you want to follow the principle of least privilege, restricting access to internal microservices so they can communicate with each other without being exposed to the host machine or external traffic.
When migrating a multi-container Docker Compose setup to Kubernetes, how do you handle volumes and networking differences?
Moving from Docker Compose to Kubernetes requires translating local volume paths and simple bridge networks into Persistent Volumes (PVs) and Services. In Docker Compose, you might use 'volumes: - ./data:/var/lib/data', which relies on local file system paths. In Kubernetes, you must use PersistentVolumeClaims to ensure storage is decoupled from the pod lifecycle, ensuring data persists across pod rescheduling. Networking changes significantly as well; instead of relying on simple service names, you must define Kubernetes Services or Ingress resources to manage load balancing and traffic routing, which provides far more advanced traffic management than standard Docker Compose networking.
Check yourself
1. When defining a service, what is the effect of using 'restart: always' compared to 'restart: unless-stopped'?
- A.Always attempts to restart regardless of manual intervention.
- B.Only restarts if the container exited with a non-zero code.
- C.Unless-stopped requires an explicit docker stop command to prevent restart on daemon startup.
- D.Always restarts only if the host machine reboots.
Show answer
A. Always attempts to restart regardless of manual intervention.
'restart: always' will always restart the container after it stops, regardless of manual intervention. 'unless-stopped' will not restart the container if it was manually stopped before the daemon restarted. The other options are incorrect because they mischaracterize the daemon's behavior.
2. Which of the following describes the correct behavior of the 'extends' keyword in Docker Compose?
- A.It downloads a remote service configuration from Docker Hub.
- B.It allows reusing a service configuration from another file to maintain DRY principles.
- C.It merges two images into a single container instance.
- D.It forces two services to run on the same virtual network bridge.
Show answer
B. It allows reusing a service configuration from another file to maintain DRY principles.
The 'extends' keyword is specifically designed for sharing common configuration across services or different compose files. It does not pull remote files, merge images, or define network topology.
3. In a docker-compose.yml file, why would you prefer 'secrets' over environment variables for sensitive data?
- A.Secrets are faster to load than environment variables.
- B.Secrets are encrypted at rest and mounted as files, reducing exposure in logs or process lists.
- C.Secrets allow for automatic rotation every 24 hours.
- D.Environment variables are deprecated in modern Docker versions.
Show answer
B. Secrets are encrypted at rest and mounted as files, reducing exposure in logs or process lists.
Secrets are mounted into the container as files in /run/secrets, which prevents them from appearing in shell histories or 'docker inspect' output. Environment variables are often leaked. The other claims regarding performance, rotation, and deprecation are false.
4. What is the primary purpose of the 'deploy' key in a compose file?
- A.To define the base operating system for the service.
- B.To configure orchestration options like replicas and resource limits when using Swarm.
- C.To specify which storage driver the volume uses.
- D.To force the build of an image before deployment.
Show answer
B. To configure orchestration options like replicas and resource limits when using Swarm.
The 'deploy' key is used for cluster-level configuration (like replicas or CPU limits) compatible with Swarm mode. It is not for image building, base OS definition, or storage driver configuration.
5. If you define 'ports: - "8080:80"', what is the specific network behavior?
- A.The container maps port 8080 on the host to port 80 inside the container.
- B.The container maps port 80 on the host to port 8080 inside the container.
- C.Both the host and container open port 8080 for internal communication.
- D.The container blocks all external traffic on port 80.
Show answer
A. The container maps port 8080 on the host to port 80 inside the container.
In 'host:container' syntax, the left side is the host port and the right side is the container port. Option 1 is inverted, option 3 is invalid syntax for port mapping, and option 4 is incorrect as this mapping exposes the container.