Docker Compose
Environment Variables and .env Files
Environment variables provide a mechanism to inject configuration into containers without modifying the underlying container image. This approach is essential for maintaining a single artifact that behaves differently across development, testing, and production environments. You should reach for this pattern whenever your application requires externalized configuration like database credentials, API endpoints, or feature toggles.
Passing Environment Variables via CLI
The fundamental way to inject configuration into a container is through the environment variable flag during the execution process. When you start a container, you essentially provide key-value pairs that the operating system inside the container maps into its active shell session. This is the simplest method because it requires no extra files or complex setups; it is purely runtime injection. From a systems perspective, this works because the container runtime intercepts these instructions and sets the environment block for the primary process being executed. The advantage here is immediate visibility; any user running the docker command can see exactly what is being configured. However, as your application grows, passing dozens of flags through the terminal becomes error-prone and unmanageable. This approach is best reserved for quick debugging sessions or simple single-container tasks where permanence is not a requirement.
# Starts a container with a single configuration variable injected at runtime
docker run -e DB_HOST=production.db.internal my-app:latestUsing Environment Files for Bulk Configuration
When a container requires a complex set of environment variables, individual command-line flags become unsustainable. Docker addresses this by allowing you to supply an external file, typically named with a .env suffix, containing key-value pairs formatted as simple text. When you pass the --env-file flag, the runtime reads this file, parses the variables, and injects them into the container environment just as if you had passed them individually via the command line. This mechanism works by decoupling configuration from the execution command, which enables cleaner automation scripts and improves security by allowing these files to be managed separately from the deployment logic. Because the shell parses these files before the container starts, you can keep your sensitive credentials out of your command history, though it is important to note that these files themselves must still be protected by file system permissions to prevent unauthorized access.
# Creates a file containing environment variables
echo "DB_USER=admin" > config.env
echo "DB_PASS=secret_password" >> config.env
# Launches the container using the file for all configuration
docker run --env-file config.env my-app:latestIntegrating Variables in Docker Compose
Docker Compose elevates environment management by providing a declarative structure to handle configuration for multiple services simultaneously. In a docker-compose.yml file, you can specify an 'environment' block under each service definition. This allows you to hardcode variables, but the real power comes from the 'env_file' directive, which tells Compose to automatically ingest variables from a local file. This mechanism is highly effective because it treats configuration as a first-class citizen of the service definition, ensuring that every time you run 'docker-compose up', the correct environment state is reconstructed. The internal logic maps the YAML definitions into the Docker API calls, ensuring that the container is initialized with the exact state required. This is the standard practice for local development environments where consistency across multiple services is crucial for preventing 'it works on my machine' scenarios during team collaboration.
services:
web:
image: my-app:latest
env_file: .env # Loads variables automatically from this file
environment:
- DEBUG=false # Hardcoded default for this serviceVariable Substitution and Shell Integration
A powerful feature of the Compose engine is its ability to perform shell variable substitution directly within the YAML file. If you have a variable set in your current shell session, you can reference it inside your compose file using the ${VARIABLE_NAME} syntax. This works because the Compose tool parses the YAML content before translating it into service instructions, effectively merging your host machine's environment with the service configuration. This is incredibly useful for dynamic scenarios, such as tagging images based on git commits or selecting different ports based on the current user's local setup. By leveraging this, you create a bridge between the host environment and the container orchestration logic. It empowers developers to build highly flexible pipelines that adapt their behavior based on the context in which they are invoked, without needing to rewrite configuration files repeatedly for different deployment targets.
# References an environment variable from the host shell session
# Usage: APP_PORT=8080 docker-compose up
services:
app:
ports:
- "${APP_PORT}:80"Precedence and Overriding Strategies
Understanding the order of precedence is critical when managing multiple sources of configuration. Generally, variables defined explicitly in the 'environment' section of a Compose file will override those defined in an 'env_file'. Furthermore, values provided at runtime through command-line arguments take precedence over static definitions in the YAML file. This hierarchical structure allows for a sophisticated layering strategy: you define sane defaults in the source-controlled .env file, specify environment-specific overrides in the docker-compose.yml, and reserve CLI flags for temporary emergency adjustments. The runtime evaluates these layers in order, essentially 'stacking' the variables until the final environment block is constructed for the container process. By mastering this precedence, you gain the ability to create robust, predictable deployment configurations that remain clean and maintainable, regardless of the complexity of the underlying service infrastructure or the deployment environment you are targeting.
# Docker Compose configuration showing specific overrides
services:
app:
env_file: base.env # Contains general defaults
environment:
- DB_HOST=prod.db.override # Explicitly overrides values in base.envKey points
- Environment variables provide a way to configure containers without changing the image.
- The --env-file flag allows for mass loading of configuration from a text file.
- Docker Compose integrates environment files directly into the service definition process.
- Shell variable substitution enables dynamic configuration based on the host environment.
- You should never commit sensitive credentials in plain text to version control systems.
- Configuration precedence determines which variable takes effect when defined in multiple places.
- Environment variables are best for configuration that varies across different deployment stages.
- Standardizing on a .env file format ensures consistency across different developer machines.
Common mistakes
- Mistake: Committing .env files to Version Control. Why it's wrong: Secrets are exposed in plain text history, leading to security breaches. Fix: Add .env to .gitignore and use Kubernetes Secrets or vault services.
- Mistake: Hardcoding environment variables in the Dockerfile. Why it's wrong: This couples the image to a specific environment, preventing the same image from being reused across dev/staging/prod. Fix: Use ARG for build-time and ENV/Docker run flags for runtime.
- Mistake: Expecting Docker to automatically read a local .env file when running a container. Why it's wrong: Docker engine does not look for .env files by default; they must be explicitly passed using --env-file. Fix: Use the --env-file flag in your docker run command.
- Mistake: Overusing ENV instructions in Dockerfiles. Why it's wrong: ENV variables baked into the image are permanent and difficult to override for specific pod instances. Fix: Reserve ENV for default settings and inject runtime configurations via Kubernetes ConfigMaps or Secrets.
- Mistake: Assuming environment variables are globally shared between containers. Why it's wrong: Environment variables are local to the specific container process, not the Docker network or the host. Fix: Pass required variables explicitly to each container definition.
Interview questions
What is the primary purpose of using an .env file when configuring Docker containers?
The primary purpose of using an .env file is to decouple application configuration from the container image itself, following the Twelve-Factor App methodology. By storing environment-specific variables like database credentials or API keys in a separate file, you ensure that the Docker image remains portable and immutable. This means you can build a single image once and deploy it across development, staging, and production environments simply by injecting different environment variables at runtime, rather than rebuilding the container.
How does Docker Compose handle .env files by default?
When you run a command like 'docker-compose up', Docker Compose automatically looks for a file named '.env' in the current project directory. It parses this file and makes the variables available for substitution within the 'docker-compose.yml' file. This is extremely useful for defining common values, such as image versions or port mappings, directly in your orchestration file. You can reference them using the '${VARIABLE_NAME}' syntax, which keeps your configuration DRY and prevents hard-coding sensitive information directly into your version-controlled YAML files.
What is the difference between setting environment variables via a Dockerfile versus at container runtime?
Setting variables via 'ENV' in a Dockerfile embeds them into the image layers at build time, meaning they are hard-coded and static; this is best for permanent configurations like PATH settings. In contrast, passing variables at runtime using the '--env' flag or 'env_file' in Docker Compose allows you to inject dynamic data, like secrets or connection strings, without modifying the image. Runtime injection is significantly more secure and flexible, as it avoids exposing sensitive data in the image layers, which are often stored in accessible registries.
How should you manage sensitive environment variables in a Kubernetes cluster versus using plain .env files?
While .env files are fine for local development, they are insecure for production Kubernetes environments. In Kubernetes, you should use 'Secrets' objects to store sensitive data like passwords or tokens. Secrets can be mounted as volumes or exposed as environment variables within your Pods. This provides an additional layer of security by separating sensitive data from the Pod definition, allowing for better access control, encryption at rest, and the ability to rotate credentials without having to redeploy your entire application container infrastructure.
Can you compare the 'ConfigMap' approach with the 'env_file' approach in Kubernetes deployments?
The 'env_file' approach is essentially a local Docker development pattern, whereas ConfigMaps are a native Kubernetes primitive. A ConfigMap acts as a centralized key-value store that can be shared across multiple pods, making it ideal for managing non-sensitive application settings. While both allow you to inject configuration, ConfigMaps provide better lifecycle management and decoupling, allowing you to update configuration without restarting the container in some scenarios. Kubernetes specifically maps these ConfigMaps directly into the Pod's environment space, ensuring consistent configuration management across distributed nodes.
Explain the order of precedence for environment variables when using Docker and Kubernetes together.
Environment variables follow a strict hierarchy of precedence to prevent configuration drift. Generally, variables defined explicitly in the container runtime (or Kubernetes manifest 'env' block) take the highest priority, overriding anything defined in the Dockerfile 'ENV' instruction. If using Kubernetes, 'valueFrom' references, such as those pulling from a Secret or ConfigMap, will override static values defined in the PodSpec. Understanding this hierarchy is critical for debugging, as developers must know exactly which source is winning if multiple variables with the same name are present across different configuration layers.
Check yourself
1. When deploying to Kubernetes, why should you prefer ConfigMaps over building environment variables directly into your Docker image?
- A.ConfigMaps are encrypted by default, whereas Docker images are not.
- B.ConfigMaps allow you to update configuration without rebuilding or redeploying the container image.
- C.Docker images cannot store environment variables, so ConfigMaps are the only option.
- D.ConfigMaps automatically sync with local .env files on the developer's machine.
Show answer
B. ConfigMaps allow you to update configuration without rebuilding or redeploying the container image.
ConfigMaps decouple configuration from the image, allowing for environment-specific adjustments without recompiling the image. Option 0 is false as ConfigMaps are plain text; option 2 is false as images can have ENV; option 3 is false as there is no automatic local sync.
2. What happens if you define an environment variable in a Dockerfile using ENV and then try to override it using the --env flag during docker run?
- A.The container fails to start due to a conflict.
- B.The Dockerfile value takes precedence and cannot be changed.
- C.The --env flag value overrides the value defined in the Dockerfile.
- D.Both values are concatenated together into a single string.
Show answer
C. The --env flag value overrides the value defined in the Dockerfile.
Runtime environment variables passed via the command line or container orchestration layers take precedence over values baked into the image via ENV. The other options describe incorrect behaviors regarding variable priority.
3. Which is the most secure method for handling sensitive database credentials when using Kubernetes?
- A.Hardcoding them as ENV variables in the Dockerfile.
- B.Storing them in a .env file and copying it into the image during the build process.
- C.Using Kubernetes Secret objects mounted as environment variables or volumes.
- D.Passing them as plain text arguments in the docker run command history.
Show answer
C. Using Kubernetes Secret objects mounted as environment variables or volumes.
Kubernetes Secrets are designed to handle sensitive data with better access control and management than hardcoding or including files in images. Hardcoding or copying .env files leaks credentials into the image layer history.
4. Why does using the --env-file flag in Docker not automatically make your application secure?
- A.The file is only read at image build time, not container start time.
- B.The environment variables are still injected into the container's environment, where they can be inspected by any process with sufficient permissions.
- C.Docker automatically uploads the .env file contents to a public registry.
- D.Environment variables are converted to clear-text logs immediately upon container startup.
Show answer
B. The environment variables are still injected into the container's environment, where they can be inspected by any process with sufficient permissions.
Environment variables are accessible to the process tree and often visible via system tools, which is why they shouldn't be used for highly sensitive secrets if more secure alternatives are available. Other options are technically incorrect regarding how Docker handles those files.
5. If you have a multi-stage Docker build, what is the best way to handle environment variables intended only for the build process?
- A.Use ENV to set them, so they persist in the final image.
- B.Use ARG, as these are not persisted in the final image layers.
- C.Create a temporary .env file and delete it in the same RUN command.
- D.Inject them via Kubernetes ConfigMaps during the build.
Show answer
B. Use ARG, as these are not persisted in the final image layers.
ARG variables are specifically designed for build-time configuration and do not persist in the final container image, unlike ENV. The other methods either leak data or are functionally incapable of passing build arguments correctly.