Working with Docker Compose? Use environment variables

Intro to Docker Compose

Much like the Docker runtime, Compose is not limited to the Docker Enterprise platform. Compose also works with Amazon Elastic Container Service (ECS), Azure Container Instances (ACI) and Kubernetes.

Additionally, Docker made the Compose specification, a cloud-agnostic community project that works with other packaging formats like Helm charts and Kubernetes YAML manifests. Furthermore, Compose can deploy applications on ACI, Amazon ECS and AWS Fargate, and can configure Kubernetes resources via the Kompose conversion tool. For example, the Azure integration enables developers to use native Docker commands to run applications in ACI via the Docker CLI or Virtual Studio Code extension.

Compose is included in Docker Desktop for Windows and MacOS and can be installed manually on Linux via the Docker website.

The Compose specification defines the following as the essential elements of a multi-container application Compose file:

• Services based on one or more container images and run within a Docker runtime to provide application functionality;
• Networks that provide IP connectivity and routing between containers;
• Volumes that store persistent data that can be shared among containers via a filesystem mount point;
• Configs that enable services to change behavior without an image file rebuild, via options stored in a file mounted to the container's file system; and
• Secrets containing sensitive configuration information such as server certificates.

Together, these components form an application project.

Compose file and syntax

The following three steps are required to deploy applications with Compose:

1. Define the application images and environment using a Dockerfile.
2. Define and configure the components of a multi-container application using a Docker Compose file.
3. Run docker-compose up to deploy and run the application.

A typical, albeit simple, example of a composite application is a two-tier website consisting of a front-end web server, a back-end database server and an associated disk volume. When implemented with containers, this consists of the following:

• two services and related image files (web and database);
• one secret (HTTPS certificate for the web front end);
• one configuration (HTTP, also for the front end);
• one persistent volume mounted by the database; and
• two networks (front end to the internet; back end between the web and database servers).

The YAML Compose file for this project looks like the following:

services:
  frontend:
    image: awesome/webapp
    ports:
      - "443:8043"
    networks:
      - front-tier
      - back-tier
    configs:
      - httpd-config
    secrets:
      - server-certificate

  backend:
    image: awesome/database
    volumes:
      - db-data:/etc/data
    networks:
      - back-tier

volumes:
  db-data:
    driver: flocker
    driver_opts:
      size: "10GiB"

configs:
  httpd-config:
    external: true

secrets:
  server-certificate:
    external: true

networks:
  # The presence of these objects is sufficient to define them
  front-tier: {}
  back-tier: {}

Compose files are typically named compose.yaml and use keywords defined in the specification. Runtime options can be set by command-line flags or environment variables in an env_file -- typically carrying a .env suffix.

Compose environment variables

As introduced above, environment variables are used in every programming language as an abstraction layer to enable IT admins to change parameters in a piece of software or configuration without changing the underlying code itself.

In Docker Compose, IT admins can use environment variables to generalize configurations for different situations, deployment environments and security contexts without editing the main project file(s) manually. Variables can either be passed as command-line arguments -- suitable for only a few parameters -- or via a .env file, preferred for more complicated configurations.

YAML files are, essentially, lists of key-value pairs. Compose environment files support both map and array syntaxes, as illustrated in Figure 1.

Because variables can be defined in several places, developers must be careful not to override items defined in an environment file. The order of evaluation precedence for variables is:

1. Compose file
2. Shell environment variables
3. Environment file
4. Dockerfile

As the syntax example in Figure 1 indicates, environment variables are best used to specify deployment-specific parameters, such as an application name, container image, volume names and mount points, IP addresses and ports, time intervals, Boolean values and runtime variables -- for example, retries=5.

Do not store security information -- such as passwords, SSH private keys, SSL certificates or any other data that should not be stored as clear text -- as environment files and variables. Instead, use Swarm Secrets, Kubernetes Secrets or a cloud service like AWS Secrets Manager or Google Cloud Secret Manager.

The main Compose file uses variable substitution to refer to parameters defined in an environment file. This file supports both $VARIABLE and ${VARIABLE} syntax:

# database server
db-container:
IMAGE:"mysql:${mysql_version}"
PORT="{db_port}"

Environment variables are a critical element of Docker Compose, as the prime advantage is reusable multi-container application configurations. Variables enable developers to modify parameters quickly for different deployment and resource needs without adjusting the underlying code and, thus, should be part of every production Docker Compose environment.