Docker Compose Orchestration: Introduction and Setup

Docker Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services, networks, and volumes, then create and start all services with a single command.

What is Docker Compose?

Docker Compose solves the complexity of managing multiple containers by providing:

Declarative Configuration: Define your entire application stack in a single YAML file
Service Orchestration: Manage dependencies between containers
Environment Management: Easy switching between development, testing, and production
Scaling: Scale services up or down with simple commands
Networking: Automatic network creation and service discovery

Installation and Setup

Installing Docker Compose

Linux:

# Download the latest version
sudo curl -L "https://github.com/docker/compose/releases/latest/download/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose

# Make it executable
sudo chmod +x /usr/local/bin/docker-compose

# Verify installation
docker-compose --version

macOS (with Homebrew):

brew install docker-compose

Windows: Docker Compose is included with Docker Desktop for Windows.

Compose File Structure

A basic docker-compose.yml file structure:

version: '3.8'

services:
  web:
    build: .
    ports:
      - "8000:8000"
    volumes:
      - .:/app
    environment:
      - DEBUG=1
    depends_on:
      - db

  db:
    image: postgres:13
    environment:
      POSTGRES_DB: myapp
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password
    volumes:
      - postgres_data:/var/lib/postgresql/data

volumes:
  postgres_data:

networks:
  default:
    driver: bridge

Your First Compose Application

Let’s create a simple web application with a database:

Project Structure

my-app/
├── docker-compose.yml
├── Dockerfile
├── app.py
├── requirements.txt
└── templates/
    └── index.html

Flask Application

app.py:

from flask import Flask, render_template
import psycopg2
import os

app = Flask(__name__)

def get_db_connection():
    conn = psycopg2.connect(
        host=os.environ.get('DB_HOST', 'db'),
        database=os.environ.get('DB_NAME', 'myapp'),
        user=os.environ.get('DB_USER', 'user'),
        password=os.environ.get('DB_PASSWORD', 'password')
    )
    return conn

@app.route('/')
def index():
    try:
        conn = get_db_connection()
        cur = conn.cursor()
        cur.execute('SELECT version();')
        db_version = cur.fetchone()
        cur.close()
        conn.close()
        return render_template('index.html', db_version=db_version[0])
    except Exception as e:
        return f"Database connection failed: {str(e)}"

if __name__ == '__main__':
    app.run(host='0.0.0.0', port=8000, debug=True)

requirements.txt:

Flask==2.3.3
psycopg2-binary==2.9.7

templates/index.html:

<!DOCTYPE html>
<html>
<head>
    <title>Docker Compose App</title>
</head>
<body>
    <h1>Hello from Docker Compose!</h1>
    <p>Database Version: {{ db_version }}</p>
</body>
</html>

Dockerfile

FROM python:3.9-slim

WORKDIR /app

COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY . .

EXPOSE 8000

CMD ["python", "app.py"]

Docker Compose Configuration

docker-compose.yml:

version: '3.8'

services:
  web:
    build: .
    ports:
      - "8000:8000"
    volumes:
      - .:/app
    environment:
      - DB_HOST=db
      - DB_NAME=myapp
      - DB_USER=user
      - DB_PASSWORD=password
    depends_on:
      - db
    restart: unless-stopped

  db:
    image: postgres:13
    environment:
      POSTGRES_DB: myapp
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - ./init.sql:/docker-entrypoint-initdb.d/init.sql
    ports:
      - "5432:5432"
    restart: unless-stopped

  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    volumes:
      - redis_data:/data
    restart: unless-stopped

volumes:
  postgres_data:
  redis_data:

networks:
  default:
    driver: bridge

Essential Compose Commands

Basic Operations

# Start all services
docker-compose up

# Start in detached mode
docker-compose up -d

# Build and start
docker-compose up --build

# Stop all services
docker-compose down

# Stop and remove volumes
docker-compose down -v

# View running services
docker-compose ps

# View logs
docker-compose logs

# Follow logs for specific service
docker-compose logs -f web

Service Management

# Start specific service
docker-compose start web

# Stop specific service
docker-compose stop web

# Restart service
docker-compose restart web

# Scale service
docker-compose up --scale web=3

# Execute command in running container
docker-compose exec web bash

# Run one-off command
docker-compose run web python manage.py migrate

Environment Configuration

Environment Files

Create .env file for environment variables:

# .env
POSTGRES_DB=myapp
POSTGRES_USER=user
POSTGRES_PASSWORD=secretpassword
DEBUG=1
SECRET_KEY=your-secret-key

Update docker-compose.yml:

version: '3.8'

services:
  web:
    build: .
    ports:
      - "8000:8000"
    env_file:
      - .env
    environment:
      - DB_HOST=db
    depends_on:
      - db

  db:
    image: postgres:13
    env_file:
      - .env
    volumes:
      - postgres_data:/var/lib/postgresql/data

Multiple Environment Files

# docker-compose.override.yml (development)
version: '3.8'

services:
  web:
    volumes:
      - .:/app
    environment:
      - DEBUG=1
    command: python app.py

  db:
    ports:
      - "5432:5432"

# docker-compose.prod.yml (production)
version: '3.8'

services:
  web:
    environment:
      - DEBUG=0
    restart: always
    command: gunicorn --bind 0.0.0.0:8000 app:app

  db:
    restart: always

Run with specific configuration:

# Development (uses docker-compose.override.yml automatically)
docker-compose up

# Production
docker-compose -f docker-compose.yml -f docker-compose.prod.yml up

Networking in Compose

Default Network

Compose automatically creates a default network for your application:

version: '3.8'

services:
  web:
    image: nginx
    # Can communicate with 'api' service using hostname 'api'
  
  api:
    image: node:16
    # Can communicate with 'web' service using hostname 'web'

Custom Networks

version: '3.8'

services:
  web:
    image: nginx
    networks:
      - frontend
      - backend

  api:
    image: node:16
    networks:
      - backend

  db:
    image: postgres:13
    networks:
      - backend

networks:
  frontend:
    driver: bridge
  backend:
    driver: bridge
    internal: true  # No external access

External Networks

version: '3.8'

services:
  web:
    image: nginx
    networks:
      - existing-network

networks:
  existing-network:
    external: true

Volume Management

Named Volumes

version: '3.8'

services:
  db:
    image: postgres:13
    volumes:
      - postgres_data:/var/lib/postgresql/data
      - postgres_config:/etc/postgresql

volumes:
  postgres_data:
    driver: local
  postgres_config:
    driver: local
    driver_opts:
      type: none
      o: bind
      device: /host/path/to/config

Bind Mounts

version: '3.8'

services:
  web:
    image: nginx
    volumes:
      - ./html:/usr/share/nginx/html:ro  # Read-only
      - ./logs:/var/log/nginx:rw         # Read-write
      - /etc/localtime:/etc/localtime:ro # Host timezone

External Volumes

version: '3.8'

services:
  db:
    image: postgres:13
    volumes:
      - existing_volume:/var/lib/postgresql/data

volumes:
  existing_volume:
    external: true

Health Checks and Dependencies

Health Checks

version: '3.8'

services:
  web:
    build: .
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8000/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

  db:
    image: postgres:13
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U user -d myapp"]
      interval: 10s
      timeout: 5s
      retries: 5

Service Dependencies

version: '3.8'

services:
  web:
    build: .
    depends_on:
      db:
        condition: service_healthy
      redis:
        condition: service_started

  db:
    image: postgres:13
    healthcheck:
      test: ["CMD-SHELL", "pg_isready"]
      interval: 10s
      timeout: 5s
      retries: 5

  redis:
    image: redis:7-alpine

Practical Example: Development Environment

Let’s create a complete development environment:

docker-compose.dev.yml:

version: '3.8'

services:
  # Frontend Development Server
  frontend:
    build:
      context: ./frontend
      dockerfile: Dockerfile.dev
    ports:
      - "3000:3000"
    volumes:
      - ./frontend:/app
      - /app/node_modules
    environment:
      - REACT_APP_API_URL=http://localhost:8000
    command: npm start

  # Backend API
  backend:
    build:
      context: ./backend
      dockerfile: Dockerfile.dev
    ports:
      - "8000:8000"
    volumes:
      - ./backend:/app
    environment:
      - DATABASE_URL=postgresql://user:password@db:5432/devdb
      - REDIS_URL=redis://redis:6379
      - DEBUG=1
    depends_on:
      - db
      - redis
    command: python manage.py runserver 0.0.0.0:8000

  # Database
  db:
    image: postgres:13
    environment:
      POSTGRES_DB: devdb
      POSTGRES_USER: user
      POSTGRES_PASSWORD: password
    volumes:
      - postgres_dev_data:/var/lib/postgresql/data
      - ./database/init.sql:/docker-entrypoint-initdb.d/init.sql
    ports:
      - "5432:5432"

  # Redis Cache
  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
    volumes:
      - redis_dev_data:/data

  # Database Admin Interface
  pgadmin:
    image: dpage/pgadmin4
    environment:
      PGADMIN_DEFAULT_EMAIL: [email protected]
      PGADMIN_DEFAULT_PASSWORD: admin
    ports:
      - "5050:80"
    volumes:
      - pgadmin_data:/var/lib/pgadmin

volumes:
  postgres_dev_data:
  redis_dev_data:
  pgadmin_data:

Start the development environment:

docker-compose -f docker-compose.dev.yml up --build

Summary

In this introduction, you’ve learned:

Core Concepts

Docker Compose Purpose: Orchestrating multi-container applications
YAML Configuration: Declarative service definitions
Service Dependencies: Managing container startup order
Environment Management: Different configurations for different stages

Essential Skills

Installation and Setup: Getting Compose ready for development
Basic Commands: Starting, stopping, and managing services
Networking: Service discovery and custom networks
Volume Management: Persistent data and bind mounts
Health Checks: Ensuring service reliability

Practical Applications

Development Environment: Complete local development stack
Environment Configuration: Using .env files and overrides
Service Communication: Inter-container networking
Data Persistence: Volume management strategies

Next Steps: In Part 2, we’ll dive deeper into core concepts including advanced service configuration, networking patterns, and volume strategies that form the foundation of production-ready Compose applications.

Continue Your Learning

This is part 1 of 6 in the comprehensive guide.

Guide Overview See all 6 parts Next → Core Concepts and Fundamentals