Docker Networking and Storage

Understanding Docker networking and storage is crucial for building multi-container applications and managing persistent data. This section covers networking concepts, volume management, and practical multi-container scenarios.

Docker Networking Fundamentals

Network Types

Docker provides several network drivers for different use cases:

# List available networks
docker network ls

# Default networks:
# bridge    - Default network for containers
# host      - Use host's network stack
# none      - Disable networking

Bridge Networks (Default)

# Create custom bridge network
docker network create my-app-network

# Inspect network details
docker network inspect my-app-network

# Run containers on custom network
docker run -d --name web --network my-app-network nginx:latest
docker run -d --name api --network my-app-network node:16-alpine

# Containers can communicate using container names as hostnames
docker exec web ping api  # This works!

Host Networking

# Use host network (container shares host's network)
docker run -d --name web --network host nginx:latest

# Container binds directly to host ports
# No port mapping needed, but less isolation

Container Communication

# Create network for multi-container app
docker network create webapp-net

# Database container
docker run -d \
  --name postgres \
  --network webapp-net \
  -e POSTGRES_DB=myapp \
  -e POSTGRES_USER=user \
  -e POSTGRES_PASSWORD=pass \
  postgres:13

# Application container
docker run -d \
  --name app \
  --network webapp-net \
  -p 8080:3000 \
  -e DATABASE_URL=postgresql://user:pass@postgres:5432/myapp \
  my-app:latest

# Containers communicate using service names
# app can reach postgres using hostname "postgres"

Advanced Networking

Port Mapping and Exposure

# Map multiple ports
docker run -d \
  --name web-server \
  -p 80:80 \
  -p 443:443 \
  -p 8080:8080 \
  nginx:latest

# Map to specific host interface
docker run -d -p 127.0.0.1:8080:80 nginx:latest

# Map random host port
docker run -d -P nginx:latest  # Maps all exposed ports to random host ports

# Check port mappings
docker port web-server

Network Aliases and DNS

# Create network with custom DNS
docker network create \
  --driver bridge \
  --subnet=172.20.0.0/16 \
  --ip-range=172.20.240.0/20 \
  custom-net

# Run container with network alias
docker run -d \
  --name database \
  --network custom-net \
  --network-alias db \
  --network-alias postgres-db \
  postgres:13

# Other containers can reach it using any alias
docker run --rm --network custom-net alpine ping db
docker run --rm --network custom-net alpine ping postgres-db

Multi-Network Containers

# Create multiple networks
docker network create frontend-net
docker network create backend-net

# Connect container to multiple networks
docker run -d --name api --network backend-net my-api:latest
docker network connect frontend-net api

# Now api container is on both networks
docker inspect api --format='{{.NetworkSettings.Networks}}'

Docker Volumes and Storage

Volume Types

Docker provides three main storage options:

Volumes - Managed by Docker, stored in Docker area
Bind Mounts - Mount host directory into container
tmpfs Mounts - Temporary filesystem in memory

Named Volumes

# Create named volume
docker volume create my-data

# List volumes
docker volume ls

# Inspect volume
docker volume inspect my-data

# Use volume in container
docker run -d \
  --name database \
  -v my-data:/var/lib/mysql \
  -e MYSQL_ROOT_PASSWORD=secret \
  mysql:8.0

# Volume persists even if container is removed
docker rm -f database
docker run -d --name new-db -v my-data:/var/lib/mysql mysql:8.0
# Data is still there!

Bind Mounts

# Mount host directory into container
docker run -d \
  --name web \
  -v /host/path/to/html:/usr/share/nginx/html:ro \
  -v /host/path/to/config:/etc/nginx/conf.d:ro \
  -p 80:80 \
  nginx:latest

# Development with live code reloading
docker run -d \
  --name dev-app \
  -v $(pwd):/app \
  -v /app/node_modules \
  -p 3000:3000 \
  node:16-alpine \
  npm run dev

Volume Management

# Backup volume data
docker run --rm \
  -v my-data:/data \
  -v $(pwd):/backup \
  alpine \
  tar czf /backup/backup.tar.gz -C /data .

# Restore volume data
docker run --rm \
  -v my-data:/data \
  -v $(pwd):/backup \
  alpine \
  tar xzf /backup/backup.tar.gz -C /data

# Copy data between volumes
docker run --rm \
  -v source-vol:/source:ro \
  -v dest-vol:/dest \
  alpine \
  cp -r /source/. /dest/

# Remove unused volumes
docker volume prune

Practical Multi-Container Applications

Example 1: WordPress with MySQL

# Create network
docker network create wordpress-net

# Create volumes
docker volume create mysql-data
docker volume create wordpress-data

# MySQL database
docker run -d \
  --name mysql \
  --network wordpress-net \
  -v mysql-data:/var/lib/mysql \
  -e MYSQL_ROOT_PASSWORD=rootpass \
  -e MYSQL_DATABASE=wordpress \
  -e MYSQL_USER=wpuser \
  -e MYSQL_PASSWORD=wppass \
  mysql:8.0

# WordPress application
docker run -d \
  --name wordpress \
  --network wordpress-net \
  -v wordpress-data:/var/www/html \
  -p 8080:80 \
  -e WORDPRESS_DB_HOST=mysql:3306 \
  -e WORDPRESS_DB_NAME=wordpress \
  -e WORDPRESS_DB_USER=wpuser \
  -e WORDPRESS_DB_PASSWORD=wppass \
  wordpress:latest

# Access WordPress at http://localhost:8080

Example 2: MEAN Stack Application

# Create network
docker network create mean-stack

# MongoDB
docker run -d \
  --name mongodb \
  --network mean-stack \
  -v mongo-data:/data/db \
  -e MONGO_INITDB_ROOT_USERNAME=admin \
  -e MONGO_INITDB_ROOT_PASSWORD=secret \
  mongo:5.0

# Node.js API
docker run -d \
  --name api \
  --network mean-stack \
  -e MONGODB_URI=mongodb://admin:secret@mongodb:27017/myapp?authSource=admin \
  -e NODE_ENV=production \
  my-api:latest

# Angular Frontend
docker run -d \
  --name frontend \
  --network mean-stack \
  -p 80:80 \
  -e API_URL=http://localhost:3000 \
  my-frontend:latest

# Nginx Reverse Proxy
cat > nginx.conf << EOF
upstream api {
    server api:3000;
}

server {
    listen 80;
    
    location /api/ {
        proxy_pass http://api/;
        proxy_set_header Host \$host;
        proxy_set_header X-Real-IP \$remote_addr;
    }
    
    location / {
        proxy_pass http://frontend:80/;
        proxy_set_header Host \$host;
    }
}
EOF

docker run -d \
  --name proxy \
  --network mean-stack \
  -p 8080:80 \
  -v $(pwd)/nginx.conf:/etc/nginx/conf.d/default.conf:ro \
  nginx:alpine

Example 3: Development Environment

# Create development network
docker network create dev-env

# PostgreSQL for development
docker run -d \
  --name dev-postgres \
  --network dev-env \
  -v postgres-dev-data:/var/lib/postgresql/data \
  -e POSTGRES_DB=devdb \
  -e POSTGRES_USER=dev \
  -e POSTGRES_PASSWORD=devpass \
  -p 5432:5432 \
  postgres:13

# Redis for caching
docker run -d \
  --name dev-redis \
  --network dev-env \
  -v redis-dev-data:/data \
  -p 6379:6379 \
  redis:6-alpine

# Application with hot reload
docker run -d \
  --name dev-app \
  --network dev-env \
  -v $(pwd):/app \
  -v /app/node_modules \
  -p 3000:3000 \
  -e DATABASE_URL=postgresql://dev:devpass@dev-postgres:5432/devdb \
  -e REDIS_URL=redis://dev-redis:6379 \
  -e NODE_ENV=development \
  node:16-alpine \
  npm run dev

# Development tools container
docker run -it --rm \
  --name dev-tools \
  --network dev-env \
  -v $(pwd):/workspace \
  -w /workspace \
  node:16-alpine \
  /bin/sh

Container Orchestration Basics

Docker Compose Preview

While we’ll cover Docker Compose in detail later, here’s a preview of how it simplifies multi-container management:

# docker-compose.yml
version: '3.8'

services:
  web:
    image: nginx:latest
    ports:
      - "80:80"
    volumes:
      - ./html:/usr/share/nginx/html:ro
    depends_on:
      - api

  api:
    build: ./api
    environment:
      - DATABASE_URL=postgresql://user:pass@db:5432/myapp
    depends_on:
      - db

  db:
    image: postgres:13
    environment:
      - POSTGRES_DB=myapp
      - POSTGRES_USER=user
      - POSTGRES_PASSWORD=pass
    volumes:
      - postgres_data:/var/lib/postgresql/data

volumes:
  postgres_data:

networks:
  default:
    driver: bridge

# Start entire stack
docker-compose up -d

# Scale services
docker-compose up -d --scale api=3

# View logs
docker-compose logs -f

# Stop stack
docker-compose down

Monitoring and Logging

Container Logs

# View logs from multiple containers
docker logs web-server
docker logs api-server
docker logs database

# Follow logs in real-time
docker logs -f --tail 100 web-server

# Aggregate logs from multiple containers
docker logs web-server 2>&1 | grep ERROR &
docker logs api-server 2>&1 | grep ERROR &
docker logs database 2>&1 | grep ERROR &

Health Checks

# Add health check to Dockerfile
FROM nginx:alpine

# Copy custom nginx config
COPY nginx.conf /etc/nginx/nginx.conf

# Add health check
HEALTHCHECK --interval=30s --timeout=3s --start-period=5s --retries=3 \
  CMD curl -f http://localhost/health || exit 1

EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]

# Run container with health check
docker run -d --name web-with-health my-nginx:latest

# Check health status
docker ps  # Shows health status
docker inspect web-with-health --format='{{.State.Health.Status}}'

Resource Monitoring

# Monitor resource usage
docker stats

# Monitor specific containers
docker stats web-server api-server database

# Get resource usage in JSON format
docker stats --no-stream --format "table {{.Container}}\t{{.CPUPerc}}\t{{.MemUsage}}"

Security Best Practices

Network Security

# Create isolated networks
docker network create --internal backend-net
docker network create frontend-net

# Backend services (no external access)
docker run -d --name database --network backend-net postgres:13
docker run -d --name cache --network backend-net redis:alpine

# API service (connected to both networks)
docker run -d --name api --network backend-net my-api:latest
docker network connect frontend-net api

# Frontend (only on frontend network)
docker run -d --name web --network frontend-net -p 80:80 nginx:latest

Volume Security

# Read-only volumes
docker run -d \
  --name web \
  -v $(pwd)/config:/etc/nginx/conf.d:ro \
  -v $(pwd)/html:/usr/share/nginx/html:ro \
  nginx:latest

# Restrict volume permissions
docker run -d \
  --name app \
  -v app-data:/data \
  --user 1000:1000 \
  my-app:latest

Troubleshooting Network and Storage Issues

Network Debugging

# Test container connectivity
docker exec container1 ping container2
docker exec container1 nslookup container2
docker exec container1 telnet container2 80

# Check network configuration
docker network inspect my-network
docker exec container ip addr show
docker exec container netstat -tlnp

# Debug DNS resolution
docker exec container cat /etc/resolv.conf
docker exec container nslookup google.com

Volume Debugging

# Check volume mounts
docker inspect container --format='{{.Mounts}}'

# Verify volume contents
docker exec container ls -la /data
docker exec container df -h

# Check volume permissions
docker exec container ls -la /data
docker exec -u root container chown -R appuser:appuser /data

Common Issues and Solutions

# Port already in use
netstat -tlnp | grep :8080
docker ps --filter "publish=8080"

# Volume permission issues
docker exec -u root container chown -R $(id -u):$(id -g) /data

# Network connectivity issues
docker network ls
docker network inspect bridge
docker exec container ping 8.8.8.8  # Test external connectivity

Performance Optimization

Network Performance

# Use host networking for high-performance applications
docker run -d --network host high-performance-app:latest

# Optimize bridge network settings
docker network create \
  --driver bridge \
  --opt com.docker.network.bridge.enable_icc=true \
  --opt com.docker.network.bridge.enable_ip_masquerade=true \
  optimized-net

Storage Performance

# Use tmpfs for temporary data
docker run -d \
  --name fast-app \
  --tmpfs /tmp:rw,noexec,nosuid,size=100m \
  my-app:latest

# Optimize volume drivers
docker volume create \
  --driver local \
  --opt type=tmpfs \
  --opt device=tmpfs \
  --opt o=size=1g \
  fast-volume

Summary

In this section, you learned:

Networking Concepts

Docker network types and drivers
Container communication and DNS
Port mapping and exposure
Multi-network container setups

Storage Management

Volume types: named volumes, bind mounts, tmpfs
Volume lifecycle and data persistence
Backup and restore strategies
Security considerations for storage

Practical Applications

Multi-container application architectures
Development environment setup
Monitoring and logging strategies
Health checks and resource monitoring

Best Practices

Network isolation and security
Volume permissions and access control
Performance optimization techniques
Troubleshooting common issues

Key Takeaways:

Use custom networks for multi-container applications
Named volumes provide data persistence and portability
Always consider security when designing network topology
Monitor resource usage and implement health checks
Use bind mounts for development, volumes for production

Next, we’ll explore Docker Compose for managing multi-container applications and advanced deployment patterns.