CI/CD Pipeline Integration

Manually building and deploying containers works for development, but production demands automation. I’ve built CI/CD pipelines that deploy hundreds of times per day while maintaining security and reliability. The key is treating your pipeline as code and building in quality gates at every step.

Pipeline Architecture Strategy

A robust container CI/CD pipeline includes several stages that fail fast and provide clear feedback:

Source Stage: Code commit triggers the pipeline
Test Stage: Unit tests, integration tests, and code quality checks
Security Stage: Vulnerability scanning and compliance checks
Build Stage: Container image creation and optimization
Deploy Stage: Progressive rollout with monitoring and rollback capabilities

GitHub Actions Container Pipeline

GitHub Actions provides excellent container support with built-in Docker registry integration:

# .github/workflows/container-pipeline.yml
name: Container CI/CD Pipeline

on:
  push:
    branches: [ main, develop ]
  pull_request:
    branches: [ main ]

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Node.js
        uses: actions/setup-node@v4
        with:
          node-version: '18'
          cache: 'npm'

      - name: Install dependencies
        run: npm ci

      - name: Run tests
        run: npm test -- --coverage

      - name: Upload coverage
        uses: codecov/codecov-action@v3

  security-scan:
    runs-on: ubuntu-latest
    needs: test
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Run Trivy scanner
        uses: aquasecurity/trivy-action@master
        with:
          scan-type: 'fs'
          format: 'sarif'
          output: 'trivy-results.sarif'

      - name: Upload scan results
        uses: github/codeql-action/upload-sarif@v2
        with:
          sarif_file: 'trivy-results.sarif'

  build-and-push:
    runs-on: ubuntu-latest
    needs: [test, security-scan]
    if: github.event_name != 'pull_request'
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Log in to registry
        uses: docker/login-action@v3
        with:
          registry: ${{ env.REGISTRY }}
          username: ${{ github.actor }}
          password: ${{ secrets.GITHUB_TOKEN }}

      - name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          platforms: linux/amd64,linux/arm64
          push: true
          tags: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

  deploy-staging:
    runs-on: ubuntu-latest
    needs: build-and-push
    if: github.ref == 'refs/heads/develop'
    environment: staging
    steps:
      - name: Deploy to staging
        run: |
          kubectl set image deployment/web-app web=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }} -n staging
          kubectl rollout status deployment/web-app -n staging

Advanced Testing Strategies

Comprehensive testing prevents issues from reaching production:

# docker-compose.test.yml
version: '3.8'

services:
  app:
    build:
      context: .
      dockerfile: Dockerfile.test
    environment:
      - NODE_ENV=test
      - DATABASE_URL=postgresql://test:test@db:5432/testdb
    depends_on:
      db:
        condition: service_healthy

  db:
    image: postgres:15-alpine
    environment:
      - POSTGRES_DB=testdb
      - POSTGRES_USER=test
      - POSTGRES_PASSWORD=test
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U test -d testdb"]
      interval: 5s
      retries: 5

  integration-tests:
    build:
      context: .
      dockerfile: Dockerfile.test
    environment:
      - API_URL=http://app:8080
    depends_on:
      - app
    command: npm run test:integration

Security Integration

Security scanning should be integrated throughout the pipeline:

#!/bin/bash
# security-scan.sh

IMAGE_NAME=${1:-myapp:latest}
SEVERITY_THRESHOLD=${2:-HIGH}

echo "Running security scan for $IMAGE_NAME"

# Scan the built image
trivy image \
  --severity $SEVERITY_THRESHOLD,CRITICAL \
  --format json \
  --output image-scan.json \
  $IMAGE_NAME

# Scan for secrets
trivy fs \
  --scanners secret \
  --format json \
  --output secret-scan.json \
  .

# Check results
CRITICAL_VULNS=$(jq '.Results[]?.Vulnerabilities[]? | select(.Severity == "CRITICAL") | length' image-scan.json | wc -l)

if [ "$CRITICAL_VULNS" -gt 0 ]; then
    echo "Critical vulnerabilities found. Failing build."
    exit 1
fi

echo "Security scan passed"

Progressive Deployment

Implement safe deployment patterns that minimize risk:

# canary-deployment.yaml
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata:
  name: web-app-rollout
spec:
  replicas: 10
  strategy:
    canary:
      steps:
      - setWeight: 10
      - pause: {duration: 2m}
      - setWeight: 50
      - pause: {duration: 5m}
      analysis:
        templates:
        - templateName: success-rate
      canaryService: web-app-canary
      stableService: web-app-stable
  selector:
    matchLabels:
      app: web-app
  template:
    spec:
      containers:
      - name: web
        image: myregistry.io/web-app:latest

Multi-Environment Pipeline

Manage deployments across environments with proper promotion gates:

# Multi-environment workflow
jobs:
  deploy-dev:
    needs: build
    environment: development
    steps:
      - name: Deploy to dev
        run: kubectl set image deployment/web-app web=${{ needs.build.outputs.image }} -n dev

  test-dev:
    needs: deploy-dev
    steps:
      - name: Run smoke tests
        run: curl -f https://dev.myapp.com/health

  deploy-staging:
    needs: test-dev
    environment: staging
    steps:
      - name: Deploy to staging
        run: kubectl set image deployment/web-app web=${{ needs.build.outputs.image }} -n staging

  deploy-production:
    needs: deploy-staging
    environment: production
    steps:
      - name: Deploy to production
        run: |
          kubectl set image deployment/web-app web=${{ needs.build.outputs.image }} -n production
          kubectl rollout status deployment/web-app -n production

Pipeline Troubleshooting

Common issues I’ve encountered and their solutions:

Flaky tests causing failures:

- name: Run tests with retry
  uses: nick-invision/retry@v2
  with:
    timeout_minutes: 10
    max_attempts: 3
    command: npm test

Docker build cache misses:

- name: Build with cache
  uses: docker/build-push-action@v5
  with:
    cache-from: type=gha
    cache-to: type=gha,mode=max

A well-designed CI/CD pipeline becomes the backbone of your containerization strategy. It should be fast, reliable, and provide clear feedback when things go wrong. In the next part, I’ll cover monitoring and observability for production container environments.