CI/CD Integration
Integrating Terraform testing into CI/CD pipelines ensures that every infrastructure change is validated before reaching production. A well-designed pipeline combines static analysis, unit testing, integration testing, and policy validation to create a comprehensive quality gate that prevents infrastructure failures and security issues.
This final part demonstrates how to build robust CI/CD pipelines that automate Terraform testing and deployment workflows.
GitHub Actions Pipeline
A comprehensive GitHub Actions workflow for Terraform testing:
# .github/workflows/terraform-test.yml
name: Terraform Test and Deploy
on:
pull_request:
paths: ['infrastructure/**', 'modules/**']
push:
branches: [main]
paths: ['infrastructure/**', 'modules/**']
env:
TF_VERSION: 1.6.0
AWS_REGION: us-west-2
jobs:
static-analysis:
name: Static Analysis
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Terraform
uses: hashicorp/setup-terraform@v3
with:
terraform_version: ${{ env.TF_VERSION }}
- name: Terraform Format Check
run: terraform fmt -check -recursive
- name: Terraform Validate
run: |
find . -name "*.tf" -exec dirname {} \; | sort -u | while read dir; do
echo "Validating $dir"
cd "$dir"
terraform init -backend=false
terraform validate
cd - > /dev/null
done
- name: Setup TFLint
uses: terraform-linters/setup-tflint@v4
with:
tflint_version: v0.50.0
- name: Run TFLint
run: |
tflint --init
tflint --recursive
- name: Run Checkov
uses: bridgecrewio/checkov-action@master
with:
directory: .
framework: terraform
output_format: sarif
output_file_path: checkov.sarif
- name: Upload SARIF file
uses: github/codeql-action/upload-sarif@v3
if: always()
with:
sarif_file: checkov.sarif
unit-tests:
name: Unit Tests
runs-on: ubuntu-latest
needs: static-analysis
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Terraform
uses: hashicorp/setup-terraform@v3
with:
terraform_version: ${{ env.TF_VERSION }}
- name: Run Unit Tests
run: |
cd test/unit
for test_dir in */; do
echo "Running unit tests in $test_dir"
cd "$test_dir"
terraform init -backend=false
terraform plan -out=test.tfplan
terraform show -json test.tfplan > plan.json
# Add custom validation logic here
cd ..
done
integration-tests:
name: Integration Tests
runs-on: ubuntu-latest
needs: unit-tests
if: github.event_name == 'pull_request'
environment: testing
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Go
uses: actions/setup-go@v4
with:
go-version: '1.21'
- name: Setup Terraform
uses: hashicorp/setup-terraform@v3
with:
terraform_version: ${{ env.TF_VERSION }}
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
aws-region: ${{ env.AWS_REGION }}
- name: Run Integration Tests
run: |
cd test/integration
go mod download
go test -v -timeout 30m ./...
env:
AWS_DEFAULT_REGION: ${{ env.AWS_REGION }}
policy-validation:
name: Policy Validation
runs-on: ubuntu-latest
needs: static-analysis
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup OPA
uses: open-policy-agent/setup-opa@v2
with:
version: latest
- name: Setup Terraform
uses: hashicorp/setup-terraform@v3
with:
terraform_version: ${{ env.TF_VERSION }}
- name: Generate Terraform Plans
run: |
find infrastructure -name "*.tf" -exec dirname {} \; | sort -u | while read dir; do
echo "Generating plan for $dir"
cd "$dir"
terraform init -backend=false
terraform plan -out=plan.tfplan
terraform show -json plan.tfplan > plan.json
cd - > /dev/null
done
- name: Run Policy Tests
run: |
find infrastructure -name "plan.json" | while read plan; do
echo "Validating policy for $plan"
opa eval -d policies/ -i "$plan" "data.terraform.deny[x]"
done
deploy-staging:
name: Deploy to Staging
runs-on: ubuntu-latest
needs: [unit-tests, policy-validation]
if: github.ref == 'refs/heads/main'
environment: staging
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Terraform
uses: hashicorp/setup-terraform@v3
with:
terraform_version: ${{ env.TF_VERSION }}
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_STAGING_ROLE_ARN }}
aws-region: ${{ env.AWS_REGION }}
- name: Terraform Plan
run: |
cd infrastructure/staging
terraform init
terraform plan -out=staging.tfplan
- name: Terraform Apply
run: |
cd infrastructure/staging
terraform apply staging.tfplan
- name: Run Smoke Tests
run: |
cd test/smoke
go test -v -timeout 10m ./...
deploy-production:
name: Deploy to Production
runs-on: ubuntu-latest
needs: deploy-staging
if: github.ref == 'refs/heads/main'
environment: production
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Terraform
uses: hashicorp/setup-terraform@v3
with:
terraform_version: ${{ env.TF_VERSION }}
- name: Configure AWS Credentials
uses: aws-actions/configure-aws-credentials@v4
with:
role-to-assume: ${{ secrets.AWS_PRODUCTION_ROLE_ARN }}
aws-region: ${{ env.AWS_REGION }}
- name: Terraform Plan
run: |
cd infrastructure/production
terraform init
terraform plan -out=production.tfplan
- name: Manual Approval
uses: trstringer/manual-approval@v1
with:
secret: ${{ github.TOKEN }}
approvers: platform-team
minimum-approvals: 2
issue-title: "Production Deployment Approval"
- name: Terraform Apply
run: |
cd infrastructure/production
terraform apply production.tfplan
- name: Run Production Tests
run: |
cd test/production
go test -v -timeout 15m ./...
GitLab CI Pipeline
A comprehensive GitLab CI pipeline with multiple stages:
# .gitlab-ci.yml
stages:
- validate
- test
- security
- deploy-staging
- deploy-production
variables:
TF_VERSION: "1.6.0"
TF_ROOT: ${CI_PROJECT_DIR}
TF_ADDRESS: ${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/terraform/state/${CI_COMMIT_REF_SLUG}
cache:
key: "${TF_ROOT}"
paths:
- ${TF_ROOT}/.terraform
before_script:
- apt-get update -qq && apt-get install -y -qq git curl unzip
- curl -fsSL https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_amd64.zip -o terraform.zip
- unzip terraform.zip && mv terraform /usr/local/bin/
- terraform --version
validate:
stage: validate
script:
- terraform fmt -check -recursive
- find . -name "*.tf" -exec dirname {} \; | sort -u | while read dir; do
cd "$dir"
terraform init -backend=false
terraform validate
cd - > /dev/null
done
rules:
- changes:
- "**/*.tf"
- "**/*.tfvars"
unit-test:
stage: test
script:
- cd test/unit
- for test_dir in */; do
echo "Testing $test_dir"
cd "$test_dir"
terraform init -backend=false
terraform plan -detailed-exitcode
cd ..
done
rules:
- changes:
- "**/*.tf"
- "**/*.tfvars"
integration-test:
stage: test
image: golang:1.21
services:
- docker:dind
before_script:
- apt-get update -qq && apt-get install -y -qq curl unzip
- curl -fsSL https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_amd64.zip -o terraform.zip
- unzip terraform.zip && mv terraform /usr/local/bin/
script:
- cd test/integration
- go mod download
- go test -v -timeout 30m ./...
rules:
- if: $CI_PIPELINE_SOURCE == "merge_request_event"
changes:
- "**/*.tf"
- "**/*.tfvars"
security-scan:
stage: security
image: bridgecrew/checkov:latest
script:
- checkov -d . --framework terraform --output cli --output json --output-file-path console,checkov-report.json
artifacts:
reports:
sast: checkov-report.json
expire_in: 1 week
rules:
- changes:
- "**/*.tf"
- "**/*.tfvars"
policy-check:
stage: security
image: openpolicyagent/opa:latest
before_script:
- apk add --no-cache curl unzip
- curl -fsSL https://releases.hashicorp.com/terraform/${TF_VERSION}/terraform_${TF_VERSION}_linux_amd64.zip -o terraform.zip
- unzip terraform.zip && mv terraform /usr/local/bin/
script:
- find infrastructure -name "*.tf" -exec dirname {} \; | sort -u | while read dir; do
cd "$dir"
terraform init -backend=false
terraform plan -out=plan.tfplan
terraform show -json plan.tfplan > plan.json
opa eval -d ../../policies/ -i plan.json "data.terraform.deny[x]"
cd - > /dev/null
done
rules:
- changes:
- "**/*.tf"
- "**/*.tfvars"
- "policies/**/*.rego"
deploy-staging:
stage: deploy-staging
environment:
name: staging
url: https://staging.example.com
before_script:
- echo $AWS_STAGING_CREDENTIALS | base64 -d > ~/.aws/credentials
script:
- cd infrastructure/staging
- terraform init -backend-config="address=${TF_ADDRESS}-staging"
- terraform plan -out=staging.tfplan
- terraform apply staging.tfplan
after_script:
- cd test/smoke
- go test -v -timeout 10m ./...
rules:
- if: $CI_COMMIT_BRANCH == "main"
changes:
- "**/*.tf"
- "**/*.tfvars"
deploy-production:
stage: deploy-production
environment:
name: production
url: https://production.example.com
before_script:
- echo $AWS_PRODUCTION_CREDENTIALS | base64 -d > ~/.aws/credentials
script:
- cd infrastructure/production
- terraform init -backend-config="address=${TF_ADDRESS}-production"
- terraform plan -out=production.tfplan
- terraform apply production.tfplan
after_script:
- cd test/production
- go test -v -timeout 15m ./...
when: manual
rules:
- if: $CI_COMMIT_BRANCH == "main"
changes:
- "**/*.tf"
- "**/*.tfvars"
Azure DevOps Pipeline
A comprehensive Azure DevOps pipeline:
# azure-pipelines.yml
trigger:
branches:
include:
- main
paths:
include:
- infrastructure/*
- modules/*
pr:
branches:
include:
- main
paths:
include:
- infrastructure/*
- modules/*
variables:
terraformVersion: '1.6.0'
awsRegion: 'us-west-2'
stages:
- stage: Validate
displayName: 'Validate and Test'
jobs:
- job: StaticAnalysis
displayName: 'Static Analysis'
pool:
vmImage: 'ubuntu-latest'
steps:
- task: TerraformInstaller@0
displayName: 'Install Terraform'
inputs:
terraformVersion: $(terraformVersion)
- script: |
terraform fmt -check -recursive
displayName: 'Terraform Format Check'
- script: |
find . -name "*.tf" -exec dirname {} \; | sort -u | while read dir; do
echo "Validating $dir"
cd "$dir"
terraform init -backend=false
terraform validate
cd - > /dev/null
done
displayName: 'Terraform Validate'
- script: |
curl -s https://raw.githubusercontent.com/terraform-linters/tflint/master/install_linux.sh | bash
tflint --init
tflint --recursive
displayName: 'TFLint'
- script: |
pip install checkov
checkov -d . --framework terraform --output cli --output sarif --output-file-path console,checkov.sarif
displayName: 'Checkov Security Scan'
- task: PublishTestResults@2
condition: always()
inputs:
testResultsFormat: 'JUnit'
testResultsFiles: 'checkov.sarif'
testRunTitle: 'Security Scan Results'
- job: UnitTests
displayName: 'Unit Tests'
dependsOn: StaticAnalysis
pool:
vmImage: 'ubuntu-latest'
steps:
- task: TerraformInstaller@0
inputs:
terraformVersion: $(terraformVersion)
- script: |
cd test/unit
for test_dir in */; do
echo "Running unit tests in $test_dir"
cd "$test_dir"
terraform init -backend=false
terraform plan -out=test.tfplan
cd ..
done
displayName: 'Run Unit Tests'
- stage: IntegrationTest
displayName: 'Integration Testing'
condition: eq(variables['Build.Reason'], 'PullRequest')
dependsOn: Validate
jobs:
- job: IntegrationTests
displayName: 'Integration Tests'
pool:
vmImage: 'ubuntu-latest'
steps:
- task: GoTool@0
inputs:
version: '1.21'
- task: TerraformInstaller@0
inputs:
terraformVersion: $(terraformVersion)
- task: AWSShellScript@1
inputs:
awsCredentials: 'AWS-Testing'
regionName: $(awsRegion)
scriptType: 'inline'
inlineScript: |
cd test/integration
go mod download
go test -v -timeout 30m ./...
displayName: 'Run Integration Tests'
- stage: DeployStaging
displayName: 'Deploy to Staging'
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
dependsOn: Validate
jobs:
- deployment: DeployStaging
displayName: 'Deploy to Staging'
environment: 'staging'
pool:
vmImage: 'ubuntu-latest'
strategy:
runOnce:
deploy:
steps:
- task: TerraformInstaller@0
inputs:
terraformVersion: $(terraformVersion)
- task: TerraformTaskV4@4
inputs:
provider: 'aws'
command: 'init'
workingDirectory: 'infrastructure/staging'
backendServiceAWS: 'AWS-Staging'
backendAWSBucketName: 'terraform-state-staging'
backendAWSKey: 'infrastructure/terraform.tfstate'
- task: TerraformTaskV4@4
inputs:
provider: 'aws'
command: 'plan'
workingDirectory: 'infrastructure/staging'
environmentServiceNameAWS: 'AWS-Staging'
- task: TerraformTaskV4@4
inputs:
provider: 'aws'
command: 'apply'
workingDirectory: 'infrastructure/staging'
environmentServiceNameAWS: 'AWS-Staging'
- script: |
cd test/smoke
go test -v -timeout 10m ./...
displayName: 'Run Smoke Tests'
- stage: DeployProduction
displayName: 'Deploy to Production'
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/main'))
dependsOn: DeployStaging
jobs:
- deployment: DeployProduction
displayName: 'Deploy to Production'
environment: 'production'
pool:
vmImage: 'ubuntu-latest'
strategy:
runOnce:
deploy:
steps:
- task: TerraformInstaller@0
inputs:
terraformVersion: $(terraformVersion)
- task: TerraformTaskV4@4
inputs:
provider: 'aws'
command: 'init'
workingDirectory: 'infrastructure/production'
backendServiceAWS: 'AWS-Production'
backendAWSBucketName: 'terraform-state-production'
backendAWSKey: 'infrastructure/terraform.tfstate'
- task: TerraformTaskV4@4
inputs:
provider: 'aws'
command: 'plan'
workingDirectory: 'infrastructure/production'
environmentServiceNameAWS: 'AWS-Production'
- task: ManualValidation@0
inputs:
notifyUsers: '[email protected]'
instructions: 'Please review the Terraform plan and approve the production deployment'
- task: TerraformTaskV4@4
inputs:
provider: 'aws'
command: 'apply'
workingDirectory: 'infrastructure/production'
environmentServiceNameAWS: 'AWS-Production'
- script: |
cd test/production
go test -v -timeout 15m ./...
displayName: 'Run Production Tests'
Testing Pipeline Optimization
Optimize pipeline performance and reliability:
#!/bin/bash
# scripts/optimize-pipeline.sh
# Parallel test execution
run_tests_parallel() {
local test_dirs=("$@")
local pids=()
for dir in "${test_dirs[@]}"; do
(
echo "Running tests in $dir"
cd "$dir"
terraform init -backend=false
terraform plan -out=test.tfplan
terraform show -json test.tfplan > plan.json
# Run custom validations
python3 ../../scripts/validate-plan.py plan.json
) &
pids+=($!)
done
# Wait for all tests to complete
for pid in "${pids[@]}"; do
wait $pid || exit 1
done
}
# Cache Terraform providers
cache_providers() {
local cache_dir="$HOME/.terraform.d/plugin-cache"
mkdir -p "$cache_dir"
export TF_PLUGIN_CACHE_DIR="$cache_dir"
# Pre-download common providers
terraform providers mirror "$cache_dir"
}
# Selective testing based on changes
selective_testing() {
local changed_files=$(git diff --name-only HEAD~1)
local test_modules=()
for file in $changed_files; do
if [[ $file == modules/* ]]; then
module_name=$(echo "$file" | cut -d'/' -f2)
test_modules+=("test/unit/$module_name")
fi
done
if [ ${#test_modules[@]} -gt 0 ]; then
run_tests_parallel "${test_modules[@]}"
else
echo "No module changes detected, running full test suite"
run_tests_parallel test/unit/*/
fi
}
# Main execution
main() {
cache_providers
selective_testing
}
main "$@"
Conclusion
Comprehensive CI/CD integration ensures that every infrastructure change is thoroughly tested before reaching production. The combination of static analysis, unit testing, integration testing, policy validation, and automated deployment creates a robust quality gate that prevents infrastructure failures and maintains security standards.
The key to successful Terraform testing in CI/CD is balancing thoroughness with speed, using parallel execution, caching, and selective testing to maintain fast feedback cycles while ensuring comprehensive validation of your infrastructure code.