Governance and Cost Management
Managing governance and costs across multiple cloud providers presents unique challenges. Each provider has different pricing models, compliance frameworks, and management tools. Effective multi-cloud governance requires unified policies, consistent tagging strategies, and comprehensive cost monitoring that works across AWS, Azure, and Google Cloud.
This final part covers the patterns and practices for implementing governance and cost management in multi-cloud Terraform environments.
Unified Tagging Strategy
Implement consistent tagging across all cloud providers:
# Global tagging strategy
locals {
# Standard tags that work across all providers
standard_tags = {
Environment = var.environment
Project = var.project_name
Owner = var.team_name
CostCenter = var.cost_center
ManagedBy = "terraform"
CreatedDate = formatdate("YYYY-MM-DD", timestamp())
LastModified = formatdate("YYYY-MM-DD", timestamp())
}
# Provider-specific tag formats
aws_tags = local.standard_tags
azure_tags = {
for k, v in local.standard_tags :
k => v
}
gcp_labels = {
for k, v in local.standard_tags :
lower(replace(k, " ", "_")) => lower(replace(v, " ", "_"))
}
}
# AWS resources with standard tags
resource "aws_instance" "web" {
count = var.providers_config.aws_enabled ? var.instance_count : 0
ami = data.aws_ami.latest[0].id
instance_type = var.instance_type
tags = merge(local.aws_tags, {
Name = "${var.project_name}-web-${count.index + 1}"
Role = "webserver"
Provider = "aws"
})
}
# Azure resources with standard tags
resource "azurerm_virtual_machine" "web" {
count = var.providers_config.azure_enabled ? var.instance_count : 0
name = "${var.project_name}-web-${count.index + 1}"
location = var.azure_location
resource_group_name = azurerm_resource_group.main[0].name
vm_size = var.azure_vm_size
tags = merge(local.azure_tags, {
Role = "webserver"
Provider = "azure"
})
}
# GCP resources with standard labels
resource "google_compute_instance" "web" {
count = var.providers_config.gcp_enabled ? var.instance_count : 0
name = "${var.project_name}-web-${count.index + 1}"
machine_type = var.gcp_machine_type
zone = var.gcp_zone
labels = merge(local.gcp_labels, {
role = "webserver"
provider = "gcp"
})
}
Multi-Cloud Policy Framework
Implement consistent policies across providers:
# Policy configuration for all providers
variable "governance_policies" {
description = "Governance policies to apply across all providers"
type = object({
allowed_regions = object({
aws = list(string)
azure = list(string)
gcp = list(string)
})
allowed_instance_types = object({
aws = list(string)
azure = list(string)
gcp = list(string)
})
required_tags = list(string)
cost_limits = object({
monthly_budget = number
alert_threshold = number
})
})
default = {
allowed_regions = {
aws = ["us-west-2", "us-east-1", "eu-west-1"]
azure = ["West US 2", "East US", "West Europe"]
gcp = ["us-west1", "us-east1", "europe-west1"]
}
allowed_instance_types = {
aws = ["t3.micro", "t3.small", "t3.medium", "t3.large"]
azure = ["Standard_B1s", "Standard_B2s", "Standard_D2s_v3"]
gcp = ["e2-micro", "e2-small", "e2-medium", "e2-standard-2"]
}
required_tags = ["Environment", "Project", "Owner", "CostCenter"]
cost_limits = {
monthly_budget = 10000
alert_threshold = 80
}
}
}
# AWS policy validation
resource "aws_instance" "web" {
count = var.providers_config.aws_enabled ? var.instance_count : 0
ami = data.aws_ami.latest[0].id
instance_type = var.aws_instance_type
lifecycle {
precondition {
condition = contains(
var.governance_policies.allowed_instance_types.aws,
var.aws_instance_type
)
error_message = "Instance type ${var.aws_instance_type} is not allowed. Allowed types: ${join(", ", var.governance_policies.allowed_instance_types.aws)}"
}
postcondition {
condition = alltrue([
for tag in var.governance_policies.required_tags :
contains(keys(self.tags), tag)
])
error_message = "All required tags must be present: ${join(", ", var.governance_policies.required_tags)}"
}
}
tags = local.aws_tags
}
# Azure policy validation
resource "azurerm_virtual_machine" "web" {
count = var.providers_config.azure_enabled ? var.instance_count : 0
name = "${var.project_name}-web-${count.index + 1}"
location = var.azure_location
resource_group_name = azurerm_resource_group.main[0].name
vm_size = var.azure_vm_size
lifecycle {
precondition {
condition = contains(
var.governance_policies.allowed_regions.azure,
var.azure_location
)
error_message = "Region ${var.azure_location} is not allowed for Azure resources."
}
precondition {
condition = contains(
var.governance_policies.allowed_instance_types.azure,
var.azure_vm_size
)
error_message = "VM size ${var.azure_vm_size} is not allowed."
}
}
tags = local.azure_tags
}
Cost Monitoring and Budgets
Implement comprehensive cost monitoring across providers:
# AWS cost monitoring
resource "aws_budgets_budget" "monthly_aws" {
count = var.providers_config.aws_enabled ? 1 : 0
name = "${var.project_name}-aws-monthly-budget"
budget_type = "COST"
limit_amount = var.governance_policies.cost_limits.monthly_budget * 0.4 # 40% allocation to AWS
limit_unit = "USD"
time_unit = "MONTHLY"
cost_filters = {
LinkedAccount = [data.aws_caller_identity.current[0].account_id]
TagKey = ["Project"]
TagValue = [var.project_name]
}
notification {
comparison_operator = "GREATER_THAN"
threshold = var.governance_policies.cost_limits.alert_threshold
threshold_type = "PERCENTAGE"
notification_type = "ACTUAL"
subscriber_email_addresses = var.budget_notification_emails
}
}
# Azure cost monitoring
resource "azurerm_consumption_budget_resource_group" "monthly_azure" {
count = var.providers_config.azure_enabled ? 1 : 0
name = "${var.project_name}-azure-monthly-budget"
resource_group_id = azurerm_resource_group.main[0].id
amount = var.governance_policies.cost_limits.monthly_budget * 0.4 # 40% allocation to Azure
time_grain = "Monthly"
time_period {
start_date = formatdate("YYYY-MM-01T00:00:00Z", timestamp())
end_date = formatdate("YYYY-MM-01T00:00:00Z", timeadd(timestamp(), "8760h")) # 1 year
}
notification {
enabled = true
threshold = var.governance_policies.cost_limits.alert_threshold
operator = "GreaterThan"
threshold_type = "Actual"
contact_emails = var.budget_notification_emails
}
}
# GCP cost monitoring
resource "google_billing_budget" "monthly_gcp" {
count = var.providers_config.gcp_enabled ? 1 : 0
billing_account = var.gcp_billing_account
display_name = "${var.project_name}-gcp-monthly-budget"
budget_filter {
projects = ["projects/${var.gcp_project_id}"]
labels = {
project = var.project_name
}
}
amount {
specified_amount {
currency_code = "USD"
units = tostring(floor(var.governance_policies.cost_limits.monthly_budget * 0.2)) # 20% allocation to GCP
}
}
threshold_rules {
threshold_percent = var.governance_policies.cost_limits.alert_threshold / 100
spend_basis = "CURRENT_SPEND"
}
all_updates_rule {
monitoring_notification_channels = var.gcp_notification_channels
disable_default_iam_recipients = false
}
}
Unified Cost Reporting
Create unified cost reporting across all providers:
#!/usr/bin/env python3
# scripts/multi_cloud_cost_report.py
import boto3
import json
import requests
from datetime import datetime, timedelta
from google.cloud import billing_v1
from azure.identity import DefaultAzureCredential
from azure.mgmt.consumption import ConsumptionManagementClient
class MultiCloudCostReporter:
def __init__(self, config):
self.config = config
self.aws_client = boto3.client('ce', region_name='us-east-1') if config.get('aws_enabled') else None
self.azure_client = ConsumptionManagementClient(
DefaultAzureCredential(),
config.get('azure_subscription_id')
) if config.get('azure_enabled') else None
self.gcp_client = billing_v1.CloudBillingClient() if config.get('gcp_enabled') else None
def get_aws_costs(self, start_date, end_date):
"""Get AWS costs for the specified period"""
if not self.aws_client:
return {"provider": "aws", "total_cost": 0, "services": []}
try:
response = self.aws_client.get_cost_and_usage(
TimePeriod={
'Start': start_date.strftime('%Y-%m-%d'),
'End': end_date.strftime('%Y-%m-%d')
},
Granularity='MONTHLY',
Metrics=['BlendedCost'],
GroupBy=[
{'Type': 'DIMENSION', 'Key': 'SERVICE'},
]
)
total_cost = 0
services = []
for result in response['ResultsByTime']:
for group in result['Groups']:
service_name = group['Keys'][0]
cost = float(group['Metrics']['BlendedCost']['Amount'])
total_cost += cost
services.append({
'service': service_name,
'cost': cost
})
return {
"provider": "aws",
"total_cost": total_cost,
"services": services
}
except Exception as e:
print(f"Error getting AWS costs: {e}")
return {"provider": "aws", "total_cost": 0, "services": []}
def get_azure_costs(self, start_date, end_date):
"""Get Azure costs for the specified period"""
if not self.azure_client:
return {"provider": "azure", "total_cost": 0, "services": []}
try:
# Azure consumption API call would go here
# This is a simplified example
return {
"provider": "azure",
"total_cost": 0, # Placeholder
"services": []
}
except Exception as e:
print(f"Error getting Azure costs: {e}")
return {"provider": "azure", "total_cost": 0, "services": []}
def get_gcp_costs(self, start_date, end_date):
"""Get GCP costs for the specified period"""
if not self.gcp_client:
return {"provider": "gcp", "total_cost": 0, "services": []}
try:
# GCP billing API call would go here
# This is a simplified example
return {
"provider": "gcp",
"total_cost": 0, # Placeholder
"services": []
}
except Exception as e:
print(f"Error getting GCP costs: {e}")
return {"provider": "gcp", "total_cost": 0, "services": []}
def generate_unified_report(self, days_back=30):
"""Generate a unified cost report across all providers"""
end_date = datetime.now()
start_date = end_date - timedelta(days=days_back)
# Get costs from all providers
aws_costs = self.get_aws_costs(start_date, end_date)
azure_costs = self.get_azure_costs(start_date, end_date)
gcp_costs = self.get_gcp_costs(start_date, end_date)
# Combine results
total_cost = aws_costs['total_cost'] + azure_costs['total_cost'] + gcp_costs['total_cost']
report = {
"report_date": datetime.now().isoformat(),
"period": {
"start": start_date.isoformat(),
"end": end_date.isoformat()
},
"total_cost": total_cost,
"providers": [aws_costs, azure_costs, gcp_costs],
"cost_breakdown": {
"aws_percentage": (aws_costs['total_cost'] / total_cost * 100) if total_cost > 0 else 0,
"azure_percentage": (azure_costs['total_cost'] / total_cost * 100) if total_cost > 0 else 0,
"gcp_percentage": (gcp_costs['total_cost'] / total_cost * 100) if total_cost > 0 else 0
}
}
return report
def save_report(self, report, filename=None):
"""Save the cost report to a file"""
if not filename:
filename = f"multi_cloud_cost_report_{datetime.now().strftime('%Y%m%d')}.json"
with open(filename, 'w') as f:
json.dump(report, f, indent=2)
print(f"Cost report saved to {filename}")
return filename
# Usage example
if __name__ == "__main__":
config = {
'aws_enabled': True,
'azure_enabled': True,
'gcp_enabled': True,
'azure_subscription_id': 'your-subscription-id',
'gcp_project_id': 'your-project-id'
}
reporter = MultiCloudCostReporter(config)
report = reporter.generate_unified_report(30)
reporter.save_report(report)
print(f"Total multi-cloud cost: ${report['total_cost']:.2f}")
for provider in report['providers']:
print(f"{provider['provider'].upper()}: ${provider['total_cost']:.2f}")
Compliance and Audit Framework
Implement unified compliance monitoring:
# Compliance monitoring module
module "compliance_monitoring" {
source = "./modules/compliance-monitoring"
providers_config = var.providers_config
project_name = var.project_name
# Compliance requirements
compliance_frameworks = [
"SOC2",
"ISO27001",
"GDPR",
"HIPAA"
]
# Audit requirements
audit_config = {
log_retention_days = 2555 # 7 years
enable_encryption = true
enable_monitoring = true
}
# Notification settings
compliance_alerts = {
email_addresses = var.compliance_notification_emails
slack_webhook = var.compliance_slack_webhook
}
}
# AWS compliance resources
resource "aws_config_configuration_recorder" "compliance" {
count = var.providers_config.aws_enabled ? 1 : 0
name = "${var.project_name}-compliance-recorder"
role_arn = aws_iam_role.config_role[0].arn
recording_group {
all_supported = true
include_global_resource_types = true
}
}
resource "aws_config_config_rule" "required_tags" {
count = var.providers_config.aws_enabled ? 1 : 0
name = "${var.project_name}-required-tags"
source {
owner = "AWS"
source_identifier = "REQUIRED_TAGS"
}
input_parameters = jsonencode({
tag1Key = "Environment"
tag2Key = "Project"
tag3Key = "Owner"
tag4Key = "CostCenter"
})
depends_on = [aws_config_configuration_recorder.compliance]
}
# Azure compliance resources
resource "azurerm_policy_assignment" "required_tags" {
count = var.providers_config.azure_enabled ? 1 : 0
name = "${var.project_name}-required-tags"
scope = azurerm_resource_group.main[0].id
policy_definition_id = "/providers/Microsoft.Authorization/policyDefinitions/1e30110a-5ceb-460c-a204-c1c3969c6d62"
parameters = jsonencode({
tagName = {
value = "Environment"
}
})
}
# GCP compliance resources
resource "google_project_organization_policy" "require_labels" {
count = var.providers_config.gcp_enabled ? 1 : 0
project = var.gcp_project_id
constraint = "constraints/gcp.resourceLocations"
list_policy {
allow {
values = var.governance_policies.allowed_regions.gcp
}
}
}
Automated Governance Enforcement
Implement automated policy enforcement:
# .github/workflows/governance-check.yml
name: Multi-Cloud Governance Check
on:
pull_request:
paths: ['infrastructure/**']
jobs:
governance-validation:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Setup Terraform
uses: hashicorp/setup-terraform@v3
with:
terraform_version: 1.6.0
- name: Setup OPA
uses: open-policy-agent/setup-opa@v2
- name: Generate Terraform Plans
run: |
find infrastructure -name "*.tf" -exec dirname {} \; | sort -u | while read dir; do
cd "$dir"
terraform init -backend=false
terraform plan -out=plan.tfplan
terraform show -json plan.tfplan > plan.json
cd - > /dev/null
done
- name: Run Multi-Cloud Policy Checks
run: |
# Check AWS resources
find infrastructure -name "plan.json" | while read plan; do
echo "Checking AWS policies for $plan"
opa eval -d policies/aws/ -i "$plan" "data.aws.deny[x]"
done
# Check Azure resources
find infrastructure -name "plan.json" | while read plan; do
echo "Checking Azure policies for $plan"
opa eval -d policies/azure/ -i "$plan" "data.azure.deny[x]"
done
# Check GCP resources
find infrastructure -name "plan.json" | while read plan; do
echo "Checking GCP policies for $plan"
opa eval -d policies/gcp/ -i "$plan" "data.gcp.deny[x]"
done
- name: Cost Impact Analysis
run: |
python3 scripts/cost_impact_analysis.py \
--terraform-plans "infrastructure/*/plan.json" \
--budget-limit ${{ vars.MONTHLY_BUDGET_LIMIT }} \
--output cost-impact.json
- name: Generate Governance Report
run: |
python3 scripts/governance_report.py \
--terraform-plans "infrastructure/*/plan.json" \
--policies policies/ \
--output governance-report.md
- name: Comment on PR
uses: actions/github-script@v7
with:
script: |
const fs = require('fs');
const report = fs.readFileSync('governance-report.md', 'utf8');
github.rest.issues.createComment({
issue_number: context.issue.number,
owner: context.repo.owner,
repo: context.repo.repo,
body: `## Multi-Cloud Governance Report\n\n${report}`
});
Conclusion
Effective multi-cloud governance and cost management require unified policies, consistent monitoring, and automated enforcement across all cloud providers. The patterns covered in this guide provide a framework for implementing governance that scales across AWS, Azure, and Google Cloud while maintaining cost control and compliance requirements.
The key to successful multi-cloud governance is treating it as a unified system rather than managing each provider separately. Consistent tagging, unified cost reporting, and automated policy enforcement ensure that your multi-cloud infrastructure remains manageable, compliant, and cost-effective as it scales.
Remember that governance is an ongoing process that requires regular review and adjustment as your multi-cloud architecture evolves and as cloud providers introduce new services and pricing models.