Multi-Provider Setup

Managing infrastructure across multiple cloud providers requires careful planning of provider configurations, authentication strategies, and resource organization. Each cloud provider has different authentication mechanisms, regional structures, and service offerings that need to be coordinated in a unified Terraform configuration.

This part covers the foundational patterns for multi-cloud Terraform configurations, from basic provider setup to advanced authentication and resource management strategies.

Multi-Provider Configuration

A typical multi-cloud setup involves configuring multiple providers with appropriate aliases and authentication:

terraform {
  required_version = ">= 1.6"
  
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.20"
    }
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.70"
    }
    google = {
      source  = "hashicorp/google"
      version = "~> 4.80"
    }
  }
}

# AWS Provider Configuration
provider "aws" {
  region = var.aws_region
  alias  = "primary"
  
  default_tags {
    tags = local.common_tags
  }
}

provider "aws" {
  region = var.aws_secondary_region
  alias  = "secondary"
  
  default_tags {
    tags = local.common_tags
  }
}

# Azure Provider Configuration
provider "azurerm" {
  features {}
  
  subscription_id = var.azure_subscription_id
  tenant_id       = var.azure_tenant_id
  
  # Use managed identity when running in Azure
  use_msi = var.use_azure_msi
}

# Google Cloud Provider Configuration
provider "google" {
  project = var.gcp_project_id
  region  = var.gcp_region
  
  # Use service account key or application default credentials
  credentials = var.gcp_credentials_file
}

provider "google" {
  project = var.gcp_project_id
  region  = var.gcp_secondary_region
  alias   = "secondary"
  
  credentials = var.gcp_credentials_file
}

Authentication Strategies

Different providers require different authentication approaches:

AWS Authentication:

# Method 1: IAM Roles (recommended for production)
provider "aws" {
  region = "us-west-2"
  
  assume_role {
    role_arn = "arn:aws:iam::123456789012:role/TerraformRole"
  }
}

# Method 2: Environment variables
# AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_SESSION_TOKEN

# Method 3: AWS CLI profiles
# AWS_PROFILE=production terraform apply

Azure Authentication:

# Method 1: Service Principal
provider "azurerm" {
  features {}
  
  subscription_id = var.azure_subscription_id
  client_id       = var.azure_client_id
  client_secret   = var.azure_client_secret
  tenant_id       = var.azure_tenant_id
}

# Method 2: Managed Identity (when running in Azure)
provider "azurerm" {
  features {}
  use_msi = true
}

# Method 3: Azure CLI authentication
# az login && terraform apply

Google Cloud Authentication:

# Method 1: Service Account Key
provider "google" {
  project     = var.gcp_project_id
  region      = var.gcp_region
  credentials = file("path/to/service-account-key.json")
}

# Method 2: Application Default Credentials
provider "google" {
  project = var.gcp_project_id
  region  = var.gcp_region
  # Uses gcloud application-default login
}

# Method 3: Workload Identity (when running in GKE)
provider "google" {
  project = var.gcp_project_id
  region  = var.gcp_region
  # Automatically uses workload identity
}

Environment-Specific Provider Configuration

Different environments often require different provider configurations:

# variables.tf
variable "environment" {
  description = "Environment name"
  type        = string
}

variable "cloud_providers" {
  description = "Cloud providers to use by environment"
  type = map(object({
    aws_enabled   = bool
    azure_enabled = bool
    gcp_enabled   = bool
    aws_region    = string
    azure_region  = string
    gcp_region    = string
  }))
  
  default = {
    dev = {
      aws_enabled   = true
      azure_enabled = false
      gcp_enabled   = false
      aws_region    = "us-west-2"
      azure_region  = "West US 2"
      gcp_region    = "us-west1"
    }
    staging = {
      aws_enabled   = true
      azure_enabled = true
      gcp_enabled   = false
      aws_region    = "us-west-2"
      azure_region  = "West US 2"
      gcp_region    = "us-west1"
    }
    production = {
      aws_enabled   = true
      azure_enabled = true
      gcp_enabled   = true
      aws_region    = "us-west-2"
      azure_region  = "West US 2"
      gcp_region    = "us-west1"
    }
  }
}

# main.tf
locals {
  config = var.cloud_providers[var.environment]
  
  common_tags = {
    Environment = var.environment
    ManagedBy   = "terraform"
    Project     = var.project_name
  }
}

# Conditional provider configuration
provider "aws" {
  count  = local.config.aws_enabled ? 1 : 0
  region = local.config.aws_region
  
  default_tags {
    tags = local.common_tags
  }
}

provider "azurerm" {
  count = local.config.azure_enabled ? 1 : 0
  features {}
}

provider "google" {
  count   = local.config.gcp_enabled ? 1 : 0
  project = var.gcp_project_id
  region  = local.config.gcp_region
}

Resource Organization Patterns

Organize multi-cloud resources for maintainability:

# AWS Resources
resource "aws_vpc" "main" {
  count      = local.config.aws_enabled ? 1 : 0
  provider   = aws
  cidr_block = "10.0.0.0/16"
  
  tags = merge(local.common_tags, {
    Name     = "${var.project_name}-aws-vpc"
    Provider = "aws"
  })
}

# Azure Resources
resource "azurerm_resource_group" "main" {
  count    = local.config.azure_enabled ? 1 : 0
  provider = azurerm
  name     = "${var.project_name}-rg"
  location = local.config.azure_region
  
  tags = merge(local.common_tags, {
    Provider = "azure"
  })
}

resource "azurerm_virtual_network" "main" {
  count               = local.config.azure_enabled ? 1 : 0
  provider            = azurerm
  name                = "${var.project_name}-vnet"
  address_space       = ["10.1.0.0/16"]
  location            = azurerm_resource_group.main[0].location
  resource_group_name = azurerm_resource_group.main[0].name
  
  tags = merge(local.common_tags, {
    Provider = "azure"
  })
}

# Google Cloud Resources
resource "google_compute_network" "main" {
  count                   = local.config.gcp_enabled ? 1 : 0
  provider                = google
  name                    = "${var.project_name}-vpc"
  auto_create_subnetworks = false
  
  labels = {
    environment = var.environment
    managed_by  = "terraform"
    project     = var.project_name
    provider    = "gcp"
  }
}

Cross-Provider Data Sharing

Share data between providers using outputs and data sources:

# outputs.tf
output "network_info" {
  description = "Network information across all providers"
  value = {
    aws = local.config.aws_enabled ? {
      vpc_id         = aws_vpc.main[0].id
      vpc_cidr_block = aws_vpc.main[0].cidr_block
      region         = local.config.aws_region
    } : null
    
    azure = local.config.azure_enabled ? {
      vnet_id           = azurerm_virtual_network.main[0].id
      vnet_address_space = azurerm_virtual_network.main[0].address_space
      resource_group    = azurerm_resource_group.main[0].name
      region           = local.config.azure_region
    } : null
    
    gcp = local.config.gcp_enabled ? {
      network_id   = google_compute_network.main[0].id
      network_name = google_compute_network.main[0].name
      region       = local.config.gcp_region
    } : null
  }
}

# Use in other configurations
data "terraform_remote_state" "network" {
  backend = "s3"
  config = {
    bucket = "company-terraform-state"
    key    = "network/terraform.tfstate"
    region = "us-west-2"
  }
}

locals {
  aws_vpc_id   = data.terraform_remote_state.network.outputs.network_info.aws.vpc_id
  azure_vnet_id = data.terraform_remote_state.network.outputs.network_info.azure.vnet_id
  gcp_network_id = data.terraform_remote_state.network.outputs.network_info.gcp.network_id
}

Provider Version Management

Pin provider versions for consistency across clouds:

terraform {
  required_version = ">= 1.6"
  
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.20"
    }
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "~> 3.70"
    }
    google = {
      source  = "hashicorp/google"
      version = "~> 4.80"
    }
    random = {
      source  = "hashicorp/random"
      version = "~> 3.4"
    }
    tls = {
      source  = "hashicorp/tls"
      version = "~> 4.0"
    }
  }
}

# Lock file ensures consistent provider versions
# terraform.lock.hcl is automatically generated

Multi-Cloud Module Structure

Organize modules for multi-cloud scenarios:

modules/
├── multi-cloud-network/
│   ├── main.tf
│   ├── variables.tf
│   ├── outputs.tf
│   ├── aws.tf
│   ├── azure.tf
│   └── gcp.tf
├── cloud-agnostic-database/
│   ├── main.tf
│   ├── variables.tf
│   ├── outputs.tf
│   ├── aws-rds.tf
│   ├── azure-sql.tf
│   └── gcp-sql.tf
└── monitoring/
    ├── main.tf
    ├── variables.tf
    ├── outputs.tf
    ├── aws-cloudwatch.tf
    ├── azure-monitor.tf
    └── gcp-monitoring.tf

Multi-cloud network module example:

# modules/multi-cloud-network/main.tf
variable "providers_config" {
  description = "Configuration for each cloud provider"
  type = object({
    aws_enabled   = bool
    azure_enabled = bool
    gcp_enabled   = bool
  })
}

variable "network_cidrs" {
  description = "CIDR blocks for each provider"
  type = object({
    aws   = string
    azure = string
    gcp   = string
  })
  
  default = {
    aws   = "10.0.0.0/16"
    azure = "10.1.0.0/16"
    gcp   = "10.2.0.0/16"
  }
}

# AWS networking resources
resource "aws_vpc" "main" {
  count      = var.providers_config.aws_enabled ? 1 : 0
  cidr_block = var.network_cidrs.aws
  
  enable_dns_hostnames = true
  enable_dns_support   = true
  
  tags = {
    Name     = "${var.name_prefix}-aws-vpc"
    Provider = "aws"
  }
}

# Azure networking resources
resource "azurerm_virtual_network" "main" {
  count               = var.providers_config.azure_enabled ? 1 : 0
  name                = "${var.name_prefix}-vnet"
  address_space       = [var.network_cidrs.azure]
  location            = var.azure_location
  resource_group_name = var.azure_resource_group_name
  
  tags = {
    Provider = "azure"
  }
}

# GCP networking resources
resource "google_compute_network" "main" {
  count                   = var.providers_config.gcp_enabled ? 1 : 0
  name                    = "${var.name_prefix}-vpc"
  auto_create_subnetworks = false
  
  labels = {
    provider = "gcp"
  }
}

Error Handling and Debugging

Multi-cloud configurations can be complex to debug:

# Enable detailed logging
export TF_LOG=DEBUG
export TF_LOG_PATH=terraform.log

# Test provider authentication
terraform providers

# Validate configuration
terraform validate

# Plan with specific providers
terraform plan -target="aws_vpc.main"
terraform plan -target="azurerm_virtual_network.main"
terraform plan -target="google_compute_network.main"

# Check provider plugin cache
ls -la .terraform/providers/

Provider-specific debugging:

# AWS debugging
aws sts get-caller-identity
aws configure list

# Azure debugging
az account show
az account list

# GCP debugging
gcloud auth list
gcloud config list
gcloud projects list

What’s Next

Multi-provider setup provides the foundation for multi-cloud infrastructure, but the real challenges emerge when you need to connect networks across different cloud providers. Cross-cloud networking requires understanding each provider’s networking model and implementing secure, performant connections.

In the next part, we’ll explore cross-cloud networking patterns, including VPN connections, private peering, and hybrid connectivity solutions that enable seamless communication across AWS, Azure, and Google Cloud.

Continue Your Learning

This is part 1 of 7 in the comprehensive guide.

Guide Overview See all 7 parts Next → Cross-Cloud Networking