By default Terraform writes terraform.tfstate next to your code. That's fine for a homelab experiment and wrong for anything shared: no locking, no history, secrets on a laptop, and nobody else can run a plan. A backend stores state somewhere shared and locks it.
Backend configuration¶
Backends are configured in the terraform block of the root module only:
terraform {
backend "azurerm" {
resource_group_name = "rg-tfstate-prod"
storage_account_name = "sttfstateprod001"
container_name = "tfstate"
key = "prod/canadacentral/network.tfstate"
use_azuread_auth = true
}
}
The backend block is evaluated before variables exist, so in Terraform it accepts only literals. (OpenTofu 1.8+ allows variables and locals here.) That limitation is why partial configuration and tools like Terragrunt exist.
Azure (azurerm)¶
terraform {
backend "azurerm" {
resource_group_name = "rg-tfstate-prod"
storage_account_name = "sttfstateprod001"
container_name = "tfstate"
key = "prod/canadacentral/network.tfstate"
use_azuread_auth = true # Entra ID auth to the blob, not account keys
# In CI with workload identity federation:
# use_oidc = true
# client_id / tenant_id / subscription_id from ARM_* env vars
}
}
Storage account hardening:
shared_access_key_enabled = falseso only Entra ID auth works; grant Storage Blob Data Contributor on the container to the pipeline identity for that environment only.- Blob versioning and soft delete (containers and blobs) on.
- Private endpoint or firewall restricted to your runners;
public_network_access_enabled = falseif your agents are inside the VNet. - A
CanNotDeleteresource lock on the storage account. - One storage account per environment (in that environment's subscription), so the dev pipeline identity physically can't read prod state.
Locking is automatic via blob leases.
AWS (s3)¶
terraform {
backend "s3" {
bucket = "acme-tfstate-prod"
key = "canadacentral/network.tfstate"
region = "ca-central-1"
encrypt = true
kms_key_id = "alias/tfstate-prod"
use_lockfile = true # S3-native locking; replaces dynamodb_table
}
}
Bucket: versioning on, public access blocked, SSE-KMS, a bucket policy that denies non-TLS, and lifecycle rules to expire old versions after a sensible window. Terraform 1.15+ can authenticate the S3 backend through aws login.
Google Cloud (gcs)¶
terraform {
backend "gcs" {
bucket = "acme-tfstate-prod"
prefix = "canadacentral/network"
}
}
Object versioning on, uniform bucket-level access, CMEK if your policy requires it.
GitLab-managed state (http)¶
GitLab projects can store Terraform state, with locking, scoped to the project's permissions. Convenient if your pipelines already run in GitLab and you'd rather not run a storage account per environment:
terraform {
backend "http" {}
}
# Partial config at init time (GitLab's terraform images set these for you)
PROJECT_ID=1234
STATE_NAME=prod-network
GITLAB=https://gitlab.example.com/api/v4/projects/${PROJECT_ID}/terraform/state/${STATE_NAME}
terraform init \
-backend-config="address=${GITLAB}" \
-backend-config="lock_address=${GITLAB}/lock" \
-backend-config="unlock_address=${GITLAB}/lock" \
-backend-config="username=gitlab-ci-token" \
-backend-config="password=${CI_JOB_TOKEN}" \
-backend-config="lock_method=POST" \
-backend-config="unlock_method=DELETE" \
-backend-config="retry_wait_min=5"
Trade-off: anyone with Maintainer on the project can read every state in it. Use separate projects for environments that need separate access.
HCP Terraform (cloud block)¶
terraform {
cloud {
organization = "acme"
workspaces {
tags = ["app:billing", "env:prod"]
}
}
}
The cloud block is more than a backend: runs can execute remotely, with variables, policies and run history in HCP Terraform.
Other backends¶
consul, pg (PostgreSQL), kubernetes (a Secret, size-limited), oss (Alibaba), cos (Tencent), oci (OCI Object Storage, Terraform 1.12+). Use them if that's where your platform already is; otherwise stick to object storage.
Partial configuration: one root, many environments¶
Leave environment-specific values out of the block and supply them at init:
# backend.tf
terraform {
backend "azurerm" {
container_name = "tfstate"
use_azuread_auth = true
}
}
# env/prod.backend.hcl
resource_group_name = "rg-tfstate-prod"
storage_account_name = "sttfstateprod001"
key = "canadacentral/network.tfstate"
terraform init -reconfigure -backend-config=env/prod.backend.hcl
terraform plan -var-file=env/prod.tfvars
The risk is pairing the wrong backend file with the wrong var file. Wrap it so the environment is one argument:
#!/usr/bin/env bash
# tf.sh <env> <terraform args...>
set -euo pipefail
env="$1"; shift
terraform init -reconfigure -input=false -backend-config="env/${env}.backend.hcl" >/dev/null
terraform "$@" -var-file="env/${env}.tfvars"
…or use directory-per-environment, or Terragrunt, which generates the backend from the folder path. See multi-environment configuration.
Bootstrapping the backend¶
The storage that holds state has to exist before any backend can use it. The classic chicken-and-egg:
- Write a tiny
bootstrap/root module that creates the resource group, storage account, container, RBAC and lock, using local state. - Apply it.
- Add a backend block pointing at what it just created and run
terraform init -migrate-state. The bootstrap stack's own state moves into its own storage. - Never let that stack's state be the only copy anyone has. Export a backup.
Alternatively create it once with the az CLI or your own tooling, documented in a runbook. For S3/GCS, Terragrunt can bootstrap buckets for you (terragrunt backend bootstrap); for azurerm it can't.
Migrating state between backends¶
# 1. Change the backend block (or -backend-config)
# 2. Terraform detects the change and offers to copy state
terraform init -migrate-state
# Renaming keys or moving to a new storage account: same command.
# Non-interactive (CI): add -force-copy
-reconfigure is different: it ignores existing state and configures the new backend from scratch. Use it when switching between environments' partial configs, not when moving state.
After migrating, run terraform plan and expect no changes. Then delete the old state, after confirming the new one has a version history.
Passing data between stacks¶
When you split state into layers (architecture), upper layers need values from lower ones.
Option 1: Data sources by naming convention (preferred)¶
data "azurerm_subnet" "aks" {
name = "snet-aks"
virtual_network_name = "vnet-${var.env}-cc"
resource_group_name = "rg-network-${var.env}-cc"
}
Reads the real world, needs only read access to the resource (not to another state), and works no matter how the producer was built. Requires predictable names, which is a good thing to have anyway. Package the lookups in a data-only module so the convention lives in one place.
Option 2: terraform_remote_state¶
data "terraform_remote_state" "network" {
backend = "azurerm"
config = {
resource_group_name = "rg-tfstate-${var.env}"
storage_account_name = "sttfstate${var.env}001"
container_name = "tfstate"
key = "canadacentral/network.tfstate"
use_azuread_auth = true
}
}
resource "azurerm_kubernetes_cluster" "this" {
# ...
default_node_pool {
vnet_subnet_id = data.terraform_remote_state.network.outputs.subnet_ids["aks"]
# ...
}
}
Only root module outputs are readable. Costs:
- The consumer needs read access to the whole producer state, secrets included.
- Producer output names become a contract across repos.
- No ordering: nothing stops you applying the consumer before the producer.
Use it within a trust boundary (same team, same environment), and treat the producer's outputs as a versioned API.
Option 3: A config store¶
The producer writes values to Azure App Configuration, Key Vault, SSM Parameter Store or Consul; consumers read them with a data source. Explicit contract, per-key access control, at the cost of another system.
Option 4: An orchestrator¶
Terragrunt dependency blocks read outputs and order runs (Terragrunt). HCP Terraform has run triggers and Stacks.
Checklist¶
- [ ] Remote backend with locking for every shared stack
- [ ] Versioning and soft delete on state storage; restore tested
- [ ] Separate state storage and identity per environment
- [ ] Entra ID / IAM auth to state, no shared keys
- [ ] State keys derived from directory paths
- [ ] Plan artifacts treated as secrets
- [ ] Backend bootstrap documented and reproducible