Terraform State Management: A Deep Dive Beyond the Basics for Azure Deployments

Without state management, Terraform is as good as normal CLI commands... so think about it now !

Normally, while developing Terraform scripts, state management is the last topic on most developers' minds. Only after development is complete does the thinking process related to state management start. This is not an optimal practice. State management is extremely crucial for the success of infrastructure as code using Terraform.

State management is important because the state file contains all the sensitive information in plain text format. It cannot be stored as part of the code in any version control system. It should be protected and secured heavily with all authentication, authorization, confidentiality and integrity.

State should also be stored centrally if multiple developers and operators are involved in managing the environment provisioned through Terraform. Having the Terraform state in a single developer’s machine will not let others manage the environment.

Terraform also uses this state to check which resources need to be added, removed or updated. There are more secondary reasons why Terraform state is important however, it is equally important to have a strategy to manage Terraform state in a way that allows

• Collaboration between teams and developers
• Ability for Terraform to access and manage the resources
• Treat it as a highly secure and confidential resource containing sensitive information to be protected at any cost.

Let’s get into the details of state management in Terraform with Azure Cloud

Strategy: Hierarchical State Organization

Most developers start with a simple Azure Storage backend configuration:

terraform {
  backend "azurerm" {
    resource_group_name  = "terraform-state-rg"
    storage_account_name = "tfstate0123456789"
    container_name       = "tfstate"
    key                  = "prod.terraform.tfstate"
  }
}

This strategy can be implemented with dynamic backend configuration:

locals {
  environment = terraform.workspace
  region      = "eastus2"
  component   = "networking"
}

terraform {
  backend "azurerm" {
    resource_group_name  = "terraform-state-rg"
    storage_account_name = "tfstate0123456789"
    container_name       = "tfstate"
    key                  = "${local.environment}/${local.region}/${local.component}/terraform.tfstate"
  }
}

This hierarchical approach provides several benefits:

• Logical organization that maps to your Azure architecture
• Clearer separation of concerns
• Easier disaster recovery
• Better alignment with team structures

Strategy: Multi-Workspace State Isolation for Development Teams

For large engineering teams, workspace-based isolation provides better control:

# Create development workspace for Team A
terraform workspace new team_a_dev

# Create development workspace for Team B
terraform workspace new team_b_dev

# List available workspaces
terraform workspace list

The workspaces can be combined with Azure RBAC for granular access control:

# Create a custom role for Team A state access
az role definition create --role-definition '{
    "Name": "Team A Terraform State Access",
    "Description": "Can read and write Team A terraform state",
    "Actions": [
        "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read",
        "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/write",
        "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/lease/action"
    ],
    "DataActions": [
        "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read",
        "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/write",
        "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/lease/action"
    ],
    "AssignableScopes": [
        "/subscriptions/{subscription-id}/resourceGroups/terraform-state-rg"
    ]
}'

# Assign the role to Team A
az role assignment create \
  --role "Team A Terraform State Access" \
  --assignee-object-id {team-a-group-id} \
  --scope "/subscriptions/{subscription-id}/resourceGroups/terraform-state-rg/providers/Microsoft.Storage/storageAccounts/tfstate0123456789/blobServices/default/containers/tfstate/blobs/team_a_dev"

Strategy: State Encryption

While Azure Storage provides encryption at rest, sensitive environments might require additional protection:

# Generate a customer-managed key in Azure Key Vault
resource "azurerm_key_vault_key" "terraform_state_key" {
  name         = "terraform-state-encryption-key"
  key_vault_id = azurerm_key_vault.terraform_kv.id
  key_type     = "RSA"
  key_size     = 2048
  key_opts     = ["decrypt", "encrypt", "sign", "verify"]
}

# Configure the storage account to use this key
resource "azurerm_storage_account_customer_managed_key" "terraform_state" {
  storage_account_id = azurerm_storage_account.terraform_state.id
  key_vault_id       = azurerm_key_vault.terraform_kv.id
  key_name           = azurerm_key_vault_key.terraform_state_key.name
}

For even more sensitive environments, we can implement client-side encryption as well:

# pre-commit hook example
import json
import os
from cryptography.fernet import Fernet

# Encrypt state before it's pushed to remote
def encrypt_state():
    # Read encryption key from secure location
    with open('.terraform-key', 'rb') as key_file:
        key = key_file.read()
    
    fernet = Fernet(key)
    
    # Read the state file
    with open('.terraform/terraform.tfstate', 'rb') as state_file:
        state_data = state_file.read()
    
    # Encrypt the state
    encrypted_state = fernet.encrypt(state_data)
    
    # Write encrypted state
    with open('.terraform/terraform.tfstate.encrypted', 'wb') as encrypted_file:
        encrypted_file.write(encrypted_state)
    
    print("State encrypted successfully")

if __name__ == "__main__":
    encrypt_state()

Conclusion

Effective state management is the foundation of successful Terraform deployments in Azure. The strategies outlined here move beyond basic storage configuration to address enterprise concerns:

• Security: Multi-layered protection for sensitive state data

• Governance: Controlled access and monitoring across teams

• Resilience: Backup, versioning, and disaster recovery

• Scalability: Patterns for growing beyond a single team or application

  
  

By implementing these advanced techniques, you can build a state management approach that supports even the most complex Azure deployments while maintaining security and operational efficiency.

Remember that state files are the single source of truth for your infrastructure – treat them with the same care you would apply to your most critical database or application code.