How to Use Terraform to Configure AWS Verified Access With an OIDC Google Provider

In this blog post, I'll walk you through AWS Verified Access and Terraform.

We will configure an AWS Verified Access endpoint for our private Application Load Balancer. We'll also discuss other private services that can be exposed through Verified Access, such as Network Load Balancers or ENI.

We'll use Google AuthN as an OIDC (OpenID Connect) user trust provider instead of IAM identity center because some users don't have access to it or can't change their IAM identity center permissions. In addition, after configuring the Verified Access endpoint, we will configure AuthZ permissions using the Cedar configuration language.

By default, I assume that you have already has any private EC2/ECS/EKS service (whether hidden via private ALB or not) to which you wish to gain access without using a VPN. We will not go over the deployment process of such a service in this article. In addition, I assume that you are familiar with AWS Certificate Manager and know how to use it to issue and store TLS certificates.

Don't worry if you're new to Terraform or AWS Verified Access; I'll walk you through everything step by step, including all the code. Prepare for an exciting learning adventure!

Prerequisites

• Basic understanding of AWS, VPC, and Terraform

• An AWS account with necessary permissions to create VPC, subnets, verified access etc.

• Configured aws cli to point to your AWS account

• Configured Terraform / OpenTofu

• Configured Google Cloud account

• At least one EC2/ECS/EKS private service with Application/Network load balancer or ENI interface.

What is AWS Verified Access?

AWS Verified Access is a service that can provide secure access to your applications without requiring the use of a virtual private network (VPN). Verified Access evaluates each application request and helps ensure that users can access each application only when they meet the specified security requirements.

Setup Google OAuth application

For OIDC AuthN, AWS trust provider requires a set of configuration options like Client ID or Client Secret. Unfortunately, AWS does not provide a guide for configuring Google OAuth and getting these values. So, let me walk you through it step by step.

Setup OAuth consent screen

1. Go to Google Cloud console → API & Services → OAuth consent screen

2. Choose the type of application.

   1. Internal: if you only want to use the OAuth consent screen for users within your Google organisation.
   2. External: if you want to use the OAuth consent screen for any Google users.

3. Click to “Create” button

4. Fill “App name“, “User support email“, “Developer contact information” and “Authorised domains” fields

5. All subsequent screens can be skipped; after all of them, click save.

Create OIDC credentials

1. Go to Google cloud console → API & Services → Credentials
2. Click to “CREATE CREDENTIALS” → “OAuth client ID“
3. Application type should be “Web Application“
4. Fill out the form as shown in the screenshot; In my case, I'll use the domain name kvendingoldo.dev.referrs.me as the AWS Verified Access endpoint name. Also, you need to add it to the Authorised JavaScript origins and Authorised redirect URIs sections.

5. You will see "OAuth client created" after clicking the save button. Client ID and Client secret should be copied. They will be used for AWS Verified Access trust provider in the future.

AWS infrastructure

Now it's time to use Terraform to create AWS Verified Access resources. First, let's take a look at our directory structure:

.
├── 01-data.tf
├── 01-meta.tf
├── 03-main.tf
└── 05-variables.tf

Terraform does not provide a rigid structure for a project by default, and all resources can be created in a single file, such as main.tf. I dislike it and recommend that you use the following layout for your resources:

Let's take a look at what's inside each of these files for our infrastructure.

01-meta.tf

provider "aws" {
  region = "us-east-2"
}

terraform {
  required_version = "1.5.7"

  required_providers {
    aws = ">= 5.0.0"
  }
}

As you may see, in this file we have two configuration blocks:

• terraform: The special configuration block type is used to configure some behaviours of Terraform itself, such as requiring a minimum Terraform version to apply your configuration.
  • It is considered best practice when working with Terraform code to specify the version of providers that you are using. By explicitly defining the version, you ensure that your code is consistently built and tested, reducing the risk of compatibility issues. If no version is specified, Terraform will retrieve the most recent version, which may result in unexpected behavior and potential compatibility issues. In my case, I set >= 5.0.0. version, which is sufficient for the example.
• aws provider block with default region us-east2.

01-data.tf

data "aws_route53_zone" "main" {
  zone_id = var.r53_zone_id
}

This file usually contains data sources. This type of resource allows Terraform to use information defined outside of Terraform, defined by another separate Terraform configuration, or modified by functions.

In the scope of your infrastructure, we'll create a Route53 record for the Verified Access endpoint, so knowing more about our Route53 DNS zone is extremely useful. As a result, we define aws_route53_zone data source, which will provide us with a wealth of information about the Route53 zone based solely on the Zone ID.

03-main.tf

locals {
  tags = merge(var.tags, {
    Name : var.svc_name
  })
}

resource "aws_verifiedaccess_instance" "main" {
  count = var.enable_va ? 1 : 0

  tags = local.tags
}

resource "aws_verifiedaccess_trust_provider" "main" {
  count = var.enable_va && var.va_trustprovider_id == null ? 1 : 0

  description = format("Verified Access Trust provider for %s app", var.svc_name)

  policy_reference_name    = replace(format("%s-%s", var.env_abbr, var.svc_name), "-", "_")
  trust_provider_type      = "user"
  user_trust_provider_type = var.va_user_trust_provider_type

  dynamic "oidc_options" {
    for_each = var.va_user_trust_provider_type == "oidc" ? [1] : []
    content {
      authorization_endpoint = var.va_authorization_endpoint
      client_id              = var.va_client_id
      client_secret          = var.va_client_secret
      issuer                 = var.va_issuer
      scope                  = var.va_scope
      token_endpoint         = var.va_token_endpoint
      user_info_endpoint     = var.va_user_info_endpoint
    }
  }

  tags = local.tags
}

resource "aws_verifiedaccess_instance_trust_provider_attachment" "main" {
  count = var.enable_va ? 1 : 0

  verifiedaccess_instance_id       = aws_verifiedaccess_instance.main[0].id
  verifiedaccess_trust_provider_id = var.va_trustprovider_id == null ? aws_verifiedaccess_trust_provider.main[0].id : var.va_trustprovider_id
}

resource "aws_verifiedaccess_group" "main" {
  count = var.enable_va ? 1 : 0

  verifiedaccess_instance_id = aws_verifiedaccess_instance.main[0].id
  policy_document            = var.va_group_policy_document

  tags = local.tags

  depends_on = [
    aws_verifiedaccess_instance_trust_provider_attachment.main[0]
  ]
}

resource "aws_verifiedaccess_endpoint" "main" {
  count = var.enable_va ? 1 : 0

  application_domain     = format("%s.%s", var.r53_va_record, data.aws_route53_zone.main.name)
  attachment_type        = "vpc"
  domain_certificate_arn = var.va_domain_cert_arn
  endpoint_domain_prefix = substr(format("%s-%s", var.env_abbr, var.svc_name), 0, 19)
  endpoint_type          = "load-balancer"
  load_balancer_options {
    load_balancer_arn = var.lb_arn
    port              = var.lb_port
    protocol          = var.lb_protocol
    subnet_ids        = var.subnet_ids
  }
  security_group_ids       = var.sg_ids
  verified_access_group_id = aws_verifiedaccess_group.main[0].id
}

resource "aws_route53_record" "va" {
  count = var.enable_va ? 1 : 0

  zone_id = var.r53_zone_id
  name    = var.r53_va_record
  type    = "CNAME"
  ttl     = 60

  records = [
    aws_verifiedaccess_endpoint.main[0].endpoint_domain
  ]
}

#
# Logging
#
resource "aws_verifiedaccess_instance_logging_configuration" "main" {
  count = var.enable_va && var.enable_va_logging ? 1 : 0

  verifiedaccess_instance_id = aws_verifiedaccess_instance.main[0].id

  access_logs {
    include_trust_context = var.va_log_include_trust_context
    log_version           = var.va_log_version


    dynamic "cloudwatch_logs" {
      for_each = var.va_log_cloudwatch_enabled ? [1] : []
      content {
        enabled   = var.va_log_cloudwatch_enabled
        log_group = var.va_log_cloudwatch_log_group_id
      }
    }

    dynamic "kinesis_data_firehose" {
      for_each = var.va_log_kinesis_enabled ? [1] : []
      content {
        enabled         = var.va_log_kinesis_enabled
        delivery_stream = var.va_log_kinesis_delivery_stream_name

      }
    }

    dynamic "s3" {
      for_each = var.va_log_s3_enabled ? [1] : []
      content {
        enabled     = var.va_log_s3_enabled
        bucket_name = var.va_log_s3_bucket_name

      }
    }
  }
}

This is the most important file because it contains all of the AWS Verified Access resources that are required. Let's go over each of them and some of the tricks you can see in the provided code snippet.

Many resources have a Terraform count based on the following rule: count = var.enable_va? 1: 0. This technique is useful when you need to quickly enable or disable a part of your Terraform infrastructure. On a large scale, you might have a lot of feature flags, like var.enable_va.

Also, when discussing the techniques demonstrated in the snippet, it is important to highlight a transformation of var.tags variable into a local variable local.tags. As you can see at the top of this file, we've merged var.tag with a tag Name for getting a new map that is stored inside of local.tags. It is useful because without it, all resources in the AWS Console UI will have an empty name.

Let's take a closer look at each AWS resource now.

aws_verifiedaccess_instance

AWS Verified Access instance is an AWS resource that assists you in organising your trust providers and Verified Access groups. Except for tags, it's a simple Terraform resource that doesn't require any additional configuration options.

aws_verifiedaccess_trust_provider

A trust provider is a AWS service that sends information about users and devices to AWS Verified Access. This information is called trust context. It can include attributes based on user identity, such as an email address or membership in the "sales" organization, or device information such as installed security patches or anti-virus software version.

Verified Access supports the following categories of trust providers:

• User identity – An identity provider (IdP) service that stores and manages digital identities for users.
• Device management – A device management system for devices such as laptops, tablets, and smartphones.

In our case, we use user identity provider (trust_provider_type = "user") together with OIDC configuration (oidc_options).

aws_verifiedaccess_instance_trust_provider_attachment

This resources attaches Verified Access Instance to Verified Access Trust Provider.

aws_verifiedaccess_group

This is a resource for managing a Verified Access Group. AWS Verified Access group is a collection of Verified Access endpoints and a group-level Verified Access policy. Each endpoint within a group shares the Verified Access policy.

You can use groups to gather endpoints that have common security requirements. This can help simplify policy administration by using one policy (policy_document option. We’ll talk about policies configuration later) for the security needs of multiple applications.

For example, you can group all sales applications together and set a group-wide access policy. You can then use this policy to define a common set of minimum security requirements for all sales applications.

When you create a group, you are required to associate the group with a Verified Access instance (verifiedaccess_instance_idoption). During the process of creating an endpoint, you will associate the endpoint with a group.

aws_verifiedaccess_endpoint

A Verified Access endpoint represents an application. Each endpoint is associated with a Verified Access group and inherits the access policy for the group. You can optionally attach an application-specific endpoint policy to each endpoint.

In this article I associate endpoint with Application Load balancer via the following block.

endpoint_type          = "load-balancer"
load_balancer_options {
  load_balancer_arn = var.lb_arn
  port              = var.lb_port
  protocol          = var.lb_protocol
  subnet_ids        = var.subnet_ids
}

It is important to highlight that the aws_verifiedaccess_endpoint has the correctly configured domain_certificate_arn option, which should point to a valid TLS certificate stored in AWS Certificate Manager. Keep in mind that the certificate must match the route53 domain record that will be created for the AWS Verified Access Endpoint.

ALB/NLB is not the only method for exposing a service via a Verified Access Endpoint. You can also expose the service outside of the private network by using any private ENI interface, such as the EC2 ENI interface, via Verified Access Endpoint.

  endpoint_type          = "network-interface"
  network_interface_options {
    network_interface_id = aws_network_interface.example.id
    port                 = 443
    protocol             = "https"
  }

aws_route53_record

This resource creates a Route53 record resource for Verified Access Endpoint. As Verified Access Endpoint has its own DNS name; the type of our record (kvendingoldo.dev.referrs.me) should be CNAME.

aws_verifiedaccess_instance_logging_configuration

This resource manages a Verified Access Logging Configuration. I strongly advise enabling this resource via var.enable_va_logging feature flag during Verified Access testing. For me, using only CloudWatch logs to find issues in Cedar rules was sufficient, but you can also configure S3 or Kinesis sources to send logs from your Verified Access Instance.

If you have any problems with your rules during testing, you will see something like this in your logs:

    "activity_id": "2",
    "activity_name": "Access Deny",
    "actor": null,
    "category_name": "Audit Activity",
    "category_uid": "3",
    "class_name": "Access Activity",
    "class_uid": "3006",
    "data": null,
    "device": null,
    "duration": "0.0",
    "end_time": "1704880589864",
    "time": "1704880589864",
    "http_request": {
        "http_method": "GET",
        "url": {
            "hostname": "3.100.226.13",
            "path": "/version",
            "port": 443,
            "scheme": "https",
            "text": "https://3.100.226.13:443/version"
        },
        "user_agent": "Mozilla/5.0 zgrab/0.x",
        "version": "HTTP/1.1"
    },
    "http_response": {
        "code": 302
    },
    "message": "",
    "metadata": {
        "uid": "Root=1-659e69cd-2832fc7d4af907443fce29dd",
        "logged_time": 1704880985929,
        "version": "1.0.0-rc.2",
        "product": {
            "name": "Verified Access",
            "vendor_name": "AWS"
        }
    },
    "ref_time": "2024-01-10T09:56:29.864879Z",
    "proxy": {
        "ip": "3.133.226.13",
        "port": 443,
        "svc_name": "Verified Access",
        "uid": "vai-0de4109b1cc38ecd4"
    },
    "severity": "Informational",
    "severity_id": "1",
    "src_endpoint": {
        "ip": "162.243.138.37",
        "port": 50388
    },
    "start_time": "1704880589864",
    "status_code": "200",
    "status_detail": "Authentication Denied",
    "status_id": "2",
    "status": "Failure",
    "type_uid": "300602",
    "type_name": "Access Activity: Access Deny",
    "unmapped": null
}

Also, I recommend creating a Cloudwatch log group using Terraform and other resources, which you can do with the following code:

resource "aws_cloudwatch_log_group" "main" {
  name              = "my_va_loggroup_for_testing"
  retention_in_days = 10
  tags              = var.tags
}

05-variables.tf

I'd like to talk about Google OIDC variables before we get into Terraform variables for our resources. I couldn't find any documentation about the correct set of these variables and for this reason, I had to get them from https://accounts.google.com/.well-known/openid-configuration.

This is our infrastructure's Terraform variables file:

#
# Common variables
#
variable "svc_name" {
  type    = string
  default = "kvendingoldo-app"
}
variable "env_abbr" {
  type    = string
  default = "dev01"
}
variable "tags" {
  default = {
    automation : "terraform"
    data_classification : "internal"
    owner : "Alexander Sharov"
  }
}

#
# Network
#
variable "subnet_ids" {
  description = "A list of subnet ids where verified access endpoint will be placed"
  type        = list(string)
}
variable "sg_ids" {
  description = "List of the the security groups IDs to associate with the Verified Access endpoint"
  type        = list(string)
}
variable "lb_arn" {
  description = "The ARN of the load balancer that will be used for the Verified Access endpoint"
  type        = string
}
variable "lb_port" {
  description = "The load balancer port that will be used for the Verified Access endpoint"
  type        = string
  default     = 80
}
variable "lb_protocol" {
  description = "The load balancer protocol that will be used for the Verified Access endpoint"
  type        = string
  default     = "http"
}

#
# Verified Access Common
#
variable "enable_va" {
  default = true
}
variable "enable_va_logging" {
  default = false
}
variable "va_domain_cert_arn" {
  default = null
}
variable "va_group_policy_document" {
  description = "The policy document that is associated with this resource"
  type        = string
  default     = null
}

#
# Verified Access Trust provider
#
variable "va_trustprovider_id" {
  default = null
}
variable "va_user_trust_provider_type" {
  description = "The type of user-based trust provider."
  type        = string
  default     = "oidc"
}

#
# Verified Access logging
#
variable "va_log_include_trust_context" {
  default = true
}
variable "va_log_version" {
  default = "ocsf-1.0.0-rc.2"
}

variable "va_log_cloudwatch_enabled" {
  default = false
}
variable "va_log_cloudwatch_log_group_id" {
  default = ""
}

variable "va_log_kinesis_enabled" {
  default = false
}
variable "va_log_kinesis_delivery_stream_name" {
  default = null
}

variable "va_log_s3_enabled" {
  default = false
}
variable "va_log_s3_bucket_name" {
  default = null
}

#
# Verified Access Trust provider OIDC Settings
#
variable "va_authorization_endpoint" {
  description = "The OIDC authorization endpoint"
  type        = string
  default     = "https://accounts.google.com/o/oauth2/v2/auth"
}
variable "va_client_id" {
  description = "The client identifier"
  type        = string
}
variable "va_client_secret" {
  description = "The client secret"
  type        = string
}
variable "va_issuer" {
  description = "The OIDC issuer"
  type        = string
  default     = "https://accounts.google.com"
}
variable "va_scope" {
  description = "OpenID Connect (OIDC) scopes are used by an application during authentication to authorize access to details of a user."
  type        = string
  default     = "openid email profile"
}
variable "va_token_endpoint" {
  description = "The OIDC token endpoint"
  type        = string
  default     = "https://oauth2.googleapis.com/token"
}
variable "va_user_info_endpoint" {
  description = "The OIDC user info endpoint"
  type        = string
  default     = "https://openidconnect.googleapis.com/v1/userinfo"
}

#
# Route53
#
variable "r53_zone_id" {
  default = ""
}
variable "r53_va_record" {
  default = "kvendingoldo"
}

Let's discuss only some of the variables that may raise questions

AuthZ policies via Cedar rules

Finally, after we have configured our infrastructure and AuthN, we can configure AuthZ policies. For such policies, AWS Verified Access Group use Cedar configuration language. E.g.:

permit(principal, action, resource)
when {
    // Returns true if the email ends in "@example.com"
    context.identity.email like "*@example.com" &&
    // Returns true if the user is part of the "finance" group
    context.identity.groups.contains("finance")
};

In my case, I'd like to restrict service access under Verified Access Endpoint to all users with the email address *@referrs.me. The rule will be as follows:

permit(principal,action,resource)
when {
    context.dev01_kvendingoldo_app.email like "*@referrs.me"

};

Pay attention to dev01_kvendingoldo_app. This variable has been configured inside of aws_verifiedaccess_trust_provider Terraform resource, policy_reference_name property:

policy_reference_name    = replace(format("%s-%s", var.env_abbr, var.svc_name), "-", "_")

To add the described rule for our infrastructure, we have to change va_group_policy_document variable:

variable "va_group_policy_document" {
  description = "The policy document that is associated with this resource"
  type        = string
  default     = <<-EOT
permit(principal,action,resource)
when {
  context.dev01_kvendingoldo_app.email like "*@referrs.me"
};
EOT
}

And then run our Terraform scripts once more. When the infrastructure is applied, it may take up to 5 minutes for the rule to start working.

Conclusion

In this blog post, I demonstrated how to use Terraform to create a Verified Access infrastructure for any private ALB/NLB or ENI interface. You can use the same infrastructure if you need a simple secure access to your service based on Google AuthN without VPN setup.

Please contact me if you have any concerns or questions. I will make every effort to respond to your questions and provide solutions.

Stay tuned for future posts where we'll talk a lot about Terraform infrastructure.

References

• AWS Verified Access documentation

• Terraform documentation, AWS Terraform provider documentation, tofuenv documentation to manage multiple OpenTofu CLI versions and Terraform Docs CLI documentation to autogenerate Terraform module documentation.

• OpenTofu documentation.

• TofuEnv repository.

• aws-letsencrypt-lambda repository.

• A walk through AWS Verified Access policies.

• Cedar configuration language.