How to Use Terraform to Configure AWS Verified Access With an OIDC Google Provider

In this blog post, I'll walk you through and Terraform. AWS Verified Access We will configure an AWS Verified Access endpoint for our private Application Load Balancer. We'll also discuss other private services that can be exposed through Verified Access, such as Network Load Balancers or ENI. We'll use Google AuthN as an OIDC (OpenID Connect) user trust provider instead of because some users don't have access to it or can't change their IAM identity center permissions. In addition, after configuring the Verified Access endpoint, we will configure AuthZ permissions using the . IAM identity center Cedar configuration language By default, I assume that you have already has any private EC2/ECS/EKS service (whether hidden via private ALB or not) to which you wish to gain access without using a VPN. We will not go over the deployment process of such a service in this article. In addition, I assume that you are familiar with AWS Certificate Manager and know how to use it to issue and store TLS certificates. Don't worry if you're new to Terraform or AWS Verified Access; I'll walk you through everything step by step, including all the code. Prepare for an exciting learning adventure! Prerequisites Basic understanding of AWS, , and VPC Terraform An with necessary permissions to create VPC, subnets, verified access etc. AWS account Configured to point to your AWS account aws cli Configured Terraform / OpenTofu Configured Google Cloud account At least one EC2/ECS/EKS private service with Application/Network load balancer or ENI interface. Terraform v1.5.7 was used in this article. It is the final Terraform version that is licensed under the Mozilla Public License v2.0 (MPL 2.0). All subsequent versions were migrated to Business Source License (BSL) v1.1. If you're building something commercial on top of Terraform, don't upgrade to the further version; instead, migrate to OpenTofu. If you need to manage multiple versions of Terraform or OpenTofu, use the / tools to switch between them quickly. tfenv tofuenv What is AWS Verified Access? is a service that can provide secure access to your applications without requiring the use of a virtual private network (VPN). Verified Access evaluates each application request and helps ensure that users can access each application only when they meet the specified security requirements. AWS Verified Access Setup Google OAuth application For OIDC AuthN, AWS trust provider requires a set of configuration options like or . Unfortunately, AWS does not provide a guide for configuring Google OAuth and getting these values. So, let me walk you through it step by step. Client ID Client Secret Setup OAuth consent screen Go to Google Cloud console → API & Services → OAuth consent screen Choose the type of application. : if you only want to use the OAuth consent screen for users within your Google organisation. Internal : if you want to use the OAuth consent screen for any Google users. External Click to “Create” button Fill “App name“, “User support email“, “Developer contact information” and “Authorised domains” fields All subsequent screens can be skipped; after all of them, click save. Create OIDC credentials Go to Google cloud console → API & Services → Credentials Click to “CREATE CREDENTIALS” → “OAuth client ID“ Application type should be “Web Application“ Fill out the form as shown in the screenshot; In my case, I'll use the domain name as the AWS Verified Access endpoint name. Also, you need to add it to the and sections. kvendingoldo.dev.referrs.me Authorised JavaScript origins Authorised redirect URIs Keep in mind that section must include the second URI: . Google AuthN will not function properly without the second URI. Autorized redirect URIs https://your_domain>/oauth2/idpresponse 5. You will see "OAuth client created" after clicking the save button. and should be copied. They will be used for AWS Verified Access trust provider in the future. Client ID Client secret AWS infrastructure Now it's time to use Terraform to create AWS Verified Access resources. First, let's take a look at our directory structure: . ├── 01-data.tf ├── 01-meta.tf ├── 03-main.tf └── 05-variables.tf Terraform does not provide a rigid structure for a project by default, and all resources can be created in a single file, such as . I dislike it and recommend that you use the following layout for your resources: main.tf File name Resources 01-data.tf Data resources 01-meta.tf Terraform configuration block and providers configuration 03-main.tf Common cloud resources that you’re going to create 03-modules.tf Terraform modules definition that you’re going to use 05-variables.tf Terraform variables 06-outputs.tf Terraform outputs Let's take a look at what's inside each of these files for our infrastructure. 01-meta.tf provider "aws" { region = "us-east-2" } terraform { required_version = "1.5.7" required_providers { aws = ">= 5.0.0" } } As you may see, in this file we have two configuration blocks: : The special type is used to configure some behaviours of Terraform itself, such as requiring a minimum Terraform version to apply your configuration. terraform configuration block It is considered best practice when working with Terraform code to specify the version of providers that you are using. By explicitly defining the version, you ensure that your code is consistently built and tested, reducing the risk of compatibility issues. If no version is specified, Terraform will retrieve the most recent version, which may result in unexpected behavior and potential compatibility issues. In my case, I set . version, which is sufficient for the example. >= 5.0.0 provider block with default region . aws us-east2 01-data.tf data "aws_route53_zone" "main" { zone_id = var.r53_zone_id } This file usually contains . This type of resource allows Terraform to use information defined outside of Terraform, defined by another separate Terraform configuration, or modified by functions. data sources In the scope of your infrastructure, we'll create a Route53 record for the Verified Access endpoint, so knowing more about our Route53 DNS zone is extremely useful. As a result, we define data source, which will provide us with a wealth of information about the Route53 zone based solely on the Zone ID. aws_route53_zone 03-main.tf locals { tags = merge(var.tags, { Name : var.svc_name }) } resource "aws_verifiedaccess_instance" "main" { count = var.enable_va ? 1 : 0 tags = local.tags } resource "aws_verifiedaccess_trust_provider" "main" { count = var.enable_va && var.va_trustprovider_id == null ? 1 : 0 description = format("Verified Access Trust provider for %s app", var.svc_name) policy_reference_name = replace(format("%s-%s", var.env_abbr, var.svc_name), "-", "_") trust_provider_type = "user" user_trust_provider_type = var.va_user_trust_provider_type dynamic "oidc_options" { for_each = var.va_user_trust_provider_type == "oidc" ? [1] : [] content { authorization_endpoint = var.va_authorization_endpoint client_id = var.va_client_id client_secret = var.va_client_secret issuer = var.va_issuer scope = var.va_scope token_endpoint = var.va_token_endpoint user_info_endpoint = var.va_user_info_endpoint } } tags = local.tags } resource "aws_verifiedaccess_instance_trust_provider_attachment" "main" { count = var.enable_va ? 1 : 0 verifiedaccess_instance_id = aws_verifiedaccess_instance.main[0].id verifiedaccess_trust_provider_id = var.va_trustprovider_id == null ? aws_verifiedaccess_trust_provider.main[0].id : var.va_trustprovider_id } resource "aws_verifiedaccess_group" "main" { count = var.enable_va ? 1 : 0 verifiedaccess_instance_id = aws_verifiedaccess_instance.main[0].id policy_document = var.va_group_policy_document tags = local.tags depends_on = [ aws_verifiedaccess_instance_trust_provider_attachment.main[0] ] } resource "aws_verifiedaccess_endpoint" "main" { count = var.enable_va ? 1 : 0 application_domain = format("%s.%s", var.r53_va_record, data.aws_route53_zone.main.name) attachment_type = "vpc" domain_certificate_arn = var.va_domain_cert_arn endpoint_domain_prefix = substr(format("%s-%s", var.env_abbr, var.svc_name), 0, 19) endpoint_type = "load-balancer" load_balancer_options { load_balancer_arn = var.lb_arn port = var.lb_port protocol = var.lb_protocol subnet_ids = var.subnet_ids } security_group_ids = var.sg_ids verified_access_group_id = aws_verifiedaccess_group.main[0].id } resource "aws_route53_record" "va" { count = var.enable_va ? 1 : 0 zone_id = var.r53_zone_id name = var.r53_va_record type = "CNAME" ttl = 60 records = [ aws_verifiedaccess_endpoint.main[0].endpoint_domain ] } # # Logging # resource "aws_verifiedaccess_instance_logging_configuration" "main" { count = var.enable_va && var.enable_va_logging ? 1 : 0 verifiedaccess_instance_id = aws_verifiedaccess_instance.main[0].id access_logs { include_trust_context = var.va_log_include_trust_context log_version = var.va_log_version dynamic "cloudwatch_logs" { for_each = var.va_log_cloudwatch_enabled ? [1] : [] content { enabled = var.va_log_cloudwatch_enabled log_group = var.va_log_cloudwatch_log_group_id } } dynamic "kinesis_data_firehose" { for_each = var.va_log_kinesis_enabled ? [1] : [] content { enabled = var.va_log_kinesis_enabled delivery_stream = var.va_log_kinesis_delivery_stream_name } } dynamic "s3" { for_each = var.va_log_s3_enabled ? [1] : [] content { enabled = var.va_log_s3_enabled bucket_name = var.va_log_s3_bucket_name } } } } This is the most important file because it contains all of the AWS Verified Access resources that are required. Let's go over each of them and some of the tricks you can see in the provided code snippet. Many resources have a Terraform based on the following rule: . This technique is useful when you need to quickly enable or disable a part of your Terraform infrastructure. On a large scale, you might have a lot of feature flags, like . count count = var.enable_va? 1: 0 var.enable_va Also, when discussing the techniques demonstrated in the snippet, it is important to highlight a transformation of variable into . As you can see at the top of this file, we've merged with a tag for getting a new map that is stored inside of . It is useful because without it, all resources in the AWS Console UI will have an empty name. var.tags a local variable local.tags var.tag Name local.tags Let's take a closer look at each AWS resource now. aws_verifiedaccess_instance AWS Verified Access instance is an AWS resource that assists you in organising your trust providers and Verified Access groups. Except for , it's a simple Terraform resource that doesn't require any additional configuration options. tags aws_verifiedaccess_trust_provider A trust provider is a AWS service that sends information about users and devices to AWS Verified Access. This information is called trust context. It can include attributes based on user identity, such as an email address or membership in the "sales" organization, or device information such as installed security patches or anti-virus software version. Verified Access supports the following categories of trust providers: – An identity provider (IdP) service that stores and manages digital identities for users. User identity – A device management system for devices such as laptops, tablets, and smartphones. Device management In our case, we use provider ( ) together with OIDC configuration ( ). user identity trust_provider_type = "user" oidc_options aws_verifiedaccess_instance_trust_provider_attachment This resources attaches Verified Access Instance to Verified Access Trust Provider. aws_verifiedaccess_group This is a resource for managing a Verified Access Group. AWS Verified Access group is a collection of Verified Access endpoints and a group-level Verified Access policy. Each endpoint within a group shares the Verified Access policy. You can use groups to gather endpoints that have common security requirements. This can help simplify policy administration by using one policy ( option. We’ll talk about policies configuration later) for the security needs of multiple applications. policy_document For example, you can group all sales applications together and set a group-wide access policy. You can then use this policy to define a common set of minimum security requirements for all sales applications. When you create a group, you are required to associate the group with a Verified Access instance ( option). During the process of creating an endpoint, you will associate the endpoint with a group. verifiedaccess_instance_id aws_verifiedaccess_endpoint A Verified Access endpoint represents an application. Each endpoint is associated with a Verified Access group and inherits the access policy for the group. You can optionally attach an application-specific endpoint policy to each endpoint. In this article I associate endpoint with Application Load balancer via the following block. endpoint_type = "load-balancer" load_balancer_options { load_balancer_arn = var.lb_arn port = var.lb_port protocol = var.lb_protocol subnet_ids = var.subnet_ids } It is important to highlight that the has the correctly configured option, which should point to a valid TLS certificate stored in AWS Certificate Manager. Keep in mind that the certificate must match the route53 domain record that will be created for the AWS Verified Access Endpoint. aws_verifiedaccess_endpoint domain_certificate_arn can be used to issue a purchased certificate, but I prefer free Let's Encrypt certificates. AWS does not natively support Let's Encrypt and Certificate Manager integration, so I created an AWS Lambda that automatically issues and renews TLS certificates. This Lambda's sources and documentation may be found at AWS Certificate Manager https://github.com/kvendingoldo/aws-letsencrypt-lambda. ALB/NLB is not the only method for exposing a service via a Verified Access Endpoint. You can also expose the service outside of the private network by using any private ENI interface, such as the EC2 ENI interface, via Verified Access Endpoint. endpoint_type = "network-interface" network_interface_options { network_interface_id = aws_network_interface.example.id port = 443 protocol = "https" } aws_route53_record This resource creates a Route53 record resource for Verified Access Endpoint. As Verified Access Endpoint has its own DNS name; the type of our record ( ) should be CNAME. kvendingoldo.dev.referrs.me aws_verifiedaccess_instance_logging_configuration This resource manages a Verified Access Logging Configuration. I strongly advise enabling this resource via feature flag during Verified Access testing. For me, using only CloudWatch logs to find issues in Cedar rules was sufficient, but you can also configure S3 or Kinesis sources to send logs from your Verified Access Instance. var.enable_va_logging If you have any problems with your rules during testing, you will see something like this in your logs: "activity_id": "2", "activity_name": "Access Deny", "actor": null, "category_name": "Audit Activity", "category_uid": "3", "class_name": "Access Activity", "class_uid": "3006", "data": null, "device": null, "duration": "0.0", "end_time": "1704880589864", "time": "1704880589864", "http_request": { "http_method": "GET", "url": { "hostname": "3.100.226.13", "path": "/version", "port": 443, "scheme": "https", "text": "https://3.100.226.13:443/version" }, "user_agent": "Mozilla/5.0 zgrab/0.x", "version": "HTTP/1.1" }, "http_response": { "code": 302 }, "message": "", "metadata": { "uid": "Root=1-659e69cd-2832fc7d4af907443fce29dd", "logged_time": 1704880985929, "version": "1.0.0-rc.2", "product": { "name": "Verified Access", "vendor_name": "AWS" } }, "ref_time": "2024-01-10T09:56:29.864879Z", "proxy": { "ip": "3.133.226.13", "port": 443, "svc_name": "Verified Access", "uid": "vai-0de4109b1cc38ecd4" }, "severity": "Informational", "severity_id": "1", "src_endpoint": { "ip": "162.243.138.37", "port": 50388 }, "start_time": "1704880589864", "status_code": "200", "status_detail": "Authentication Denied", "status_id": "2", "status": "Failure", "type_uid": "300602", "type_name": "Access Activity: Access Deny", "unmapped": null } Also, I recommend creating a Cloudwatch log group using Terraform and other resources, which you can do with the following code: resource "aws_cloudwatch_log_group" "main" { name = "my_va_loggroup_for_testing" retention_in_days = 10 tags = var.tags } 05-variables.tf I'd like to talk about Google OIDC variables before we get into Terraform variables for our resources. I couldn't find any documentation about the correct set of these variables and for this reason, I had to get them from . https://accounts.google.com/.well-known/openid-configuration Use as a source of truth for Google AuthN configuration options. https://accounts.google.com/.well-known/openid-configuration This is our infrastructure's Terraform variables file: # # Common variables # variable "svc_name" { type = string default = "kvendingoldo-app" } variable "env_abbr" { type = string default = "dev01" } variable "tags" { default = { automation : "terraform" data_classification : "internal" owner : "Alexander Sharov" } } # # Network # variable "subnet_ids" { description = "A list of subnet ids where verified access endpoint will be placed" type = list(string) } variable "sg_ids" { description = "List of the the security groups IDs to associate with the Verified Access endpoint" type = list(string) } variable "lb_arn" { description = "The ARN of the load balancer that will be used for the Verified Access endpoint" type = string } variable "lb_port" { description = "The load balancer port that will be used for the Verified Access endpoint" type = string default = 80 } variable "lb_protocol" { description = "The load balancer protocol that will be used for the Verified Access endpoint" type = string default = "http" } # # Verified Access Common # variable "enable_va" { default = true } variable "enable_va_logging" { default = false } variable "va_domain_cert_arn" { default = null } variable "va_group_policy_document" { description = "The policy document that is associated with this resource" type = string default = null } # # Verified Access Trust provider # variable "va_trustprovider_id" { default = null } variable "va_user_trust_provider_type" { description = "The type of user-based trust provider." type = string default = "oidc" } # # Verified Access logging # variable "va_log_include_trust_context" { default = true } variable "va_log_version" { default = "ocsf-1.0.0-rc.2" } variable "va_log_cloudwatch_enabled" { default = false } variable "va_log_cloudwatch_log_group_id" { default = "" } variable "va_log_kinesis_enabled" { default = false } variable "va_log_kinesis_delivery_stream_name" { default = null } variable "va_log_s3_enabled" { default = false } variable "va_log_s3_bucket_name" { default = null } # # Verified Access Trust provider OIDC Settings # variable "va_authorization_endpoint" { description = "The OIDC authorization endpoint" type = string default = "https://accounts.google.com/o/oauth2/v2/auth" } variable "va_client_id" { description = "The client identifier" type = string } variable "va_client_secret" { description = "The client secret" type = string } variable "va_issuer" { description = "The OIDC issuer" type = string default = "https://accounts.google.com" } variable "va_scope" { description = "OpenID Connect (OIDC) scopes are used by an application during authentication to authorize access to details of a user." type = string default = "openid email profile" } variable "va_token_endpoint" { description = "The OIDC token endpoint" type = string default = "https://oauth2.googleapis.com/token" } variable "va_user_info_endpoint" { description = "The OIDC user info endpoint" type = string default = "https://openidconnect.googleapis.com/v1/userinfo" } # # Route53 # variable "r53_zone_id" { default = "" } variable "r53_va_record" { default = "kvendingoldo" } Let's discuss only some of the variables that may raise questions Variable name Description va_domain_cert_arn ARN of a certificate stored in AWS Certificate Manager. It should be generated prior to infrastructure provisioning. va_group_policy_document The AuthZ policy document. We’ll take a look at this variable deeper in the following section. va_trustprovider_id ID of existing Trust Provider. If it’s not empty, a new Trust Provider won’t be created. va_client_id Google OAuth Client Id va_client_secret Google OAuth Secret Id r53_zone_id ID of Route53 zone. In my case, it’s ID of zone that is managed via AWS Route53. dev.referrs.me r53_va_record Name of a record inside of zone. In my case the value is r53_zone_id kvendingoldo AuthZ policies via Cedar rules Finally, after we have configured our infrastructure and AuthN, we can configure . For such policies, AWS Verified Access Group use Cedar configuration language. E.g.: AuthZ policies permit(principal, action, resource) when { // Returns true if the email ends in "@example.com" context.identity.email like "*@example.com" && // Returns true if the user is part of the "finance" group context.identity.groups.contains("finance") }; In my case, I'd like to restrict service access under Verified Access Endpoint to all users with the email address . The rule will be as follows: *@referrs.me permit(principal,action,resource) when { context.dev01_kvendingoldo_app.email like "*@referrs.me" }; Pay attention to . This variable has been configured inside of Terraform resource, property: dev01_kvendingoldo_app aws_verifiedaccess_trust_provider policy_reference_name policy_reference_name = replace(format("%s-%s", var.env_abbr, var.svc_name), "-", "_") To add the described rule for our infrastructure, we have to change variable: va_group_policy_document variable "va_group_policy_document" { description = "The policy document that is associated with this resource" type = string default = <<-EOT permit(principal,action,resource) when { context.dev01_kvendingoldo_app.email like "*@referrs.me" }; EOT } And then run our Terraform scripts once more. When the infrastructure is applied, it may take up to 5 minutes for the rule to start working. Conclusion In this blog post, I demonstrated how to use Terraform to create a Verified Access infrastructure for any private ALB/NLB or ENI interface. You can use the same infrastructure if you need a simple secure access to your service based on Google AuthN without VPN setup. Please contact me if you have any concerns or questions. I will make every effort to respond to your questions and provide solutions. Stay tuned for future posts where we'll talk a lot about Terraform infrastructure. References documentation AWS Verified Access documentation, documentation, tofuenv documentation to manage multiple OpenTofu CLI versions and CLI documentation to autogenerate Terraform module documentation. Terraform AWS Terraform provider Terraform Docs documentation. OpenTofu . TofuEnv repository repository. aws-letsencrypt-lambda . A walk through AWS Verified Access policies . Cedar configuration language