Referencing one subnet per AZ with Terraform

April 21, 2022

I recently had a Terraform use case in AWS where I needed to obtain a list of subnets from a VPC, but I only wanted one subnet per availability zone. Some quick searching around on DuckDuckGo didn’t yield an immediate solution to this problem, so I’m sharing my findings in the hope that it will help someone else.

If you’re interested in the solution, feel free to skip directly to “The Solution” below.

There are arguably better ways to architect this, such as adding a unique tag to only one subnet in an AZ. However, this provides one approach to solving the problem if that isn’t an option. It’s also an interesting thought experiment.

The Problem

I was using the Terraform VPC module to create a new VPC with public and private subnets. I then wanted to pass the public subnets to an AWS ALB resource. Ideally, the ALB would have a network mapping in each availability zone for fault tolerance.

The problem is that an ALB can only be attached to one subnet in each availability zone:

╷
│ Error: error creating application Load Balancer: InvalidConfigurationRequest: A load balancer cannot be attached to multiple subnets in the same Availability Zone

In my environment, each availability zone (us-west-1a and us-west-1c) has two subnets:

❯ aws --region us-west-1  ec2 describe-subnets --filters Name=vpc-id,Values=vpc-0db31fcb510846b2a Name=tag:public,Values=public | jq -r '.[][] | .AvailabilityZone, .SubnetId'
us-west-1a
subnet-0d8ee0c3b94df5735
us-west-1a
subnet-03e675359912ef292
us-west-1c
subnet-0167ac2ac63e71317
us-west-1c
subnet-03dc67203dac221c9

I needed a way to obtain only one subnet per AZ so that I could pass these subnets to the ALB resource. In my case, these subnets were provisioned with the VPC module, but that isn’t relevant to the problem: the VPC and subnets could be provisioned in any manner. As long as there is more than one subnet per AZ, it would cause a problem in my use case.

The Solution

The solution to this problem involves a few steps:

1. Obtain a list of all subnets in each availability zone for a given VPC
2. Filter those subnets such that there is only one per availability zone
3. Ensure that created resources ignore changes to the provided subnets, as the ordering is non-determinstic

Obtaining a list of all subnets in each availabilty zone can be accomplished by combining the aws_availability_zone data source and the aws_subnets data source. The aws_subnets data source supports the use of advanced filters that correspond to the underlying AWS API.

The example below performs the following:

1. Obtain a list of all availabilty zones using the aws_availability_zones data source
2. Obtain a list of subnets using the aws_subnets data source and a for_each for each availability zone. These subnets are filtered on three criteria:
   1. The subnet must be in the desired VPC
   2. It must be tagged with the public tag
   3. It must be in the availabilty zone being processed in the current loop iteration

data "aws_availability_zones" "available" {
  state = "available"
}

data "aws_subnets" "filtered_public" {
  for_each = toset(data.aws_availability_zones.available.zone_ids)

  filter {
    name   = "vpc-id"
    values = [var.vpc_id]
  }

  filter {
    name   = "tag-key"
    values = ["public"]
  }

  filter {
    name   = "availability-zone-id"
    values = ["${each.value}"]
  }
}

This yields a map data structure that looks like this (I have removed some of the fields for brevity):

filtered_public_subnets = {
  "usw1-az1" = {
...
    "ids" = tolist([
      "subnet-0d8ee0c3b94df5735",
      "subnet-03e675359912ef292",
    ])
...
  }
  "usw1-az3" = {
...
    "ids" = tolist([
      "subnet-0167ac2ac63e71317",
      "subnet-03dc67203dac221c9",
    ])
...
  }
}

The top-level keys for this map are the availability zones being processed in the for_each (usw1-az1 and usw1-az3). This is useful, but I am only interested in the ids list within these maps, and I only want one element from each. These can be extracted using a local variable and a loop:

locals {
  public_subnet_ids = [for k, v in data.aws_subnets.filtered_public : v.ids[0]]
}

This logic creates a list by iterating over each value in the map (usw1-az1 and usw1-az2) and extracting the first element ([0]) of the ids list. The resulting data structure is a list of subnet IDs, one per AZ:

public_subnet_ids = [
  "subnet-0d8ee0c3b94df5735",
  "subnet-0167ac2ac63e71317",
]

This is now in a format that can be passed as a list of subnets to other resources, such as the aws_lb that I was originally trying to create. It’s very important to note that the ids use a tolist, which means that ordering is not guaranteed across Terraform runs (a question about this was even raised on the AWS provider’s issue tracker). This makes the use of the lifecycle meta-argument necessary to ensure that changes to the list of subnets do not cause a change to the resource itself:

resource "aws_lb" "alb" {
  name               = "example-alb"
...
  subnets = local.public_subnet_ids

  lifecycle {
    ignore_changes = [ subnets ]
  }
}

Wrapping Up

Looping and filtering are easy concepts in most programming languages, but they are more difficult to grok when using the Terraform DSL. This is a pretty simple use case, but it took me longer than I’d like to admit to figure it out. Hopefully this solution will help others in a similar position who need to filter the raw output of a data source so that it is more easily consumed by another resource. However, you must always keep in mind that values may not be deterministic, and be prepared to handle those scenarios accordingly.

If anyone has a better solution to this problem, then please get in touch!