VPC: CIDR Design and Cost Trade-offs for a 2-AZ, 3-Tier Network
By the end of the previous chapters, we had a dedicated IAM user set up for running Terraform. From here, we start actually building up AWS resources. The first thing we create is the VPC. ALB, ECS, and RDS all need to specify which subnet they belong to at creation time, and the VPC that owns that subnet has to exist first.1
A VPC itself is just a resource holding a single CIDR block, and the real design decisions show up in how you divide it into subnets. This chapter covers the reasoning behind the 10.0.0.0/16 CIDR that Tasuku uses, why we place three tiers — public, private-app, and private-data — across two Availability Zones, and why we provision two NAT Gateways as the egress point for the private subnets.
4.1. VPC and CIDR basics
A VPC (Virtual Private Cloud) is a logically isolated virtual network within your AWS account. At creation time, you specify only a single IPv4 CIDR block, and you then carve subnets out of that address space.
There are constraints on the CIDR block size. AWS allows a range from /16 (65,536 IP addresses) to /28 (16 addresses); you can't go any larger or smaller.2 The convention is to use a private IP address range (RFC 1918), choosing from 10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16.
Subnets are subject to the same /16–/28 constraint. However, for subnets, AWS reserves 5 IP addresses — the first 4 and the last 1 of the CIDR block — so the number of usable IPs ends up 5 fewer than the block size. For a /24, that's 256 minus 5, or 251.3 4 The 5 reserved addresses are: the network address, one for the VPC router, one for the DNS server, one reserved for future AWS use, and the broadcast address.3
For the VPC's overall CIDR design, the AWS Well-Architected Framework recommends leaving headroom rather than using up the entire space.5 If you make a VPC or subnet too small, you'll end up either adding another CIDR block later or rebuilding the subnet from scratch.
4.2. Tasuku's CIDR design
We assign 10.0.0.0/16 to Tasuku's network. It falls within the RFC 1918 range and is the same value used as-is in AWS's own example VPCs.2 Out of the 65,536 IP address space, this chapter actually uses only six /24 subnets.
| Tier | AZ1 | AZ2 | Purpose |
|---|---|---|---|
| public | 10.0.0.0/24 | 10.0.1.0/24 | ALB (Chapter 6) |
| private-app | 10.0.10.0/24 | 10.0.11.0/24 | ECS Fargate (Chapters 8 and 10) |
| private-data | 10.0.20.0/24 | 10.0.21.0/24 | RDS (Chapter 9) |
We don't number the subnets back-to-back. Instead, we leave each tier's numbering in steps of 10, leaving gaps such as 10.0.2.0/24 through 10.0.9.0/24 open. This is so that when we separate environments in Chapter 11, or add subnets in the future, we won't have to renumber what's already there.
A /24 has 251 IP addresses.3 Since each Fargate task occupies exactly one ENI (Elastic Network Interface),6 the number of running tasks directly consumes those 251 addresses. But across all six subnets we're only using 6 out of a possible 256 /24 blocks, so there's plenty of headroom in the CIDR space itself.
4.3. A 2-AZ, 3-tier subnet layout
Tasuku's network places three tiers — public, private-app, and private-data — across two Availability Zones.
public is the subnet with a route to the internet gateway, and it's where we place the ALB. private-app has no route to the internet gateway and can only send outbound traffic through a NAT Gateway; it's where we place the ECS Fargate tasks. private-data, like private-app, has no route to the internet gateway, but we don't give it a route to the NAT Gateway either; it's where we place RDS.
The reason for splitting into three tiers is that each tier needs a different direction of traffic. The ALB has to accept inbound connections from an unspecified number of clients, so the public subnet is the only option. ECS Fargate tasks don't need inbound traffic, but they do need outbound traffic to pull container images and ship logs.7 RDS is a managed service that shouldn't be reachable from the internet,8 and in this setup RDS doesn't need to initiate outbound connections to the internet either.
We don't hardcode a literal string like ap-northeast-1a for the Availability Zone. Instead, we use the first two entries from the list retrieved at runtime via the aws_availability_zones data source. AWS assigns the physical location of Availability Zones to a random name on a per-account basis, so us-east-1a in one account doesn't necessarily point to the same physical location as us-east-1a in another account.9 If you hardcode a specific name in your code, running it under a different account could deploy resources into a zone you didn't intend.
Note that the code doesn't check whether this list contains at least two Availability Zones. Since ap-northeast-1, which Tasuku uses, has four Availability Zones, this causes no real harm here,10 but if you bring this same code into a region with only one Availability Zone, check the count of the list first.
4.4. The internet gateway and public subnet routes
An internet gateway (IGW) is the doorway between a VPC and the internet. You attach exactly one to a VPC, and all it takes is specifying vpc_id.11
The routes for both public subnets simply point to the same IGW. Since an IGW isn't tied to an Availability Zone, there's no point in preparing separate route tables for the two subnets — they share a single one. The route consists of just this one line:
0.0.0.0/0 → Internet Gateway
4.5. NAT Gateway: the egress point for private subnets and cost trade-offs
A NAT Gateway incurs costs along two axes: a charge for the time it's running, and a charge for the volume of data it processes.12 With this resource, we introduce the first resource in this series that's "billed by time, even when you're not using it." We won't state specific prices here, since they're subject to revision. If you're trying this yourself, check current rates on the AWS pricing page, and delete the resource once you're done.
The ECS Fargate tasks placed in the private-app subnet need outbound traffic to pull container images and ship logs.7 Since the private-app subnet has no route to the IGW, this outbound traffic goes through a NAT Gateway instead.
There are two options for how many NAT Gateways to use.
The first option is to share a single NAT Gateway across both Availability Zones. This cuts the time-based charge in half, but if the Availability Zone hosting that NAT Gateway becomes unavailable, the private-app subnet in the other, still-healthy Availability Zone also loses its route out to the internet.
The second option is to provision one NAT Gateway per Availability Zone, two in total. Multiple pieces of official AWS documentation agree that a NAT Gateway's high availability is confined to a single Availability Zone, and that achieving fault tolerance across Availability Zones requires one NAT Gateway per zone.13 14 15 16
Tasuku goes with the two-NAT-Gateway configuration, because in Chapter 1 we decided to build a setup that keeps running even if one Availability Zone becomes unavailable.17 Letting the NAT Gateway alone become a single point of failure spanning Availability Zones would contradict that premise.
The extra cost is limited to the time-based charge. The charge for the amount of data a NAT Gateway processes, viewed within a single Availability Zone, doesn't depend on the number of NAT Gateways — it's determined solely by the total volume of traffic passing through that Availability Zone.18 Going from one to two only adds the fixed cost of the additional running time.
AWS also offers an availability mode called "Regional NAT Gateway." In this scheme, a single logical NAT Gateway automatically deploys itself into whichever Availability Zone your workload runs in, and in Terraform it can be written as a single aws_nat_gateway.19 However, this feature reserves bandwidth per Availability Zone,20 and the fact that the code looks like a single resource is a separate matter from whether the actual billing ends up cheaper than provisioning two NAT Gateways, one per Availability Zone. We haven't been able to confirm the latter as of this writing. We won't make this option — whose cost structure isn't settled — the primary teaching material; Tasuku doesn't adopt it, and we mention it here only as one alternative.
4.6. Route table design
Tasuku's network provisions four route tables.
| Route table | Target subnets | Route |
|---|---|---|
| public | public-az1, public-az2 (shared) | 0.0.0.0/0 → IGW |
| private-app-az1 | private-app-az1 | 0.0.0.0/0 → NAT Gateway (az1) |
| private-app-az2 | private-app-az2 | 0.0.0.0/0 → NAT Gateway (az2) |
| private-data | private-data-az1, private-data-az2 (shared) | None |
The reason one table is enough for public is as described in the previous section. private-app can only point to the NAT Gateway in its own Availability Zone, so it needs one table per Availability Zone.
For private-data, we intentionally add no routes beyond the implicit local route for traffic within the VPC (an undeletable route to the CIDR block that's automatically included in every route table). This is because RDS shouldn't be accessible from the internet,8 and in this setup it has no need to initiate communication out to the internet either.21 The point of splitting into the three tiers — public, private-app, and private-data — shows up directly in these differences in route table contents: public can reach anywhere, private-app can only reach out through the NAT Gateway, and private-data can't reach anywhere outside the VPC (connections from ECS tasks are still reachable, since they're internal VPC traffic) — three distinct levels.
None of the subnets are left on the "main route table" that's implicitly used when a VPC is created. All six subnets are explicitly associated with the route table they belong to.
4.7. Security group chain approach (implementation in later chapters)
The route table design so far has been about deciding "how far traffic can reach." Which traffic is actually allowed is the security group's job.
Tasuku ultimately needs three security groups, and the chain of permissions looks like this:
| Security group | Allowed source | Created in |
|---|---|---|
| For ALB | Internet (port 80 → port 443) | Chapter 6 → 7 |
| For ECS tasks | ALB SG only | Chapter 8 |
| For RDS | ECS task SG only | Chapter 9 |
At this point in the chapter, none of ALB, ECS, or RDS exist yet. Since we can't write a rule allowing traffic to a resource that doesn't exist, we defer creating the actual security groups to the chapters that create each corresponding resource. This is the same approach we took in Chapter 3, where we limited ourselves to a preview table for IAM roles.
The only thing we actually touch in this chapter is the default security group that AWS automatically creates with the VPC. Left unconfigured, the default security group allows all traffic from within the same security group, and it also leaves all outbound traffic allowed. To limit the impact if this default security group ever gets mistakenly attached to a resource we don't intend to use, we close it off by leaving both its ingress and egress rules empty.22 23
4.8. Terraform implementation
Now we write the design so far into vpc.tf. We add the VPC's overall CIDR as a variable in the existing variables.tf, and we keep this chapter's main resources — the VPC, subnets, and routing — together in a single vpc.tf file. As with budget.tf in Chapter 2 and iam.tf in Chapter 3, the policy is to gather the resources that each chapter primarily adds into one file. Alongside this, we introduce a new outputs.tf in this chapter to collect the values that later chapters will reference.
First, we gather the definitions of the six subnets as a local value.
locals {
# Definitions for the six subnets across 2 AZs × 3 tiers (public / private-app / private-data)
# Align az_key ("az1"/"az2") as the for_each key across all resources to keep the AZ mapping consistent
subnets = {
"public-az1" = { tier = "public", az_key = "az1", cidr = "10.0.0.0/24", public_ip_launch = true }
"public-az2" = { tier = "public", az_key = "az2", cidr = "10.0.1.0/24", public_ip_launch = true }
"private-app-az1" = { tier = "private-app", az_key = "az1", cidr = "10.0.10.0/24", public_ip_launch = false }
"private-app-az2" = { tier = "private-app", az_key = "az2", cidr = "10.0.11.0/24", public_ip_launch = false }
"private-data-az1" = { tier = "private-data", az_key = "az1", cidr = "10.0.20.0/24", public_ip_launch = false }
"private-data-az2" = { tier = "private-data", az_key = "az2", cidr = "10.0.21.0/24", public_ip_launch = false }
}
# az1 → names[0], az2 → names[1] (the order returned by data.aws_availability_zones.available)
az_key_to_index = { az1 = 0, az2 = 1 }
# Reverse lookup of subnet keys by az_key, one per tier. Aligning the NAT Gateway's and
# route table's for_each on this az_key structurally prevents mixing up AZs
public_subnets_by_az = { for k, v in local.subnets : v.az_key => k if v.tier == "public" }
private_app_subnets_by_az = { for k, v in local.subnets : v.az_key => k if v.tier == "private-app" }
private_data_subnet_keys = { for k, v in local.subnets : k => k if v.tier == "private-data" }
}
for_each is the first meta-argument we use in this series. Instead of writing six similarly shaped subnets as individual blocks, we pass a map of definitions to a single resource block and let Terraform handle expanding it into that many blocks. The key point is that we align the for_each keys for public and private-app on the shared strings "az1"/"az2". This shared key structurally prevents mix-ups, such as a NAT Gateway in one Availability Zone ending up associated with the route table of a different Availability Zone.
The list of Availability Zones is retrieved at runtime as a data source.
data "aws_availability_zones" "available" {
state = "available"
}
Now we create the VPC itself and the subnets.
resource "aws_vpc" "main" {
cidr_block = var.vpc_cidr
# DNS hostnames default to false, so set this explicitly
enable_dns_support = true
enable_dns_hostnames = true
tags = {
Name = "${local.name_prefix}-vpc"
}
}
resource "aws_subnet" "this" {
for_each = local.subnets
vpc_id = aws_vpc.main.id
cidr_block = each.value.cidr
availability_zone = data.aws_availability_zones.available.names[local.az_key_to_index[each.value.az_key]]
map_public_ip_on_launch = each.value.public_ip_launch
tags = {
Name = "${local.name_prefix}-${each.key}"
}
}
enable_dns_hostnames defaults to false, so we explicitly set it to true.24 aws_subnet's map_public_ip_on_launch also defaults to false,25 and only the two public subnets pass true via each.value.public_ip_launch.
Here's the internet gateway and the route table for the public subnets.
resource "aws_internet_gateway" "main" {
vpc_id = aws_vpc.main.id
tags = {
Name = "${local.name_prefix}-igw"
}
}
resource "aws_route_table" "public" {
vpc_id = aws_vpc.main.id
route {
cidr_block = "0.0.0.0/0"
gateway_id = aws_internet_gateway.main.id
}
tags = {
Name = "${local.name_prefix}-public-rt"
}
}
resource "aws_route_table_association" "public" {
for_each = local.public_subnets_by_az
subnet_id = aws_subnet.this[each.value].id
route_table_id = aws_route_table.public.id
}
Next, the NAT Gateways and the route tables for private-app.
resource "aws_eip" "nat" {
for_each = local.public_subnets_by_az
domain = "vpc"
tags = {
Name = "${local.name_prefix}-nat-eip-${each.key}"
}
depends_on = [aws_internet_gateway.main]
}
resource "aws_nat_gateway" "this" {
for_each = local.public_subnets_by_az
allocation_id = aws_eip.nat[each.key].id
subnet_id = aws_subnet.this[each.value].id
tags = {
Name = "${local.name_prefix}-nat-${each.key}"
}
depends_on = [aws_internet_gateway.main]
}
resource "aws_route_table" "private_app" {
for_each = local.private_app_subnets_by_az
vpc_id = aws_vpc.main.id
route {
cidr_block = "0.0.0.0/0"
nat_gateway_id = aws_nat_gateway.this[each.key].id
}
tags = {
Name = "${local.name_prefix}-private-app-rt-${each.key}"
}
}
resource "aws_route_table_association" "private_app" {
for_each = local.private_app_subnets_by_az
subnet_id = aws_subnet.this[each.value].id
route_table_id = aws_route_table.private_app[each.key].id
}
We explicitly add a depends_on on the internet gateway to both aws_eip and aws_nat_gateway. EIPs and NAT Gateways internally assume the internet gateway already exists, and relying on Terraform's implicit dependency resolution order can sometimes cause errors, so making the dependency explicit is the officially recommended approach.26 27
Here's the route table for private-data, and the default security group.
resource "aws_route_table" "private_data" {
vpc_id = aws_vpc.main.id
tags = {
Name = "${local.name_prefix}-private-data-rt"
}
}
resource "aws_route_table_association" "private_data" {
for_each = local.private_data_subnet_keys
subnet_id = aws_subnet.this[each.value].id
route_table_id = aws_route_table.private_data.id
}
resource "aws_default_security_group" "main" {
vpc_id = aws_vpc.main.id
}
aws_default_security_group is different in nature from the other resources. Rather than creating something new, it brings the default security group that AWS automatically creates with the VPC under Terraform's management, and by writing no ingress or egress rules, it removes all of the existing rules.28
Finally, we gather the values that later chapters will reference into outputs.tf.
output "vpc_id" {
description = "ID of the VPC created in Chapter 4 (referenced in Chapters 6, 8, and 9)"
value = aws_vpc.main.id
}
output "public_subnet_ids" {
description = "List of IDs of the public subnets (for ALB, referenced in Chapter 6)"
value = [for k, v in aws_subnet.this : v.id if local.subnets[k].tier == "public"]
}
output "private_app_subnet_ids" {
description = "List of IDs of the private-app subnets (for ECS Fargate, referenced in Chapters 8 and 10)"
value = [for k, v in aws_subnet.this : v.id if local.subnets[k].tier == "private-app"]
}
output "private_data_subnet_ids" {
description = "List of IDs of the private-data subnets (for RDS, referenced in Chapter 9)"
value = [for k, v in aws_subnet.this : v.id if local.subnets[k].tier == "private-data"]
}
output "nat_gateway_public_ips" {
description = "Public IPs of the NAT Gateways (for verification, and for future external API allow-lists)"
value = [for k, v in aws_nat_gateway.this : v.public_ip]
}
Since this series uses a single, non-modularized configuration through Chapter 11, later chapters can reference values directly, like aws_subnet.this["public-az1"].id, without going through outputs.tf. We still set up outputs.tf so that we can check values with terraform output, and to keep the values this chapter exposes gathered in one place.
4.9. Verification
Run terraform fmt -check -recursive -diff and terraform validate to confirm there are no issues with syntax or the provider schema.29 These two commands only guarantee consistency of syntax and schema — they don't guarantee the result of actually running apply against AWS.
Running terraform plan should show the following 22 additions in the plan.
aws_vpc: 1aws_subnet: 6aws_internet_gateway: 1aws_route_table(public): 1 +aws_route_table_association: 2aws_eip: 2aws_nat_gateway: 2aws_route_table(private-app): 2 +aws_route_table_association: 2aws_route_table(private-data): 1 +aws_route_table_association: 2
Because aws_default_security_group doesn't create a new resource but instead has the special behavior of adopting the existing default security group,28 it may show up separately from the 22 items above, listed as a change instead.
After apply, you can check the VPC ID and the NAT Gateways' public IPs with terraform output. The NAT Gateways' public IPs will be referenced from later chapters in situations such as adding an external API to an IP-based allow-list.
Summary
- The VPC's real design decisions show up in the CIDR allocation and how subnets are divided. Tasuku carved six subnets out of
10.0.0.0/16, across 2 Availability Zones × 3 tiers (public / private-app / private-data) - We provision two NAT Gateways, one per Availability Zone. This costs more than a single-NAT-Gateway setup, but it's necessary for the "keeps running even if one Availability Zone fails" configuration we decided on in Chapter 1
- There are four route tables, and the differing nature of the three tiers — public, private-app, and private-data — shows up directly in the differing contents of their routes
- For security groups, this chapter only closes off the default one; the actual ALB, ECS, and RDS security groups are each created in the chapter that uses them
- We introduced
for_eachfor the first time in this series, using it to define the six identically shaped subnets and the NAT Gateways together
Next steps
- Chapter 5: S3: Bucket design principles and best practices for managing tfstate
- Chapter 3: IAM: Reading and writing policy JSON, and a practical approach to Terraform execution permissions
- Series table of contents: The structure of all 12 chapters and how to read through them