VPC peering and Transit Gateway: connecting many VPCs
Connect VPCs with peering and with AWS Transit Gateway, understand non-transitive routing, design segmented Transit Gateway route tables for dev, test and prod, and see why peering stops scaling in a bank.
Assumes: .NET 10 LTS (file-based app, no AWS calls) ยท AWS CLI v2 ยท CDK v2 (Amazon.CDK.Lib 2.277.0) ยท Terraform AWS provider v6 ยท Region us-east-1. Verified against official sources on 2026-11-18.
Learning objectives
By the end of this lesson you can:
- Create a VPC peering connection, add the routes on both sides, and explain why peering is not transitive.
- Calculate how many peering connections a full mesh needs and why that breaks down for a bank.
- Explain Transit Gateway attachments, route tables, associations and propagations, and predict where a packet goes.
- Design segmented Transit Gateway routing so prod cannot talk to dev, while both reach shared services and on-premises.
- Know the limits that matter: quotas, bandwidth, MTU, appliance mode, cross-Region peering, and cost per attachment and per GB.
- Simulate a Transit Gateway design offline with a small C# tool before you spend money.
Prerequisites
- CIDR planning โ non-overlapping ranges are a precondition for everything here.
- Private networking patterns โ PrivateLink as the alternative for single-service connectivity.
- Route tables from the module overview lessons.
- Tools: .NET 10 SDK (the simulator needs no AWS account), AWS CLI v2 with
learn-sandboxif you deploy.
Why this matters in banking
Harbourline started with three VPCs: dev, test and prod. Peering them was easy. Two years later there are 24 VPCs across eight accounts โ Payments, Accounts, Cards, Data Platform, the network hub, shared services, each in dev/test/prod โ plus two data centres and an EU subsidiary. Every new VPC needs to reach shared services (DNS, logging, CI runners), the data centres (core banking) and sometimes each other.
With peering, that becomes a web of point-to-point links that nobody can draw from memory. Worse, it is easy to connect the wrong things: a dev VPC peered to prod "for a quick test" gives every dev workload a route to production databases. Banks need a hub where connectivity policy lives in one place, segments are explicit (prod, non-prod, shared, on-premises), and every change is reviewable. That hub is AWS Transit Gateway.
The idea in plain language
VPC peering is a private cable between exactly two VPCs. Each side adds a route saying "to reach the other VPC's range, use this cable". It's cheap and fast, but each cable only connects the two VPCs at its ends. If A is peered with B, and B with C, A cannot reach C through B โ peering is non-transitive.
Transit Gateway is a Regional router that you attach VPCs, VPNs and Direct Connect to. Each attachment is like plugging a network into a port on the router. The router has route tables; each attachment is associated with exactly one route table (which decides where its traffic can go), and can propagate its routes into any number of route tables (which decides who can reach it).
Analogy: peering is a private door between two neighbouring offices. Transit Gateway is the building's central reception: every office has one door to reception, and reception's rulebook (route tables) decides which office visitors may be sent to. Want the trading floor to never see the interns' floor? One rule at reception, instead of bricking up dozens of doors.
What problem the AWS services solve
| VPC peering | AWS Transit Gateway | |
|---|---|---|
| Topology | Point-to-point between two VPCs | Hub-and-spoke for up to 5,000 attachments |
| Transitive routing | No | Yes, controlled by route tables |
| Connects VPN / Direct Connect | No (edge-to-edge routing is not allowed) | Yes |
| Overlapping CIDRs | Not allowed | Not routable (same problem) |
| Cross-Region | Yes (inter-Region peering) | Yes (Transit Gateway peering, static routes) |
| Cross-account | Yes (requester/accepter) | Yes (share with AWS RAM) |
| Security group referencing | Yes, within the same Region | Supported on VPC attachments when enabled โ VERIFY current scope |
| Bandwidth | No aggregate limit per peering beyond instance limits | Up to 100 Gbps per VPC attachment per AZ (each direction) |
| MTU | 9001 within a Region โ VERIFY; 1500 inter-Region | 8500 between VPCs/DX/peering; 1500 over VPN |
| Price (us-east-1, checked 2026-11-18) | No hourly charge; data within one AZ free; USD 0.01/GB each direction across AZs; inter-Region at data-transfer rates | USD 0.05 per attachment-hour + USD 0.02 per GB processed |
| Quotas | 50 active peerings per VPC, adjustable to 125 | 5 TGWs per Region per account; 5,000 attachments; 20 route tables per TGW (adjustable) |
Why peering does not scale
A full mesh of n VPCs needs n(nโ1)/2 peering connections, and each VPC's route tables need a route per peer:
| VPCs | Peering connections | Routes per VPC route table |
|---|---|---|
| 3 | 3 | 2 |
| 10 | 45 | 9 |
| 24 | 276 | 23 |
| 50 | 1,225 | 49 โ and 50 peerings per VPC is the default quota |
Add on-premises: peering cannot carry traffic from a VPN or Direct Connect in one VPC to another VPC (no edge-to-edge routing), so every VPC would need its own VPN. That is the point at which banks move to Transit Gateway.
Peering still has a place: it has no hourly cost and no per-GB processing within an AZ, so it is ideal for a few high-volume pairs (for example, a data platform VPC and a large analytics VPC moving terabytes a day), even inside a Transit Gateway estate.
Microsoft / Azure comparison
| You know (Microsoft/Azure) | AWS equivalent | What's different |
|---|---|---|
| VNet peering | VPC peering | Both non-transitive by default; Azure allows "use remote gateways" for shared VPN, AWS peering never transits edge connections |
| Hub VNet with NVA / Azure Firewall and UDRs | Transit Gateway + inspection VPC | TGW is a managed router; no VM in the path unless you add one |
| Azure Virtual WAN hub | Transit Gateway (Regional) / AWS Cloud WAN (global, policy-based) | Cloud WAN builds TGW-like core networks across Regions from a policy document |
| Azure Virtual Network Manager connectivity configurations | TGW route tables + RAM sharing, or Cloud WAN segments | AWS segments are route tables (TGW) or named segments (Cloud WAN) |
| Global VNet peering | Inter-Region VPC peering / TGW peering | TGW peering uses static routes only |
| Gateway transit | TGW with VPN/DX attachments | Central, routed, with per-attachment control |
Architecture
Peering: non-transitive
Dev can reach shared services; shared services can reach prod; dev cannot reach prod through shared services. That is sometimes exactly what you want โ but it is a property of the tool, not a policy you chose.
Transit Gateway with segmented route tables
Propagations (who appears in which table):
| Route table | Associated attachments | Propagated into it | Effect |
|---|---|---|---|
| rt-prod | Payments, Accounts | Payments, Accounts, Hub, DX | Prod VPCs reach each other, shared services and on-premises |
| rt-nonprod | Test, Dev | Test, Dev, Hub | Non-prod reaches each other and shared services โ not prod, not on-premises |
| rt-shared | Hub | All VPCs, DX | Shared services (DNS, CI runners, inspection) reach everything |
| rt-onprem | DX gateway | Payments, Accounts, Hub | Data centres reach prod and shared services only |
The rule to remember: association decides where an attachment's traffic can go; propagation decides who can come back. Isolation between prod and non-prod is simply "no prod routes in rt-nonprod and no non-prod routes in rt-prod".
What happens to one packet
Even though the dev VPC route table sends the whole AWS block 10.16.0.0/12 to the Transit Gateway (a single summarised route from the IP plan), the TGW route table associated with the dev attachment has no route for 10.17.0.0/16, so the packet is dropped. Segmentation lives in one place.
Step-by-step implementation
1. Peering (two VPCs, same account)
- Create the peering connection from the requester VPC to the accepter VPC.
- Accept it (in the accepter account if cross-account).
- Add a route in each side's route tables: destination = the other VPC's CIDR, target =
pcx-โฆ. - Update security groups โ within one Region you can reference the peer VPC's security groups.
- Optionally enable DNS resolution so private DNS names of the peer resolve to private IPs.
2. Transit Gateway: create it with defaults off
Create the TGW with default route table association and default route table propagation disabled. With both on, every new attachment joins one big table and can reach everything โ the opposite of segmentation. Choose a private ASN (64512โ65534 range or 4200000000โ4294967294) for BGP with VPN/DX.
3. Attach VPCs
For each VPC, create a VPC attachment using one small dedicated subnet per AZ (a /28 is enough). Put the attachment in every AZ your workloads use; traffic from an AZ without an attachment subnet cannot use the TGW from that AZ.
4. Create route tables, associations and propagations
Create rt-prod, rt-nonprod, rt-shared, rt-onprem, associate each attachment with exactly one, and enable propagations per the table above.
5. Update VPC route tables
In each spoke VPC, route the AWS supernet (and on-premises ranges) to the TGW: 10.16.0.0/12 โ tgw-โฆ, 10.0.0.0/13 โ tgw-โฆ (covers both data centres' /14s). The TGW route tables then decide what is actually reachable.
6. Share across accounts
In a multi-account estate, the TGW lives in the network account and is shared with workload accounts through AWS Resource Access Manager (RAM). Workload accounts create attachments; the network account accepts and associates them (with auto-accept off). See multi-account networking.
7. Verify
Use Reachability Analyzer (it understands TGW route tables โ see the codes such as TGW_RTB_NO_ROUTE_TO_TGW_ATTACHMENT in VPC Flow Logs) and Transit Gateway flow logs for evidence.
AWS Console instructions
- Peering: Console โ VPC โ Peering connections โ Create peering connection โ requester VPC, accepter (this or another account, this or another Region) โ Create. Then Actions โ Accept request. Then edit route tables on both sides.
- Transit Gateway: Console โ VPC โ Transit gateways โ Create transit gateway โ untick Default route table association and Default route table propagation, set the ASN, leave DNS support on.
- Attachments: Transit gateway attachments โ Create โ type VPC, pick one subnet per AZ, tick Appliance mode only for inspection VPCs.
- Route tables: Transit gateway route tables โ Create โ then on each table use the Associations and Propagations tabs.
AWS CLI
$p = '--profile', 'learn-sandbox', '--region', 'us-east-1'
# --- VPC peering (same account, same Region) ---
$pcx = aws ec2 create-vpc-peering-connection @p --vpc-id vpc-0dev00000000000 --peer-vpc-id vpc-0hub00000000000 `
--tag-specifications "ResourceType=vpc-peering-connection,Tags=[{Key=Name,Value=hl-dev-to-hub},{Key=Project,Value=cea-lab},{Key=Lab,Value=m02-l12}]" `
--query VpcPeeringConnection.VpcPeeringConnectionId --output text
aws ec2 accept-vpc-peering-connection @p --vpc-peering-connection-id $pcx
aws ec2 create-route @p --route-table-id rtb-0dev0000000000a --destination-cidr-block 10.16.0.0/16 --vpc-peering-connection-id $pcx
aws ec2 create-route @p --route-table-id rtb-0hub0000000000a --destination-cidr-block 10.20.0.0/16 --vpc-peering-connection-id $pcx
aws ec2 modify-vpc-peering-connection-options @p --vpc-peering-connection-id $pcx `
--requester-peering-connection-options AllowDnsResolutionFromRemoteVpc=true `
--accepter-peering-connection-options AllowDnsResolutionFromRemoteVpc=true
# --- Transit Gateway with default association/propagation OFF ---
$tgw = aws ec2 create-transit-gateway @p --description "Harbourline hub" `
--options AmazonSideAsn=64600,DefaultRouteTableAssociation=disable,DefaultRouteTablePropagation=disable,DnsSupport=enable,AutoAcceptSharedAttachments=disable `
--tag-specifications "ResourceType=transit-gateway,Tags=[{Key=Name,Value=hl-tgw},{Key=Project,Value=cea-lab},{Key=Lab,Value=m02-l12}]" `
--query TransitGateway.TransitGatewayId --output text
# Attach the dev VPC using one /28 per AZ
$attDev = aws ec2 create-transit-gateway-vpc-attachment @p --transit-gateway-id $tgw --vpc-id vpc-0dev00000000000 `
--subnet-ids subnet-0devtgw00000000a subnet-0devtgw00000000b `
--query TransitGatewayVpcAttachment.TransitGatewayAttachmentId --output text
# Route table for non-prod, association and propagation
$rtNon = aws ec2 create-transit-gateway-route-table @p --transit-gateway-id $tgw `
--tag-specifications "ResourceType=transit-gateway-route-table,Tags=[{Key=Name,Value=rt-nonprod}]" `
--query TransitGatewayRouteTable.TransitGatewayRouteTableId --output text
aws ec2 associate-transit-gateway-route-table @p --transit-gateway-route-table-id $rtNon --transit-gateway-attachment-id $attDev
aws ec2 enable-transit-gateway-route-table-propagation @p --transit-gateway-route-table-id $rtNon --transit-gateway-attachment-id $attDev
# One summarised route in the dev VPC towards the TGW
aws ec2 create-route @p --route-table-id rtb-0dev0000000000a --destination-cidr-block 10.16.0.0/12 --transit-gateway-id $tgw
# What does rt-nonprod contain?
aws ec2 search-transit-gateway-routes @p --transit-gateway-route-table-id $rtNon `
--filters Name=state,Values=active,blackhole --query "Routes[].[DestinationCidrBlock,Type,State,TransitGatewayAttachments[0].ResourceId]" --output tableA blackhole static route (create-transit-gateway-route --blackhole) explicitly drops traffic for a range โ useful to make an intended block visible in the route table rather than relying on absence.
Infrastructure as Code
using Amazon.CDK.AWS.EC2;
using Constructs;
namespace Harbourline.Networking.Transit;
/// <summary>Transit Gateway with defaults off and one route table per segment (L1 constructs).</summary>
public sealed class TransitHub : Construct
{
public TransitHub(Construct scope, string id) : base(scope, id)
{
Gateway = new CfnTransitGateway(this, "Tgw", new CfnTransitGatewayProps
{
AmazonSideAsn = 64600,
DefaultRouteTableAssociation = "disable",
DefaultRouteTablePropagation = "disable",
AutoAcceptSharedAttachments = "disable",
DnsSupport = "enable",
Description = "Harbourline hub",
});
foreach (var segment in new[] { "prod", "nonprod", "shared", "onprem" })
{
RouteTables[segment] = new CfnTransitGatewayRouteTable(this, $"Rt-{segment}", new CfnTransitGatewayRouteTableProps
{
TransitGatewayId = Gateway.Ref,
Tags = [new Amazon.CDK.CfnTag { Key = "Name", Value = $"rt-{segment}" }],
});
}
}
public CfnTransitGateway Gateway { get; }
public Dictionary<string, CfnTransitGatewayRouteTable> RouteTables { get; } = [];
/// <summary>Attach a VPC, associate it with one segment and propagate it into the listed segments.</summary>
public CfnTransitGatewayAttachment Attach(string name, string vpcId, string[] subnetIds, string associateWith, params string[] propagateTo)
{
var attachment = new CfnTransitGatewayAttachment(this, $"Att-{name}", new CfnTransitGatewayAttachmentProps
{
TransitGatewayId = Gateway.Ref,
VpcId = vpcId,
SubnetIds = subnetIds,
Tags = [new Amazon.CDK.CfnTag { Key = "Name", Value = $"att-{name}" }],
});
_ = new CfnTransitGatewayRouteTableAssociation(this, $"Assoc-{name}", new CfnTransitGatewayRouteTableAssociationProps
{
TransitGatewayAttachmentId = attachment.Ref,
TransitGatewayRouteTableId = RouteTables[associateWith].Ref,
});
foreach (var segment in propagateTo)
{
_ = new CfnTransitGatewayRouteTablePropagation(this, $"Prop-{name}-{segment}", new CfnTransitGatewayRouteTablePropagationProps
{
TransitGatewayAttachmentId = attachment.Ref,
TransitGatewayRouteTableId = RouteTables[segment].Ref,
});
}
return attachment;
}
}Usage: hub.Attach("dev", devVpcId, devTgwSubnets, associateWith: "nonprod", "nonprod", "shared"); โ dev's traffic uses rt-nonprod, and dev is reachable from non-prod and shared services only.
Complete code
Before deploying a USD 0.05-per-attachment-hour router, simulate it. This .NET 10 file-based app models TGW route tables with associations, propagations, static and blackhole routes, and answers "can A reach B?" with longest-prefix match in both directions โ exactly how TGW forwards. It runs with no AWS account.
using System.Net;
// Usage: dotnet run tgwsim.cs
// Models Harbourline's segmented Transit Gateway and checks a reachability matrix.
var attachments = new Dictionary<string, Attachment>
{
["payments"] = new("payments", "10.17.0.0/16", Associate: "prod", PropagateTo: ["prod", "shared", "onprem"]),
["accounts"] = new("accounts", "10.18.0.0/16", Associate: "prod", PropagateTo: ["prod", "shared", "onprem"]),
["test"] = new("test", "10.19.0.0/16", Associate: "nonprod", PropagateTo: ["nonprod", "shared"]),
["dev"] = new("dev", "10.20.0.0/16", Associate: "nonprod", PropagateTo: ["nonprod", "shared"]),
["hub"] = new("hub", "10.16.0.0/16", Associate: "shared", PropagateTo: ["prod", "nonprod", "shared", "onprem"]),
["dc"] = new("dc", "10.0.0.0/13", Associate: "onprem", PropagateTo: ["prod", "shared"]),
};
// Static routes: (route table, CIDR, target attachment or null for blackhole)
var staticRoutes = new List<(string Table, string Cidr, string? Target)>
{
("nonprod", "10.17.0.0/16", null), // explicit blackholes make the policy visible
("nonprod", "10.18.0.0/16", null),
};
// Build each route table: propagated routes first, then static routes (static wins on an exact-prefix tie in TGW).
var tables = new Dictionary<string, List<Route>>();
foreach (var a in attachments.Values)
{
foreach (var t in a.PropagateTo)
{
Get(t).Add(new Route(Cidr.Parse(a.Cidr), a.Name, Static: false));
}
}
foreach (var (table, cidr, target) in staticRoutes)
{
var parsed = Cidr.Parse(cidr);
Get(table).RemoveAll(r => r.Prefix.Equals(parsed));
Get(table).Add(new Route(parsed, target, Static: true));
}
// Expected policy: who must and must not talk.
(string From, string To, bool Expected)[] checks =
[
("payments", "accounts", true),
("payments", "hub", true),
("payments", "dc", true),
("dev", "test", true),
("dev", "hub", true),
("dev", "payments", false),
("dev", "dc", false),
("dc", "test", false),
("dc", "accounts", true),
];
var failures = 0;
foreach (var (from, to, expected) in checks)
{
var src = attachments[from];
var dst = attachments[to];
var forward = Lookup(src.Associate, FirstHost(dst.Cidr));
var back = Lookup(dst.Associate, FirstHost(src.Cidr));
var ok = forward == to && back == from;
var verdict = ok == expected ? "PASS" : "FAIL";
if (ok != expected)
{
failures++;
}
Console.WriteLine($"{verdict} {from,-9} -> {to,-9} expected {(expected ? "reachable" : "blocked"),-9} " +
$"forward via rt-{src.Associate}: {forward ?? "drop"}, return via rt-{dst.Associate}: {back ?? "drop"}");
}
Console.WriteLine(failures == 0 ? "All policy checks passed." : $"{failures} policy check(s) failed.");
return failures == 0 ? 0 : 3;
List<Route> Get(string table) => tables.TryGetValue(table, out var list) ? list : tables[table] = [];
// Longest-prefix match; a blackhole route (Target null) that wins means "drop".
string? Lookup(string table, IPAddress address) =>
Get(table).Where(r => r.Prefix.Contains(address))
.OrderByDescending(r => r.Prefix.Length)
.ThenByDescending(r => r.Static)
.FirstOrDefault()?.Target;
static IPAddress FirstHost(string cidr)
{
var c = Cidr.Parse(cidr);
var bytes = c.Network.GetAddressBytes();
bytes[3] += 10; // any address inside the range will do
return new IPAddress(bytes);
}
record Attachment(string Name, string Cidr, string Associate, string[] PropagateTo);
record Route(Cidr Prefix, string? Target, bool Static);
readonly record struct Cidr(IPAddress Network, int Length)
{
public static Cidr Parse(string text)
{
var parts = text.Split('/');
return new Cidr(IPAddress.Parse(parts[0]), int.Parse(parts[1]));
}
public bool Contains(IPAddress address)
{
var mask = Length == 0 ? 0u : uint.MaxValue << (32 - Length);
return (ToUInt(address) & mask) == (ToUInt(Network) & mask);
}
private static uint ToUInt(IPAddress a)
{
var b = a.GetAddressBytes();
return (uint)(b[0] << 24 | b[1] << 16 | b[2] << 8 | b[3]);
}
}Expected output:
PASS payments -> accounts expected reachable forward via rt-prod: accounts, return via rt-prod: payments
PASS payments -> hub expected reachable forward via rt-prod: hub, return via rt-shared: payments
PASS payments -> dc expected reachable forward via rt-prod: dc, return via rt-onprem: payments
PASS dev -> test expected reachable forward via rt-nonprod: test, return via rt-nonprod: dev
PASS dev -> hub expected reachable forward via rt-nonprod: hub, return via rt-shared: dev
PASS dev -> payments expected blocked forward via rt-nonprod: drop, return via rt-prod: drop
PASS dev -> dc expected blocked forward via rt-nonprod: drop, return via rt-onprem: drop
PASS dc -> test expected blocked forward via rt-onprem: drop, return via rt-nonprod: drop
PASS dc -> accounts expected reachable forward via rt-onprem: accounts, return via rt-prod: dc
All policy checks passed.Code explained line by line
attachmentsโ each attachment has its CIDR, the one route table it is associated with, and the tables it propagates into. This mirrors the Terraformspokesmap, so the simulator and the IaC can share a source of truth.dcuses10.0.0.0/13, the summary of both data centres' /14s, as a Direct Connect gateway would advertise via BGP.staticRoutesadds blackhole routes for prod ranges inrt-nonprod.Target = nullmeans drop.- The first
foreachbuilds propagated routes; the second removes any propagated route with the same prefix and adds the static one โ in TGW, a static route takes precedence over a propagated route for an identical prefix. checksis the connectivity policy written as data: pairs and whether they must be reachable.- For each pair the simulator does two lookups: the forward lookup in the source's associated table, and the return lookup in the destination's associated table. Real traffic needs both โ a one-way route is a classic TGW mistake.
Lookupimplements longest-prefix match: among routes containing the address, the longest prefix wins; on a tie, static beats propagated.FirstHostpicks an address inside the destination range (network + 10) to look up.- The
Cidrrecord struct does the bit arithmetic: build a mask from the prefix length and compare network bits. - Exit code 3 on any failure lets the simulator run in CI whenever someone edits the connectivity map.
Try changing dev's PropagateTo to include "prod": dev -> payments still fails on the forward path (rt-nonprod has no route to prod, and the blackhole wins), demonstrating that you need both directions for reachability โ and that blackholes protect against well-meant mistakes.
Security considerations
โ Learning shortcut โ not for production
Labs often create the Transit Gateway with default association and propagation enabled because it "just works". In production that puts every attachment in one routing domain, so any VPC can reach any other. Always disable both defaults and associate/propagate explicitly, as in the IaC above.
- Routing is not filtering. A route allows packets to arrive; security groups and NACLs still decide whether they are accepted. Keep SGs tight even inside a segment.
- Auto-accept off. With RAM sharing, set
AutoAcceptSharedAttachments=disableso the network team reviews every new attachment. - Inspect east-west where it matters. Route traffic between segments (or between sensitive VPCs) through an inspection VPC with Network Firewall; enable appliance mode on that attachment (see AWS Network Firewall).
- Log it. Enable Transit Gateway flow logs to the central log archive; they record traffic across attachments the VPC flow logs may not see.
- Least privilege on the hub. Only the network team's pipeline role should be allowed
ec2:*TransitGatewayRoute*,ec2:AssociateTransitGatewayRouteTableand similar actions; deny them elsewhere with an SCP. - Peering and SG references. Cross-VPC security-group references over peering (same Region) are convenient, but they bind two teams' security groups together; document them.
Banking use case
Harbourline's network team runs the Transit Gateway in the network hub account and shares it with all workload accounts through RAM. The connectivity map lives in Git as the Terraform spokes map, the tgwsim.cs checks run in the pull-request pipeline, and every change is approved by the network team and security.
- Segments: prod, non-prod, shared, on-premises, plus a separate card-data segment whose route table contains only the shared inspection VPC and the card-processing VPCs.
- On-premises: a Direct Connect gateway attachment carries
10.0.0.0/13from the data centres. Only prod and shared services see it, so test environments can never reach core banking by accident. - High-volume pair: the Data Platform VPC and the analytics VPC exchange about 20 TB a month. That pair also has a VPC peering connection (no per-GB processing within an AZ), with a more-specific route on both sides so their bulk traffic bypasses the TGW.
- Multi-Region: the EU subsidiary's Transit Gateway in eu-west-1 is peered with the us-east-1 TGW; static routes allow only shared services to cross. Data-residency rules for EU customer data are decided by Harbourline's data-protection officer, not by routing alone.
Common mistakes
- Forgetting the return route โ the classic "ping one way" problem. Check both associated tables (the simulator does).
- Leaving default association/propagation on, creating one flat network.
- No TGW attachment subnet in an AZ where workloads run โ traffic from that AZ can't use the TGW.
- Overlapping CIDRs โ peering is refused; TGW routes become ambiguous. Fix the IP plan, or use PrivateLink/NAT.
- Expecting peering to be transitive, or to carry VPN/DX traffic to the peer (edge-to-edge routing).
- MTU surprises โ jumbo frames over peering versus 8500 on TGW; mismatch during migration drops large packets.
- TGW peering with dynamic routing assumptions โ TGW peering uses static routes only.
- Underestimating per-GB processing โ bulk data through a TGW costs USD 0.02/GB on every pass.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
Peering stays pending-acceptance | Not accepted in the accepter account/Region | Accept it there; requests expire after 7 days |
| Peering created, no traffic | Missing route on one side, or SG/NACL | Add routes both ways; reference peer SG or CIDR |
| Peer's private DNS names resolve to public IPs | DNS resolution option off | Enable AllowDnsResolutionFromRemoteVpc on both sides |
TGW attachment pendingAcceptance | Shared TGW with auto-accept off | Network account accepts the attachment |
| One-way connectivity via TGW | Return route missing in the destination's associated table | Propagate the source into that table (or add a static route) |
| Some AZs can't use the TGW | No attachment subnet in that AZ | Modify the attachment to add a subnet per AZ |
| Traffic dropped despite route | A blackhole or a more specific route wins | search-transit-gateway-routes and check longest-prefix |
| Inspection drops intermittent flows | Appliance mode off on inspection VPC attachment | Enable appliance mode |
| Large transfers stall after migration from peering | MTU mismatch | Align MTU; rely on PMTUD only on VPC/Connect attachments |
Cost and cleanup
Prices (us-east-1, checked 2026-11-18):
- VPC peering: no hourly charge. Data within the same AZ is free; across AZs USD 0.01/GB in each direction; inter-Region peering at inter-Region data-transfer rates.
- Transit Gateway: USD 0.05 per attachment-hour and USD 0.02 per GB processed. Data arriving from a TGW peering attachment is not charged processing.
Examples:
- Harbourline's 24 VPCs + 1 DX gateway attachment: 25 ร 0.05 ร 730 = USD 912.50/month in attachment-hours, plus 0.02 ร traffic (10 TB/month = USD 200).
- The same 20 TB/month Data Platform โ analytics pair: via TGW USD 400/month in processing; via same-AZ peering USD 0; cross-AZ peering USD 400 (0.01 each direction) โ so keep the heavy flows AZ-local or accept the cost.
- Lab: one TGW with two VPC attachments for one hour โ USD 0.10 plus negligible data.
Free alternative: run tgwsim.cs, extend the policy checks, and draw the route tables. Create peering between two free lab VPCs if you want hands-on routing โ peering itself has no hourly cost.
Deploy-and-destroy (optional, under USD 0.20): two lab VPCs, one TGW, two attachments, one route table, test with Reachability Analyzer, delete within the hour:
$p = '--profile', 'learn-sandbox', '--region', 'us-east-1'
# Remove VPC routes that point to the TGW or peering first
aws ec2 delete-route @p --route-table-id rtb-0dev0000000000a --destination-cidr-block 10.16.0.0/12
# Delete attachments (wait until state = deleted), then route tables, then the TGW
aws ec2 delete-transit-gateway-vpc-attachment @p --transit-gateway-attachment-id tgw-attach-0dev000000000000
aws ec2 delete-transit-gateway-route-table @p --transit-gateway-route-table-id tgw-rtb-0nonprod00000000
aws ec2 delete-transit-gateway @p --transit-gateway-id tgw-0hub00000000000
# Peering
aws ec2 delete-vpc-peering-connection @p --vpc-peering-connection-id pcx-0devhub0000000000
# Nothing left?
aws ec2 describe-transit-gateways @p --filters Name=state,Values=available,pending --query "TransitGateways[].TransitGatewayId"
aws resourcegroupstaggingapi get-resources @p --tag-filters Key=Project,Values=cea-lab --query "ResourceTagMappingList[].ResourceARN"Exercises
- ๐ข Guided: Run
tgwsim.cs. Then add acardsVPC (10.21.0.0/16) in a newcdesegment that may reach onlyhubanddc, and add checks proving it cannot reachpaymentsordev. - ๐ก Challenge: Peer two free lab VPCs (
10.20.0.0/16and a new10.19.0.0/16), add routes, enable DNS resolution, and use Reachability Analyzer to show a path from an ENI in one to an ENI in the other. Then add a third VPC peered only with the second and prove (with an analysis) that the first cannot reach it. - ๐ด Live problem: After onboarding a new Fraud Analytics VPC, the Fraud team can reach shared services (DNS, CI runners) without problems, but every call from the Payments VPC to the Fraud API times out. Security groups allow the traffic. The new attachment is associated with
rt-prodand propagates only intort-shared. Explain and fix.
Assignment
Commit to your portfolio: the Terraform spokes map for a five-VPC Harbourline estate, the tgwsim.cs simulator with a checks file that encodes your connectivity policy, and a README with the segment diagram and a monthly cost estimate.
Solutions
["cards"] = new("cards", "10.21.0.0/16", Associate: "cde", PropagateTo: ["shared", "onprem"]),and make hub and dc propagate into cde (add "cde" to their PropagateTo). Checks:
("cards", "hub", true),
("cards", "dc", true),
("cards", "payments", false),
("cards", "dev", false),cards โ payments fails on the forward lookup (rt-cde has no 10.17 route) and the return lookup (rt-prod has no 10.21 route). Why: a dedicated segment is the routing equivalent of the PCI DSS cardholder-data-environment boundary; adequacy of that segmentation is for your QSA to judge.
Create the peering, accept it, then add 10.19.0.0/16 โ pcx in the first VPC's route tables and 10.20.0.0/16 โ pcx in the second's. Allow the test port in the destination SG from the peer CIDR (or the peer SG). Reachability Analyzer from ENI A to ENI B: reachable, path through pcx. With a third VPC peered only to the second: an analysis from A to C returns not reachable with an explanation such as NO_ROUTE_TO_DESTINATION (A has no route to C's CIDR) โ and even adding a route via the first peering fails with PCX_REQUIRES_ADDRESS_IN_VPC_CIDR, because peering only carries traffic for the peer VPC's own range. That is non-transitivity, shown by evidence.
Trace both directions with the association rule:
- Fraud โ shared services: Fraud's traffic uses
rt-prod(its association), which contains the hub route (hub propagates intort-prod). Replies from the hub usert-shared, which contains Fraud (Fraud propagates intort-shared). Both directions work. - Payments โ Fraud: Payments' traffic uses
rt-prod. Fraud does not propagate intort-prod, so there is no route for the Fraud CIDR and the TGW drops the packet. Reachability Analyzer from a Payments ENI to the Fraud API ENI reports a Transit Gateway route-table explanation code onrt-prod.
Fix: enable propagation of the Fraud attachment into rt-prod (and into rt-onprem if the data centres need it). Add fraud to tgwsim.cs and to the Terraform spokes map, run the simulator, then confirm with Reachability Analyzer. Lesson: for each pair, check the table associated with each end; a missing propagation breaks exactly the flows that start on the other side.
Knowledge check
Answer: No. Peering is non-transitive; A needs its own peering with C (or a Transit Gateway).
Answer: Association chooses the one route table used to forward traffic arriving from an attachment. Propagation inserts an attachment's routes into a route table, so traffic using that table can reach the attachment.
Answer: With them on, every attachment joins one shared table and can reach every other โ no segmentation by default.
Answer: For a small number of high-volume VPC pairs, because peering has no hourly charge or per-GB processing within an AZ.
Answer: No. The spoke can send to the TGW; the TGW route table associated with its attachment decides what is actually reachable.
Interview questions
- "Why doesn't peering scale?" โ n(nโ1)/2 connections, per-VPC route sprawl, the 50-per-VPC default quota, no transitive or edge routing, and no central place to see or change policy.
- "How would you keep prod and non-prod apart on one Transit Gateway?" โ Separate route tables per segment, defaults off, explicit associations/propagations, blackhole routes to make the block visible, and a policy-as-code test like the simulator.
- "What is appliance mode and when do you need it?" โ It keeps both directions of a flow on the same AZ's attachment ENI; needed for stateful appliances (inspection VPCs with Network Firewall or third-party firewalls).
- "Transit Gateway vs Cloud WAN?" โ TGW is a Regional router you wire up yourself; Cloud WAN builds a global network from a policy with segments and attachment rules โ useful when you have many Regions and want central policy.
Certification practice
A company has 30 VPCs in one Region and two on-premises data centres connected by AWS Direct Connect. All VPCs must reach the data centres, production VPCs must not reach development VPCs, and the network team wants to manage routing centrally. Which solution has the LEAST operational overhead?
A. Full-mesh VPC peering and a Site-to-Site VPN from each VPC B. A transit gateway with a Direct Connect gateway attachment, separate transit gateway route tables for production and development, and propagations configured per segment C. A shared services VPC peered to all VPCs, with the Direct Connect virtual interface attached to the shared services VPC D. AWS PrivateLink endpoint services in each VPC
Answer: B. TGW centralises routing, supports DX, and route tables segment prod from dev. A needs 435 peerings and 30 VPNs. C fails because peering does not carry DX/VPN traffic to other VPCs (no edge-to-edge routing) and is non-transitive. D exposes individual services, not networks.
VPC A and VPC B are attached to a transit gateway with default association and propagation disabled. Each attachment is associated with its own route table, rt-a and rt-b. VPC B's attachment propagates into rt-a, but VPC A's attachment propagates into no route tables. Both VPC route tables send the other VPC's CIDR to the transit gateway, and security groups allow the traffic. Instances in A cannot connect to B. What should the CloudOps engineer do?
A. Enable propagation of VPC A's attachment into rt-b
B. Enable appliance mode on VPC A's attachment
C. Enable DNS support on the transit gateway
D. Change the transit gateway ASN
Answer: A. Requests from A use rt-a, which has B's route, so they reach B. Replies from B use rt-b, which has no route to A's CIDR, so they are dropped and the connection never completes. Propagating A into rt-b adds the return route. B only affects AZ symmetry. C affects DNS name resolution, not IP routing. D matters for BGP with VPN or Direct Connect.
A security team wants to ensure that no workload account can connect its VPCs to the organization's transit gateway or to other VPCs without network team approval. Which combination of controls is MOST effective? (Choose two.)
A. Share the transit gateway with AWS RAM and disable automatic acceptance of shared attachments
B. An SCP that denies ec2:CreateVpcPeeringConnection and ec2:AcceptVpcPeeringConnection for workload accounts, except for the network pipeline role
C. Enable VPC Flow Logs in every account
D. Enable default route table propagation on the transit gateway
E. Use security groups that deny traffic from other VPCs
Answer: A and B. A forces review of every attachment; B stops peering from bypassing the hub. C detects, it doesn't prevent. D creates flat connectivity. E: security groups have no deny rules.
Completion checklist
- I can explain non-transitive peering and edge-to-edge routing
- I can predict TGW forwarding from associations and propagations
- I ran
tgwsim.csand extended its policy checks - I know the TGW and peering pricing and when each wins
- If I deployed anything, I deleted attachments, route tables, TGW and peerings and the tag check is clean
Key takeaways
- Peering is a cheap, fast, two-VPC link โ non-transitive, no edge routing, and a mesh grows as nยฒ.
- Transit Gateway is the Regional hub: attachments, route tables, associations (where my traffic goes) and propagations (who can reach me).
- Segment with separate route tables and defaults off; blackhole routes make blocks explicit.
- Check both directions; most TGW incidents are a missing return route.
- Cost: USD 0.05 per attachment-hour plus USD 0.02 per GB โ keep very heavy flows on peering within an AZ.
Further reading
- How VPC peering connections work
- VPC peering quotas
- AWS Transit Gateway quotas
- AWS Transit Gateway pricing
- Amazon VPC pricing (peering data transfer)
Next steps: connect the hub to the data centres in hybrid connectivity.
Independent learning resource โ not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.
Private networking patterns: keeping traffic off the internet end to end
Design workloads whose traffic never touches the public internet โ NAT-free subnets, gateway and interface endpoints, private DNS, private API Gateway APIs, PrivateLink services, VPC Lattice โ and enforce it with endpoint and resource policies.
Hybrid connectivity: Site-to-Site VPN, Direct Connect and hybrid DNS
Connect AWS to on-premises data centres with Site-to-Site VPN and AWS Direct Connect, choose a resilient connection model for a bank, and make names resolve both ways with Route 53 Resolver endpoints and forwarding rules.