Skip to Content
Cloud Engineering Academy
2. AWS Networking & Infrastructure

VPC peering and Transit Gateway: connecting many VPCs

Connect VPCs with peering and with AWS Transit Gateway, understand non-transitive routing, design segmented Transit Gateway route tables for dev, test and prod, and see why peering stops scaling in a bank.

intermediateโฑ 1 h 40 min๐Ÿ”ด Paid โ€” optionalSAA-C03SOA-C03SCS-C03

Assumes: .NET 10 LTS (file-based app, no AWS calls) ยท AWS CLI v2 ยท CDK v2 (Amazon.CDK.Lib 2.277.0) ยท Terraform AWS provider v6 ยท Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Create a VPC peering connection, add the routes on both sides, and explain why peering is not transitive.
  • Calculate how many peering connections a full mesh needs and why that breaks down for a bank.
  • Explain Transit Gateway attachments, route tables, associations and propagations, and predict where a packet goes.
  • Design segmented Transit Gateway routing so prod cannot talk to dev, while both reach shared services and on-premises.
  • Know the limits that matter: quotas, bandwidth, MTU, appliance mode, cross-Region peering, and cost per attachment and per GB.
  • Simulate a Transit Gateway design offline with a small C# tool before you spend money.

Prerequisites

  • CIDR planning โ€” non-overlapping ranges are a precondition for everything here.
  • Private networking patterns โ€” PrivateLink as the alternative for single-service connectivity.
  • Route tables from the module overview lessons.
  • Tools: .NET 10 SDK (the simulator needs no AWS account), AWS CLI v2 with learn-sandbox if you deploy.

Why this matters in banking

Harbourline started with three VPCs: dev, test and prod. Peering them was easy. Two years later there are 24 VPCs across eight accounts โ€” Payments, Accounts, Cards, Data Platform, the network hub, shared services, each in dev/test/prod โ€” plus two data centres and an EU subsidiary. Every new VPC needs to reach shared services (DNS, logging, CI runners), the data centres (core banking) and sometimes each other.

With peering, that becomes a web of point-to-point links that nobody can draw from memory. Worse, it is easy to connect the wrong things: a dev VPC peered to prod "for a quick test" gives every dev workload a route to production databases. Banks need a hub where connectivity policy lives in one place, segments are explicit (prod, non-prod, shared, on-premises), and every change is reviewable. That hub is AWS Transit Gateway.

The idea in plain language

VPC peering is a private cable between exactly two VPCs. Each side adds a route saying "to reach the other VPC's range, use this cable". It's cheap and fast, but each cable only connects the two VPCs at its ends. If A is peered with B, and B with C, A cannot reach C through B โ€” peering is non-transitive.

Transit Gateway is a Regional router that you attach VPCs, VPNs and Direct Connect to. Each attachment is like plugging a network into a port on the router. The router has route tables; each attachment is associated with exactly one route table (which decides where its traffic can go), and can propagate its routes into any number of route tables (which decides who can reach it).

Analogy: peering is a private door between two neighbouring offices. Transit Gateway is the building's central reception: every office has one door to reception, and reception's rulebook (route tables) decides which office visitors may be sent to. Want the trading floor to never see the interns' floor? One rule at reception, instead of bricking up dozens of doors.

What problem the AWS services solve

VPC peeringAWS Transit Gateway
TopologyPoint-to-point between two VPCsHub-and-spoke for up to 5,000 attachments
Transitive routingNoYes, controlled by route tables
Connects VPN / Direct ConnectNo (edge-to-edge routing is not allowed)Yes
Overlapping CIDRsNot allowedNot routable (same problem)
Cross-RegionYes (inter-Region peering)Yes (Transit Gateway peering, static routes)
Cross-accountYes (requester/accepter)Yes (share with AWS RAM)
Security group referencingYes, within the same RegionSupported on VPC attachments when enabled โš  VERIFY current scope
BandwidthNo aggregate limit per peering beyond instance limitsUp to 100 Gbps per VPC attachment per AZ (each direction)
MTU9001 within a Region โš  VERIFY; 1500 inter-Region8500 between VPCs/DX/peering; 1500 over VPN
Price (us-east-1, checked 2026-11-18)No hourly charge; data within one AZ free; USD 0.01/GB each direction across AZs; inter-Region at data-transfer ratesUSD 0.05 per attachment-hour + USD 0.02 per GB processed
Quotas50 active peerings per VPC, adjustable to 1255 TGWs per Region per account; 5,000 attachments; 20 route tables per TGW (adjustable)

Why peering does not scale

A full mesh of n VPCs needs n(nโˆ’1)/2 peering connections, and each VPC's route tables need a route per peer:

VPCsPeering connectionsRoutes per VPC route table
332
10459
2427623
501,22549 โ€” and 50 peerings per VPC is the default quota

Add on-premises: peering cannot carry traffic from a VPN or Direct Connect in one VPC to another VPC (no edge-to-edge routing), so every VPC would need its own VPN. That is the point at which banks move to Transit Gateway.

Peering still has a place: it has no hourly cost and no per-GB processing within an AZ, so it is ideal for a few high-volume pairs (for example, a data platform VPC and a large analytics VPC moving terabytes a day), even inside a Transit Gateway estate.

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
VNet peeringVPC peeringBoth non-transitive by default; Azure allows "use remote gateways" for shared VPN, AWS peering never transits edge connections
Hub VNet with NVA / Azure Firewall and UDRsTransit Gateway + inspection VPCTGW is a managed router; no VM in the path unless you add one
Azure Virtual WAN hubTransit Gateway (Regional) / AWS Cloud WAN (global, policy-based)Cloud WAN builds TGW-like core networks across Regions from a policy document
Azure Virtual Network Manager connectivity configurationsTGW route tables + RAM sharing, or Cloud WAN segmentsAWS segments are route tables (TGW) or named segments (Cloud WAN)
Global VNet peeringInter-Region VPC peering / TGW peeringTGW peering uses static routes only
Gateway transitTGW with VPN/DX attachmentsCentral, routed, with per-attachment control

Architecture

Peering: non-transitive

Dev can reach shared services; shared services can reach prod; dev cannot reach prod through shared services. That is sometimes exactly what you want โ€” but it is a property of the tool, not a policy you chose.

Transit Gateway with segmented route tables

Propagations (who appears in which table):

Route tableAssociated attachmentsPropagated into itEffect
rt-prodPayments, AccountsPayments, Accounts, Hub, DXProd VPCs reach each other, shared services and on-premises
rt-nonprodTest, DevTest, Dev, HubNon-prod reaches each other and shared services โ€” not prod, not on-premises
rt-sharedHubAll VPCs, DXShared services (DNS, CI runners, inspection) reach everything
rt-onpremDX gatewayPayments, Accounts, HubData centres reach prod and shared services only

The rule to remember: association decides where an attachment's traffic can go; propagation decides who can come back. Isolation between prod and non-prod is simply "no prod routes in rt-nonprod and no non-prod routes in rt-prod".

What happens to one packet

Even though the dev VPC route table sends the whole AWS block 10.16.0.0/12 to the Transit Gateway (a single summarised route from the IP plan), the TGW route table associated with the dev attachment has no route for 10.17.0.0/16, so the packet is dropped. Segmentation lives in one place.

Step-by-step implementation

1. Peering (two VPCs, same account)

  1. Create the peering connection from the requester VPC to the accepter VPC.
  2. Accept it (in the accepter account if cross-account).
  3. Add a route in each side's route tables: destination = the other VPC's CIDR, target = pcx-โ€ฆ.
  4. Update security groups โ€” within one Region you can reference the peer VPC's security groups.
  5. Optionally enable DNS resolution so private DNS names of the peer resolve to private IPs.

2. Transit Gateway: create it with defaults off

Create the TGW with default route table association and default route table propagation disabled. With both on, every new attachment joins one big table and can reach everything โ€” the opposite of segmentation. Choose a private ASN (64512โ€“65534 range or 4200000000โ€“4294967294) for BGP with VPN/DX.

3. Attach VPCs

For each VPC, create a VPC attachment using one small dedicated subnet per AZ (a /28 is enough). Put the attachment in every AZ your workloads use; traffic from an AZ without an attachment subnet cannot use the TGW from that AZ.

4. Create route tables, associations and propagations

Create rt-prod, rt-nonprod, rt-shared, rt-onprem, associate each attachment with exactly one, and enable propagations per the table above.

5. Update VPC route tables

In each spoke VPC, route the AWS supernet (and on-premises ranges) to the TGW: 10.16.0.0/12 โ†’ tgw-โ€ฆ, 10.0.0.0/13 โ†’ tgw-โ€ฆ (covers both data centres' /14s). The TGW route tables then decide what is actually reachable.

6. Share across accounts

In a multi-account estate, the TGW lives in the network account and is shared with workload accounts through AWS Resource Access Manager (RAM). Workload accounts create attachments; the network account accepts and associates them (with auto-accept off). See multi-account networking.

7. Verify

Use Reachability Analyzer (it understands TGW route tables โ€” see the codes such as TGW_RTB_NO_ROUTE_TO_TGW_ATTACHMENT in VPC Flow Logs) and Transit Gateway flow logs for evidence.

AWS Console instructions

  • Peering: Console โ†’ VPC โ†’ Peering connections โ†’ Create peering connection โ†’ requester VPC, accepter (this or another account, this or another Region) โ†’ Create. Then Actions โ†’ Accept request. Then edit route tables on both sides.
  • Transit Gateway: Console โ†’ VPC โ†’ Transit gateways โ†’ Create transit gateway โ†’ untick Default route table association and Default route table propagation, set the ASN, leave DNS support on.
  • Attachments: Transit gateway attachments โ†’ Create โ†’ type VPC, pick one subnet per AZ, tick Appliance mode only for inspection VPCs.
  • Route tables: Transit gateway route tables โ†’ Create โ†’ then on each table use the Associations and Propagations tabs.

AWS CLI

$p = '--profile', 'learn-sandbox', '--region', 'us-east-1'

# --- VPC peering (same account, same Region) ---
$pcx = aws ec2 create-vpc-peering-connection @p --vpc-id vpc-0dev00000000000 --peer-vpc-id vpc-0hub00000000000 `
  --tag-specifications "ResourceType=vpc-peering-connection,Tags=[{Key=Name,Value=hl-dev-to-hub},{Key=Project,Value=cea-lab},{Key=Lab,Value=m02-l12}]" `
  --query VpcPeeringConnection.VpcPeeringConnectionId --output text
aws ec2 accept-vpc-peering-connection @p --vpc-peering-connection-id $pcx
aws ec2 create-route @p --route-table-id rtb-0dev0000000000a --destination-cidr-block 10.16.0.0/16 --vpc-peering-connection-id $pcx
aws ec2 create-route @p --route-table-id rtb-0hub0000000000a --destination-cidr-block 10.20.0.0/16 --vpc-peering-connection-id $pcx
aws ec2 modify-vpc-peering-connection-options @p --vpc-peering-connection-id $pcx `
  --requester-peering-connection-options AllowDnsResolutionFromRemoteVpc=true `
  --accepter-peering-connection-options AllowDnsResolutionFromRemoteVpc=true

# --- Transit Gateway with default association/propagation OFF ---
$tgw = aws ec2 create-transit-gateway @p --description "Harbourline hub" `
  --options AmazonSideAsn=64600,DefaultRouteTableAssociation=disable,DefaultRouteTablePropagation=disable,DnsSupport=enable,AutoAcceptSharedAttachments=disable `
  --tag-specifications "ResourceType=transit-gateway,Tags=[{Key=Name,Value=hl-tgw},{Key=Project,Value=cea-lab},{Key=Lab,Value=m02-l12}]" `
  --query TransitGateway.TransitGatewayId --output text

# Attach the dev VPC using one /28 per AZ
$attDev = aws ec2 create-transit-gateway-vpc-attachment @p --transit-gateway-id $tgw --vpc-id vpc-0dev00000000000 `
  --subnet-ids subnet-0devtgw00000000a subnet-0devtgw00000000b `
  --query TransitGatewayVpcAttachment.TransitGatewayAttachmentId --output text

# Route table for non-prod, association and propagation
$rtNon = aws ec2 create-transit-gateway-route-table @p --transit-gateway-id $tgw `
  --tag-specifications "ResourceType=transit-gateway-route-table,Tags=[{Key=Name,Value=rt-nonprod}]" `
  --query TransitGatewayRouteTable.TransitGatewayRouteTableId --output text
aws ec2 associate-transit-gateway-route-table @p --transit-gateway-route-table-id $rtNon --transit-gateway-attachment-id $attDev
aws ec2 enable-transit-gateway-route-table-propagation @p --transit-gateway-route-table-id $rtNon --transit-gateway-attachment-id $attDev

# One summarised route in the dev VPC towards the TGW
aws ec2 create-route @p --route-table-id rtb-0dev0000000000a --destination-cidr-block 10.16.0.0/12 --transit-gateway-id $tgw

# What does rt-nonprod contain?
aws ec2 search-transit-gateway-routes @p --transit-gateway-route-table-id $rtNon `
  --filters Name=state,Values=active,blackhole --query "Routes[].[DestinationCidrBlock,Type,State,TransitGatewayAttachments[0].ResourceId]" --output table

A blackhole static route (create-transit-gateway-route --blackhole) explicitly drops traffic for a range โ€” useful to make an intended block visible in the route table rather than relying on absence.

Infrastructure as Code

TransitHub.cs
using Amazon.CDK.AWS.EC2;
using Constructs;

namespace Harbourline.Networking.Transit;

/// <summary>Transit Gateway with defaults off and one route table per segment (L1 constructs).</summary>
public sealed class TransitHub : Construct
{
    public TransitHub(Construct scope, string id) : base(scope, id)
    {
        Gateway = new CfnTransitGateway(this, "Tgw", new CfnTransitGatewayProps
        {
            AmazonSideAsn = 64600,
            DefaultRouteTableAssociation = "disable",
            DefaultRouteTablePropagation = "disable",
            AutoAcceptSharedAttachments = "disable",
            DnsSupport = "enable",
            Description = "Harbourline hub",
        });

        foreach (var segment in new[] { "prod", "nonprod", "shared", "onprem" })
        {
            RouteTables[segment] = new CfnTransitGatewayRouteTable(this, $"Rt-{segment}", new CfnTransitGatewayRouteTableProps
            {
                TransitGatewayId = Gateway.Ref,
                Tags = [new Amazon.CDK.CfnTag { Key = "Name", Value = $"rt-{segment}" }],
            });
        }
    }

    public CfnTransitGateway Gateway { get; }
    public Dictionary<string, CfnTransitGatewayRouteTable> RouteTables { get; } = [];

    /// <summary>Attach a VPC, associate it with one segment and propagate it into the listed segments.</summary>
    public CfnTransitGatewayAttachment Attach(string name, string vpcId, string[] subnetIds, string associateWith, params string[] propagateTo)
    {
        var attachment = new CfnTransitGatewayAttachment(this, $"Att-{name}", new CfnTransitGatewayAttachmentProps
        {
            TransitGatewayId = Gateway.Ref,
            VpcId = vpcId,
            SubnetIds = subnetIds,
            Tags = [new Amazon.CDK.CfnTag { Key = "Name", Value = $"att-{name}" }],
        });

        _ = new CfnTransitGatewayRouteTableAssociation(this, $"Assoc-{name}", new CfnTransitGatewayRouteTableAssociationProps
        {
            TransitGatewayAttachmentId = attachment.Ref,
            TransitGatewayRouteTableId = RouteTables[associateWith].Ref,
        });

        foreach (var segment in propagateTo)
        {
            _ = new CfnTransitGatewayRouteTablePropagation(this, $"Prop-{name}-{segment}", new CfnTransitGatewayRouteTablePropagationProps
            {
                TransitGatewayAttachmentId = attachment.Ref,
                TransitGatewayRouteTableId = RouteTables[segment].Ref,
            });
        }

        return attachment;
    }
}

Usage: hub.Attach("dev", devVpcId, devTgwSubnets, associateWith: "nonprod", "nonprod", "shared"); โ€” dev's traffic uses rt-nonprod, and dev is reachable from non-prod and shared services only.

Complete code

Before deploying a USD 0.05-per-attachment-hour router, simulate it. This .NET 10 file-based app models TGW route tables with associations, propagations, static and blackhole routes, and answers "can A reach B?" with longest-prefix match in both directions โ€” exactly how TGW forwards. It runs with no AWS account.

tgwsim.cs
using System.Net;

// Usage: dotnet run tgwsim.cs
// Models Harbourline's segmented Transit Gateway and checks a reachability matrix.

var attachments = new Dictionary<string, Attachment>
{
    ["payments"] = new("payments", "10.17.0.0/16", Associate: "prod",    PropagateTo: ["prod", "shared", "onprem"]),
    ["accounts"] = new("accounts", "10.18.0.0/16", Associate: "prod",    PropagateTo: ["prod", "shared", "onprem"]),
    ["test"]     = new("test",     "10.19.0.0/16", Associate: "nonprod", PropagateTo: ["nonprod", "shared"]),
    ["dev"]      = new("dev",      "10.20.0.0/16", Associate: "nonprod", PropagateTo: ["nonprod", "shared"]),
    ["hub"]      = new("hub",      "10.16.0.0/16", Associate: "shared",  PropagateTo: ["prod", "nonprod", "shared", "onprem"]),
    ["dc"]       = new("dc",       "10.0.0.0/13",  Associate: "onprem",  PropagateTo: ["prod", "shared"]),
};

// Static routes: (route table, CIDR, target attachment or null for blackhole)
var staticRoutes = new List<(string Table, string Cidr, string? Target)>
{
    ("nonprod", "10.17.0.0/16", null),   // explicit blackholes make the policy visible
    ("nonprod", "10.18.0.0/16", null),
};

// Build each route table: propagated routes first, then static routes (static wins on an exact-prefix tie in TGW).
var tables = new Dictionary<string, List<Route>>();
foreach (var a in attachments.Values)
{
    foreach (var t in a.PropagateTo)
    {
        Get(t).Add(new Route(Cidr.Parse(a.Cidr), a.Name, Static: false));
    }
}

foreach (var (table, cidr, target) in staticRoutes)
{
    var parsed = Cidr.Parse(cidr);
    Get(table).RemoveAll(r => r.Prefix.Equals(parsed));
    Get(table).Add(new Route(parsed, target, Static: true));
}

// Expected policy: who must and must not talk.
(string From, string To, bool Expected)[] checks =
[
    ("payments", "accounts", true),
    ("payments", "hub", true),
    ("payments", "dc", true),
    ("dev", "test", true),
    ("dev", "hub", true),
    ("dev", "payments", false),
    ("dev", "dc", false),
    ("dc", "test", false),
    ("dc", "accounts", true),
];

var failures = 0;
foreach (var (from, to, expected) in checks)
{
    var src = attachments[from];
    var dst = attachments[to];
    var forward = Lookup(src.Associate, FirstHost(dst.Cidr));
    var back = Lookup(dst.Associate, FirstHost(src.Cidr));
    var ok = forward == to && back == from;
    var verdict = ok == expected ? "PASS" : "FAIL";
    if (ok != expected)
    {
        failures++;
    }

    Console.WriteLine($"{verdict}  {from,-9} -> {to,-9} expected {(expected ? "reachable" : "blocked"),-9} " +
                      $"forward via rt-{src.Associate}: {forward ?? "drop"}, return via rt-{dst.Associate}: {back ?? "drop"}");
}

Console.WriteLine(failures == 0 ? "All policy checks passed." : $"{failures} policy check(s) failed.");
return failures == 0 ? 0 : 3;

List<Route> Get(string table) => tables.TryGetValue(table, out var list) ? list : tables[table] = [];

// Longest-prefix match; a blackhole route (Target null) that wins means "drop".
string? Lookup(string table, IPAddress address) =>
    Get(table).Where(r => r.Prefix.Contains(address))
              .OrderByDescending(r => r.Prefix.Length)
              .ThenByDescending(r => r.Static)
              .FirstOrDefault()?.Target;

static IPAddress FirstHost(string cidr)
{
    var c = Cidr.Parse(cidr);
    var bytes = c.Network.GetAddressBytes();
    bytes[3] += 10; // any address inside the range will do
    return new IPAddress(bytes);
}

record Attachment(string Name, string Cidr, string Associate, string[] PropagateTo);

record Route(Cidr Prefix, string? Target, bool Static);

readonly record struct Cidr(IPAddress Network, int Length)
{
    public static Cidr Parse(string text)
    {
        var parts = text.Split('/');
        return new Cidr(IPAddress.Parse(parts[0]), int.Parse(parts[1]));
    }

    public bool Contains(IPAddress address)
    {
        var mask = Length == 0 ? 0u : uint.MaxValue << (32 - Length);
        return (ToUInt(address) & mask) == (ToUInt(Network) & mask);
    }

    private static uint ToUInt(IPAddress a)
    {
        var b = a.GetAddressBytes();
        return (uint)(b[0] << 24 | b[1] << 16 | b[2] << 8 | b[3]);
    }
}

Expected output:

PASS  payments  -> accounts  expected reachable forward via rt-prod: accounts, return via rt-prod: payments
PASS  payments  -> hub       expected reachable forward via rt-prod: hub, return via rt-shared: payments
PASS  payments  -> dc        expected reachable forward via rt-prod: dc, return via rt-onprem: payments
PASS  dev       -> test      expected reachable forward via rt-nonprod: test, return via rt-nonprod: dev
PASS  dev       -> hub       expected reachable forward via rt-nonprod: hub, return via rt-shared: dev
PASS  dev       -> payments  expected blocked   forward via rt-nonprod: drop, return via rt-prod: drop
PASS  dev       -> dc        expected blocked   forward via rt-nonprod: drop, return via rt-onprem: drop
PASS  dc        -> test      expected blocked   forward via rt-onprem: drop, return via rt-nonprod: drop
PASS  dc        -> accounts  expected reachable forward via rt-onprem: accounts, return via rt-prod: dc
All policy checks passed.

Code explained line by line

  1. attachments โ€” each attachment has its CIDR, the one route table it is associated with, and the tables it propagates into. This mirrors the Terraform spokes map, so the simulator and the IaC can share a source of truth.
  2. dc uses 10.0.0.0/13, the summary of both data centres' /14s, as a Direct Connect gateway would advertise via BGP.
  3. staticRoutes adds blackhole routes for prod ranges in rt-nonprod. Target = null means drop.
  4. The first foreach builds propagated routes; the second removes any propagated route with the same prefix and adds the static one โ€” in TGW, a static route takes precedence over a propagated route for an identical prefix.
  5. checks is the connectivity policy written as data: pairs and whether they must be reachable.
  6. For each pair the simulator does two lookups: the forward lookup in the source's associated table, and the return lookup in the destination's associated table. Real traffic needs both โ€” a one-way route is a classic TGW mistake.
  7. Lookup implements longest-prefix match: among routes containing the address, the longest prefix wins; on a tie, static beats propagated.
  8. FirstHost picks an address inside the destination range (network + 10) to look up.
  9. The Cidr record struct does the bit arithmetic: build a mask from the prefix length and compare network bits.
  10. Exit code 3 on any failure lets the simulator run in CI whenever someone edits the connectivity map.

Try changing dev's PropagateTo to include "prod": dev -> payments still fails on the forward path (rt-nonprod has no route to prod, and the blackhole wins), demonstrating that you need both directions for reachability โ€” and that blackholes protect against well-meant mistakes.

Security considerations

โš  Learning shortcut โ€” not for production

Labs often create the Transit Gateway with default association and propagation enabled because it "just works". In production that puts every attachment in one routing domain, so any VPC can reach any other. Always disable both defaults and associate/propagate explicitly, as in the IaC above.

  • Routing is not filtering. A route allows packets to arrive; security groups and NACLs still decide whether they are accepted. Keep SGs tight even inside a segment.
  • Auto-accept off. With RAM sharing, set AutoAcceptSharedAttachments=disable so the network team reviews every new attachment.
  • Inspect east-west where it matters. Route traffic between segments (or between sensitive VPCs) through an inspection VPC with Network Firewall; enable appliance mode on that attachment (see AWS Network Firewall).
  • Log it. Enable Transit Gateway flow logs to the central log archive; they record traffic across attachments the VPC flow logs may not see.
  • Least privilege on the hub. Only the network team's pipeline role should be allowed ec2:*TransitGatewayRoute*, ec2:AssociateTransitGatewayRouteTable and similar actions; deny them elsewhere with an SCP.
  • Peering and SG references. Cross-VPC security-group references over peering (same Region) are convenient, but they bind two teams' security groups together; document them.

Banking use case

Harbourline's network team runs the Transit Gateway in the network hub account and shares it with all workload accounts through RAM. The connectivity map lives in Git as the Terraform spokes map, the tgwsim.cs checks run in the pull-request pipeline, and every change is approved by the network team and security.

  • Segments: prod, non-prod, shared, on-premises, plus a separate card-data segment whose route table contains only the shared inspection VPC and the card-processing VPCs.
  • On-premises: a Direct Connect gateway attachment carries 10.0.0.0/13 from the data centres. Only prod and shared services see it, so test environments can never reach core banking by accident.
  • High-volume pair: the Data Platform VPC and the analytics VPC exchange about 20 TB a month. That pair also has a VPC peering connection (no per-GB processing within an AZ), with a more-specific route on both sides so their bulk traffic bypasses the TGW.
  • Multi-Region: the EU subsidiary's Transit Gateway in eu-west-1 is peered with the us-east-1 TGW; static routes allow only shared services to cross. Data-residency rules for EU customer data are decided by Harbourline's data-protection officer, not by routing alone.

Common mistakes

  • Forgetting the return route โ€” the classic "ping one way" problem. Check both associated tables (the simulator does).
  • Leaving default association/propagation on, creating one flat network.
  • No TGW attachment subnet in an AZ where workloads run โ†’ traffic from that AZ can't use the TGW.
  • Overlapping CIDRs โ€” peering is refused; TGW routes become ambiguous. Fix the IP plan, or use PrivateLink/NAT.
  • Expecting peering to be transitive, or to carry VPN/DX traffic to the peer (edge-to-edge routing).
  • MTU surprises โ€” jumbo frames over peering versus 8500 on TGW; mismatch during migration drops large packets.
  • TGW peering with dynamic routing assumptions โ€” TGW peering uses static routes only.
  • Underestimating per-GB processing โ€” bulk data through a TGW costs USD 0.02/GB on every pass.

Troubleshooting

SymptomLikely causeFix
Peering stays pending-acceptanceNot accepted in the accepter account/RegionAccept it there; requests expire after 7 days
Peering created, no trafficMissing route on one side, or SG/NACLAdd routes both ways; reference peer SG or CIDR
Peer's private DNS names resolve to public IPsDNS resolution option offEnable AllowDnsResolutionFromRemoteVpc on both sides
TGW attachment pendingAcceptanceShared TGW with auto-accept offNetwork account accepts the attachment
One-way connectivity via TGWReturn route missing in the destination's associated tablePropagate the source into that table (or add a static route)
Some AZs can't use the TGWNo attachment subnet in that AZModify the attachment to add a subnet per AZ
Traffic dropped despite routeA blackhole or a more specific route winssearch-transit-gateway-routes and check longest-prefix
Inspection drops intermittent flowsAppliance mode off on inspection VPC attachmentEnable appliance mode
Large transfers stall after migration from peeringMTU mismatchAlign MTU; rely on PMTUD only on VPC/Connect attachments

Cost and cleanup

Prices (us-east-1, checked 2026-11-18):

  • VPC peering: no hourly charge. Data within the same AZ is free; across AZs USD 0.01/GB in each direction; inter-Region peering at inter-Region data-transfer rates.
  • Transit Gateway: USD 0.05 per attachment-hour and USD 0.02 per GB processed. Data arriving from a TGW peering attachment is not charged processing.

Examples:

  • Harbourline's 24 VPCs + 1 DX gateway attachment: 25 ร— 0.05 ร— 730 = USD 912.50/month in attachment-hours, plus 0.02 ร— traffic (10 TB/month = USD 200).
  • The same 20 TB/month Data Platform โ†” analytics pair: via TGW USD 400/month in processing; via same-AZ peering USD 0; cross-AZ peering USD 400 (0.01 each direction) โ€” so keep the heavy flows AZ-local or accept the cost.
  • Lab: one TGW with two VPC attachments for one hour โ‰ˆ USD 0.10 plus negligible data.

Free alternative: run tgwsim.cs, extend the policy checks, and draw the route tables. Create peering between two free lab VPCs if you want hands-on routing โ€” peering itself has no hourly cost.

Deploy-and-destroy (optional, under USD 0.20): two lab VPCs, one TGW, two attachments, one route table, test with Reachability Analyzer, delete within the hour:

$p = '--profile', 'learn-sandbox', '--region', 'us-east-1'
# Remove VPC routes that point to the TGW or peering first
aws ec2 delete-route @p --route-table-id rtb-0dev0000000000a --destination-cidr-block 10.16.0.0/12
# Delete attachments (wait until state = deleted), then route tables, then the TGW
aws ec2 delete-transit-gateway-vpc-attachment @p --transit-gateway-attachment-id tgw-attach-0dev000000000000
aws ec2 delete-transit-gateway-route-table @p --transit-gateway-route-table-id tgw-rtb-0nonprod00000000
aws ec2 delete-transit-gateway @p --transit-gateway-id tgw-0hub00000000000
# Peering
aws ec2 delete-vpc-peering-connection @p --vpc-peering-connection-id pcx-0devhub0000000000
# Nothing left?
aws ec2 describe-transit-gateways @p --filters Name=state,Values=available,pending --query "TransitGateways[].TransitGatewayId"
aws resourcegroupstaggingapi get-resources @p --tag-filters Key=Project,Values=cea-lab --query "ResourceTagMappingList[].ResourceARN"

Exercises

  • ๐ŸŸข Guided: Run tgwsim.cs. Then add a cards VPC (10.21.0.0/16) in a new cde segment that may reach only hub and dc, and add checks proving it cannot reach payments or dev.
  • ๐ŸŸก Challenge: Peer two free lab VPCs (10.20.0.0/16 and a new 10.19.0.0/16), add routes, enable DNS resolution, and use Reachability Analyzer to show a path from an ENI in one to an ENI in the other. Then add a third VPC peered only with the second and prove (with an analysis) that the first cannot reach it.
  • ๐Ÿ”ด Live problem: After onboarding a new Fraud Analytics VPC, the Fraud team can reach shared services (DNS, CI runners) without problems, but every call from the Payments VPC to the Fraud API times out. Security groups allow the traffic. The new attachment is associated with rt-prod and propagates only into rt-shared. Explain and fix.

Assignment

Commit to your portfolio: the Terraform spokes map for a five-VPC Harbourline estate, the tgwsim.cs simulator with a checks file that encodes your connectivity policy, and a README with the segment diagram and a monthly cost estimate.

Solutions

Knowledge check

Interview questions

  • "Why doesn't peering scale?" โ€” n(nโˆ’1)/2 connections, per-VPC route sprawl, the 50-per-VPC default quota, no transitive or edge routing, and no central place to see or change policy.
  • "How would you keep prod and non-prod apart on one Transit Gateway?" โ€” Separate route tables per segment, defaults off, explicit associations/propagations, blackhole routes to make the block visible, and a policy-as-code test like the simulator.
  • "What is appliance mode and when do you need it?" โ€” It keeps both directions of a flow on the same AZ's attachment ENI; needed for stateful appliances (inspection VPCs with Network Firewall or third-party firewalls).
  • "Transit Gateway vs Cloud WAN?" โ€” TGW is a Regional router you wire up yourself; Cloud WAN builds a global network from a policy with segments and attachment rules โ€” useful when you have many Regions and want central policy.

Certification practice

Completion checklist

  • I can explain non-transitive peering and edge-to-edge routing
  • I can predict TGW forwarding from associations and propagations
  • I ran tgwsim.cs and extended its policy checks
  • I know the TGW and peering pricing and when each wins
  • If I deployed anything, I deleted attachments, route tables, TGW and peerings and the tag check is clean

Key takeaways

  • Peering is a cheap, fast, two-VPC link โ€” non-transitive, no edge routing, and a mesh grows as nยฒ.
  • Transit Gateway is the Regional hub: attachments, route tables, associations (where my traffic goes) and propagations (who can reach me).
  • Segment with separate route tables and defaults off; blackhole routes make blocks explicit.
  • Check both directions; most TGW incidents are a missing return route.
  • Cost: USD 0.05 per attachment-hour plus USD 0.02 per GB โ€” keep very heavy flows on peering within an AZ.

Further reading

Next steps: connect the hub to the data centres in hybrid connectivity.

Independent learning resource โ€” not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page