Skip to Content
Cloud Engineering Academy
2. AWS Networking & Infrastructure

Hybrid connectivity: Site-to-Site VPN, Direct Connect and hybrid DNS

Connect AWS to on-premises data centres with Site-to-Site VPN and AWS Direct Connect, choose a resilient connection model for a bank, and make names resolve both ways with Route 53 Resolver endpoints and forwarding rules.

intermediateโฑ 1 h 50 min๐ŸŸก Low costSAA-C03SOA-C03SCS-C03

Assumes: .NET 10 LTS (file-based app) ยท AWS SDK for .NET v4 (AWSSDK.EC2 4.1.4) ยท AWS CLI v2 ยท CDK v2 (Amazon.CDK.Lib 2.277.0) ยท Terraform AWS provider v6 ยท Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Explain how a Site-to-Site VPN works on AWS โ€” customer gateway, two tunnels, virtual private gateway or Transit Gateway โ€” and choose static or dynamic (BGP) routing.
  • Explain AWS Direct Connect: dedicated versus hosted connections, private/transit/public virtual interfaces, Direct Connect gateways, LAGs and MACsec.
  • Pick a resiliency model (development and test, high, maximum) and design VPN as a backup for Direct Connect.
  • Make DNS work in both directions with Route 53 Resolver inbound and outbound endpoints and forwarding rules shared across accounts.
  • Monitor tunnel health from C# and estimate the monthly cost of a bank's hybrid network.

Prerequisites

  • CIDR planning โ€” the on-premises ranges 10.0.0.0/14 (LDN-DC1) and 10.4.0.0/14 (MAN-DC2).
  • VPC peering and Transit Gateway โ€” VPN and Direct Connect usually terminate on a Transit Gateway.
  • The DNS and Route 53 lesson listed in the module overview.
  • Tools: AWS CLI v2 with the learn-sandbox profile, .NET 10 SDK.

Why this matters in banking

Harbourline's core banking platform runs on-premises in London (LDN-DC1) and Manchester (MAN-DC2), and it will for years. The Accounts API on AWS must read balances from it; the payments hub on-premises must call fraud scoring on AWS; nightly batch files flow both ways; staff in branches use internal web apps hosted on AWS. All of this traffic carries customer data, and much of it is time-critical (a payment that misses a scheme cut-off is a customer complaint and sometimes a regulatory report).

So the bank needs connections to AWS that are private (not across the open internet unprotected), predictable (latency and bandwidth that don't depend on internet weather), resilient (no single cable, device or building whose failure stops payments) and observable (alarms before customers notice). And it needs DNS to work across the boundary: AWS workloads must resolve corebanking.ldn.harbourline.internal, and on-premises systems must resolve the private names of AWS services.

The idea in plain language

There are two ways to connect a data centre to AWS:

  • Site-to-Site VPN builds encrypted IPsec tunnels over the internet between your router and AWS. Quick to set up (hours), cheap, encrypted by design โ€” but performance depends on the internet path and each tunnel is limited (about 1.25 Gbps for a standard tunnel).
  • AWS Direct Connect is a physical, dedicated connection from your network to AWS at a Direct Connect location (a colocation data centre), often via a telecoms partner. Predictable latency and high bandwidth (1 to 400 Gbps), lower data-out prices โ€” but it takes weeks to provision, and it is not encrypted by default.

Banks typically use both: Direct Connect as the primary path, VPN as an encrypted backup (or IPsec running over Direct Connect when encryption is required on the link).

Routing between your network and AWS is usually exchanged with BGP (Border Gateway Protocol) โ€” the protocol routers use to tell each other which address ranges they can reach. For this course you only need the idea: each side advertises its prefixes, and when a link fails, its routes are withdrawn and traffic moves to the remaining links automatically.

Analogy: VPN is an armoured van on public roads โ€” secure and available today, but stuck in traffic sometimes. Direct Connect is the bank's own private rail line to the cash centre โ€” fast and predictable, but it takes months to lay, and you still want guards on board (encryption) if the cargo is sensitive. A sensible bank has the rail line and a van on standby.

What problem the AWS services solve

Site-to-Site VPNAWS Direct Connect
TransportIPsec over the internet (or over DX)Private fibre to a DX location
Time to provisionMinutes to hoursWeeks (partner, cross-connect, LOA-CFA)
BandwidthUp to 1.25 Gbps per standard tunnel; up to 5 Gbps per large-bandwidth tunnel; ECMP across tunnels on Transit Gateway1, 10, 100, 400 Gbps dedicated; 50 Mbpsโ€“25 Gbps hosted (partner)
EncryptionAlways (IPsec)Not by default โ€” add MACsec (layer 2, on supported dedicated ports) or IPsec over DX
Terminates onVirtual private gateway (one VPC) or Transit Gateway / Cloud WANPrivate VIF โ†’ VGW or DX gateway; transit VIF โ†’ DX gateway โ†’ Transit Gateway; public VIF โ†’ AWS public endpoints
RoutingStatic or BGPBGP only
MTU1446 bytes (MSS 1406), no Path MTU DiscoveryUp to 9001 (private VIF) / 8500 (transit VIF) โš  VERIFY
Price (checked 2026-11-18)USD 0.05 per connection-hour (two tunnels included); large-bandwidth 5 Gbps USD 0.60/hour; data out at internet ratesPort-hours: 1G USD 0.30/h, 10G USD 2.25/h, 100G USD 22.50/h, 400G USD 85/h (dedicated); data out from US to a US location USD 0.02/GB; data in free

Sharp edges:

  • A VPN connection to a virtual private gateway reaches one VPC. For many VPCs, terminate on a Transit Gateway.
  • VPN to a VGW is limited to 100 routes advertised from your side; a VPN on a Transit Gateway accepts up to 1,000 dynamic routes. Summarise your on-premises ranges.
  • ECMP (spreading traffic across several tunnels) works only on Transit Gateway with dynamic (BGP) routing.
  • Direct Connect alone is a single point of failure unless you buy redundancy (see resiliency models).
  • Direct Connect is a layer-2/3 circuit, not a security control: you still need encryption decisions, firewalls and monitoring.

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
VPN Gateway (S2S)Site-to-Site VPN on a VGW or Transit GatewayAWS gives two tunnels per connection, each in a different AZ endpoint
Local network gatewayCustomer gateway (CGW) resourceDescribes your router's public IP and BGP ASN
ExpressRoute circuitDirect Connect connection (dedicated or hosted)DX ports are per location; you order cross-connects
ExpressRoute private peeringPrivate VIF / transit VIFTransit VIF connects to Transit Gateway via a DX gateway
ExpressRoute Microsoft peeringPublic VIFReaches AWS public endpoints (e.g. S3) over DX
ExpressRoute Global ReachDX SiteLinkSite-to-site over the AWS backbone โš  VERIFY pricing
ExpressRoute GatewayDirect Connect gatewayGlobal object; associates with VGWs or Transit Gateways in many Regions
ExpressRoute MACsec / IPsec over ERMACsec on DX dedicated ports / VPN over DXSame choices, similar trade-offs
Azure DNS Private Resolver (inbound/outbound endpoints, forwarding rulesets)Route 53 Resolver inbound/outbound endpoints, forwarding rulesRules shared across accounts with AWS RAM

Architecture

Harbourline's target hybrid design

What makes this "maximum resiliency": two connections at each of two separate DX locations, terminating on different AWS devices, each data centre feeding a different location, plus VPN backup over the internet. Any single router, cable, DX device or whole DX location can fail without losing connectivity.

Resiliency models

ModelShapeProtects againstAWS SLA (per the DX SLA, if all its conditions are met)
Development and testTwo connections, separate devices, one locationDevice failureโ€” (not for critical workloads)
High resiliencyOne connection at each of two locationsDevice, fibre cut, one location failure99.9%
Maximum resiliencyTwo connections at each of two locations (four total)Device and location failure at the same time99.99%
VPN onlyTwo tunnels, one connection (ideally two CGWs)Tunnel endpoint failureSeparate VPN SLA โš  VERIFY

The Direct Connect Resiliency Toolkit in the console orders connections to match a model, prevents two connections landing on the same AWS device, and offers a failover test that brings down a BGP session so you can prove traffic moves.

Routing preference: DX first, VPN second

When the same prefix is learned over DX and VPN, AWS prefers Direct Connect over VPN for traffic to on-premises. On your side, you influence how AWS sends traffic with BGP attributes (for example, AS-path prepending to make a path less attractive) and how your network sends traffic to AWS with local preference. Keep the prefixes identical on both paths (or the VPN prefixes less specific): a more specific route over VPN would win over DX because longest-prefix match beats everything โš  VERIFY the full route-priority order in the Transit Gateway and DX documentation.

Hybrid DNS with Route 53 Resolver

  • Inbound endpoint: IP addresses in your VPC that on-premises DNS servers forward to. They answer from Route 53 private hosted zones and the VPC's view of AWS names (for example, private-DNS names of interface endpoints in the hub).
  • Outbound endpoint + forwarding rules: the VPC resolver forwards queries for chosen domains (for example ldn.harbourline.internal) to your on-premises DNS servers.
  • Share rules with AWS RAM so every workload account's VPCs use the same forwarding rules, while the endpoints live once in the hub account.
  • Each endpoint has at least two IP addresses (ENIs) in different AZs.

Step-by-step implementation

1. Agree the addressing and BGP plan

Prefixes AWS will learn: 10.0.0.0/13 (both data centres summarised) or the two /14s. Prefixes on-premises will learn: 10.16.0.0/12 (all of AWS). ASNs: Harbourline on-premises 65010/65020; Transit Gateway 64600. Write it down โ€” mismatched prefixes and ASNs cause most turn-up failures.

2. Create the customer gateway

One customer gateway per on-premises router: its public IP and BGP ASN (or a certificate for devices with dynamic IPs).

3. Create the VPN connection on the Transit Gateway

Choose dynamic routing, let AWS generate pre-shared keys (store them in Secrets Manager), and pick modern tunnel options (IKEv2, AES-256-GCM, SHA-384, DH groups 19/20 โš  VERIFY current recommended set). Download the device configuration for your router vendor.

4. Bring up both tunnels and BGP

Configure the router from the downloaded file. Both tunnels should show UP with BGP routes received. Configure the router to use both tunnels (ECMP on TGW requires dynamic routing on both sides).

5. Order Direct Connect

Use the Resiliency Toolkit, choose the model, locations and speed. You receive a Letter of Authorization and Connecting Facility Assignment (LOA-CFA); your partner or colocation provider uses it to order the cross-connect. Then create a Direct Connect gateway, transit VIFs on each connection, and associate the DX gateway with the Transit Gateway with allowed prefixes (what AWS advertises to you).

6. Route in the Transit Gateway

Associate the DX gateway and VPN attachments with rt-onprem and propagate them into the route tables that should reach on-premises (prod and shared โ€” see VPC peering and Transit Gateway).

7. Hybrid DNS

Create inbound and outbound Resolver endpoints in the hub VPC, forwarding rules for on-premises domains, share the rules with RAM, associate them with every spoke VPC, and configure conditional forwarders on-premises to the inbound endpoint IPs.

8. Test failover and monitor

Run the DX failover test, shut a VPN tunnel, and watch traffic move. Alarm on TunnelState, DX ConnectionState and BGP status.

AWS Console instructions

  • Customer gateway: Console โ†’ VPC โ†’ Virtual private network โ†’ Customer gateways โ†’ Create โ†’ BGP ASN, IP address, name hl-cgw-ldn-a, tags.
  • VPN connection: Site-to-Site VPN connections โ†’ Create โ†’ Target gateway type Transit gateway โ†’ choose the TGW and CGW โ†’ Routing Dynamic (requires BGP) โ†’ tunnel options โ†’ Create. Then Download configuration for your device.
  • Direct Connect: Console โ†’ Direct Connect โ†’ Create a connection โ†’ Connection wizard (Resiliency Toolkit) โ†’ choose model, speed, locations. Direct Connect gateways โ†’ Create. Virtual interfaces โ†’ Create โ†’ Transit.
  • Resolver endpoints: Console โ†’ Route 53 โ†’ Resolver โ†’ Inbound endpoints / Outbound endpoints โ†’ Create โ†’ VPC, security group (TCP and UDP 53), two subnets in different AZs. Rules โ†’ Create rule โ†’ Forward, domain, outbound endpoint, target IPs. Console โ†’ RAM โ†’ Create resource share to share the rule.

AWS CLI

$p = '--profile', 'learn-sandbox', '--region', 'us-east-1'

# Customer gateway for the London edge router (documentation IP range)
$cgw = aws ec2 create-customer-gateway @p --type ipsec.1 --bgp-asn 65010 --ip-address 198.51.100.10 `
  --tag-specifications "ResourceType=customer-gateway,Tags=[{Key=Name,Value=hl-cgw-ldn-a},{Key=Project,Value=cea-lab},{Key=Lab,Value=m02-l13}]" `
  --query CustomerGateway.CustomerGatewayId --output text

# VPN on the Transit Gateway with dynamic routing
$vpn = aws ec2 create-vpn-connection @p --type ipsec.1 --customer-gateway-id $cgw --transit-gateway-id tgw-0hub00000000000 `
  --options "StaticRoutesOnly=false,TunnelOptions=[{IKEVersions=[{Value=ikev2}]},{IKEVersions=[{Value=ikev2}]}]" `
  --tag-specifications "ResourceType=vpn-connection,Tags=[{Key=Name,Value=hl-vpn-ldn-a},{Key=Project,Value=cea-lab},{Key=Lab,Value=m02-l13}]" `
  --query VpnConnection.VpnConnectionId --output text

# Tunnel health (UP/DOWN) and routes learned
aws ec2 describe-vpn-connections @p --vpn-connection-ids $vpn `
  --query "VpnConnections[0].VgwTelemetry[].{Outside:OutsideIpAddress,Status:Status,Routes:AcceptedRouteCount,Since:LastStatusChange}" --output table

# Device configuration for a vendor (list types, then download)
aws ec2 get-vpn-connection-device-types @p --query "VpnConnectionDeviceTypes[?Vendor=='Generic']"

# Direct Connect: list connections and virtual interfaces (read-only)
aws directconnect describe-connections @p --query "connections[].{Id:connectionId,Name:connectionName,State:connectionState,Bandwidth:bandwidth,Location:location}"
aws directconnect describe-virtual-interfaces @p --query "virtualInterfaces[].{Id:virtualInterfaceId,Type:virtualInterfaceType,State:virtualInterfaceState,Bgp:bgpPeers[0].bgpStatus}"

# Route 53 Resolver: outbound endpoint and a forwarding rule for the London data centre zone
$out = aws route53resolver create-resolver-endpoint @p --creator-request-id hl-out-1 --name hl-outbound `
  --direction OUTBOUND --security-group-ids sg-0resolver0000000 `
  --ip-addresses SubnetId=subnet-0hubres00000000a SubnetId=subnet-0hubres00000000b `
  --query ResolverEndpoint.Id --output text
aws route53resolver create-resolver-rule @p --creator-request-id hl-rule-ldn --name hl-ldn-internal `
  --rule-type FORWARD --domain-name ldn.harbourline.internal --resolver-endpoint-id $out `
  --target-ips Ip=10.0.10.53,Port=53 Ip=10.0.11.53,Port=53

Infrastructure as Code

HybridDns.cs
using Amazon.CDK.AWS.EC2;
using Amazon.CDK.AWS.Route53Resolver;
using Constructs;

namespace Harbourline.Networking.Hybrid;

/// <summary>Inbound and outbound Resolver endpoints in the hub VPC plus one forwarding rule per on-premises zone.</summary>
public sealed class HybridDns : Construct
{
    public HybridDns(Construct scope, string id, IVpc hubVpc, string[] onPremDnsServers, string[] forwardedDomains)
        : base(scope, id)
    {
        var sg = new SecurityGroup(this, "ResolverSg", new SecurityGroupProps
        {
            Vpc = hubVpc,
            Description = "Route 53 Resolver endpoints: DNS to and from Harbourline networks only",
            AllowAllOutbound = false,
        });
        var harbourline = Peer.Ipv4("10.0.0.0/8");
        sg.AddIngressRule(harbourline, Port.Tcp(53), "DNS/TCP from Harbourline");
        sg.AddIngressRule(harbourline, Port.Udp(53), "DNS/UDP from Harbourline");
        foreach (var server in onPremDnsServers)
        {
            sg.AddEgressRule(Peer.Ipv4($"{server}/32"), Port.Tcp(53), "DNS/TCP to on-prem resolver");
            sg.AddEgressRule(Peer.Ipv4($"{server}/32"), Port.Udp(53), "DNS/UDP to on-prem resolver");
        }

        var subnets = hubVpc.SelectSubnets(new SubnetSelection { SubnetGroupName = "resolver" }).SubnetIds;
        var ipAddresses = subnets.Select(s => (object)new CfnResolverEndpoint.IpAddressRequestProperty { SubnetId = s }).ToArray();

        Inbound = new CfnResolverEndpoint(this, "Inbound", new CfnResolverEndpointProps
        {
            Name = "hl-inbound",
            Direction = "INBOUND",
            SecurityGroupIds = [sg.SecurityGroupId],
            IpAddresses = ipAddresses,
        });

        var outbound = new CfnResolverEndpoint(this, "Outbound", new CfnResolverEndpointProps
        {
            Name = "hl-outbound",
            Direction = "OUTBOUND",
            SecurityGroupIds = [sg.SecurityGroupId],
            IpAddresses = ipAddresses,
        });

        foreach (var domain in forwardedDomains)
        {
            var rule = new CfnResolverRule(this, $"Rule-{domain}", new CfnResolverRuleProps
            {
                Name = "fwd-" + domain.Replace('.', '-'),
                RuleType = "FORWARD",
                DomainName = domain,
                ResolverEndpointId = outbound.AttrResolverEndpointId,
                TargetIps = onPremDnsServers.Select(ip => (object)new CfnResolverRule.TargetAddressProperty { Ip = ip, Port = "53" }).ToArray(),
            });

            _ = new CfnResolverRuleAssociation(this, $"Assoc-{domain}", new CfnResolverRuleAssociationProps
            {
                ResolverRuleId = rule.AttrResolverRuleId,
                VpcId = hubVpc.VpcId,
            });
        }
    }

    public CfnResolverEndpoint Inbound { get; }
}

Share the rules to workload accounts with Amazon.CDK.AWS.RAM.CfnResourceShare and associate them in each spoke VPC (in that account's stack).

Complete code

The SOC needs to know a tunnel is down before payments slow down. This .NET 10 file-based app checks every Site-to-Site VPN connection in the account, prints each tunnel's state and accepted routes, and exits with 3 if any connection has fewer than two tunnels up โ€” suitable for a scheduled pipeline job or a container on a timer. (In production you would also alarm on the CloudWatch TunnelState metric.)

vpnhealth.cs
#:package AWSSDK.EC2@4.1.4
#:package AWSSDK.SSO@4.1.0.2
#:package AWSSDK.SSOOIDC@4.1.0.2

using Amazon;
using Amazon.EC2;
using Amazon.EC2.Model;
using Amazon.Runtime.CredentialManagement;

// Usage: dotnet run vpnhealth.cs [--min-up 2] [--tag-key Name --tag-prefix hl-vpn-]
var minUp = 2;
string? tagPrefix = null;
for (var i = 0; i + 1 < args.Length; i += 2)
{
    switch (args[i])
    {
        case "--min-up": minUp = int.Parse(args[i + 1]); break;
        case "--tag-prefix": tagPrefix = args[i + 1]; break;
    }
}

if (!new CredentialProfileStoreChain().TryGetAWSCredentials("learn-sandbox", out var credentials))
{
    Console.Error.WriteLine("Profile learn-sandbox not found. Run: aws sso login --profile learn-sandbox");
    return 1;
}

using var ec2 = new AmazonEC2Client(credentials, RegionEndpoint.USEast1);
var response = await ec2.DescribeVpnConnectionsAsync(new DescribeVpnConnectionsRequest
{
    Filters = [new Filter { Name = "state", Values = ["available"] }],
});

var degraded = 0;
foreach (var vpn in response.VpnConnections ?? [])
{
    var name = vpn.Tags?.FirstOrDefault(t => t.Key == "Name")?.Value ?? vpn.VpnConnectionId;
    if (tagPrefix is not null && !name.StartsWith(tagPrefix, StringComparison.Ordinal))
    {
        continue;
    }

    var tunnels = vpn.VgwTelemetry ?? [];
    var up = tunnels.Count(t => t.Status == TelemetryStatus.UP);
    var target = vpn.TransitGatewayId ?? vpn.VpnGatewayId ?? "unattached";
    var routing = vpn.Options?.StaticRoutesOnly == true ? "static" : "BGP";
    var healthy = up >= minUp;
    if (!healthy)
    {
        degraded++;
    }

    Console.WriteLine($"{(healthy ? "OK  " : "WARN")} {name,-22} {vpn.VpnConnectionId} -> {target} ({routing}) tunnels up {up}/{tunnels.Count}");
    foreach (var t in tunnels.OrderBy(t => t.OutsideIpAddress))
    {
        var since = t.LastStatusChange is { } changed ? $"{DateTime.UtcNow - changed.ToUniversalTime():d\\.hh\\:mm} ago" : "unknown";
        Console.WriteLine($"       {t.OutsideIpAddress,-16} {t.Status,-5} routes={t.AcceptedRouteCount,-4} changed {since} {t.StatusMessage}");
    }
}

if (response.VpnConnections is null or { Count: 0 })
{
    Console.WriteLine("No available VPN connections in us-east-1.");
}

Console.WriteLine(degraded == 0 ? "RESULT: all VPN connections have redundant tunnels up." : $"RESULT: {degraded} connection(s) degraded.");
return degraded == 0 ? 0 : 3;

Example output against a lab VPN whose customer gateway does not exist (both tunnels down, as expected):

WARN hl-vpn-ldn-a           vpn-0abc12345def67890 -> tgw-0hub00000000000 (BGP) tunnels up 0/2
       3.210.x.x        DOWN  routes=0    changed 0.00:12 ago
       52.4.x.x         DOWN  routes=0    changed 0.00:12 ago
RESULT: 1 connection(s) degraded.

Code explained line by line

  1. #:package AWSSDK.EC2@4.1.4 โ€” VPN connections are part of the EC2 API in the SDK.
  2. The for loop reads two optional flags: --min-up (how many tunnels must be up; two is the redundant baseline) and --tag-prefix (to check only Harbourline's connections).
  3. Credentials come from the SSO profile, never from code.
  4. DescribeVpnConnectionsAsync with a state = available filter skips connections being deleted.
  5. VgwTelemetry holds one entry per tunnel โ€” despite the name, it is filled for VPNs on a Transit Gateway too. In SDK v4 it may be null, hence ?? [].
  6. TelemetryStatus.UP is the SDK's constant class for the tunnel state.
  7. AcceptedRouteCount shows how many BGP routes AWS learned over that tunnel โ€” a tunnel that is UP with zero routes usually means BGP is not established or the router filters advertisements.
  8. LastStatusChange tells you whether a tunnel is flapping (changed minutes ago) or stable.
  9. Exit code 3 when any connection has fewer than minUp tunnels up, so a scheduler can page someone.

Security considerations

โš  Learning shortcut โ€” not for production

The optional lab VPN uses AWS-generated pre-shared keys that you can see in the downloaded configuration file. In production, store and rotate them in Secrets Manager (or use certificate-based authentication with AWS Private CA), never email the configuration file, and restrict ec2:GetVpnConnectionDeviceSampleConfiguration and ec2:DescribeVpnConnections (which can expose keys) to the network team.

  • Encrypt Direct Connect traffic if policy requires encryption in transit on all external links. Options: MACsec on supported dedicated ports (line-rate, layer 2, between your device and the AWS device), or IPsec VPN over a transit/private VIF (limited by VPN tunnel throughput). Many banks also rely on TLS end to end; agree the policy with security.
  • Filter routes. Advertise only the summary prefixes you intend; on the DX gateway association use allowed prefixes; on your routers filter what you accept. A leaked default route can send AWS traffic through your data centre, or vice versa.
  • Segment at the Transit Gateway. The on-premises attachments sit in their own route table and are propagated only into segments that should reach the data centres (see the previous lesson).
  • Inspect. Route on-premises traffic to sensitive VPCs through an inspection VPC (AWS Network Firewall).
  • DNS is a trust boundary. Restrict Resolver endpoint security groups to known DNS servers; log queries with Resolver query logging; consider DNS Firewall on outbound queries.
  • Monitor. Alarms on TunnelState, DX ConnectionState, VirtualInterfaceBpsEgress, BGP state changes, and CloudTrail events for VPN/DX/route changes.

Banking use case

Harbourline's network hub account owns the Transit Gateway, the DX gateway and the Resolver endpoints:

  • Primary path: maximum resiliency DX โ€” two 10G connections at each of two DX locations, in LAGs, with MACsec enabled; transit VIFs to the DX gateway; DX gateway associated with the TGW advertising 10.16.0.0/12.
  • Backup path: two Site-to-Site VPN connections (London and Manchester edge routers) to the TGW, BGP with ECMP, advertising the same summaries. Quarterly failover tests use the DX failover test and a planned VPN tunnel shutdown; results go to the operational-resilience team as evidence for the bank's important business services mapping.
  • DNS: on-premises DNS forwards aws.harbourline.internal and AWS service names to the inbound endpoint; forwarding rules for ldn.harbourline.internal and man.harbourline.internal are shared to all accounts with RAM.
  • Change control: BGP and prefix changes are pull requests against the network repo; the TGW simulator from the previous lesson includes the dc attachment so segmentation is tested before each change.

Whether this design satisfies the bank's operational-resilience or outsourcing obligations is for Harbourline's risk and compliance functions to decide; the architecture provides the redundancy and evidence they need.

Common mistakes

  • One DX connection and calling it resilient. A single port, cross-connect or location is a single point of failure.
  • VPN on a VGW for many VPCs. A VGW serves one VPC; use a Transit Gateway.
  • More specific prefixes on the VPN than on DX, so traffic unexpectedly prefers the VPN.
  • Static routing with two tunnels and assuming automatic failover/ECMP. Use BGP.
  • Forgetting MTU/MSS on VPN (1446/1406) โ†’ large packets fail mysteriously.
  • Treating DX as encrypted. It isn't by default.
  • Resolver forwarding loops: on-premises forwards a zone to the inbound endpoint, and a rule in AWS forwards the same zone back on-premises.
  • Resolver endpoint security group that blocks UDP 53 or TCP 53 (large responses and zone transfers use TCP).

Troubleshooting

SymptomLikely causeFix
Tunnel DOWNIKE mismatch, wrong PSK, firewall blocking UDP 500/4500 or ESPCompare tunnel options with the device config; check StatusMessage; enable VPN tunnel logs to CloudWatch Logs
Tunnel UP, 0 accepted routesBGP not established or routes filteredCheck BGP neighbour/ASN; advertise prefixes; check route filters
Traffic only uses one tunnelStatic routing, or ECMP not configuredDynamic routing on TGW, ECMP on the router
Large file transfers fail over VPNMTU/MSSSet MSS clamp 1406 or lower per algorithm guidance
DX VIF downBGP not up, VLAN or IP mismatch, LOA not completedCheck VIF BGP peer config and cross-connect status
On-prem can't resolve AWS namesNo conditional forwarder to inbound endpoint, or SG blocks 53Configure forwarder; allow TCP/UDP 53 from on-prem DNS
AWS can't resolve on-prem namesRule not associated with the VPC, or outbound SG blocks egressAssociate the shared rule; allow egress 53 to target IPs
Traffic prefers VPN over DXMore specific prefixes over VPNAdvertise identical prefixes on both

Cost and cleanup

Prices (checked 2026-11-18):

  • Site-to-Site VPN: USD 0.05 per connection-hour (about USD 36/month); large-bandwidth (5 Gbps) USD 0.60/hour; accelerated VPN adds two Global Accelerators (USD 0.05/hour total) plus data-transfer premium; each tunnel's public IPv4 address bills at standard rates. Data out at internet rates (first 100 GB/month free, then from USD 0.09/GB).
  • Transit Gateway VPN attachment: USD 0.05 per hour plus USD 0.02 per GB processed (see previous lesson).
  • Direct Connect: dedicated port-hours 1G USD 0.30, 10G USD 2.25, 100G USD 22.50, 400G USD 85; hosted from USD 0.03/h (50 Mbps); data out from US Regions to US locations USD 0.02/GB; data in free. Partner and cross-connect fees are extra and outside AWS pricing.
  • Route 53 Resolver endpoints: USD 0.125 per ENI-hour (an endpoint needs at least two: USD 182.50/month per endpoint) plus USD 0.40 per million queries through endpoints.

Harbourline estimate: 4 ร— 10G DX ports = 4 ร— 2.25 ร— 730 = USD 6,570/month; 2 VPN connections + 2 TGW VPN attachments โ‰ˆ USD 146; 1 DX gateway attachment USD 36.50; inbound + outbound Resolver endpoints (2 ENIs each) = USD 365; data out 30 TB via DX at USD 0.02 = USD 600. Roughly USD 7,700/month before partner and colocation charges.

Free alternative: design-only โ€” the BGP/addressing plan, the resiliency diagram, the Resolver rule design and the cost estimate above. All exam-relevant knowledge is in the design.

Optional deploy-and-destroy (about USD 0.06 for an hour): create a customer gateway with a documentation IP and a VPN connection to a virtual private gateway on a lab VPC, download the configuration, run vpnhealth.cs (tunnels DOWN is expected), then delete. No Transit Gateway needed.

$p = '--profile', 'learn-sandbox', '--region', 'us-east-1'
aws ec2 delete-vpn-connection @p --vpn-connection-id vpn-0abc12345def67890
# wait until the VPN connection is 'deleted', then:
aws ec2 delete-customer-gateway @p --customer-gateway-id cgw-0abc12345def67890
aws ec2 detach-vpn-gateway @p --vpn-gateway-id vgw-0abc12345def67890 --vpc-id vpc-0123456789abcdef0
aws ec2 delete-vpn-gateway @p --vpn-gateway-id vgw-0abc12345def67890
aws resourcegroupstaggingapi get-resources @p --tag-filters Key=Project,Values=cea-lab --query "ResourceTagMappingList[].ResourceARN"

Resolver endpoints bill per ENI-hour from creation: if you try them, delete them in the same session (aws route53resolver delete-resolver-endpoint --resolver-endpoint-id โ€ฆ) after removing rule associations.

Exercises

  • ๐ŸŸข Guided: Draw Harbourline's hybrid design for the high resiliency model (one connection at each of two locations) with VPN backup, including prefixes advertised in each direction and the TGW route table each attachment is associated with.
  • ๐ŸŸก Challenge: Design hybrid DNS for three zones: aws.harbourline.internal (Route 53 private hosted zone in the hub), ldn.harbourline.internal and man.harbourline.internal (on-premises). List the endpoints, rules, RAM shares, VPC associations and on-premises conditional forwarders, and show how you avoid a forwarding loop.
  • ๐Ÿ”ด Live problem: On Monday morning branch staff report that the internal statements portal (hosted on AWS) is slow and sometimes times out when downloading PDFs larger than about 2 MB; small pages are fine. The network team says "DX had planned maintenance at location 1 overnight". Diagnose.

Assignment

Write a two-page hybrid connectivity design for Harbourline: addressing and BGP plan, resiliency model and justification, encryption decision, DNS design, monitoring and alarms, failover test plan, and monthly cost. Add the vpnhealth.cs tool with a README to your portfolio repository.

Solutions

Knowledge check

Interview questions

  • "VPN or Direct Connect for a bank?" โ€” Both: DX for predictable bandwidth and latency, VPN as an encrypted backup and for quick starts. Encryption on DX via MACsec or IPsec depending on policy.
  • "How do you make Direct Connect highly available?" โ€” Two DX locations, separate devices, ideally two connections per location (maximum resiliency), BGP failover, VPN backup, and regular failover tests with evidence.
  • "How does an app on AWS resolve an on-premises hostname?" โ€” Route 53 Resolver outbound endpoint with a forwarding rule for the on-prem domain, shared to accounts via RAM and associated with the VPC.
  • "Why might traffic prefer the VPN over Direct Connect?" โ€” More specific prefixes advertised over the VPN; longest-prefix match wins before path preference.

Certification practice

Completion checklist

  • I can explain VPN tunnels, customer gateways and VGW versus TGW termination
  • I can describe dedicated/hosted DX, the three VIF types and the DX gateway
  • I can draw the high and maximum resiliency models with VPN backup
  • I can design Resolver inbound/outbound endpoints and shared forwarding rules
  • If I created a lab VPN or Resolver endpoint, I deleted it and the tag check is clean

Key takeaways

  • VPN: encrypted, fast to set up, bandwidth-limited per tunnel; DX: private, predictable, high bandwidth, not encrypted by default.
  • Terminate on a Transit Gateway for many VPCs; use BGP for failover and ECMP.
  • Resilience comes from separate devices and separate locations, plus a tested backup path.
  • Hybrid DNS: inbound for on-prem โ†’ AWS, outbound + rules (shared via RAM) for AWS โ†’ on-prem.
  • Monitor tunnels, BGP and DX state, and test failover with evidence.

Further reading

Next steps: record and investigate what crosses these links with VPC Flow Logs, then put it all together in multi-account network architecture.

Independent learning resource โ€” not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page