Skip to Content
Cloud Engineering Academy
3. .NET Application Development on AWS

ASP.NET Core Web API on AWS: the baseline

Prepare an ASP.NET Core API to run behind an Application Load Balancer: Kestrel settings, trusted forwarded headers, TLS terminated at the load balancer, health endpoints, graceful shutdown and the structure of Harbourline's reference Accounts API.

intermediate⏱ 1 h 30 min🟢 FreeDVA-C03SAA-C03

Assumes: .NET 10 LTS (ASP.NET Core 10) · xUnit v3 4.0.3 · Microsoft.AspNetCore.Mvc.Testing 10.0.15 · Application Load Balancer · Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Describe the request path from a customer's browser to Kestrel behind an Application Load Balancer (ALB), and which component owns TLS, routing, health checks and client identity at each hop.
  • Configure the forwarded headers middleware so the app sees the real client IP and scheme, and only trusts those headers from the load balancer's network.
  • Set Kestrel limits and keep-alive values that match the ALB's idle timeout, so you avoid intermittent 502 errors.
  • Expose separate liveness and readiness endpoints and explain which one the load balancer should call.
  • Make the app shut down gracefully within the orchestrator's stop window, so deployments drop no requests.
  • Find your way around Harbourline's reference Accounts API (labs/m03-accounts-api) and say why each part of its startup pipeline exists.

Prerequisites

Why this matters in banking

Harbourline's fraud team noticed that every login attempt in the audit log came from the same three IP addresses, all starting with 10.0.. Those were the load balancer's private addresses. For six weeks, the velocity rules that block "too many failed logins from one IP" were blocking the load balancer, and real attackers were invisible. The fix was four lines of configuration. Nobody had added it, because everything "worked" in testing.

The opposite mistake is just as bad. Another team switched forwarded headers on with "trust everything", so anyone on the internet could send X-Forwarded-For: 10.20.30.40 and appear to come from the office network that an internal admin endpoint allowed.

Other baseline problems show up as operational noise: a few 502 errors an hour that nobody can reproduce (keep-alive mismatch), requests dropped at every deployment (no graceful shutdown), and containers restarted in a loop because the health check called the database (liveness and readiness mixed up). Each of these is a small setting. Together they decide whether your API is boring to run, which is the goal.

The idea in plain language

Kestrel is the web server inside every ASP.NET Core app. In AWS you almost never expose Kestrel directly to the internet. A load balancer sits in front of it:

  • The ALB accepts HTTPS from customers on port 443, holds the TLS certificate (from AWS Certificate Manager, ACM), and forwards each request over plain HTTP inside the VPC to one of several healthy copies of your app on port 8080.
  • Because the ALB opens a new connection to your app, the app sees the ALB as the client. The ALB adds X-Forwarded-For (the real client IP) and X-Forwarded-Proto (https) headers so the app can recover the original details.
  • The ALB repeatedly calls a health check URL on each copy and stops sending traffic to copies that fail.

Analogy: the ALB is the bank's reception desk. Customers show ID at reception (TLS), and reception walks them to a free adviser (a healthy target), with a visitor slip that says who they are and how they arrived (forwarded headers). The adviser should only believe slips written by reception, not slips customers bring from home. And before an adviser goes home, reception stops sending them customers, so they can finish the conversations they already have (graceful shutdown).

What problem the baseline solves

Splitting responsibilities between the ALB and the app:

ConcernOwned by the ALBOwned by your ASP.NET Core app
TLS certificate and protocols✅ ACM certificate, TLS security policyOnly if you choose end-to-end TLS to targets
HTTP → HTTPS redirect✅ Listener rule on port 80❌ Do not use UseHttpsRedirection behind the ALB unless headers are trusted
Real client IP and schemeAdds X-Forwarded-For / -Proto / -PortApplies them via UseForwardedHeaders, trusting only the VPC range
Routing to healthy copies✅ Target group, health checksProvides /health/ready
Idle connection handlingIdle timeout (default 60 s)Keep-alive timeout longer than the ALB's
Request size and header limitsSome fixed limitsKestrel MaxRequestBodySize, header timeouts
HSTS, security headers, ProblemDetails❌✅
Authentication and authorisationOptional (OIDC on the ALB)✅ JWT bearer and policies (later in this module)
Draining during deploymentsDeregistration delayHostOptions.ShutdownTimeout, finish in-flight requests

Limitations to know:

  • The ALB does not validate the certificate on an HTTPS target. End-to-end TLS inside the VPC gives encryption in transit, not strong authentication of the target.
  • ALB health checks are simple HTTP calls. They cannot tell "the database is slow" from "the app is dead" unless your endpoints make that distinction.
  • X-Forwarded-For can contain values the client made up. Only the rightmost entry, added by your ALB, is trustworthy without other trusted proxies.

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
IIS + ARR or Windows NLB in front of web serversApplication Load Balancer + target groupManaged, multi-AZ, scales automatically; billed per hour plus LCUs
Azure Application GatewayApplication Load BalancerPath and host routing, ACM certificates; WAF is a separate attachment (AWS WAF)
App Service front ends adding X-Forwarded-*ALB adding X-Forwarded-For, -Proto, -PortYou must enable and scope forwarded-headers processing in the app yourself
App Service health check pathTarget group health check (path, interval, thresholds)Health checks decide routing and, in ECS, task replacement
App Service managed certificates / Key Vault certificatesAWS Certificate Manager (public certificates free on ALB)Certificates stay on the ALB; you never install them in the container
web.config request limitsKestrel limits in code or configurationKestrel runs in the container; there is no IIS layer
App Service "Always On" and slot swap drainingDeregistration delay + ECS stop timeout + HostOptions.ShutdownTimeoutThree timers must agree; see the shutdown diagram

Architecture

Security-boundary view:

  • Internet → ALB: TLS terminates at the ALB, with a TLS security policy that allows only TLS 1.2 and 1.3. AWS WAF filters common attacks before the ALB routes anything. Port 80 only redirects.
  • ALB → app: inside the VPC. The app's security group allows port 8080 only from the ALB's security group, so nothing else can reach Kestrel or forge forwarded headers. The app trusts X-Forwarded-* only from the VPC range as a second layer.
  • App → AWS services: through VPC endpoints or a NAT gateway, signed with the task role's temporary credentials.

What happens during a deployment or scale-in:

Step-by-step implementation

Run the reference API locally

cd labs/m03-accounts-api
dotnet run --project src/Harbourline.Accounts.Api
curl http://localhost:5080/health/live
curl http://localhost:5080/v1/accounts/ACC-0000000001

In Development, the reference API uses the in-memory repository and Auth:Mode=Disabled. It refuses to start that way in any other environment unless you explicitly set a learning-shortcut flag. Notice the X-Correlation-ID header on every response and the JSON log lines in the console.

Trust forwarded headers from the VPC only

Configure ForwardedHeadersOptions with ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto, clear the defaults, and add the VPC CIDR as a known network. Call app.UseForwardedHeaders() first in the pipeline.

Set Kestrel limits and keep-alive

Keep KeepAliveTimeout above the ALB idle timeout (60 s by default). Set MaxRequestBodySize to what your API really accepts. Turn off the Server header.

Separate liveness from readiness

/health/live returns 200 if the process is running and has no dependencies. /health/ready checks what the app needs to serve requests. The ALB uses readiness. Container restarts use liveness, or nothing.

Make shutdown graceful

Set HostOptions.ShutdownTimeout a few seconds below the orchestrator's stop timeout. Set the target group's deregistration delay to something realistic for your request durations (30 s for a fast API, not the 300 s default).

Simulate the ALB locally

Run the baseline app with Hosting__TrustedProxyNetworks__0=127.0.0.1/32 and Diagnostics__EnableRequestInfo=true, then:

curl -H "X-Forwarded-For: 203.0.113.50" -H "X-Forwarded-Proto: https" http://localhost:5000/diagnostics/request
# {"scheme":"https","clientIp":"203.0.113.50","host":"localhost:5000"}

Remove the trusted network setting and repeat: the scheme becomes http and the client IP becomes 127.0.0.1. The headers are ignored, as they should be from an untrusted source.

AWS Console instructions

You will create a full ALB in Deploying .NET to ECS on Fargate. These are the settings to check on any ALB fronting .NET:

  • EC2 → Load Balancers → (your ALB) → Listeners and rules. Port 443 HTTPS with an ACM certificate and a TLS 1.3/1.2 security policy (for example ELBSecurityPolicy-TLS13-1-2-2021-06; newer policies may exist ⚠ VERIFY the current recommended policy). Port 80 has one default action: Redirect to HTTPS 443, status 301.
  • (your ALB) → Attributes → Edit.
    • Connection idle timeout: 60 s unless you have long downloads.
    • Drop invalid header fields: On.
    • X-Forwarded-For header: Append (default).
    • Desync mitigation mode: Defensive (default) or Strictest.
  • EC2 → Target groups → (your TG) → Health checks. Path /health/ready, success code 200, interval 15 s, timeout 5 s, healthy threshold 2, unhealthy threshold 3.
  • (your TG) → Attributes. Deregistration delay 30 s.
  • (your TG) → Targets. Each target should show healthy. If one shows unhealthy, the Health status details column gives the reason, for example Health checks failed with these codes: [404].

AWS CLI

# Look up your ALB and target group (names from the Fargate lesson).
ALB_ARN=$(aws elbv2 describe-load-balancers --names hl-m03-accounts-alb \
  --query "LoadBalancers[0].LoadBalancerArn" --output text --profile learn-sandbox)
TG_ARN=$(aws elbv2 describe-target-groups --names hl-m03-accounts-tg \
  --query "TargetGroups[0].TargetGroupArn" --output text --profile learn-sandbox)

# Baseline ALB attributes: idle timeout, drop malformed headers, append client IP to XFF.
aws elbv2 modify-load-balancer-attributes --load-balancer-arn "$ALB_ARN" --profile learn-sandbox --attributes \
  Key=idle_timeout.timeout_seconds,Value=60 \
  Key=routing.http.drop_invalid_header_fields.enabled,Value=true \
  Key=routing.http.xff_header_processing.mode,Value=append

# Health check on the readiness endpoint.
aws elbv2 modify-target-group --target-group-arn "$TG_ARN" --profile learn-sandbox \
  --health-check-path /health/ready --health-check-interval-seconds 15 --health-check-timeout-seconds 5 \
  --healthy-threshold-count 2 --unhealthy-threshold-count 3 --matcher HttpCode=200

# Drain for 30 s instead of the 300 s default.
aws elbv2 modify-target-group-attributes --target-group-arn "$TG_ARN" --profile learn-sandbox \
  --attributes Key=deregistration_delay.timeout_seconds,Value=30

# Why is a target unhealthy?
aws elbv2 describe-target-health --target-group-arn "$TG_ARN" --profile learn-sandbox \
  --query "TargetHealthDescriptions[].{Target:Target.Id,State:TargetHealth.State,Reason:TargetHealth.Reason,Detail:TargetHealth.Description}"

Infrastructure as Code

The Fargate lesson builds the whole service. Here are just the baseline settings for the ALB, listeners and target group.

AlbBaseline.cs (excerpt inside a Stack)
// vpc, certificate and the Fargate service are created elsewhere in the stack.
var alb = new ApplicationLoadBalancer(this, "Alb", new ApplicationLoadBalancerProps
{
    Vpc = vpc,
    InternetFacing = true,
    VpcSubnets = new SubnetSelection { SubnetType = SubnetType.PUBLIC },
    IdleTimeout = Duration.Seconds(60),
    DropInvalidHeaderFields = true,
});

// HTTP only redirects.
alb.AddRedirect(new ApplicationLoadBalancerRedirectConfig { SourcePort = 80, TargetPort = 443 });

var https = alb.AddListener("Https", new BaseApplicationListenerProps
{
    Port = 443,
    Certificates = [ListenerCertificate.FromCertificateManager(certificate)],
    SslPolicy = SslPolicy.RECOMMENDED_TLS,
    Open = true,
});

var targets = https.AddTargets("Api", new AddApplicationTargetsProps
{
    Port = 8080,
    Protocol = ApplicationProtocol.HTTP,
    Targets = [fargateService],
    DeregistrationDelay = Duration.Seconds(30),
    HealthCheck = new Amazon.CDK.AWS.ElasticLoadBalancingV2.HealthCheck
    {
        Path = "/health/ready",
        HealthyHttpCodes = "200",
        Interval = Duration.Seconds(15),
        Timeout = Duration.Seconds(5),
        HealthyThresholdCount = 2,
        UnhealthyThresholdCount = 3,
    },
});

The lab's compiled version of these settings is in labs/m03-accounts-api/infra/cdk/Harbourline.Accounts.Infra/FargateStack.cs. It uses the higher-level ApplicationLoadBalancedFargateService pattern and then adjusts the health check and deregistration delay in the same way.

Complete code

A small, self-contained baseline API that shows every setting in one file. The reference Accounts API in labs/m03-accounts-api applies the same ideas at full size. This project compiles with TreatWarningsAsErrors, and the tests below pass (verified with dotnet test in the .NET 10 SDK container).

Harbourline.Baseline.Api.csproj
<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
    <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
    <RootNamespace>Harbourline.Baseline.Api</RootNamespace>
    <InvariantGlobalization>true</InvariantGlobalization>
  </PropertyGroup>
</Project>
appsettings.json
{
  "Logging": { "LogLevel": { "Default": "Information", "Microsoft.AspNetCore": "Warning" } },
  "AllowedHosts": "*",
  "Accounts": { "Source": "InMemory" },
  "Hosting": { "TrustedProxyNetworks": [ "10.0.0.0/16" ] },
  "Diagnostics": { "EnableRequestInfo": false }
}
Program.cs
using System.Net;
using Microsoft.AspNetCore.Diagnostics.HealthChecks;
using Microsoft.AspNetCore.HttpOverrides;
using Microsoft.Extensions.Diagnostics.HealthChecks;

var builder = WebApplication.CreateBuilder(args);

// 1. Kestrel: HTTP only inside the VPC (TLS ends at the load balancer), small limits, no Server header.
builder.WebHost.ConfigureKestrel(kestrel =>
{
    kestrel.AddServerHeader = false;
    kestrel.Limits.MaxRequestBodySize = 1 * 1024 * 1024;           // 1 MB is plenty for this API
    kestrel.Limits.KeepAliveTimeout = TimeSpan.FromSeconds(75);     // must be LONGER than the ALB idle timeout (60 s)
    kestrel.Limits.RequestHeadersTimeout = TimeSpan.FromSeconds(15);
});

// 2. Graceful shutdown: finish in-flight requests before the orchestrator's SIGKILL (ECS default 30 s).
builder.Services.Configure<HostOptions>(o => o.ShutdownTimeout = TimeSpan.FromSeconds(25));

// 3. Trust X-Forwarded-For / X-Forwarded-Proto only from the VPC's address range.
builder.Services.Configure<ForwardedHeadersOptions>(o =>
{
    o.ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto;
    o.KnownIPNetworks.Clear();
    o.KnownProxies.Clear();
    foreach (var cidr in builder.Configuration.GetSection("Hosting:TrustedProxyNetworks").Get<string[]>() ?? [])
    {
        o.KnownIPNetworks.Add(System.Net.IPNetwork.Parse(cidr));
    }
});

// 4. Health: liveness has no dependencies; readiness checks what the app needs to serve traffic.
builder.Services.AddHealthChecks()
    .AddCheck("self", () => HealthCheckResult.Healthy(), tags: ["live"])
    .AddCheck("config", () => string.IsNullOrEmpty(builder.Configuration["Accounts:Source"])
        ? HealthCheckResult.Unhealthy("Accounts:Source is not configured.")
        : HealthCheckResult.Healthy(), tags: ["ready"]);

// 5. Errors as RFC 9457 ProblemDetails; HSTS for browsers that reach us over HTTPS.
builder.Services.AddProblemDetails();
builder.Services.AddHsts(o => { o.MaxAge = TimeSpan.FromDays(365); o.IncludeSubDomains = true; });

var app = builder.Build();

app.UseForwardedHeaders();           // first: everything after it sees the real client IP and scheme
app.UseExceptionHandler();
app.UseStatusCodePages();
if (!app.Environment.IsDevelopment())
{
    app.UseHsts();                   // only emitted when Request.IsHttps, i.e. X-Forwarded-Proto was trusted
}

app.MapHealthChecks("/health/live", new HealthCheckOptions { Predicate = r => r.Tags.Contains("live") });
app.MapHealthChecks("/health/ready", new HealthCheckOptions { Predicate = r => r.Tags.Contains("ready") });

app.MapGet("/v1/accounts/{accountId}", (string accountId) =>
    accountId == "ACC-0000000001"
        ? Results.Ok(new { accountId, currency = "GBP", product = "current", status = "open", balance = 1250.75m })
        : Results.Problem(statusCode: 404, title: "Account not found"));

// Troubleshooting aid for load-balancer set-ups. Off unless explicitly enabled.
if (app.Configuration.GetValue<bool>("Diagnostics:EnableRequestInfo"))
{
    app.MapGet("/diagnostics/request", (HttpContext ctx) => new
    {
        scheme = ctx.Request.Scheme,
        clientIp = ctx.Connection.RemoteIpAddress?.ToString(),
        host = ctx.Request.Host.Value,
    });
}

app.Run();

public partial class Program;

The tests prove the security-critical behaviour: headers are honoured from the VPC range and ignored from anywhere else.

tests/Harbourline.Baseline.Api.Tests/BaselineTests.cs
using System.Net;
using System.Net.Http.Json;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Mvc.Testing;
using Microsoft.Extensions.DependencyInjection;

namespace Harbourline.Baseline.Api.Tests;

public class ForwardedHeaderTests
{
    private static CancellationToken Ct => TestContext.Current.CancellationToken;

    /// <summary>TestServer has no TCP connection, so pretend the caller is a given address (the ALB or an attacker).</summary>
    private sealed class FakeRemoteIp(string ip) : IStartupFilter
    {
        public Action<IApplicationBuilder> Configure(Action<IApplicationBuilder> next) => app =>
        {
            app.Use((ctx, nextMiddleware) =>
            {
                ctx.Connection.RemoteIpAddress = IPAddress.Parse(ip);
                return nextMiddleware(ctx);
            });
            next(app);
        };
    }

    private sealed record RequestInfo(string Scheme, string? ClientIp, string Host);

    private static HttpClient ClientFrom(string remoteIp)
    {
        var factory = new WebApplicationFactory<Program>().WithWebHostBuilder(b =>
        {
            b.UseSetting("Diagnostics:EnableRequestInfo", "true");
            b.ConfigureServices(s => s.AddSingleton<IStartupFilter>(new FakeRemoteIp(remoteIp)));
        });
        return factory.CreateClient();
    }

    private static HttpRequestMessage Forwarded() => new(HttpMethod.Get, "/diagnostics/request")
    {
        Headers = { { "X-Forwarded-For", "203.0.113.50" }, { "X-Forwarded-Proto", "https" } },
    };

    [Fact]
    public async Task Headers_from_the_load_balancer_subnet_are_trusted()
    {
        var response = await ClientFrom("10.0.1.25").SendAsync(Forwarded(), Ct);
        var info = await response.Content.ReadFromJsonAsync<RequestInfo>(Ct);

        Assert.Equal("https", info!.Scheme);
        Assert.Equal("203.0.113.50", info.ClientIp);
    }

    [Fact]
    public async Task Headers_from_anywhere_else_are_ignored()
    {
        var response = await ClientFrom("198.51.100.7").SendAsync(Forwarded(), Ct);
        var info = await response.Content.ReadFromJsonAsync<RequestInfo>(Ct);

        Assert.Equal("http", info!.Scheme);
        Assert.Equal("198.51.100.7", info.ClientIp);
    }

    [Fact]
    public async Task Health_and_problem_details_work()
    {
        using var factory = new WebApplicationFactory<Program>();
        var client = factory.CreateClient();

        Assert.Equal(HttpStatusCode.OK, (await client.GetAsync("/health/live", Ct)).StatusCode);
        Assert.Equal(HttpStatusCode.OK, (await client.GetAsync("/health/ready", Ct)).StatusCode);
        var missing = await client.GetAsync("/v1/accounts/ACC-0000000999", Ct);
        Assert.Equal(HttpStatusCode.NotFound, missing.StatusCode);
        Assert.Equal("application/problem+json", missing.Content.Headers.ContentType?.MediaType);
        Assert.Equal(HttpStatusCode.NotFound, (await client.GetAsync("/diagnostics/request", Ct)).StatusCode);
    }
}

The test project file is the same shape as in the previous lesson: xUnit v3 4.0.3, Microsoft.AspNetCore.Mvc.Testing 10.0.15, OutputType Exe, and a project reference to the API.

Code explained line by line

Kestrel (block 1)

  1. AddServerHeader = false stops Kestrel from sending Server: Kestrel. That gives scanners one less fingerprint, and many bank penetration-test reports flag the header.
  2. MaxRequestBodySize = 1 MB. The default is about 28.6 MB. An account API never needs that, and smaller limits reduce the cost of abusive requests. Set larger limits per endpoint ([RequestSizeLimit]) where uploads are expected.
  3. KeepAliveTimeout = 75 s. AWS recommends that the target's idle timeout is longer than the ALB's (60 s by default). Otherwise Kestrel may close an idle connection at the moment the ALB reuses it, and the customer gets a 502. Kestrel's default (130 s) already satisfies this. Setting it explicitly documents the dependency: if someone raises the ALB idle timeout to 120 s for slow exports, this value must rise too.
  4. RequestHeadersTimeout = 15 s limits slow-header attacks. The ALB also protects you here, but defence in depth is cheap.

Shutdown (block 2)

  1. HostOptions.ShutdownTimeout = 25 s. When the container receives SIGTERM, the .NET host stops accepting new connections and waits up to this long for in-flight requests and hosted services. The ECS stop timeout defaults to 30 s, after which the container is killed. 25 s leaves a margin for logging to flush.

Forwarded headers (block 3)

  1. ForwardedHeaders = XForwardedFor | XForwardedProto processes the two headers you need: client IP for audit and fraud rules, scheme for HSTS, redirects and absolute links. X-Forwarded-Host is deliberately not trusted, because host header injection is a known attack.
  2. KnownIPNetworks.Clear(); KnownProxies.Clear(); removes the defaults (loopback), so trust is exactly what configuration says. In .NET 10, KnownIPNetworks (based on System.Net.IPNetwork) is the property to use; the older KnownNetworks is obsolete.
  3. KnownIPNetworks.Add(IPNetwork.Parse(cidr)) trusts the VPC range (10.0.0.0/16). The ALB's nodes have private addresses inside the VPC, so only connections from inside the VPC can set the client IP. Combined with a security group that only admits the ALB, that is two layers.
  4. ForwardLimit is left at its default of 1. The middleware reads only the rightmost X-Forwarded-For entry, the one your ALB appended. If a client sends its own X-Forwarded-For: 10.20.30.40, the ALB appends the real address after it, and the middleware still picks the real one. Raise ForwardLimit only if you add another trusted proxy in front, such as CloudFront, and then trust that proxy's ranges too.

Health (block 4)

  1. "self" tagged live has no dependencies. If it fails, the process is broken and a restart helps.
  2. "config" tagged ready stands in for real dependency checks (the reference API checks DynamoDB). If a dependency is down, a restart does not help, so readiness failing should take the instance out of the load balancer, not kill it.

Errors and HSTS (block 5)

  1. AddProblemDetails() + UseExceptionHandler() + UseStatusCodePages() make every error, including a bare 404, an RFC 9457 application/problem+json body with no stack trace.
  2. UseHsts() adds Strict-Transport-Security only to responses for HTTPS requests. Behind the ALB the request is HTTP, so without trusted X-Forwarded-Proto the header would never be sent. This is a good self-test that forwarded headers work.

Pipeline order

  1. UseForwardedHeaders() first. Anything that reads the client IP or scheme (logging, rate limiting, HSTS, authentication redirects) must run after it.
  2. Health endpoints are anonymous and mapped before business endpoints. In the reference API they are explicitly .AllowAnonymous() because the business endpoints require authorisation by default.
  3. /diagnostics/request exists only when enabled. It is a real troubleshooting aid for load balancer set-ups, but it reveals network details, so it is off by default and must never be enabled in production.

Tests

  1. FakeRemoteIp (an IStartupFilter) sets RemoteIpAddress before the app's own middleware, because the in-memory TestServer has no TCP connection. It lets one test act as "the ALB" (10.0.1.25) and another as "an attacker on the internet" (198.51.100.7).
  2. The two forwarded-header tests are the regression tests that would have caught both incidents from the introduction.

Tour of the reference Accounts API

Open labs/m03-accounts-api/src/Harbourline.Accounts.Api/Program.cs. Its numbered sections map onto this module:

Step in Program.csWhat it doesLesson that goes deeper
1. AddAwsConfigurationSources()appsettings → environment → Parameter Store → AppConfigConfiguration (lesson 10)
2. Serilog to stdout as compact JSONEvery host ships stdout to CloudWatch LogsLogging and errors (lesson 14)
3. UsePlatformPort(), AddAWSLambdaHosting(...), AddTrustedForwardedHeaders(...)Same binary runs on Beanstalk (port from PORT), in containers (8080) and in LambdaThis lesson and Hosting options compared
4. AddDefaultAWSOptions(...)SDK defaults from the AWS sectionAWS clients and DI
5. Data, pseudonymisation, auth, health checksIAccountRepository (in-memory or DynamoDB), HMAC pseudonyms for logs, Cognito JWT, live/readySecrets, auth and health-check lessons
6. ProblemDetails + GlobalExceptionHandlerRFC 9457 errors carrying the correlation IDLogging and errors
7. JSON optionscamelCase properties, snake_case enum values (credit_card)CONVENTIONS.md §2
Middleware orderUseForwardedHeaders → correlation ID → request logging → exception handler → authentication → authorisationThis lesson

The lessons named by number are in this module's overview.

Security considerations

⚠ Learning shortcut — not for production

The baseline app and the lab's Fargate stack terminate TLS at the ALB and send plain HTTP to port 8080 inside the VPC. Many banks accept this when the private subnets, security groups and VPC flow logs are tightly controlled. If your PCI DSS scope or internal policy requires encryption between the load balancer and targets, use an HTTPS target group with a certificate in the container, remembering that the ALB does not validate target certificates. Discuss the choice with your security and compliance teams; do not assume either answer.

  • Never use ASPNETCORE_FORWARDEDHEADERS_ENABLED=true on its own behind an internet-facing ALB. It turns on forwarded-header processing with the known-proxy lists cleared, meaning any caller that can reach Kestrel is trusted. It is meant for platforms where only the proxy can reach the app. Use explicit KnownIPNetworks instead.
  • Security groups first, forwarded headers second. If Kestrel is reachable only from the ALB's security group, nobody else can forge headers at all.
  • Turn on "drop invalid header fields" on the ALB, so malformed header names never reach Kestrel.
  • Use a modern TLS policy on the HTTPS listener and redirect port 80. Add AWS WAF in front of internet-facing banking APIs (WAF and Shield).
  • No stack traces to clients. ProblemDetails with a correlation ID is enough for support to find the log line.
  • Health endpoints must not leak. Return statuses, not exception messages, hostnames or table names. The reference API's WriteAsync returns only status names and a duration.
  • Log the forwarded client IP, then think about retention. IP addresses are personal data under UK GDPR. Keep them for as long as your fraud and security purposes need, with documented retention, rather than forever.

Banking use case

Harbourline's Open Banking Account Information API serves regulated third-party providers. Its baseline:

  • An internet-facing ALB with AWS WAF and a TLS 1.2/1.3 policy. Mutual TLS for third-party certificates is handled in front of the API (see Certificates, TLS and mTLS).
  • Kestrel trusts X-Forwarded-For only from the VPC range, and the API's rate limiter partitions by the real client IP and by the provider's client ID.
  • /health/ready checks DynamoDB with a 2-second timeout. During a DynamoDB incident, targets are marked unhealthy and the ALB returns 503, which the providers' clients are built to retry. Containers are not restarted, because restarting would not fix DynamoDB.
  • Deployments drain over 30 seconds. Monitoring shows zero 5xx responses during the weekly release window, which the operational resilience team uses as evidence for the API's availability target.

Common mistakes

  • Not calling UseForwardedHeaders → every request appears to come from the ALB → audit logs and fraud rules see one IP → configure and call it first.
  • Trusting all proxies (ASPNETCORE_FORWARDEDHEADERS_ENABLED, or clearing KnownIPNetworks without adding any) → clients can spoof their IP → trust only the VPC range.
  • UseHttpsRedirection behind the ALB without trusted forwarded headers → every request looks like HTTP → infinite redirect loop → redirect at the ALB listener instead.
  • Keep-alive shorter than the ALB idle timeout → a few random 502s an hour → keep Kestrel's timeout above the ALB's.
  • Health check on / or on an endpoint that needs authentication → 401/404 → targets never become healthy → use an anonymous /health/ready.
  • Database check on the liveness endpoint → a database blip restarts every container at once → dependencies belong in readiness only.
  • Deregistration delay of 300 s with an ECS stop timeout of 30 s → slow deployments, and requests longer than 30 s killed anyway → align the three timers.
  • Binding Kestrel to localhost in the container → the ALB cannot connect → listen on all interfaces (the official images set ASPNETCORE_HTTP_PORTS=8080, which does this).

Troubleshooting

SymptomLikely causeFix
Logs show 10.0.x.x as the client IP for every requestForwarded headers not processed, or the ALB subnet is not in KnownIPNetworksAdd the VPC CIDR; call UseForwardedHeaders() first
Infinite redirect (ERR_TOO_MANY_REDIRECTS)UseHttpsRedirection sees HTTP because X-Forwarded-Proto is not trustedTrust the headers or remove app-level redirection
Occasional 502 Bad Gateway with healthy targetsApp closes idle connections before the ALB doesKestrel KeepAliveTimeout above the ALB idle timeout
504 Gateway TimeoutA request ran longer than the ALB idle timeout with no data sentSpeed up the request, make it asynchronous (202 + polling) or raise both timeouts
Targets unhealthy, reason Health checks failed with these codes: [404]Wrong health check path, or the endpoint requires authorisationPoint at /health/ready and make it anonymous
Targets unhealthy, reason Request timed outSecurity group does not allow the ALB, the app listens on the wrong port, or readiness is too slowAllow 8080 from the ALB security group; check ASPNETCORE_HTTP_PORTS; time-box the dependency check
5xx spikes during every deploymentNo draining or shutdown too shortDeregistration delay plus ShutdownTimeout below the stop timeout
413 Payload Too LargeMaxRequestBodySize too small for that endpointRaise it per endpoint with [RequestSizeLimit]
HSTS header missing in productionRequest.IsHttps is false because X-Forwarded-Proto is not trustedSame fix as the client-IP problem

Cost and cleanup

This lesson is free: everything runs locally and in tests.

If you apply the CLI or IaC settings to the ALB from the Fargate lesson, they cost nothing extra. The ALB itself costs USD 0.0225 per hour plus LCU charges in us-east-1 (new accounts get 750 ALB hours per month in the free tier). Follow that lesson's teardown: deleting the stack removes the ALB, listeners and target group.

# Check no load balancer from this module is still running.
aws elbv2 describe-load-balancers --profile learn-sandbox --region us-east-1 \
  --query "LoadBalancers[?starts_with(LoadBalancerName, 'hl-m03')].LoadBalancerName"

Exercises

  • 🟢 Guided: Run the baseline API locally, simulate the ALB with curl as in the last step, and confirm the scheme and client IP change only when 127.0.0.1/32 is trusted. Then call /health/ready with Accounts__Source="" and see readiness fail while liveness still passes.
  • 🟡 Challenge: Add ASP.NET Core rate limiting (AddRateLimiter) with a fixed window of 100 requests per minute per client IP, and write a test proving that two different forwarded IPs from the ALB range get separate limits, while a spoofed X-Forwarded-For from outside the range does not.
  • 🔴 Live problem: After moving the Accounts API from an on-premises IIS farm to ECS, the fraud team reports that the "new device from a new country" rule fires for nearly every login, and the operations team sees about ten 502 errors per hour. No code changed during the move. Investigate and write up the fixes.

Assignment

Write an ADR (architecture decision record) for Harbourline titled "TLS termination for internal APIs". Compare TLS terminated at the ALB with HTTP to targets against end-to-end TLS to targets. Cover security, certificate management, PCI DSS scope discussion points, operational effort and cost. State a recommendation and the conditions under which you would change it. Put it in your portfolio repository's docs/adr/ folder.

Solutions

Knowledge check

Interview questions

  • "How do you get the client IP in ASP.NET Core behind an AWS load balancer?" — Configure ForwardedHeadersOptions for X-Forwarded-For and X-Forwarded-Proto, trust only the VPC range or the ALB's subnets, call UseForwardedHeaders() first, and restrict the app's security group to the ALB. Never trust all proxies.
  • "Where do you terminate TLS for an API on ECS, and what are the trade-offs?" — Usually at the ALB with an ACM certificate (free, automatic renewal, central TLS policy). End-to-end TLS to targets adds encryption inside the VPC for stricter policies, but the ALB does not validate target certificates and you must manage them in the container.
  • "What is the difference between liveness and readiness checks?" — Liveness: is the process alive (restart if not). Readiness: can it serve traffic now (route around it if not). Dependencies belong in readiness only.
  • "How do you avoid dropped requests during deployments?" — Deregistration delay to drain, a graceful shutdown timeout below the stop timeout, minimum healthy percent at 100% during rolling updates, and a deployment circuit breaker to roll back bad releases.

Certification practice

Completion checklist

  • I can draw the request path and say where TLS ends and who adds which headers
  • My forwarded-header tests pass for both trusted and untrusted callers
  • I can explain the three shutdown timers and the keep-alive rule
  • I separated liveness and readiness and know which one the ALB calls
  • I ran the reference Accounts API locally and traced its startup pipeline

Key takeaways

  • The ALB terminates TLS and forwards plain HTTP inside the VPC. The app learns the real client and scheme from X-Forwarded-For/-Proto, which it must trust only from the VPC range.
  • Security groups that admit only the ALB are the first control. Forwarded-header trust lists are the second.
  • Keep Kestrel's keep-alive above the ALB idle timeout to avoid random 502s.
  • Readiness for routing, liveness for restarts, and never put dependencies in liveness.
  • Align deregistration delay, ShutdownTimeout and the stop timeout for deployments with zero dropped requests.

Further reading

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page