Skip to Content
Cloud Engineering Academy
3. .NET Application Development on AWS

Scaling .NET apps on AWS

Make an ASP.NET Core API stateless, size its connection pools, move session state to ElastiCache or DynamoDB, and scale it automatically with target tracking on ECS and concurrency controls on Lambda.

intermediate⏱ 2 h 🟡 Low costDVA-C03SAA-C03

Assumes: .NET 10 LTS · AWS SDK for .NET v4 · Amazon.CDK.Lib 2.277.0 · AWS CLI v2 · Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Explain the difference between scaling out (more copies) and scaling up (bigger copies), and why AWS-hosted .NET APIs almost always scale out.
  • Find and remove the hidden state that stops an ASP.NET Core app from running as many identical copies: in-memory sessions, local files, Data Protection keys, in-process caches and sticky sessions.
  • Configure target tracking auto scaling for an ECS service on CPU and on ALB requests per target, and explain cooldowns, minimum and maximum capacity.
  • Explain how Lambda scales (concurrency, the per-function scaling rate, reserved and provisioned concurrency) and why that can overwhelm a database.
  • Size connection pools (database, HttpClient, AWS SDK) so that tasks × pool size stays below what the database can accept, and know when to add RDS Proxy.
  • Store session state and shared caches in ElastiCache (Valkey) or DynamoDB through IDistributedCache.

Prerequisites

  • Module 3 overview — the hosting options and the Accounts API
  • Module 2: Networking — load balancers, target groups and private subnets
  • Optional but useful: the module project code in labs/m03-accounts-api (its Fargate stack already contains the scaling policy discussed here)
  • Tools: AWS CLI v2 with the learn-sandbox profile, .NET 10 SDK or Docker

Why this matters in banking

On the 25th of each month Harbourline Bank's salary payments land. Between 06:00 and 08:00 several hundred thousand customers open the mobile app to check their balance. Traffic to the Accounts API goes from 40 requests per second to 1,500 in twenty minutes. Then, by 10:00, it is back to normal.

Three things can go wrong, and all three have happened at real banks:

  1. Nothing scales. The fixed fleet of four servers saturates, response times climb from 80 ms to 8 seconds, the mobile app times out and customers phone the contact centre. Complaints go to the regulator's statistics.
  2. The app scales but breaks. New copies start, but customers are logged out at random, or see "anti-forgery token could not be decrypted", because session data and encryption keys lived in the memory of one server.
  3. The app scales and takes the database down. Twenty tasks each open a pool of 100 connections to a database that accepts 400. Everything fails at once, including the payments system that shares the database.

Scaling is therefore not a single setting. It is an architecture property: stateless code, bounded pools, a scaling policy and limits that protect what sits behind the API.

The idea in plain language

Scaling up means giving one server more CPU and memory. Scaling out means running more identical copies behind a load balancer. Cloud platforms are built for scaling out: copies can be added in a minute and removed when they are no longer needed, so you only pay for what the traffic needs.

Scaling out has one precondition: any copy must be able to serve any request. If a request depends on something that only one copy has (a session in memory, a file on the local disk, a key in a local folder) the second copy gives a wrong answer. An app that keeps nothing between requests, except in shared external stores, is called stateless.

Auto scaling is the robot that decides how many copies to run. You tell it a target (for example "keep average CPU at 60%") and limits (between 2 and 10 copies). It watches a CloudWatch metric and adds or removes copies to stay near the target, like a thermostat.

Analogy: a bank branch on salary day. Opening more counters (scaling out) works only if every cashier can serve any customer, which means the customer's file is in the central system, not in one cashier's drawer (statelessness). The branch manager watches the queue length and opens or closes counters (target tracking). And there is a limit to how many cashiers the back office can support at once (connection pools and the database).

What problem AWS auto scaling solves

On-premises, capacity is bought for the peak and idles the rest of the month. In AWS you can match capacity to load, but only if three layers cooperate:

LayerServiceWhat scales
Containers on ECSApplication Auto Scaling with an ECS service as the scalable targetThe service's desired count of tasks
EC2 instances (Beanstalk, ECS on EC2)EC2 Auto Scaling groupsThe number of instances
LambdaBuilt into LambdaConcurrent execution environments, automatically per request
DynamoDBOn-demand mode, or Application Auto Scaling for provisioned capacityRead and write capacity
AuroraAurora Serverless v2 capacity units and Aurora replicasDatabase capacity and read replicas

Limitations and sharp edges you should know before relying on it:

  • Auto scaling reacts to metrics that are one or more minutes old. A new Fargate task then needs time to pull the image, start .NET and pass health checks. Expect two to four minutes from "load rises" to "new task serves traffic". For a known peak like salary day, use scheduled scaling to raise the minimum in advance.
  • Scaling out never fixes a bottleneck behind the API. If the database is the limit, more tasks make it worse.
  • Every scaling policy needs a maximum. Without one, a bug (an endless retry loop) or an attack can scale you into a large bill.
  • Lambda scales very fast and has no built-in notion of "the database can only take 200 connections". You must limit it yourself.

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
App Service "Scale out" rules (autoscale settings)Application Auto Scaling target tracking on an ECS serviceTarget tracking computes the steps for you; you set a target value, not "add 1 instance when CPU > 70%"
VM Scale Sets autoscaleEC2 Auto Scaling groupsSame idea; Beanstalk creates one for you in load-balanced environments
Azure Functions Consumption plan scalingLambda concurrency scalingLambda scales per function at up to 1,000 extra concurrent executions every 10 seconds; you cap it with reserved concurrency
Functions Premium "always ready" instancesLambda provisioned concurrencyBilled per hour while configured
ARR affinity (sticky sessions) on App ServiceALB target group stickinessAvailable, but a design smell for APIs: it hides state instead of removing it
Azure Cache for RedisAmazon ElastiCache (Valkey, Redis OSS, Memcached)Valkey is the open-source Redis fork; same protocol, works with StackExchange.Redis
Session state in Azure SQL / Cosmos DBDynamoDB via IDistributedCache providerUse DynamoDB TTL to expire items
Data Protection keys in Blob Storage + Key VaultData Protection keys in SSM Parameter Store (Amazon.AspNetCore.DataProtection.SSM) or in RedisEncrypt with KMS through SecureString parameters
Azure SQL connection limits per tierRDS max_connections (depends on instance memory) + RDS ProxyRDS Proxy pools and shares connections between many clients

Architecture

How a scale-out event plays out:

Security boundary view. Scaling must not change the security posture. New tasks use the same task role (least privilege), run in the same private subnets with the same security groups, and connect to ElastiCache with in-transit encryption and an authentication token or IAM authentication. The external stores you add for state (cache, session table) hold personal data, so they need encryption at rest, retention limits and access logging like any other data store.

Making an ASP.NET Core app stateless

Before you add any scaling policy, look for these five kinds of hidden state. Each one has a standard fix.

Hidden stateSymptom after scaling outFix
AddSession() with the default in-memory storeUsers lose their session when the next request lands on another taskIDistributedCache backed by ElastiCache or DynamoDB
IMemoryCache holding data that must be consistentDifferent tasks return different answers until their caches expireKeep IMemoryCache only for data where short inconsistency is fine; use a shared cache or none for the rest
Data Protection keys in the local file system"The key was not found in the key ring", anti-forgery and auth cookie failures, after deployments or on other tasksPersist keys to a shared store and protect them with KMS
Uploaded or generated files on local diskDownload link works only on one task, files vanish when the task stopsWrite to S3; return a pre-signed URL
Background timers in the web process (e.g. "send reminders every hour")The job runs N times when N tasks runMove to EventBridge Scheduler or a single worker service (see Background processing)

You saw the Data Protection warning in the lab's test output: ASP.NET Core logs Storing keys in a directory ... that may not be persisted outside of the container. The Accounts API is a pure bearer-token API with no cookies, so the warning is harmless there. A web app that issues cookies or anti-forgery tokens must fix it before running two tasks.

Sticky sessions (ALB target group stickiness) route a client back to the same task with a cookie. They make the symptoms disappear but keep the problem: when that task is replaced during a deployment or a scale-in, the session is gone. They also unbalance load. Treat them as a temporary crutch for a legacy app, not a design.

Scaling policies on ECS

ECS service auto scaling is provided by Application Auto Scaling. The scalable target is the ECS service; the scalable dimension is ecs:service:DesiredCount. You choose one of three policy types:

Policy typeHow it worksUse it for
Target trackingKeep a metric near a target value. AWS creates and manages the CloudWatch alarmsThe default for APIs
Step scalingYou define alarm thresholds and how many tasks to add or remove per stepUnusual metrics, or when you need asymmetric reactions
Scheduled scalingChange min/max capacity at a time (cron or one-off)Known peaks: salary day, month-end batch, market open
Predictive scalingForecasts load from history and scales ahead of itStrongly cyclical traffic ⚠ VERIFY current ECS support in your Region before relying on it

Three predefined metrics are available for target tracking on ECS services:

  • ECSServiceAverageCPUUtilization — average CPU of the service's tasks.
  • ECSServiceAverageMemoryUtilization — average memory.
  • ALBRequestCountPerTarget — requests per target per minute for one target group. Needs a ResourceLabel identifying the load balancer and target group.

Which metric for a .NET API? CPU works well for compute-bound APIs. Many banking APIs are I/O bound instead: they wait on databases and downstream services, so CPU stays low while latency climbs. For those, requests per target is a better signal, because it tracks load directly. Measure how many requests per minute one task handles at your latency objective (with a load test) and set the target at about 70% of that. Memory is rarely a good scaling metric for .NET: the garbage collector holds memory it does not need, so memory does not drop when load drops and the service never scales in.

Cooldowns and stability. After a scale-out, Application Auto Scaling waits for the scale-out cooldown before deciding to add more, so that the new tasks have time to absorb load. Scale-in is deliberately slower and more conservative. Target tracking only scales in when the metric is well below the target for a sustained period. Set a scale-in cooldown long enough (a few minutes) to avoid "flapping": adding and removing tasks repeatedly.

Minimum and maximum. For production, run at least two tasks in two Availability Zones so that one task or one AZ can fail. Set the maximum from two numbers: the most you are willing to pay, and the most the database can serve (see the connection-pool section).

How Lambda scales

Lambda scales without a policy. Each concurrent request needs its own execution environment; if none is free, Lambda creates one (a cold start). The number of environments busy at the same time is the function's concurrency.

  • An account has a Regional concurrency quota shared by all functions. The default is 1,000; new accounts often start much lower (10 is common), and you request an increase through Service Quotas ⚠ VERIFY the value shown in your sandbox account.
  • Each function can scale by up to 1,000 additional concurrent executions every 10 seconds, until the account quota is reached.
  • Reserved concurrency sets both a guaranteed share and a hard maximum for one function. It is how you stop a function from opening 1,000 database connections. Lambda refuses a reservation that would leave the unreserved pool below 100, which is why a new account with a quota of 10 cannot reserve anything.
  • Provisioned concurrency keeps environments initialised in advance, removing cold starts for that many concurrent requests. It is billed per hour while configured, and it can itself be scaled with Application Auto Scaling (metric LambdaProvisionedConcurrencyUtilization) or scheduled before a known peak.

Requests beyond the available concurrency are throttled (HTTP 429 from API Gateway, TooManyRequestsException from a direct invoke). For asynchronous and queue-driven invocations Lambda retries for you; for synchronous API calls the client sees the error.

Connection pools: the part that breaks databases

Every copy of your app keeps pools of open connections. Scaling out multiplies them.

connections at peak = max tasks × pool size per task
                    = 10 × 100 (Npgsql/SqlClient default Max Pool Size) = 1,000

An Aurora PostgreSQL instance's max_connections depends on its memory; a small instance accepts a few hundred at most. The fix has three parts:

  1. Size the pool deliberately. Set Maximum Pool Size (Npgsql) or Max Pool Size (SqlClient) in the connection string so that max tasks × pool size is below the database limit with headroom for admin tools and other apps. A pool of 10–20 per task is plenty for most APIs, because each request holds a connection for milliseconds.
  2. Add RDS Proxy when many short-lived clients (Lambda in particular) connect. The proxy keeps a smaller set of real database connections and shares them between client connections. It also speeds up failover.
  3. Cap the compute. Set the ECS maximum task count, or Lambda reserved concurrency, from the database budget, not from hope.

HTTP connections. HttpClient should be created through IHttpClientFactory (or a long-lived instance with SocketsHttpHandler.PooledConnectionLifetime set) so that sockets are reused and DNS changes are picked up. Creating new HttpClient() per request exhausts ephemeral ports under load, the classic SocketException: Address already in use on Linux containers.

AWS SDK clients. Register SDK clients as singletons through AddAWSService<T>(), as the Accounts API does. A singleton client reuses its HTTP connections. Creating a client per request is slow (credential resolution, TLS handshakes) and can trigger throttling on the credential endpoint.

Session state and shared caches

When the app does need per-user state, put it in a store every task can reach. ASP.NET Core's IDistributedCache abstraction lets you switch the store without changing app code.

OptionPackageStrengthsWatch out for
ElastiCache Serverless for ValkeyMicrosoft.Extensions.Caching.StackExchangeRedis 10.0.15Sub-millisecond reads, no nodes to size, scales automaticallyBilled per GB-hour of stored data (minimum 100 MB) and per ECPU; VPC-only access
ElastiCache node-based (Valkey)SamePredictable hourly price for steady loadYou choose node size and replicas; plan failover
DynamoDBAWS.AspNetCore.DistributedCacheProvider 1.0.3Serverless, pay per request, no VPC needed (works from Lambda without VPC)Single-digit millisecond latency, not sub-millisecond; enable TTL to expire items

Valkey is a Linux Foundation fork of Redis OSS. ElastiCache offers it at a lower price than Redis OSS (AWS states 33% lower for Serverless and 20% lower for node-based clusters), and it speaks the same protocol, so StackExchange.Redis works unchanged.

Step-by-step implementation

Audit the app for hidden state

Search the code for AddSession, AddMemoryCache, File.Write, Path.GetTempPath, System.Timers.Timer and PeriodicTimer. For each hit, decide: remove it, move it to a shared store, or prove it is safe to have per-task copies (for example a 30-second cache of reference data).

Move Data Protection keys and sessions to shared stores

Use the Program.cs in the Complete code section: Data Protection keys go to Parameter Store (encrypted with KMS), sessions and the shared cache go to Valkey.

Bound every pool

Set the database pool size in the connection string, create HTTP clients through IHttpClientFactory, register AWS clients once. Write the arithmetic (max tasks × pool) into the service's runbook.

Load test one task

Run a load test against a single task (for example with the lab's tools/latency-probe.cs for latency, and a load tool such as k6 or bombardier for throughput) to find the requests per minute one task sustains at your p95 objective.

Add target tracking, limits and a scheduled boost

Create a target tracking policy on ALBRequestCountPerTarget (or CPU), set min 2 and a max derived from the database budget, then add a scheduled action that raises the minimum before salary-day mornings.

Watch it work

Generate load, watch the DesiredTaskCount and RunningTaskCount metrics in CloudWatch (Container Insights) and the scaling activities in the ECS console. Confirm that scale-in happens after the load stops.

AWS Console instructions

  • ECS → Clusters → hl-m03-cluster → Services → hl-m03-accounts-api → Update service → Service auto scaling. Choose Use service auto scaling, set minimum, desired and maximum tasks, then add a Target tracking policy: pick ECSServiceAverageCPUUtilization or ALBRequestCountPerTarget, enter the target value and cooldowns.
  • ECS → service → Deployments and events / Events tab shows each scaling action with the reason.
  • CloudWatch → Alarms lists the two alarms that target tracking created (names start with TargetTracking-service/...). Do not edit or delete them; change the policy instead.
  • Lambda → Functions → your function → Configuration → Concurrency and recursion detection sets reserved concurrency; Versions/Aliases → Provisioned concurrency sets provisioned concurrency on an alias or version.
  • ElastiCache → Valkey caches → Create Valkey cache → Serverless creates a serverless cache; choose the VPC, private subnets and a security group that only allows the API tasks' security group on port 6379.

AWS CLI

# 1. Register the ECS service as a scalable target (min 2, max 10 tasks)
aws application-autoscaling register-scalable-target \
  --service-namespace ecs \
  --resource-id service/hl-m03-cluster/hl-m03-accounts-api \
  --scalable-dimension ecs:service:DesiredCount \
  --min-capacity 2 --max-capacity 10 \
  --profile learn-sandbox

# 2. Target tracking on requests per target (replace the resource label:
#    app/<alb-name>/<alb-id>/targetgroup/<tg-name>/<tg-id>)
aws application-autoscaling put-scaling-policy \
  --service-namespace ecs \
  --resource-id service/hl-m03-cluster/hl-m03-accounts-api \
  --scalable-dimension ecs:service:DesiredCount \
  --policy-name requests-per-target \
  --policy-type TargetTrackingScaling \
  --target-tracking-scaling-policy-configuration '{
    "TargetValue": 1000,
    "PredefinedMetricSpecification": {
      "PredefinedMetricType": "ALBRequestCountPerTarget",
      "ResourceLabel": "app/hl-m03-alb/0123456789abcdef/targetgroup/hl-m03-tg/fedcba9876543210"
    },
    "ScaleOutCooldown": 60,
    "ScaleInCooldown": 300
  }' \
  --profile learn-sandbox

# 3. Salary-day boost: at 05:30 UTC on the 25th raise the minimum to 6
aws application-autoscaling put-scheduled-action \
  --service-namespace ecs \
  --resource-id service/hl-m03-cluster/hl-m03-accounts-api \
  --scalable-dimension ecs:service:DesiredCount \
  --scheduled-action-name salary-day-boost \
  --schedule "cron(30 5 25 * ? *)" \
  --scalable-target-action MinCapacity=6,MaxCapacity=10 \
  --profile learn-sandbox

# 4. See what auto scaling decided and why
aws application-autoscaling describe-scaling-activities \
  --service-namespace ecs \
  --resource-id service/hl-m03-cluster/hl-m03-accounts-api \
  --max-items 5 --profile learn-sandbox

# 5. Lambda: cap a function at 50 concurrent executions (protects the database)
aws lambda put-function-concurrency \
  --function-name hl-m03-accounts-api \
  --reserved-concurrent-executions 50 \
  --profile learn-sandbox

# 6. Check the account's concurrency quota and how much is unreserved
aws lambda get-account-settings --query AccountLimit --profile learn-sandbox

A second scheduled action (for example at 11:00 on the 25th) should set the minimum back to 2; otherwise the boost stays forever.

Infrastructure as Code

The lab's Fargate stack already contains a CPU target tracking policy. This version adds request-based tracking and a scheduled boost.

ScalingSnippet.cs (inside FargateStack, after the service is created)
using Amazon.CDK;
using Amazon.CDK.AWS.ApplicationAutoScaling;
using Amazon.CDK.AWS.ECS;

// service is the ApplicationLoadBalancedFargateService from FargateStack.cs
var scaling = service.Service.AutoScaleTaskCount(new EnableScalingProps
{
    MinCapacity = 2,
    MaxCapacity = 10, // from the database budget: 10 tasks x 15 connections = 150 < max_connections
});

// I/O-bound API: track requests per task rather than CPU.
scaling.ScaleOnRequestCount("RequestsPerTarget", new RequestCountScalingProps
{
    RequestsPerTarget = 1000,          // per minute, about 70% of what one task handles at p95 target
    TargetGroup = service.TargetGroup,
    ScaleOutCooldown = Duration.Seconds(60),
    ScaleInCooldown = Duration.Seconds(300),
});

// Belt and braces: also track CPU. The policy asking for MORE capacity wins.
scaling.ScaleOnCpuUtilization("CpuTarget", new CpuUtilizationScalingProps
{
    TargetUtilizationPercent = 60,
});

// Salary day: raise the floor before customers wake up, lower it afterwards.
scaling.ScaleOnSchedule("SalaryDayBoost", new ScalingSchedule
{
    Schedule = Schedule.Cron(new CronOptions { Minute = "30", Hour = "5", Day = "25" }),
    MinCapacity = 6,
});
scaling.ScaleOnSchedule("SalaryDayEnd", new ScalingSchedule
{
    Schedule = Schedule.Cron(new CronOptions { Minute = "0", Hour = "11", Day = "25" }),
    MinCapacity = 2,
});

Complete code

A stateless ASP.NET Core web app with shared Data Protection keys, distributed session and cache in Valkey, a bounded database pool and a correctly managed HttpClient. It runs locally against a Valkey container (docker run -p 6379:6379 valkey/valkey:8) with no AWS account when DataProtection:UseParameterStore is false.

Harbourline.Statements.Web.csproj
<Project Sdk="Microsoft.NET.Sdk.Web">
  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
    <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
    <RootNamespace>Harbourline.Statements.Web</RootNamespace>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Microsoft.Extensions.Caching.StackExchangeRedis" Version="10.0.15" />
    <PackageReference Include="Microsoft.AspNetCore.DataProtection.StackExchangeRedis" Version="10.0.15" />
    <PackageReference Include="Amazon.AspNetCore.DataProtection.SSM" Version="4.0.3" />
    <PackageReference Include="AWSSDK.Extensions.NETCore.Setup" Version="4.1.0.2" />
  </ItemGroup>
</Project>
Program.cs
using Microsoft.AspNetCore.DataProtection;
using StackExchange.Redis;

var builder = WebApplication.CreateBuilder(args);

// ---- 1. One shared connection to Valkey/Redis for the whole process (it multiplexes internally).
var cacheEndpoint = builder.Configuration["Cache:Endpoint"] ?? "localhost:6379";
var redisOptions = ConfigurationOptions.Parse(cacheEndpoint);
redisOptions.Ssl = builder.Configuration.GetValue("Cache:UseTls", false);   // true for ElastiCache
redisOptions.AbortOnConnectFail = false;                                     // keep retrying in the background
redisOptions.ConnectTimeout = 2000;
var redis = await ConnectionMultiplexer.ConnectAsync(redisOptions);
builder.Services.AddSingleton<IConnectionMultiplexer>(redis);

// ---- 2. IDistributedCache + session state in Valkey (shared by every task).
builder.Services.AddStackExchangeRedisCache(o =>
{
    o.ConnectionMultiplexerFactory = () => Task.FromResult<IConnectionMultiplexer>(redis);
    o.InstanceName = "hl-statements:";   // key prefix, so several apps can share one cache safely
});
builder.Services.AddSession(o =>
{
    o.IdleTimeout = TimeSpan.FromMinutes(15);   // banking sessions should be short
    o.Cookie.HttpOnly = true;
    o.Cookie.SecurePolicy = CookieSecurePolicy.Always;
    o.Cookie.SameSite = SameSiteMode.Strict;
    o.Cookie.Name = "__Host-hl-session";
});

// ---- 3. Data Protection keys: one key ring for all tasks.
var dp = builder.Services.AddDataProtection().SetApplicationName("Harbourline.Statements.Web");
if (builder.Configuration.GetValue("DataProtection:UseParameterStore", false))
{
    // Keys stored as SecureString parameters under this path (encrypted with KMS).
    builder.Services.AddDefaultAWSOptions(builder.Configuration.GetAWSOptions());
    dp.PersistKeysToAWSSystemsManager("/hl-statements/data-protection");
}
else
{
    dp.PersistKeysToStackExchangeRedis(redis, "hl-statements:dp-keys");
}

// ---- 4. Outbound HTTP through the factory: pooled sockets, recycled every 5 minutes for DNS changes.
builder.Services.AddHttpClient("core-banking", c =>
{
    c.BaseAddress = new Uri(builder.Configuration["CoreBanking:BaseUrl"] ?? "https://core-banking.internal.example.net/");
    c.Timeout = TimeSpan.FromSeconds(5);
})
.ConfigurePrimaryHttpMessageHandler(() => new SocketsHttpHandler
{
    PooledConnectionLifetime = TimeSpan.FromMinutes(5),
    MaxConnectionsPerServer = 50,   // bound per task: 10 tasks x 50 = 500 connections to core banking
});

var app = builder.Build();
app.UseSession();

// Remembers the last statement month the customer viewed — works on ANY task.
app.MapGet("/statements/last-viewed", (HttpContext ctx) =>
    Results.Ok(new { month = ctx.Session.GetString("lastMonth") ?? "none" }));

app.MapPost("/statements/{month}/view", (string month, HttpContext ctx) =>
{
    if (!System.Text.RegularExpressions.Regex.IsMatch(month, "^20[0-9]{2}-(0[1-9]|1[0-2])$"))
    {
        return Results.Problem(statusCode: 400, title: "Invalid month", detail: "Use YYYY-MM.");
    }
    ctx.Session.SetString("lastMonth", month);
    return Results.NoContent();
});

// Which task answered? Proves requests are spread and sessions survive.
app.MapGet("/whoami", () => Results.Ok(new { task = Environment.MachineName }));

app.MapGet("/health/live", () => Results.Ok());
app.MapGet("/health/ready", async (IConnectionMultiplexer mux) =>
    mux.IsConnected && await mux.GetDatabase().PingAsync() < TimeSpan.FromSeconds(1)
        ? Results.Ok()
        : Results.StatusCode(503));

app.Run();
appsettings.Production.json
{
  "Cache": {
    "Endpoint": "hl-statements-abc123.serverless.use1.cache.amazonaws.com:6379",
    "UseTls": true
  },
  "DataProtection": { "UseParameterStore": true },
  "ConnectionStrings": {
    "Statements": "Host=hl-statements-proxy.proxy-xxxx.us-east-1.rds.amazonaws.com;Database=statements;Username=app;Maximum Pool Size=15;Timeout=5;Command Timeout=10"
  }
}

Code explained line by line

  1. ConfigurationOptions.Parse(cacheEndpoint) — the StackExchange.Redis client reads a host:port string. ElastiCache Serverless gives you an endpoint name and requires TLS, hence Ssl = true in production.
  2. AbortOnConnectFail = false — if Valkey is briefly unreachable at start-up the client keeps retrying in the background instead of crashing the app; the readiness check reports the outage.
  3. AddSingleton<IConnectionMultiplexer>(redis) — one multiplexer per process. It is thread-safe and pipelines commands over few connections; creating one per request is the Redis equivalent of new HttpClient() per request.
  4. AddStackExchangeRedisCache with ConnectionMultiplexerFactory — the IDistributedCache implementation reuses our multiplexer. InstanceName prefixes every key so two apps never collide.
  5. AddSession — session data is now stored through IDistributedCache, i.e. in Valkey. The cookie only carries a session id. __Host- prefix, Secure, HttpOnly and SameSite=Strict harden the cookie.
  6. SetApplicationName — all tasks must use the same application name, otherwise they derive different keys from the same key ring and cannot read each other's cookies.
  7. PersistKeysToAWSSystemsManager — from Amazon.AspNetCore.DataProtection.SSM: each key is stored as a SecureString parameter, encrypted by KMS. The task role needs ssm:GetParametersByPath and ssm:PutParameter on that path.
  8. PersistKeysToStackExchangeRedis — the alternative for local runs (or when a cache is already in place). Keys in Redis are not encrypted at rest by the framework, so for production prefer the SSM option or add ProtectKeysWith....
  9. AddHttpClient("core-banking", ...) — IHttpClientFactory manages handler lifetimes. PooledConnectionLifetime forces connections to be re-created periodically, so DNS changes (a failover) are noticed.
  10. MaxConnectionsPerServer = 50 — a per-task bound on outbound connections. Multiply by your maximum task count and check that the downstream system can take it.
  11. Maximum Pool Size=15 in the connection string — Npgsql's pool limit per process (default 100). Ten tasks × 15 = 150 connections, reaching the database through RDS Proxy.
  12. /whoami — returns the container's host name so you can see different tasks answering while the session value stays the same.

Security considerations

⚠ Learning shortcut — not for production

Running Valkey locally without TLS or authentication, and storing Data Protection keys in Redis without extra encryption, is fine on your laptop. In AWS: enable in-transit encryption (Serverless caches require TLS), use IAM authentication or an AUTH token stored in Secrets Manager, allow port 6379 only from the API's security group, and keep Data Protection keys in KMS-encrypted SecureString parameters.

  • Session data is personal data. Store an identifier and minimal UI state, never full customer records or card data. Set short idle timeouts and TTLs (minutes, not days), consistent with your retention policy.
  • Caches leak across tenants if keys are guessable. Prefix keys per application and include the authenticated customer in any per-user key, so one user can never read another's cached entry.
  • Scaling limits are a security control. A maximum task count and Lambda reserved concurrency limit the blast radius (and bill) of a denial-of-wallet attack. Pair them with AWS WAF rate-based rules at the edge.
  • New tasks inherit everything. Because tasks are cattle, fix configuration in the task definition and IaC, never by hand on a running task.

Banking use case

Harbourline's mobile banking backend runs the Accounts API on ECS Fargate across two AZs:

  • Minimum 2 tasks, maximum 12, tracking ALBRequestCountPerTarget at 1,200 per minute (load tests showed one 0.5 vCPU task holds p95 under 150 ms up to about 1,700).
  • A scheduled action raises the minimum to 8 at 05:30 UTC on the 25th and on the last working day of the month, and lowers it at 11:00.
  • The API reads balances from DynamoDB on-demand (scales by itself) and statement metadata from Aurora PostgreSQL through RDS Proxy, with Maximum Pool Size=15: 12 × 15 = 180 client connections, which the proxy multiplexes onto fewer database connections.
  • A Lambda function that sends "salary received" push notifications reads from an SQS queue. Its reserved concurrency is 20 so that it cannot flood the notification provider's rate limit.
  • The capacity plan, the pool arithmetic and the scaling limits are reviewed by the change advisory board as part of operational resilience testing.

Common mistakes

  • Scaling on memory for .NET → the GC keeps memory reserved, so the metric never falls and the service never scales in → scale on CPU or requests.
  • No maximum, or a maximum chosen at random → a retry storm scales the fleet and the bill → derive the maximum from the database and budget.
  • Health check grace period too short → new .NET tasks are killed while still starting, then replaced, in a loop → set the grace period to cover real start-up time (lesson 12).
  • new HttpClient() per request → socket exhaustion only under load, never in tests → use IHttpClientFactory.
  • Forgetting Data Protection → login and anti-forgery failures that look random → shared key ring with the same application name.
  • Using sticky sessions to "fix" state → sessions still vanish at deployments and scale-in → move the state out.
  • Lambda in front of a relational database without limits → hundreds of connections in seconds → reserved concurrency plus RDS Proxy.

Troubleshooting

SymptomLikely causeFix
Service never scales out although latency is highScaling on CPU but the API is I/O bound (CPU stays low)Track ALBRequestCountPerTarget or add a custom latency-based step policy
Service scales out and in every few minutesCooldowns too short, or target too close to actual usageIncrease the scale-in cooldown; lower the target value
Desired count rises but running count does notTasks fail to start: image pull, secrets, subnet IPs, Fargate quotaECS service events and stopped-task reasons
Users logged out after a deploymentData Protection keys or sessions were in the containerShared key ring and distributed session store
Npgsql.NpgsqlException: The connection pool has been exhaustedPool too small for concurrency, or connections not disposedawait using every connection; check pool size and slow queries
FATAL: remaining connection slots are reserved / too many connectionstasks × pool above max_connectionsLower pool size, cap tasks, add RDS Proxy
API Gateway returns 429 under load with LambdaAccount or reserved concurrency exhaustedCheck ConcurrentExecutions and Throttles; raise quota or reservation
RedisConnectionException ... It was not possible to connectSecurity group, TLS setting or endpoint wrongAllow 6379 from the task security group; Ssl=true for Serverless

Cost and cleanup

  • Application Auto Scaling itself is free; you pay for the tasks it starts (Fargate ARM in us-east-1 is about USD 0.0324 per vCPU-hour and 0.0036 per GB-hour) and the CloudWatch alarms it creates.
  • ElastiCache Serverless for Valkey in us-east-1 bills about USD 0.084 per GB-hour of data stored, with a 100 MB minimum (roughly USD 0.0084 per hour at the minimum), plus USD 0.0023 per million ECPUs. Delete it the same day. Free alternative: run Valkey in Docker locally.
  • Provisioned concurrency bills every hour it is configured. Remove it after experiments.
# Remove scaling configuration from the lab service (deleting the scalable target removes its policies)
aws application-autoscaling deregister-scalable-target \
  --service-namespace ecs \
  --resource-id service/hl-m03-cluster/hl-m03-accounts-api \
  --scalable-dimension ecs:service:DesiredCount --profile learn-sandbox

# Remove a Lambda reservation
aws lambda delete-function-concurrency --function-name hl-m03-accounts-api --profile learn-sandbox

# Delete a serverless cache created for the exercises
aws elasticache delete-serverless-cache --serverless-cache-name hl-m03-sessions --profile learn-sandbox

# Then destroy the lab stacks (see labs/m03-accounts-api/README.md) and confirm nothing tagged remains
aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=cea-lab \
  --query "ResourceTagMappingList[].ResourceARN" --profile learn-sandbox

Exercises

  • 🟢 Guided: Run the Complete code app twice locally on different ports (--urls http://localhost:5101 and 5102) against one Valkey container. Set a month through the first instance and read it through the second (copy the session cookie). Then switch to AddDistributedMemoryCache() and observe that the second instance no longer sees it.
  • 🟡 Challenge: In the lab's FargateStack.cs, replace CPU tracking with request-count tracking plus a scheduled boost, and set the maximum from a stated database budget of 200 connections with a pool size of 15. Explain your numbers in a code comment.
  • 🔴 Live problem: Ticket from operations: "Every month-end at 07:10 the Statements API returns 500s for about four minutes. CPU on the tasks peaks at 35%. Aurora shows 'too many connections'. Auto scaling added 9 tasks." Write the incident analysis and the fix.

Assignment

Write a one-page capacity plan for the Accounts API in your portfolio repository: expected peak requests per second, requests per task at your latency objective (from a load test or a stated assumption), min/max tasks, scaling metric and target, pool sizes and the arithmetic against database limits, scheduled actions for known peaks, and the monthly cost at baseline and at peak.

Solutions

Knowledge check

Interview questions

  • "How do you make a .NET API horizontally scalable?" — Remove in-process state: sessions and caches to ElastiCache or DynamoDB, files to S3, Data Protection keys to a shared store, timers to a scheduler. Then put it behind a load balancer with health checks and an auto scaling policy with sensible limits.
  • "What metric would you scale a banking API on?" — Requests per target when the API is I/O bound, CPU when it is compute bound, usually both (the larger capacity wins). Plus scheduled actions for known peaks. Not memory for .NET.
  • "Lambda versus a database: what's the risk?" — Lambda can create hundreds of environments in seconds, each with its own connection. Cap it with reserved concurrency, use RDS Proxy, or use DynamoDB which has no connection limit.
  • "Sticky sessions: yes or no?" — No for APIs. They hide state, unbalance load, and break during deployments and scale-in. Acceptable briefly for a legacy app while state is moved out.

Certification practice

Completion checklist

  • I can list the five kinds of hidden state and the fix for each
  • I can explain target tracking, cooldowns, and min/max capacity
  • I can compute tasks × pool size and choose a maximum task count from it
  • I can explain Lambda concurrency, reserved and provisioned concurrency
  • I ran the session-sharing exercise locally
  • I removed any scaling targets, caches and provisioned concurrency I created

Key takeaways

  • Scale out, not up, and make the app stateless first: sessions, caches, files, Data Protection keys and timers all leave the process.
  • Target tracking on requests per target or CPU with a real maximum is the default for ECS; add scheduled actions for known banking peaks.
  • Lambda scales fast by itself; reserved concurrency and RDS Proxy protect what is behind it.
  • Every pool multiplies by the number of copies. Write the arithmetic down.
  • ElastiCache for Valkey and DynamoDB both back IDistributedCache; choose by latency need and network shape.

Further reading

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page