Scaling .NET apps on AWS
Make an ASP.NET Core API stateless, size its connection pools, move session state to ElastiCache or DynamoDB, and scale it automatically with target tracking on ECS and concurrency controls on Lambda.
Assumes: .NET 10 LTS · AWS SDK for .NET v4 · Amazon.CDK.Lib 2.277.0 · AWS CLI v2 · Region us-east-1. Verified against official sources on 2026-11-18.
Learning objectives
By the end of this lesson you can:
- Explain the difference between scaling out (more copies) and scaling up (bigger copies), and why AWS-hosted .NET APIs almost always scale out.
- Find and remove the hidden state that stops an ASP.NET Core app from running as many identical copies: in-memory sessions, local files, Data Protection keys, in-process caches and sticky sessions.
- Configure target tracking auto scaling for an ECS service on CPU and on ALB requests per target, and explain cooldowns, minimum and maximum capacity.
- Explain how Lambda scales (concurrency, the per-function scaling rate, reserved and provisioned concurrency) and why that can overwhelm a database.
- Size connection pools (database,
HttpClient, AWS SDK) so thattasks × pool sizestays below what the database can accept, and know when to add RDS Proxy. - Store session state and shared caches in ElastiCache (Valkey) or DynamoDB through
IDistributedCache.
Prerequisites
- Module 3 overview — the hosting options and the Accounts API
- Module 2: Networking — load balancers, target groups and private subnets
- Optional but useful: the module project code in
labs/m03-accounts-api(its Fargate stack already contains the scaling policy discussed here) - Tools: AWS CLI v2 with the
learn-sandboxprofile, .NET 10 SDK or Docker
Why this matters in banking
On the 25th of each month Harbourline Bank's salary payments land. Between 06:00 and 08:00 several hundred thousand customers open the mobile app to check their balance. Traffic to the Accounts API goes from 40 requests per second to 1,500 in twenty minutes. Then, by 10:00, it is back to normal.
Three things can go wrong, and all three have happened at real banks:
- Nothing scales. The fixed fleet of four servers saturates, response times climb from 80 ms to 8 seconds, the mobile app times out and customers phone the contact centre. Complaints go to the regulator's statistics.
- The app scales but breaks. New copies start, but customers are logged out at random, or see "anti-forgery token could not be decrypted", because session data and encryption keys lived in the memory of one server.
- The app scales and takes the database down. Twenty tasks each open a pool of 100 connections to a database that accepts 400. Everything fails at once, including the payments system that shares the database.
Scaling is therefore not a single setting. It is an architecture property: stateless code, bounded pools, a scaling policy and limits that protect what sits behind the API.
The idea in plain language
Scaling up means giving one server more CPU and memory. Scaling out means running more identical copies behind a load balancer. Cloud platforms are built for scaling out: copies can be added in a minute and removed when they are no longer needed, so you only pay for what the traffic needs.
Scaling out has one precondition: any copy must be able to serve any request. If a request depends on something that only one copy has (a session in memory, a file on the local disk, a key in a local folder) the second copy gives a wrong answer. An app that keeps nothing between requests, except in shared external stores, is called stateless.
Auto scaling is the robot that decides how many copies to run. You tell it a target (for example "keep average CPU at 60%") and limits (between 2 and 10 copies). It watches a CloudWatch metric and adds or removes copies to stay near the target, like a thermostat.
Analogy: a bank branch on salary day. Opening more counters (scaling out) works only if every cashier can serve any customer, which means the customer's file is in the central system, not in one cashier's drawer (statelessness). The branch manager watches the queue length and opens or closes counters (target tracking). And there is a limit to how many cashiers the back office can support at once (connection pools and the database).
What problem AWS auto scaling solves
On-premises, capacity is bought for the peak and idles the rest of the month. In AWS you can match capacity to load, but only if three layers cooperate:
| Layer | Service | What scales |
|---|---|---|
| Containers on ECS | Application Auto Scaling with an ECS service as the scalable target | The service's desired count of tasks |
| EC2 instances (Beanstalk, ECS on EC2) | EC2 Auto Scaling groups | The number of instances |
| Lambda | Built into Lambda | Concurrent execution environments, automatically per request |
| DynamoDB | On-demand mode, or Application Auto Scaling for provisioned capacity | Read and write capacity |
| Aurora | Aurora Serverless v2 capacity units and Aurora replicas | Database capacity and read replicas |
Limitations and sharp edges you should know before relying on it:
- Auto scaling reacts to metrics that are one or more minutes old. A new Fargate task then needs time to pull the image, start .NET and pass health checks. Expect two to four minutes from "load rises" to "new task serves traffic". For a known peak like salary day, use scheduled scaling to raise the minimum in advance.
- Scaling out never fixes a bottleneck behind the API. If the database is the limit, more tasks make it worse.
- Every scaling policy needs a maximum. Without one, a bug (an endless retry loop) or an attack can scale you into a large bill.
- Lambda scales very fast and has no built-in notion of "the database can only take 200 connections". You must limit it yourself.
Microsoft / Azure comparison
| You know (Microsoft/Azure) | AWS equivalent | What's different |
|---|---|---|
| App Service "Scale out" rules (autoscale settings) | Application Auto Scaling target tracking on an ECS service | Target tracking computes the steps for you; you set a target value, not "add 1 instance when CPU > 70%" |
| VM Scale Sets autoscale | EC2 Auto Scaling groups | Same idea; Beanstalk creates one for you in load-balanced environments |
| Azure Functions Consumption plan scaling | Lambda concurrency scaling | Lambda scales per function at up to 1,000 extra concurrent executions every 10 seconds; you cap it with reserved concurrency |
| Functions Premium "always ready" instances | Lambda provisioned concurrency | Billed per hour while configured |
| ARR affinity (sticky sessions) on App Service | ALB target group stickiness | Available, but a design smell for APIs: it hides state instead of removing it |
| Azure Cache for Redis | Amazon ElastiCache (Valkey, Redis OSS, Memcached) | Valkey is the open-source Redis fork; same protocol, works with StackExchange.Redis |
| Session state in Azure SQL / Cosmos DB | DynamoDB via IDistributedCache provider | Use DynamoDB TTL to expire items |
| Data Protection keys in Blob Storage + Key Vault | Data Protection keys in SSM Parameter Store (Amazon.AspNetCore.DataProtection.SSM) or in Redis | Encrypt with KMS through SecureString parameters |
| Azure SQL connection limits per tier | RDS max_connections (depends on instance memory) + RDS Proxy | RDS Proxy pools and shares connections between many clients |
Architecture
How a scale-out event plays out:
Security boundary view. Scaling must not change the security posture. New tasks use the same task role (least privilege), run in the same private subnets with the same security groups, and connect to ElastiCache with in-transit encryption and an authentication token or IAM authentication. The external stores you add for state (cache, session table) hold personal data, so they need encryption at rest, retention limits and access logging like any other data store.
Making an ASP.NET Core app stateless
Before you add any scaling policy, look for these five kinds of hidden state. Each one has a standard fix.
| Hidden state | Symptom after scaling out | Fix |
|---|---|---|
AddSession() with the default in-memory store | Users lose their session when the next request lands on another task | IDistributedCache backed by ElastiCache or DynamoDB |
IMemoryCache holding data that must be consistent | Different tasks return different answers until their caches expire | Keep IMemoryCache only for data where short inconsistency is fine; use a shared cache or none for the rest |
| Data Protection keys in the local file system | "The key was not found in the key ring", anti-forgery and auth cookie failures, after deployments or on other tasks | Persist keys to a shared store and protect them with KMS |
| Uploaded or generated files on local disk | Download link works only on one task, files vanish when the task stops | Write to S3; return a pre-signed URL |
| Background timers in the web process (e.g. "send reminders every hour") | The job runs N times when N tasks run | Move to EventBridge Scheduler or a single worker service (see Background processing) |
You saw the Data Protection warning in the lab's test output: ASP.NET Core logs Storing keys in a directory ... that may not be persisted outside of the container. The Accounts API is a pure bearer-token API with no cookies, so the warning is harmless there. A web app that issues cookies or anti-forgery tokens must fix it before running two tasks.
Sticky sessions (ALB target group stickiness) route a client back to the same task with a cookie. They make the symptoms disappear but keep the problem: when that task is replaced during a deployment or a scale-in, the session is gone. They also unbalance load. Treat them as a temporary crutch for a legacy app, not a design.
Scaling policies on ECS
ECS service auto scaling is provided by Application Auto Scaling. The scalable target is the ECS service; the scalable dimension is ecs:service:DesiredCount. You choose one of three policy types:
| Policy type | How it works | Use it for |
|---|---|---|
| Target tracking | Keep a metric near a target value. AWS creates and manages the CloudWatch alarms | The default for APIs |
| Step scaling | You define alarm thresholds and how many tasks to add or remove per step | Unusual metrics, or when you need asymmetric reactions |
| Scheduled scaling | Change min/max capacity at a time (cron or one-off) | Known peaks: salary day, month-end batch, market open |
| Predictive scaling | Forecasts load from history and scales ahead of it | Strongly cyclical traffic ⚠ VERIFY current ECS support in your Region before relying on it |
Three predefined metrics are available for target tracking on ECS services:
ECSServiceAverageCPUUtilization— average CPU of the service's tasks.ECSServiceAverageMemoryUtilization— average memory.ALBRequestCountPerTarget— requests per target per minute for one target group. Needs aResourceLabelidentifying the load balancer and target group.
Which metric for a .NET API? CPU works well for compute-bound APIs. Many banking APIs are I/O bound instead: they wait on databases and downstream services, so CPU stays low while latency climbs. For those, requests per target is a better signal, because it tracks load directly. Measure how many requests per minute one task handles at your latency objective (with a load test) and set the target at about 70% of that. Memory is rarely a good scaling metric for .NET: the garbage collector holds memory it does not need, so memory does not drop when load drops and the service never scales in.
Cooldowns and stability. After a scale-out, Application Auto Scaling waits for the scale-out cooldown before deciding to add more, so that the new tasks have time to absorb load. Scale-in is deliberately slower and more conservative. Target tracking only scales in when the metric is well below the target for a sustained period. Set a scale-in cooldown long enough (a few minutes) to avoid "flapping": adding and removing tasks repeatedly.
Minimum and maximum. For production, run at least two tasks in two Availability Zones so that one task or one AZ can fail. Set the maximum from two numbers: the most you are willing to pay, and the most the database can serve (see the connection-pool section).
How Lambda scales
Lambda scales without a policy. Each concurrent request needs its own execution environment; if none is free, Lambda creates one (a cold start). The number of environments busy at the same time is the function's concurrency.
- An account has a Regional concurrency quota shared by all functions. The default is 1,000; new accounts often start much lower (10 is common), and you request an increase through Service Quotas ⚠ VERIFY the value shown in your sandbox account.
- Each function can scale by up to 1,000 additional concurrent executions every 10 seconds, until the account quota is reached.
- Reserved concurrency sets both a guaranteed share and a hard maximum for one function. It is how you stop a function from opening 1,000 database connections. Lambda refuses a reservation that would leave the unreserved pool below 100, which is why a new account with a quota of 10 cannot reserve anything.
- Provisioned concurrency keeps environments initialised in advance, removing cold starts for that many concurrent requests. It is billed per hour while configured, and it can itself be scaled with Application Auto Scaling (metric
LambdaProvisionedConcurrencyUtilization) or scheduled before a known peak.
Requests beyond the available concurrency are throttled (HTTP 429 from API Gateway, TooManyRequestsException from a direct invoke). For asynchronous and queue-driven invocations Lambda retries for you; for synchronous API calls the client sees the error.
Connection pools: the part that breaks databases
Every copy of your app keeps pools of open connections. Scaling out multiplies them.
connections at peak = max tasks × pool size per task
= 10 × 100 (Npgsql/SqlClient default Max Pool Size) = 1,000An Aurora PostgreSQL instance's max_connections depends on its memory; a small instance accepts a few hundred at most. The fix has three parts:
- Size the pool deliberately. Set
Maximum Pool Size(Npgsql) orMax Pool Size(SqlClient) in the connection string so thatmax tasks × pool sizeis below the database limit with headroom for admin tools and other apps. A pool of 10–20 per task is plenty for most APIs, because each request holds a connection for milliseconds. - Add RDS Proxy when many short-lived clients (Lambda in particular) connect. The proxy keeps a smaller set of real database connections and shares them between client connections. It also speeds up failover.
- Cap the compute. Set the ECS maximum task count, or Lambda reserved concurrency, from the database budget, not from hope.
HTTP connections. HttpClient should be created through IHttpClientFactory (or a long-lived instance with SocketsHttpHandler.PooledConnectionLifetime set) so that sockets are reused and DNS changes are picked up. Creating new HttpClient() per request exhausts ephemeral ports under load, the classic SocketException: Address already in use on Linux containers.
AWS SDK clients. Register SDK clients as singletons through AddAWSService<T>(), as the Accounts API does. A singleton client reuses its HTTP connections. Creating a client per request is slow (credential resolution, TLS handshakes) and can trigger throttling on the credential endpoint.
Session state and shared caches
When the app does need per-user state, put it in a store every task can reach. ASP.NET Core's IDistributedCache abstraction lets you switch the store without changing app code.
| Option | Package | Strengths | Watch out for |
|---|---|---|---|
| ElastiCache Serverless for Valkey | Microsoft.Extensions.Caching.StackExchangeRedis 10.0.15 | Sub-millisecond reads, no nodes to size, scales automatically | Billed per GB-hour of stored data (minimum 100 MB) and per ECPU; VPC-only access |
| ElastiCache node-based (Valkey) | Same | Predictable hourly price for steady load | You choose node size and replicas; plan failover |
| DynamoDB | AWS.AspNetCore.DistributedCacheProvider 1.0.3 | Serverless, pay per request, no VPC needed (works from Lambda without VPC) | Single-digit millisecond latency, not sub-millisecond; enable TTL to expire items |
Valkey is a Linux Foundation fork of Redis OSS. ElastiCache offers it at a lower price than Redis OSS (AWS states 33% lower for Serverless and 20% lower for node-based clusters), and it speaks the same protocol, so StackExchange.Redis works unchanged.
Step-by-step implementation
Audit the app for hidden state
Search the code for AddSession, AddMemoryCache, File.Write, Path.GetTempPath, System.Timers.Timer and PeriodicTimer. For each hit, decide: remove it, move it to a shared store, or prove it is safe to have per-task copies (for example a 30-second cache of reference data).
Move Data Protection keys and sessions to shared stores
Use the Program.cs in the Complete code section: Data Protection keys go to Parameter Store (encrypted with KMS), sessions and the shared cache go to Valkey.
Bound every pool
Set the database pool size in the connection string, create HTTP clients through IHttpClientFactory, register AWS clients once. Write the arithmetic (max tasks × pool) into the service's runbook.
Load test one task
Run a load test against a single task (for example with the lab's tools/latency-probe.cs for latency, and a load tool such as k6 or bombardier for throughput) to find the requests per minute one task sustains at your p95 objective.
Add target tracking, limits and a scheduled boost
Create a target tracking policy on ALBRequestCountPerTarget (or CPU), set min 2 and a max derived from the database budget, then add a scheduled action that raises the minimum before salary-day mornings.
Watch it work
Generate load, watch the DesiredTaskCount and RunningTaskCount metrics in CloudWatch (Container Insights) and the scaling activities in the ECS console. Confirm that scale-in happens after the load stops.
AWS Console instructions
- ECS → Clusters →
hl-m03-cluster→ Services →hl-m03-accounts-api→ Update service → Service auto scaling. Choose Use service auto scaling, set minimum, desired and maximum tasks, then add a Target tracking policy: pickECSServiceAverageCPUUtilizationorALBRequestCountPerTarget, enter the target value and cooldowns. - ECS → service → Deployments and events / Events tab shows each scaling action with the reason.
- CloudWatch → Alarms lists the two alarms that target tracking created (names start with
TargetTracking-service/...). Do not edit or delete them; change the policy instead. - Lambda → Functions → your function → Configuration → Concurrency and recursion detection sets reserved concurrency; Versions/Aliases → Provisioned concurrency sets provisioned concurrency on an alias or version.
- ElastiCache → Valkey caches → Create Valkey cache → Serverless creates a serverless cache; choose the VPC, private subnets and a security group that only allows the API tasks' security group on port 6379.
AWS CLI
# 1. Register the ECS service as a scalable target (min 2, max 10 tasks)
aws application-autoscaling register-scalable-target \
--service-namespace ecs \
--resource-id service/hl-m03-cluster/hl-m03-accounts-api \
--scalable-dimension ecs:service:DesiredCount \
--min-capacity 2 --max-capacity 10 \
--profile learn-sandbox
# 2. Target tracking on requests per target (replace the resource label:
# app/<alb-name>/<alb-id>/targetgroup/<tg-name>/<tg-id>)
aws application-autoscaling put-scaling-policy \
--service-namespace ecs \
--resource-id service/hl-m03-cluster/hl-m03-accounts-api \
--scalable-dimension ecs:service:DesiredCount \
--policy-name requests-per-target \
--policy-type TargetTrackingScaling \
--target-tracking-scaling-policy-configuration '{
"TargetValue": 1000,
"PredefinedMetricSpecification": {
"PredefinedMetricType": "ALBRequestCountPerTarget",
"ResourceLabel": "app/hl-m03-alb/0123456789abcdef/targetgroup/hl-m03-tg/fedcba9876543210"
},
"ScaleOutCooldown": 60,
"ScaleInCooldown": 300
}' \
--profile learn-sandbox
# 3. Salary-day boost: at 05:30 UTC on the 25th raise the minimum to 6
aws application-autoscaling put-scheduled-action \
--service-namespace ecs \
--resource-id service/hl-m03-cluster/hl-m03-accounts-api \
--scalable-dimension ecs:service:DesiredCount \
--scheduled-action-name salary-day-boost \
--schedule "cron(30 5 25 * ? *)" \
--scalable-target-action MinCapacity=6,MaxCapacity=10 \
--profile learn-sandbox
# 4. See what auto scaling decided and why
aws application-autoscaling describe-scaling-activities \
--service-namespace ecs \
--resource-id service/hl-m03-cluster/hl-m03-accounts-api \
--max-items 5 --profile learn-sandbox
# 5. Lambda: cap a function at 50 concurrent executions (protects the database)
aws lambda put-function-concurrency \
--function-name hl-m03-accounts-api \
--reserved-concurrent-executions 50 \
--profile learn-sandbox
# 6. Check the account's concurrency quota and how much is unreserved
aws lambda get-account-settings --query AccountLimit --profile learn-sandboxA second scheduled action (for example at 11:00 on the 25th) should set the minimum back to 2; otherwise the boost stays forever.
Infrastructure as Code
The lab's Fargate stack already contains a CPU target tracking policy. This version adds request-based tracking and a scheduled boost.
using Amazon.CDK;
using Amazon.CDK.AWS.ApplicationAutoScaling;
using Amazon.CDK.AWS.ECS;
// service is the ApplicationLoadBalancedFargateService from FargateStack.cs
var scaling = service.Service.AutoScaleTaskCount(new EnableScalingProps
{
MinCapacity = 2,
MaxCapacity = 10, // from the database budget: 10 tasks x 15 connections = 150 < max_connections
});
// I/O-bound API: track requests per task rather than CPU.
scaling.ScaleOnRequestCount("RequestsPerTarget", new RequestCountScalingProps
{
RequestsPerTarget = 1000, // per minute, about 70% of what one task handles at p95 target
TargetGroup = service.TargetGroup,
ScaleOutCooldown = Duration.Seconds(60),
ScaleInCooldown = Duration.Seconds(300),
});
// Belt and braces: also track CPU. The policy asking for MORE capacity wins.
scaling.ScaleOnCpuUtilization("CpuTarget", new CpuUtilizationScalingProps
{
TargetUtilizationPercent = 60,
});
// Salary day: raise the floor before customers wake up, lower it afterwards.
scaling.ScaleOnSchedule("SalaryDayBoost", new ScalingSchedule
{
Schedule = Schedule.Cron(new CronOptions { Minute = "30", Hour = "5", Day = "25" }),
MinCapacity = 6,
});
scaling.ScaleOnSchedule("SalaryDayEnd", new ScalingSchedule
{
Schedule = Schedule.Cron(new CronOptions { Minute = "0", Hour = "11", Day = "25" }),
MinCapacity = 2,
});Complete code
A stateless ASP.NET Core web app with shared Data Protection keys, distributed session and cache in Valkey, a bounded database pool and a correctly managed HttpClient. It runs locally against a Valkey container (docker run -p 6379:6379 valkey/valkey:8) with no AWS account when DataProtection:UseParameterStore is false.
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<RootNamespace>Harbourline.Statements.Web</RootNamespace>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.Extensions.Caching.StackExchangeRedis" Version="10.0.15" />
<PackageReference Include="Microsoft.AspNetCore.DataProtection.StackExchangeRedis" Version="10.0.15" />
<PackageReference Include="Amazon.AspNetCore.DataProtection.SSM" Version="4.0.3" />
<PackageReference Include="AWSSDK.Extensions.NETCore.Setup" Version="4.1.0.2" />
</ItemGroup>
</Project>using Microsoft.AspNetCore.DataProtection;
using StackExchange.Redis;
var builder = WebApplication.CreateBuilder(args);
// ---- 1. One shared connection to Valkey/Redis for the whole process (it multiplexes internally).
var cacheEndpoint = builder.Configuration["Cache:Endpoint"] ?? "localhost:6379";
var redisOptions = ConfigurationOptions.Parse(cacheEndpoint);
redisOptions.Ssl = builder.Configuration.GetValue("Cache:UseTls", false); // true for ElastiCache
redisOptions.AbortOnConnectFail = false; // keep retrying in the background
redisOptions.ConnectTimeout = 2000;
var redis = await ConnectionMultiplexer.ConnectAsync(redisOptions);
builder.Services.AddSingleton<IConnectionMultiplexer>(redis);
// ---- 2. IDistributedCache + session state in Valkey (shared by every task).
builder.Services.AddStackExchangeRedisCache(o =>
{
o.ConnectionMultiplexerFactory = () => Task.FromResult<IConnectionMultiplexer>(redis);
o.InstanceName = "hl-statements:"; // key prefix, so several apps can share one cache safely
});
builder.Services.AddSession(o =>
{
o.IdleTimeout = TimeSpan.FromMinutes(15); // banking sessions should be short
o.Cookie.HttpOnly = true;
o.Cookie.SecurePolicy = CookieSecurePolicy.Always;
o.Cookie.SameSite = SameSiteMode.Strict;
o.Cookie.Name = "__Host-hl-session";
});
// ---- 3. Data Protection keys: one key ring for all tasks.
var dp = builder.Services.AddDataProtection().SetApplicationName("Harbourline.Statements.Web");
if (builder.Configuration.GetValue("DataProtection:UseParameterStore", false))
{
// Keys stored as SecureString parameters under this path (encrypted with KMS).
builder.Services.AddDefaultAWSOptions(builder.Configuration.GetAWSOptions());
dp.PersistKeysToAWSSystemsManager("/hl-statements/data-protection");
}
else
{
dp.PersistKeysToStackExchangeRedis(redis, "hl-statements:dp-keys");
}
// ---- 4. Outbound HTTP through the factory: pooled sockets, recycled every 5 minutes for DNS changes.
builder.Services.AddHttpClient("core-banking", c =>
{
c.BaseAddress = new Uri(builder.Configuration["CoreBanking:BaseUrl"] ?? "https://core-banking.internal.example.net/");
c.Timeout = TimeSpan.FromSeconds(5);
})
.ConfigurePrimaryHttpMessageHandler(() => new SocketsHttpHandler
{
PooledConnectionLifetime = TimeSpan.FromMinutes(5),
MaxConnectionsPerServer = 50, // bound per task: 10 tasks x 50 = 500 connections to core banking
});
var app = builder.Build();
app.UseSession();
// Remembers the last statement month the customer viewed — works on ANY task.
app.MapGet("/statements/last-viewed", (HttpContext ctx) =>
Results.Ok(new { month = ctx.Session.GetString("lastMonth") ?? "none" }));
app.MapPost("/statements/{month}/view", (string month, HttpContext ctx) =>
{
if (!System.Text.RegularExpressions.Regex.IsMatch(month, "^20[0-9]{2}-(0[1-9]|1[0-2])$"))
{
return Results.Problem(statusCode: 400, title: "Invalid month", detail: "Use YYYY-MM.");
}
ctx.Session.SetString("lastMonth", month);
return Results.NoContent();
});
// Which task answered? Proves requests are spread and sessions survive.
app.MapGet("/whoami", () => Results.Ok(new { task = Environment.MachineName }));
app.MapGet("/health/live", () => Results.Ok());
app.MapGet("/health/ready", async (IConnectionMultiplexer mux) =>
mux.IsConnected && await mux.GetDatabase().PingAsync() < TimeSpan.FromSeconds(1)
? Results.Ok()
: Results.StatusCode(503));
app.Run();{
"Cache": {
"Endpoint": "hl-statements-abc123.serverless.use1.cache.amazonaws.com:6379",
"UseTls": true
},
"DataProtection": { "UseParameterStore": true },
"ConnectionStrings": {
"Statements": "Host=hl-statements-proxy.proxy-xxxx.us-east-1.rds.amazonaws.com;Database=statements;Username=app;Maximum Pool Size=15;Timeout=5;Command Timeout=10"
}
}Code explained line by line
ConfigurationOptions.Parse(cacheEndpoint)— the StackExchange.Redis client reads ahost:portstring. ElastiCache Serverless gives you an endpoint name and requires TLS, henceSsl = truein production.AbortOnConnectFail = false— if Valkey is briefly unreachable at start-up the client keeps retrying in the background instead of crashing the app; the readiness check reports the outage.AddSingleton<IConnectionMultiplexer>(redis)— one multiplexer per process. It is thread-safe and pipelines commands over few connections; creating one per request is the Redis equivalent ofnew HttpClient()per request.AddStackExchangeRedisCachewithConnectionMultiplexerFactory— theIDistributedCacheimplementation reuses our multiplexer.InstanceNameprefixes every key so two apps never collide.AddSession— session data is now stored throughIDistributedCache, i.e. in Valkey. The cookie only carries a session id.__Host-prefix,Secure,HttpOnlyandSameSite=Strictharden the cookie.SetApplicationName— all tasks must use the same application name, otherwise they derive different keys from the same key ring and cannot read each other's cookies.PersistKeysToAWSSystemsManager— fromAmazon.AspNetCore.DataProtection.SSM: each key is stored as aSecureStringparameter, encrypted by KMS. The task role needsssm:GetParametersByPathandssm:PutParameteron that path.PersistKeysToStackExchangeRedis— the alternative for local runs (or when a cache is already in place). Keys in Redis are not encrypted at rest by the framework, so for production prefer the SSM option or addProtectKeysWith....AddHttpClient("core-banking", ...)—IHttpClientFactorymanages handler lifetimes.PooledConnectionLifetimeforces connections to be re-created periodically, so DNS changes (a failover) are noticed.MaxConnectionsPerServer = 50— a per-task bound on outbound connections. Multiply by your maximum task count and check that the downstream system can take it.Maximum Pool Size=15in the connection string — Npgsql's pool limit per process (default 100). Ten tasks × 15 = 150 connections, reaching the database through RDS Proxy./whoami— returns the container's host name so you can see different tasks answering while the session value stays the same.
Security considerations
⚠ Learning shortcut — not for production
Running Valkey locally without TLS or authentication, and storing Data Protection keys in Redis without extra encryption, is fine on your laptop. In AWS: enable in-transit encryption (Serverless caches require TLS), use IAM authentication or an AUTH token stored in Secrets Manager, allow port 6379 only from the API's security group, and keep Data Protection keys in KMS-encrypted SecureString parameters.
- Session data is personal data. Store an identifier and minimal UI state, never full customer records or card data. Set short idle timeouts and TTLs (minutes, not days), consistent with your retention policy.
- Caches leak across tenants if keys are guessable. Prefix keys per application and include the authenticated customer in any per-user key, so one user can never read another's cached entry.
- Scaling limits are a security control. A maximum task count and Lambda reserved concurrency limit the blast radius (and bill) of a denial-of-wallet attack. Pair them with AWS WAF rate-based rules at the edge.
- New tasks inherit everything. Because tasks are cattle, fix configuration in the task definition and IaC, never by hand on a running task.
Banking use case
Harbourline's mobile banking backend runs the Accounts API on ECS Fargate across two AZs:
- Minimum 2 tasks, maximum 12, tracking
ALBRequestCountPerTargetat 1,200 per minute (load tests showed one 0.5 vCPU task holds p95 under 150 ms up to about 1,700). - A scheduled action raises the minimum to 8 at 05:30 UTC on the 25th and on the last working day of the month, and lowers it at 11:00.
- The API reads balances from DynamoDB on-demand (scales by itself) and statement metadata from Aurora PostgreSQL through RDS Proxy, with
Maximum Pool Size=15: 12 × 15 = 180 client connections, which the proxy multiplexes onto fewer database connections. - A Lambda function that sends "salary received" push notifications reads from an SQS queue. Its reserved concurrency is 20 so that it cannot flood the notification provider's rate limit.
- The capacity plan, the pool arithmetic and the scaling limits are reviewed by the change advisory board as part of operational resilience testing.
Common mistakes
- Scaling on memory for .NET → the GC keeps memory reserved, so the metric never falls and the service never scales in → scale on CPU or requests.
- No maximum, or a maximum chosen at random → a retry storm scales the fleet and the bill → derive the maximum from the database and budget.
- Health check grace period too short → new .NET tasks are killed while still starting, then replaced, in a loop → set the grace period to cover real start-up time (lesson 12).
new HttpClient()per request → socket exhaustion only under load, never in tests → useIHttpClientFactory.- Forgetting Data Protection → login and anti-forgery failures that look random → shared key ring with the same application name.
- Using sticky sessions to "fix" state → sessions still vanish at deployments and scale-in → move the state out.
- Lambda in front of a relational database without limits → hundreds of connections in seconds → reserved concurrency plus RDS Proxy.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| Service never scales out although latency is high | Scaling on CPU but the API is I/O bound (CPU stays low) | Track ALBRequestCountPerTarget or add a custom latency-based step policy |
| Service scales out and in every few minutes | Cooldowns too short, or target too close to actual usage | Increase the scale-in cooldown; lower the target value |
| Desired count rises but running count does not | Tasks fail to start: image pull, secrets, subnet IPs, Fargate quota | ECS service events and stopped-task reasons |
| Users logged out after a deployment | Data Protection keys or sessions were in the container | Shared key ring and distributed session store |
Npgsql.NpgsqlException: The connection pool has been exhausted | Pool too small for concurrency, or connections not disposed | await using every connection; check pool size and slow queries |
FATAL: remaining connection slots are reserved / too many connections | tasks × pool above max_connections | Lower pool size, cap tasks, add RDS Proxy |
| API Gateway returns 429 under load with Lambda | Account or reserved concurrency exhausted | Check ConcurrentExecutions and Throttles; raise quota or reservation |
RedisConnectionException ... It was not possible to connect | Security group, TLS setting or endpoint wrong | Allow 6379 from the task security group; Ssl=true for Serverless |
Cost and cleanup
- Application Auto Scaling itself is free; you pay for the tasks it starts (Fargate ARM in us-east-1 is about USD 0.0324 per vCPU-hour and 0.0036 per GB-hour) and the CloudWatch alarms it creates.
- ElastiCache Serverless for Valkey in us-east-1 bills about USD 0.084 per GB-hour of data stored, with a 100 MB minimum (roughly USD 0.0084 per hour at the minimum), plus USD 0.0023 per million ECPUs. Delete it the same day. Free alternative: run Valkey in Docker locally.
- Provisioned concurrency bills every hour it is configured. Remove it after experiments.
# Remove scaling configuration from the lab service (deleting the scalable target removes its policies)
aws application-autoscaling deregister-scalable-target \
--service-namespace ecs \
--resource-id service/hl-m03-cluster/hl-m03-accounts-api \
--scalable-dimension ecs:service:DesiredCount --profile learn-sandbox
# Remove a Lambda reservation
aws lambda delete-function-concurrency --function-name hl-m03-accounts-api --profile learn-sandbox
# Delete a serverless cache created for the exercises
aws elasticache delete-serverless-cache --serverless-cache-name hl-m03-sessions --profile learn-sandbox
# Then destroy the lab stacks (see labs/m03-accounts-api/README.md) and confirm nothing tagged remains
aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=cea-lab \
--query "ResourceTagMappingList[].ResourceARN" --profile learn-sandboxExercises
- 🟢 Guided: Run the Complete code app twice locally on different ports (
--urls http://localhost:5101and5102) against one Valkey container. Set a month through the first instance and read it through the second (copy the session cookie). Then switch toAddDistributedMemoryCache()and observe that the second instance no longer sees it. - 🟡 Challenge: In the lab's
FargateStack.cs, replace CPU tracking with request-count tracking plus a scheduled boost, and set the maximum from a stated database budget of 200 connections with a pool size of 15. Explain your numbers in a code comment. - 🔴 Live problem: Ticket from operations: "Every month-end at 07:10 the Statements API returns 500s for about four minutes. CPU on the tasks peaks at 35%. Aurora shows 'too many connections'. Auto scaling added 9 tasks." Write the incident analysis and the fix.
Assignment
Write a one-page capacity plan for the Accounts API in your portfolio repository: expected peak requests per second, requests per task at your latency objective (from a load test or a stated assumption), min/max tasks, scaling metric and target, pool sizes and the arithmetic against database limits, scheduled actions for known peaks, and the monthly cost at baseline and at peak.
Solutions
With the Redis-backed IDistributedCache, both instances read and write the same key (hl-statements:<session id>), so the month set through port 5101 is returned by 5102 when you send the same cookie. Data Protection matters here too: the session cookie is protected with the key ring, and both instances share it via PersistKeysToStackExchangeRedis and the same application name. With AddDistributedMemoryCache() each process has its own dictionary, so the second instance returns none. If you also remove the shared key ring, the second instance cannot even decrypt the cookie and silently starts a new session. That is exactly what happens behind a load balancer.
Budget: 200 connections, keep 20% headroom for admin tools and failover = 160. With a pool of 15 per task, 160 / 15 = 10.6, so MaxCapacity = 10. Code: AutoScaleTaskCount(new EnableScalingProps { MinCapacity = 2, MaxCapacity = 10 }), then ScaleOnRequestCount("RequestsPerTarget", new RequestCountScalingProps { RequestsPerTarget = 1000, TargetGroup = service.TargetGroup }), then two ScaleOnSchedule calls as in the IaC section. Why request count: the API waits on DynamoDB and the database, so CPU underestimates load. Why two schedules: a scheduled action only changes capacity limits at one moment; without the second one the higher minimum would stay.
Analysis: the service scaled correctly, which is the problem. Nine extra tasks, each with the default pool of 100, tried to open connections at the same moment; the database ran out of connection slots and every task failed, including the ones that were working. CPU at 35% shows the API was not compute bound: it was waiting on the database, so scaling out could not help. Fix: (1) set Maximum Pool Size explicitly (for example 15); (2) put RDS Proxy between the tasks and Aurora; (3) derive the maximum task count from the connection budget; (4) add a scheduled action so capacity is already there before 07:00 rather than reacting during the spike; (5) look at the month-end queries: if one statement report causes long-held connections, move it to a read replica or a background job. Add an alarm on DatabaseConnections approaching the limit.
Knowledge check
Answer: The .NET garbage collector keeps memory it has already reserved, so memory use stays high after load drops. A memory-based policy scales out but rarely scales back in. CPU or requests per target follow load in both directions.
Answer: The default Maximum Pool Size is 100, so 12 tasks can open up to 1,200 connections, more than twice the limit. Under load the database rejects new connections and requests fail on every task. Set an explicit pool size, cap the task count and consider RDS Proxy.
Answer: It reserves a share of the account's concurrency for one function and caps that function at the same number. Lambda requires at least 100 unreserved concurrent executions to remain, so an account whose quota is lower than that (new accounts are often at 10) cannot reserve any.
Answer: Cookies, anti-forgery tokens and TempData are encrypted with Data Protection keys. If each task generates its own keys, a cookie encrypted by one task cannot be decrypted by another, so users see random authentication or anti-forgery failures. Persist keys to a shared store and use the same application name everywhere.
Answer: For predictable peaks (salary day, month-end, market open). Target tracking reacts after metrics change and new tasks take minutes to become healthy; a scheduled increase of the minimum has capacity ready before the peak, while target tracking still handles the unexpected part.
Interview questions
- "How do you make a .NET API horizontally scalable?" — Remove in-process state: sessions and caches to ElastiCache or DynamoDB, files to S3, Data Protection keys to a shared store, timers to a scheduler. Then put it behind a load balancer with health checks and an auto scaling policy with sensible limits.
- "What metric would you scale a banking API on?" — Requests per target when the API is I/O bound, CPU when it is compute bound, usually both (the larger capacity wins). Plus scheduled actions for known peaks. Not memory for .NET.
- "Lambda versus a database: what's the risk?" — Lambda can create hundreds of environments in seconds, each with its own connection. Cap it with reserved concurrency, use RDS Proxy, or use DynamoDB which has no connection limit.
- "Sticky sessions: yes or no?" — No for APIs. They hide state, unbalance load, and break during deployments and scale-in. Acceptable briefly for a legacy app while state is moved out.
Certification practice
A company runs an ASP.NET Core web application on Amazon ECS with Fargate behind an Application Load Balancer. After enabling auto scaling, users report that they are randomly logged out. The application stores sessions with the default in-memory provider. Which change fixes the problem with the LEAST operational overhead?
A. Enable target group stickiness on the ALB. B. Store session state in Amazon ElastiCache (Valkey) through the distributed cache provider. C. Increase the minimum number of tasks so scaling events happen less often. D. Store session state on an Amazon EFS file system mounted into every task.
Answer: B. A shared, managed in-memory store removes the state from the tasks. A only hides the issue and still loses sessions when tasks are replaced. C reduces frequency but does not fix it. D works but adds latency and file-locking complexity compared with a purpose-built cache.
An AWS Lambda function behind Amazon API Gateway queries an Amazon Aurora PostgreSQL database. During a marketing campaign the database reports too many connections and the API fails. Which combination of actions addresses this? (Choose TWO.)
A. Configure reserved concurrency on the Lambda function. B. Increase the Lambda function's memory. C. Place Amazon RDS Proxy between the function and the database. D. Enable API Gateway caching for POST requests. E. Move the function into a public subnet.
Answer: A and C. Reserved concurrency caps how many environments (and connections) the function can create; RDS Proxy pools and shares database connections. B speeds each invocation but does not limit connections. D does not cache POSTs by default and would be wrong for writes. E is irrelevant to connections and worse for security.
A bank's API on ECS uses target tracking on CPU. Every month on the 25th at 06:00, traffic rises tenfold within 15 minutes and latency is high for the first ten minutes before scaling catches up. What is the MOST effective improvement?
A. Lower the CPU target value from 60% to 20% permanently. B. Add a scheduled scaling action that raises the minimum capacity before 06:00 on the 25th. C. Replace Fargate with EC2 launch type. D. Increase the ALB idle timeout.
Answer: B. The peak is predictable, so capacity can be in place before it starts while target tracking handles the rest. A works but wastes money every day. C does not make scaling faster. D is unrelated to capacity.
A .NET service on ECS calls a partner REST API. Under load, it logs SocketException: Address already in use and requests fail, although CPU and memory are low. What is the MOST likely cause?
A. The task role lacks permissions.
B. The code creates a new HttpClient for each request.
C. The ALB deregistration delay is too short.
D. The partner API is throttling the service.
Answer: B. Each disposed HttpClient leaves sockets in TIME_WAIT, so ephemeral ports run out under load. Use IHttpClientFactory or a long-lived client with PooledConnectionLifetime. A would cause AccessDenied, C affects deployments, D would return HTTP 429, not a socket error.
Completion checklist
- I can list the five kinds of hidden state and the fix for each
- I can explain target tracking, cooldowns, and min/max capacity
- I can compute
tasks × pool sizeand choose a maximum task count from it - I can explain Lambda concurrency, reserved and provisioned concurrency
- I ran the session-sharing exercise locally
- I removed any scaling targets, caches and provisioned concurrency I created
Key takeaways
- Scale out, not up, and make the app stateless first: sessions, caches, files, Data Protection keys and timers all leave the process.
- Target tracking on requests per target or CPU with a real maximum is the default for ECS; add scheduled actions for known banking peaks.
- Lambda scales fast by itself; reserved concurrency and RDS Proxy protect what is behind it.
- Every pool multiplies by the number of copies. Write the arithmetic down.
- ElastiCache for Valkey and DynamoDB both back
IDistributedCache; choose by latency need and network shape.
Further reading
Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.
Lab 3.5: Parameter Store and Secrets Manager with caching and rotation
Layer configuration from appsettings, environment variables and Parameter Store, change a feature flag without a restart, read a pseudonymisation key from Secrets Manager through the client-side cache, rotate it, and see exactly when the API notices.
Logging and errors for .NET on AWS
Write structured JSON logs with correlation IDs from ASP.NET Core, ship them to CloudWatch Logs from Beanstalk, ECS and Lambda, query them with Logs Insights, and return safe RFC 9457 ProblemDetails from a global exception handler.