Amazon ElastiCache (Valkey/Redis OSS)
Use ElastiCache for Valkey safely in a bank: cache-aside with .NET HybridCache, distributed session state, atomic rate-limit counters with a Lua script, choosing serverless or node-based caches, securing them with TLS, RBAC or IAM authentication — and knowing what must never be cached.
Assumes: .NET 10 LTS · StackExchange.Redis 3.4.35 · Microsoft.Extensions.Caching.StackExchangeRedis 10.0.15 · Microsoft.Extensions.Caching.Hybrid 10.11.0 · Valkey 8.1 (Docker) · ElastiCache for Valkey · Region us-east-1. Verified against official sources on 2026-11-18.
Learning objectives
By the end of this lesson you can:
- Explain what an in-memory cache is for, and choose between ElastiCache Serverless and node-based clusters, and between the Valkey, Redis OSS and Memcached engines.
- Implement cache-aside in ASP.NET Core with
HybridCache(local L1 plus Valkey L2), including expiry, invalidation and stampede protection. - Build an atomic rate limiter for OTP and login attempts with a Lua script and StackExchange.Redis.
- Secure a cache: private subnets, security groups, TLS, at-rest encryption, RBAC users or IAM authentication.
- List what a bank must never cache — and justify each item.
Prerequisites
- Amazon DynamoDB fundamentals — keys and access patterns
- Module 2: Networking — private subnets, security groups
- Module 3: .NET on AWS — ASP.NET Core configuration and DI
- Docker Desktop and the .NET 10 SDK
Why this matters in banking
Harbourline's product catalogue page (interest rates, fees, eligibility rules) is read 40,000 times a minute on the day a new savings rate launches. Every read hits Aurora, and the database's CPU sits at 90%, slowing payments, which share the cluster. A cache would absorb those reads in microseconds.
The same week, a penetration tester finds that the OTP endpoint for adding a new payee has no attempt limit: a six-digit code can be brute-forced. The fix needs a counter that is shared by every API instance, atomic, and expires on its own — again, a job for an in-memory data store.
And the security architect's first question in the design review: "What exactly will be in that cache, who can read it, and what happens when it's stale?" A cache that holds a customer's balance from five minutes ago, or a card number in plain text, turns a performance fix into an incident. This lesson covers all three angles.
The idea in plain language
A cache is a fast copy of data that lives somewhere slow. You read from the cache first; if it is not there (a miss), you read from the source and put a copy in the cache for next time (cache-aside). Every copy gets an expiry time (TTL), so stale data eventually disappears.
Valkey is an open-source, in-memory key-value store — a community fork of Redis created in 2024 after Redis changed its licence. It speaks the same protocol, so the Redis clients you know (StackExchange.Redis) work unchanged. Amazon ElastiCache runs Valkey (or Redis OSS, or Memcached) for you: patching, failover, backups, scaling.
Analogy: The cache is the branch's cash drawer; the database is the vault. Most withdrawals are served from the drawer in seconds. When the drawer runs low (a miss), the clerk fetches from the vault. The drawer is counted and topped up regularly (TTL). But you never keep the master keys or customers' PINs in the drawer — anything that would be a disaster if the drawer were stolen stays in the vault.
What problem ElastiCache solves
| Use | Why a cache fits | Example at Harbourline |
|---|---|---|
| Read offload (cache-aside) | Microsecond reads; protects the database | Product catalogue, FX display rates, branch list |
| Distributed session / state | All API instances see the same state | Multi-step onboarding wizard state |
| Counters and rate limits | Atomic INCR with expiry | OTP attempts, API quotas per partner |
| Short-lived tokens | TTL on keys | One-time links, step-up challenge IDs |
| Leaderboards, sorted sets, streams | Rich data structures | Fraud "velocity" counters per card per hour |
Serverless vs node-based
| ElastiCache Serverless | Node-based cluster | |
|---|---|---|
| Capacity | Scales automatically; no nodes to pick | You choose node type, shards and replicas |
| Pricing, Valkey, us-east-1 (checked 2026-11-18) | USD 0.084 per GB-hour of data stored (minimum 100 MB per cache) + USD 0.0023 per million ElastiCache Processing Units (ECPUs) | Per node-hour by instance type ⚠ VERIFY current node prices |
| Minimum monthly cost | About USD 6 for an idle Valkey cache at the 100 MB minimum (0.1 GB × 0.084 × 730 h) | Smallest node running 24/7 |
| Availability | Data replicated across multiple AZs automatically | You configure Multi-AZ with replicas |
| Encryption in transit | Always on (TLS required) | Optional at creation — always turn it on |
| Good for | Spiky or unknown load, small teams, labs | Large steady workloads, fine tuning, data tiering |
AWS prices Valkey lower than the other engines: about 33% lower on serverless and 20% lower on node-based caches. For new work, choose Valkey unless you depend on a Redis OSS-only feature.
Engines
| Engine | Use when |
|---|---|
| Valkey (7.2 up to 9.x on ElastiCache) | Default for new caches; cheapest; active open-source development |
| Redis OSS (up to 7.1 on ElastiCache) | Existing clusters; upgrade path to Valkey is supported in place |
| Memcached | Simple string caching with multithreading; no persistence, replication or rich types — rarely the right choice for a bank |
Honest limitations:
- A cache is not a system of record by default. Node failure, failover, eviction under memory pressure, or a flush loses data. (ElastiCache for Valkey 9.0 added an optional Multi-AZ transactional log for durability ⚠ VERIFY availability and trade-offs before relying on it — and even then, Harbourline keeps money in the ledger database.)
- Memory is expensive. Cache what is hot, not everything.
- Every cache adds a staleness window to every read it serves. Decide it per data type.
Microsoft / Azure comparison
| You know (Microsoft/Azure) | AWS equivalent | What's different |
|---|---|---|
| Azure Cache for Redis / Azure Managed Redis | Amazon ElastiCache for Valkey (or Redis OSS) | Valkey is the open-source fork AWS promotes; same client libraries |
IDistributedCache with AddStackExchangeRedisCache | Same package, pointed at the ElastiCache endpoint | Use ssl=true and the configuration endpoint |
IMemoryCache | Still useful as L1 in each instance | HybridCache combines both |
| Entra ID authentication for Azure Cache for Redis | ElastiCache IAM authentication | Short-lived SigV4 token as the password; TLS required |
| Access keys for Azure Cache for Redis | RBAC users with passwords (ACL), stored in Secrets Manager | Prefer IAM auth or per-app users over the default user |
| Private Endpoint | Cache in private subnets + security group | Node-based caches are VPC-only; serverless caches use VPC endpoints |
| ASP.NET Core session with Redis | Same, with ElastiCache | Keep session data minimal; no PII beyond what's needed |
Microsoft.AspNetCore.RateLimiting (in-process) | In-process limiter + Valkey counter for cross-instance limits | The built-in limiter is per instance; a shared counter is global |
Architecture
Security-boundary view
Patterns
Cache-aside with HybridCache
HybridCache (from Microsoft.Extensions.Caching.Hybrid, .NET 9+) gives you:
- an in-process L1 (
IMemoryCache) for the hottest keys, - an optional L2 (any
IDistributedCache— here Valkey viaAddStackExchangeRedisCache), - stampede protection: if 500 requests miss the same key at once, only one calls the database,
- tags for group invalidation (
product-catalogue).
Invalidation strategy
| Data | Strategy | TTL |
|---|---|---|
| Product catalogue | Event-driven invalidation by tag on publish + TTL as backstop | 10 min |
| FX display rates (not dealing rates) | TTL only; show "as at" time | 60 s |
| Branch list | TTL only | 24 h |
| Customer profile (name, preferences) | Invalidate on update + short TTL | 5 min |
| Account balance | Do not cache for decisions; if displayed from cache, show "as at" | — |
Rate limiting with a Lua script
A correct counter needs two operations — increment and, on the first hit, set the expiry — to happen atomically, or a crash between them leaves a counter that never expires (locking the customer out forever). A Lua script runs atomically on the server:
-- KEYS[1] = counter key, ARGV[1] = window seconds
local current = redis.call('INCR', KEYS[1])
if current == 1 then
redis.call('EXPIRE', KEYS[1], ARGV[1])
end
return currentWhat must never be cached
| Never cache | Why |
|---|---|
| Full card numbers (PAN), CVV/CVC, PIN blocks, full track data | PCI DSS forbids storing sensitive authentication data after authorisation, and a cache would bring itself into PCI scope. Use tokens |
| Balances used to authorise payments | Stale data → overdrafts and double spends. Read from the ledger (lesson 9) |
| Consent and access-revocation state used for authorisation | A revoked third party must lose access immediately |
| Passwords, OTP values, private keys, API secrets | Secrets belong in Secrets Manager/KMS; OTPs are stored as hashes with the attempt counter |
| Sanctions screening results beyond their validity | Lists change; stale "clear" results are a regulatory breach |
| Full customer records "for speed" | Data minimisation under UK GDPR; every copy is another place to secure, erase and report on |
Step-by-step implementation
Run Valkey locally
docker run --name hl-m06-valkey -p 6379:6379 -d valkey/valkey:8.1 valkey-server --requirepass LocalOnly_ChangeMe1
docker exec -it hl-m06-valkey valkey-cli -a LocalOnly_ChangeMe1 PINGPONG means it is running. Locally there is no TLS; in AWS you will always use TLS.
Create the API project
dotnet new web -n Harbourline.Catalogue.Api -f net10.0
cd Harbourline.Catalogue.Api
dotnet add package Microsoft.Extensions.Caching.StackExchangeRedis --version 10.0.15
dotnet add package Microsoft.Extensions.Caching.Hybrid --version 10.11.0
dotnet add package StackExchange.Redis --version 3.4.35Add the code from Complete code and run
dotnet run
curl http://localhost:5080/products/savings-easy-access # first call: miss (slow)
curl http://localhost:5080/products/savings-easy-access # second call: hit (fast)Watch the console log: the first request logs "loading from database".
Test the rate limiter
Call the OTP endpoint six times with a wrong code:
1..6 | ForEach-Object { curl -s -X POST http://localhost:5080/payees/verify-otp -H "Content-Type: application/json" -d '{"customerId":"CUS-00012345","challengeId":"chl-01","code":"000000"}' }Attempts 1–5 return 400 "invalid code"; attempt 6 returns 429 Too Many Requests with Retry-After.
Inspect keys
docker exec -it hl-m06-valkey valkey-cli -a LocalOnly_ChangeMe1 --scan --pattern "hl:*"
docker exec -it hl-m06-valkey valkey-cli -a LocalOnly_ChangeMe1 TTL "hl:otp:CUS-00012345:chl-01"Never use KEYS * on a production cache — it blocks the server. --scan iterates safely.
AWS Console instructions
- Console → ElastiCache → Valkey caches → Create Valkey cache.
- Deployment option: Serverless for this lesson. Name
hl-m06-cache. - Settings: engine version (latest Valkey), VPC your lab VPC, subnets the private data subnets, security group
sg-cache(inbound TCP 6379 fromsg-apionly). - Encryption key: Customer managed key if your organisation requires it; otherwise the AWS owned key. Encryption in transit is always on for serverless.
- Access control: choose User group access control list and attach a user group containing an application user (IAM-authenticated or password) — do not leave the default user with full access.
- Usage limits (optional but recommended): set a maximum data storage and maximum ECPUs per second so a bug cannot run up an unbounded bill.
- Tags:
Project=cea-lab,Lab=m06-cache,Environment=learn,Owner=<you>. Create. - The cache's Endpoint looks like
hl-m06-cache-xxxxxx.serverless.use1.cache.amazonaws.com:6379.
AWS CLI
# Create an application user with a password (stored in Secrets Manager, not typed here).
PASSWORD=$(aws secretsmanager get-secret-value --secret-id hl-m06/cache/app-user \
--query SecretString --output text --profile learn-sandbox --region us-east-1)
aws elasticache create-user --user-id hl-m06-app --user-name hl-m06-app --engine valkey \
--passwords "$PASSWORD" --access-string "on ~hl:* -@all +@read +@write +@string +@hash +@scripting -@dangerous" \
--profile learn-sandbox --region us-east-1
# A user group must include a 'default' user; create a disabled one first.
aws elasticache create-user --user-id hl-m06-default-off --user-name default --engine valkey \
--no-password-required --access-string "off -@all" --profile learn-sandbox --region us-east-1
aws elasticache create-user-group --user-group-id hl-m06-users --engine valkey \
--user-ids hl-m06-default-off hl-m06-app --profile learn-sandbox --region us-east-1
# Create a serverless Valkey cache with usage limits.
aws elasticache create-serverless-cache --serverless-cache-name hl-m06-cache --engine valkey \
--subnet-ids subnet-0aaa1111bbbb2222c subnet-0ddd3333eeee4444f \
--security-group-ids sg-0cache000000000001 --user-group-id hl-m06-users \
--cache-usage-limits 'DataStorage={Maximum=1,Unit=GB},ECPUPerSecond={Maximum=5000}' \
--tags Key=Project,Value=cea-lab Key=Lab,Value=m06-cache Key=Environment,Value=learn \
--profile learn-sandbox --region us-east-1
# Get the endpoint.
aws elasticache describe-serverless-caches --serverless-cache-name hl-m06-cache \
--query "ServerlessCaches[0].Endpoint" --profile learn-sandbox --region us-east-1The access string on ~hl:* … -@dangerous limits the user to keys starting with hl: and removes administrative commands such as FLUSHALL and KEYS. Check exact ACL category names against the Valkey ACL documentation for your engine version.
Infrastructure as Code
using Amazon.CDK;
using Amazon.CDK.AWS.EC2;
using Amazon.CDK.AWS.ElastiCache;
using Constructs;
namespace Harbourline.Infra;
public sealed class CacheStack : Stack
{
public CacheStack(Construct scope, string id, IVpc vpc, ISecurityGroup apiSg, IStackProps? props = null) : base(scope, id, props)
{
var cacheSg = new SecurityGroup(this, "CacheSg", new SecurityGroupProps
{
Vpc = vpc,
Description = "Valkey: 6379 from API tasks only",
AllowAllOutbound = false
});
cacheSg.AddIngressRule(apiSg, Port.Tcp(6379), "API to Valkey (TLS)");
// L1 construct: ElastiCache serverless is not yet available as an L2 in all CDK versions.
_ = new CfnServerlessCache(this, "Cache", new CfnServerlessCacheProps
{
ServerlessCacheName = "hl-m06-cache",
Engine = "valkey",
SubnetIds = vpc.SelectSubnets(new SubnetSelection { SubnetType = SubnetType.PRIVATE_ISOLATED }).SubnetIds,
SecurityGroupIds = new[] { cacheSg.SecurityGroupId },
UserGroupId = "hl-m06-users", // created separately (CfnUser / CfnUserGroup)
CacheUsageLimits = new CfnServerlessCache.CacheUsageLimitsProperty
{
DataStorage = new CfnServerlessCache.DataStorageProperty { Maximum = 1, Unit = "GB" },
EcpuPerSecond = new CfnServerlessCache.ECPUPerSecondProperty { Maximum = 5000 }
},
Tags = new[]
{
new CfnTag { Key = "Project", Value = "cea-lab" },
new CfnTag { Key = "Lab", Value = "m06-cache" },
new CfnTag { Key = "Environment", Value = "learn" }
}
});
}
}Complete code
using System.Text.Json;
using Microsoft.Extensions.Caching.Hybrid;
using StackExchange.Redis;
var builder = WebApplication.CreateBuilder(args);
// Locally: "localhost:6379,password=...". In AWS: "<endpoint>:6379,ssl=true,user=hl-m06-app,password=<from Secrets Manager>,abortConnect=false".
var valkeyConfig = builder.Configuration.GetConnectionString("Valkey")
?? throw new InvalidOperationException("ConnectionStrings:Valkey is not configured");
// One multiplexer per process: it is thread-safe and expensive to create.
var multiplexer = await ConnectionMultiplexer.ConnectAsync(valkeyConfig);
builder.Services.AddSingleton<IConnectionMultiplexer>(multiplexer);
// L2 distributed cache on Valkey, reusing the same multiplexer.
builder.Services.AddStackExchangeRedisCache(o =>
{
o.ConnectionMultiplexerFactory = () => Task.FromResult<IConnectionMultiplexer>(multiplexer);
o.InstanceName = "hl:catalogue:";
});
// HybridCache = L1 in-memory + L2 Valkey + stampede protection.
builder.Services.AddHybridCache(o =>
{
o.DefaultEntryOptions = new HybridCacheEntryOptions
{
Expiration = TimeSpan.FromMinutes(10), // L2 (Valkey) TTL
LocalCacheExpiration = TimeSpan.FromMinutes(1) // L1 TTL: short, so instances converge quickly
};
o.MaximumPayloadBytes = 64 * 1024;
});
builder.Services.AddSingleton<IProductRepository, FakeProductRepository>();
builder.Services.AddSingleton<OtpAttemptLimiter>();
builder.Services.AddProblemDetails();
var app = builder.Build();
// ---------- Cache-aside ----------
app.MapGet("/products/{productCode}", async (string productCode, HybridCache cache, IProductRepository repo, CancellationToken ct) =>
{
var product = await cache.GetOrCreateAsync(
$"product:{productCode}",
async token => await repo.GetAsync(productCode, token),
tags: ["product-catalogue"],
cancellationToken: ct);
return product is null ? Results.NotFound() : Results.Ok(product);
});
// Called by the catalogue publishing workflow after a rate change.
app.MapPost("/admin/catalogue/invalidate", async (HybridCache cache, CancellationToken ct) =>
{
await cache.RemoveByTagAsync("product-catalogue", ct);
return Results.NoContent();
}); // In production: protect with an admin policy, or trigger from an EventBridge event instead of HTTP.
// ---------- Rate-limited OTP verification ----------
app.MapPost("/payees/verify-otp", async (OtpRequest req, OtpAttemptLimiter limiter, CancellationToken ct) =>
{
var decision = await limiter.RegisterAttemptAsync(req.CustomerId, req.ChallengeId);
if (!decision.Allowed)
{
return Results.Problem(statusCode: StatusCodes.Status429TooManyRequests,
title: "Too many attempts", detail: "Please request a new code.",
extensions: new Dictionary<string, object?> { ["retryAfterSeconds"] = decision.RetryAfterSeconds });
}
// Verify against the stored HASH of the OTP (not shown); never cache the OTP value itself.
var valid = req.Code == "123456"; // demo only
return valid ? Results.Ok(new { verified = true }) : Results.Problem(statusCode: 400, title: "Invalid code");
});
app.Run();
// ---------- Types ----------
public sealed record Product(string ProductCode, string Name, decimal AerPercent, string Currency, DateTimeOffset EffectiveFrom);
public sealed record OtpRequest(string CustomerId, string ChallengeId, string Code);
public interface IProductRepository
{
Task<Product?> GetAsync(string productCode, CancellationToken ct);
}
public sealed class FakeProductRepository(ILogger<FakeProductRepository> log) : IProductRepository
{
public async Task<Product?> GetAsync(string productCode, CancellationToken ct)
{
log.LogInformation("Loading {ProductCode} from database", productCode);
await Task.Delay(300, ct); // simulate a slow query
return productCode == "savings-easy-access"
? new Product(productCode, "Easy Access Saver", 4.10m, "GBP", new DateTimeOffset(2026, 11, 1, 0, 0, 0, TimeSpan.Zero))
: null;
}
}
public sealed record RateDecision(bool Allowed, long Attempts, int RetryAfterSeconds);
public sealed class OtpAttemptLimiter(IConnectionMultiplexer valkey)
{
private const int MaxAttempts = 5;
private const int WindowSeconds = 900; // 15 minutes
// INCR and EXPIRE in one atomic server-side step.
private static readonly LuaScript IncrementWithExpiry = LuaScript.Prepare(
"""
local current = redis.call('INCR', @key)
if current == 1 then
redis.call('EXPIRE', @key, @window)
end
return current
""");
public async Task<RateDecision> RegisterAttemptAsync(string customerId, string challengeId)
{
var db = valkey.GetDatabase();
var key = (RedisKey)$"hl:otp:{customerId}:{challengeId}";
var attempts = (long)await db.ScriptEvaluateAsync(IncrementWithExpiry, new { key, window = WindowSeconds });
if (attempts <= MaxAttempts) return new RateDecision(true, attempts, 0);
var ttl = await db.KeyTimeToLiveAsync(key);
return new RateDecision(false, attempts, (int)(ttl?.TotalSeconds ?? WindowSeconds));
}
}{
"ConnectionStrings": {
"Valkey": "localhost:6379,password=LocalOnly_ChangeMe1,abortConnect=false"
},
"Urls": "http://localhost:5080"
}Code explained line by line
ConnectionMultiplexer.ConnectAsync(...)once, registered as a singleton — StackExchange.Redis multiplexes all commands over a few connections. Creating one per request exhausts sockets and adds TLS handshakes.abortConnect=false— the app starts even if the cache is briefly unavailable, and the multiplexer reconnects in the background. A cache outage must degrade performance, not availability.AddStackExchangeRedisCachewithConnectionMultiplexerFactory— reuses the same multiplexer for theIDistributedCacheL2 instead of opening a second set of connections.InstanceName = "hl:catalogue:"— prefixes every key, so the ACL~hl:*pattern works and teams sharing a cache do not collide.AddHybridCachewithExpirationandLocalCacheExpiration— the L1 TTL is shorter than L2 so that, after an invalidation, other instances converge within a minute even if they missed the invalidation.MaximumPayloadBytes— refuses to cache huge objects; a 5 MB value in a cache is usually a bug.cache.GetOrCreateAsync(key, factory, tags: …)— the cache-aside pattern in one call. On a miss, only one caller per key runs the factory; others await its result (stampede protection).RemoveByTagAsync("product-catalogue")— invalidates every entry with the tag. Trigger it from the event that publishes new rates.LuaScript.Prepare(...)with@keyand@window— StackExchange.Redis maps the anonymous object's properties to script parameters and caches the script on the server.INCRthenEXPIREonly whencurrent == 1— the window starts at the first attempt; the key disappears after 15 minutes. Atomic, so no orphaned counters.KeyTimeToLiveAsync— tells the client how long to wait (Retry-After).Results.Problem(...)— ProblemDetails (RFC 9457) responses, per Harbourline conventions.req.Code == "123456"— demo only. Real OTPs are random, stored as a salted hash with an expiry, and compared in constant time.
Security considerations
⚠ Learning shortcut — not for production
The local container uses a single password on the default user, no TLS, and a password in appsettings.Development.json. In AWS: TLS always (ssl=true), a dedicated RBAC user with a restricted access string or IAM authentication, the default user disabled, credentials from Secrets Manager (or short-lived IAM tokens), the cache in private subnets reachable only from the API's security group, and encryption at rest with a KMS key.
IAM authentication (Valkey 7.2+ or Redis OSS 7.0+) replaces passwords with a short-lived SigV4-signed token:
- TLS must be enabled; the ElastiCache user must have authentication mode
iamand identical user name and user ID. - The caller's IAM role needs
elasticache:Connecton both the cache ARN and the user ARN. - Tokens are valid for 15 minutes; connections are closed after 12 hours unless re-authenticated with a fresh token.
- AWS's Valkey GLIDE client (Python, Java and others) handles token generation and refresh built-in. For StackExchange.Redis in .NET there is no AWS-provided helper at the time of writing ⚠ VERIFY; you would generate a SigV4 pre-signed
connectrequest yourself and refresh before expiry, or use an RBAC user whose password Secrets Manager rotates.
Other controls:
- Data classification. Treat the cache as holding the most sensitive data it could ever contain. If customer names are cached, the cache is in GDPR scope; if card data were cached, it would be in PCI DSS scope (do not do this).
- Restrict dangerous commands. The ACL removes
FLUSHALL,KEYS,CONFIGandDEBUGfrom the application user. - No public access. Node-based caches live in your VPC. ElastiCache for Valkey 9.0 introduced optional public endpoints for serverless caches (IAM authentication and TLS 1.3 required); keep Harbourline's caches private.
- Logging. Enable slow log and engine log delivery to CloudWatch Logs with retention set; never log cached values.
Banking use case
Harbourline's catalogue and security features after the change:
| Feature | Cache use | Staleness budget | Fallback if the cache is down |
|---|---|---|---|
| Product catalogue | HybridCache, tag invalidation on publish | ≤ 1 min after publish | Read Aurora directly with a circuit breaker |
| FX display rates on the app | 60 s TTL; "as at 10:41" label | 60 s | Hide rates rather than show very old ones |
| OTP attempt limit | Lua counter, 5 per 15 min per challenge | None (authoritative while it exists) | Fail closed: refuse OTP verification and ask the customer to retry later |
| Onboarding wizard state | Hash per session, 30 min TTL, minimal fields | — | Ask the customer to restart the step |
The fallback column matters: for performance caches, a cache outage should fail open to the database; for security counters, it must fail closed.
Common mistakes
- Caching balances "for performance" → the home screen is slow → cache-aside with a clear "as at" and never use it for payment decisions.
- One
ConnectionMultiplexerper request → copied from a blog post → singleton. - Non-atomic counters (
INCRthen a separateEXPIREcall) → a crash in between leaves a permanent lockout → Lua script orSET key 1 EX 900 NX+INCR. - No TTL on keys → memory fills, the eviction policy starts deleting things you need → always set an expiry.
KEYS *in production → blocks the single-threaded command loop → useSCAN.- Assuming the cache survives → failover or eviction loses data → anything important lives in a database.
- Leaving the
defaultuser enabled → full access with one password → disable it and use per-application users. - Large values (whole statements) → slow network transfers and memory pressure → cache small, hot objects only.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
RedisConnectionException: It was not possible to connect | Security group, wrong subnet, or missing ssl=true | Allow 6379 from the API SG; add ssl=true for ElastiCache |
NOAUTH Authentication required / WRONGPASS | Missing or wrong user/password; IAM token expired | Check the user group; regenerate the token |
NOPERM this user has no permissions to run the 'flushall' command | ACL working as intended | Do not run admin commands from the app |
Timeouts under load (Timeout performing GET) | Thread-pool starvation, large values, or synchronous calls | Use async APIs; check ThreadPool min threads; reduce payload sizes |
MOVED / CROSSSLOT errors | Multi-key command across hash slots in cluster mode | Keep related keys in one slot with a hash tag, e.g. hl:otp: plus braces around the customer ID |
| Stale product data after publishing | L1 TTL too long, or invalidation missed | Shorten LocalCacheExpiration; trigger tag removal from the publish event |
| Serverless bill higher than expected | Large values or chatty access patterns consuming ECPUs | Check ElastiCacheProcessingUnits and BytesUsedForCache metrics; set usage limits |
Cost and cleanup
The local Docker run is free. A serverless Valkey cache in AWS costs about USD 0.0115 per hour at the 100 MB minimum (0.1 GB × USD 0.084) plus negligible ECPUs for a lab — roughly USD 0.03 for a two-hour session. Accounts on the legacy Free Tier (created before 15 July 2025) get 750 hours per month of cache.t3.micro node usage for 12 months; serverless is not covered. Newer accounts use their Free Tier credits.
# Local
docker rm -f hl-m06-valkey# AWS
aws elasticache delete-serverless-cache --serverless-cache-name hl-m06-cache --profile learn-sandbox --region us-east-1
# Wait until it is gone before deleting the user group.
aws elasticache describe-serverless-caches --serverless-cache-name hl-m06-cache --profile learn-sandbox --region us-east-1
aws elasticache delete-user-group --user-group-id hl-m06-users --profile learn-sandbox --region us-east-1
aws elasticache delete-user --user-id hl-m06-app --profile learn-sandbox --region us-east-1
aws elasticache delete-user --user-id hl-m06-default-off --profile learn-sandbox --region us-east-1
aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=cea-lab \
--query "ResourceTagMappingList[].ResourceARN" --profile learn-sandbox --region us-east-1Deleting a serverless cache can create a final snapshot only if you ask for one; snapshots are billed for storage.
Exercises
- 🟢 Guided: Run the API, measure the first and second request times for the product endpoint, call the invalidation endpoint, and show the next request is a miss again.
- 🟡 Challenge: Implement a sliding-window rate limiter for the Partner Gateway (100 requests per partner per rolling minute) using a sorted set (
ZADD,ZREMRANGEBYSCORE,ZCARD) in one Lua script. Compare its memory use with the fixed-window counter. - 🔴 Live problem: After a cache failover in production, the fraud team sees a spike of successful OTP verifications on a handful of challenge IDs. What happened, and what would you change?
Assignment
Write docs/caching-policy.md for your portfolio: a table of every cached data type with owner, TTL, invalidation trigger, data classification, fail-open/fail-closed behaviour and the "never cache" list with reasons. Add the API code and a xUnit v3 test for OtpAttemptLimiter using a Testcontainers Valkey container.
Solutions
Typical numbers: first call about 300–320 ms (the simulated database delay), second call 1–3 ms (L1 hit). After POST /admin/catalogue/invalidate, the next call logs "Loading … from database" again.
Why: RemoveByTagAsync invalidates both L1 (in this instance) and L2. Other instances' L1 entries expire within LocalCacheExpiration (1 minute) — which is why it should be short for data that changes on events.
private static readonly LuaScript SlidingWindow = LuaScript.Prepare(
"""
redis.call('ZREMRANGEBYSCORE', @key, 0, @now - @windowMs)
local count = redis.call('ZCARD', @key)
if count < tonumber(@limit) then
redis.call('ZADD', @key, @now, @member)
redis.call('PEXPIRE', @key, @windowMs)
return 1
end
return 0
""");
var allowed = (int)await db.ScriptEvaluateAsync(SlidingWindow, new
{
key = (RedisKey)$"hl:partner-rl:{partnerId}",
now = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds(),
windowMs = 60_000,
limit = 100,
member = Guid.CreateVersion7().ToString()
}) == 1;Why: the sorted set stores one member per request in the window, so it is exact but uses memory proportional to the limit (100 members per partner). The fixed window uses one integer per key but allows bursts of up to twice the limit across a window boundary. For partner quotas, exactness is worth the memory; for OTPs, the fixed window is enough.
What happened: the OTP attempt counters lived only in the cache. During failover, recent writes that had not yet replicated to the new primary were lost (or the cache was briefly empty), so attackers' attempt counts reset to zero, giving them fresh attempts. The limiter was designed as if the cache were durable.
Changes:
- Treat the counter as a security control: on cache errors, fail closed (already in the design table — verify the code actually does this).
- Bind attempts to the challenge, and invalidate the challenge itself after N failures in the database (the authoritative store). The cache counter then becomes an optimisation, not the only control.
- Shorten OTP validity (e.g. 5 minutes) so the reset window gives little advantage.
- Alert on failovers (
ElastiCacheevents to EventBridge) and on bursts of OTP failures per customer. - Raise an incident record and assess whether any payee was added fraudulently.
Knowledge check
Answer: Valkey is protocol-compatible (same clients), actively developed as open source, and AWS prices it lower (about 33% on serverless, 20% on node-based). Choose Redis OSS only for compatibility with an existing cluster or a feature you have verified Valkey lacks.
Answer: An in-process L1 layer, stampede protection (one factory call per key on concurrent misses), tag-based invalidation and a simpler GetOrCreateAsync API.
Answer: If the process fails between them, the counter has no expiry and the customer is locked out permanently (or, with other orderings, the limit never resets correctly). A Lua script or SET … NX EX makes it atomic.
Answer: Tokens are valid for 15 minutes; IAM-authenticated connections are disconnected after 12 hours unless re-authenticated with a new token. TLS is mandatory.
Answer: Full card numbers or CVV (PCI DSS), balances used to authorise payments (staleness), and secrets or OTP values (secrets belong in Secrets Manager; OTPs are stored hashed). Also consent revocation state used for authorisation.
Interview questions
- "Explain cache-aside and its failure modes." — Read cache, on miss read source and populate with TTL. Failure modes: stale data (mitigate with TTL and event invalidation), stampede on popular keys (single-flight), cache outage (fail open to the source with a circuit breaker), and caching negative results unintentionally.
- "How do you secure ElastiCache?" — Private subnets and SG rules, TLS in transit, KMS at rest, RBAC users with restricted access strings or IAM authentication, disabled default user, no admin commands for apps, logs to CloudWatch with retention.
- "Serverless or node-based?" — Serverless for unknown or spiky load and low operational effort, with usage limits to cap cost; node-based for large steady workloads where you want control over instance types, data tiering and reserved pricing.
- "When is a cache a bad idea?" — When correctness depends on freshness (balances for payments, consent), when data is sensitive enough to widen compliance scope, or when the hit rate would be low.
Certification practice
An application's RDS for PostgreSQL database is overloaded by repeated reads of a product catalogue that changes a few times a day. Writes are unaffected. What is the most effective way to reduce database load with minimal code change?
A. Add a read replica and send all reads to it. B. Implement cache-aside with Amazon ElastiCache for Valkey and invalidate entries when the catalogue changes. C. Increase the DB instance size. D. Enable Multi-AZ.
Answer: B. A cache serves repeated reads in microseconds and removes them from the database. A replica helps but still runs every query (A). A larger instance is costlier and only postpones the problem (C). Multi-AZ improves availability, not read capacity (D).
A company wants its ECS tasks to authenticate to ElastiCache for Valkey without storing long-lived passwords. What should a solutions architect configure?
A. Security group rules that allow only the ECS tasks' security group.
B. ElastiCache IAM authentication with an IAM-enabled ElastiCache user and elasticache:Connect permissions on the task role, with in-transit encryption enabled.
C. An AUTH token stored in an environment variable.
D. At-rest encryption with a customer managed KMS key.
Answer: B. IAM authentication uses short-lived tokens derived from the task role's credentials. Security groups control network reachability, not identity (A). An AUTH token is a long-lived password (C). KMS at rest does not authenticate clients (D).
Completion checklist
- I can explain cache-aside, TTL, invalidation and stampede protection
- I ran the API and saw a miss, a hit and an invalidation
- I triggered the OTP limiter and got a 429
- I can list what must never be cached at a bank, with reasons
- I deleted the local container and any AWS cache, users and user group
Key takeaways
- ElastiCache for Valkey is the default choice for new caches: Redis-compatible, cheaper, actively developed; serverless for simplicity, node-based for control.
- Use
HybridCachefor cache-aside in .NET: L1 + Valkey L2, stampede protection, tag invalidation. - Make counters atomic with Lua; decide fail-open vs fail-closed per use.
- Secure the cache like a database: private, TLS, least-privilege users or IAM auth, encryption at rest.
- Never cache card data, secrets, authorisation-critical balances or consent state.
Further reading
Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.
DynamoDB for banking correctness
Make DynamoDB safe for money: conditional writes for balance guards and optimistic locking, an idempotency-key store that rejects duplicate payments, TransactWriteItems for transfers, Streams with an idempotent .NET 10 Lambda for audit events, and TTL for expiring records.
Amazon DocumentDB
Decide when a document model fits a banking workload — product catalogues, onboarding applications — and when PostgreSQL JSONB or DynamoDB is the better answer; then connect to Amazon DocumentDB from .NET 10 with TLS, schema validation and the MongoDB-compatibility caveats in mind.