DynamoDB for banking correctness
Make DynamoDB safe for money: conditional writes for balance guards and optimistic locking, an idempotency-key store that rejects duplicate payments, TransactWriteItems for transfers, Streams with an idempotent .NET 10 Lambda for audit events, and TTL for expiring records.
Assumes: .NET 10 LTS · AWS SDK for .NET v4 (AWSSDK.DynamoDBv2 4.1.0.2) · Amazon.Lambda.DynamoDBEvents 4.0.0 · Lambda runtime dotnet10 · DynamoDB Local · Region us-east-1. Verified against official sources on 2026-11-18.
Learning objectives
By the end of this lesson you can:
- Use condition expressions to guard balances and to implement optimistic locking with a version attribute.
- Build an idempotency-key store that guarantees a payment request is processed at most once, even when the client, load balancer or Lambda retries.
- Post a transfer with
TransactWriteItems: debit, credit, two ledger items and the idempotency record, all or nothing — and decodeTransactionCanceledExceptionreasons. - Consume DynamoDB Streams with a .NET 10 Lambda that writes audit events, handles partial batch failures and is itself idempotent.
- Use TTL to expire idempotency records and short-lived data, and explain exactly what TTL does and does not guarantee.
Prerequisites
- Amazon DynamoDB fundamentals — keys, capacity, the
hl-m06-bankingtable - Transactions, isolation levels and consistency — lost updates and retries
- Docker Desktop with DynamoDB Local, .NET 10 SDK, AWS CLI v2 (
learn-sandbox) for the optional Streams deployment
Why this matters in banking
At 12:03 a Harbourline customer taps Send £250 to their landlord. The mobile network drops after the request leaves the phone. The app does what every good app does: it retries. The API gateway also has a retry policy. And the Lambda behind it timed out after writing the payment but before returning. Without idempotency, the landlord gets £750 and the customer gets an overdraft fee.
Duplicate money movement is not an edge case — at scale it happens every day. Regulators expect firms to prevent and correct it, and the UK's payment schemes expect each instruction to be processed once. DynamoDB gives you the building blocks to make this impossible by construction: atomic conditional writes, multi-item transactions, a change stream and automatic expiry. This lesson assembles them into patterns you can reuse in every payment-like endpoint.
The idea in plain language
- Conditional write: "Write this item only if this condition is true at the moment of writing." DynamoDB checks and writes atomically. There is no gap for another request to sneak in.
- Idempotency key: a unique ID the client sends with a request (
Idempotency-Keyheader). The server records "I've seen this key" with a conditional write the first time. A repeat with the same key gets the original result, not a second payment. - Transaction: up to 100 actions on up to 100 different items that all succeed or all fail.
- Stream: an ordered log of every change to items in a table, kept for 24 hours, that a Lambda function can read.
- TTL (time to live): an attribute holding an expiry time; DynamoDB deletes the item some time after it passes, for free.
Analogy: An idempotency key is the cheque number. The bank's clerk keeps a book of cheque numbers already paid. Before paying, they check and write the number in one motion. A photocopy of the same cheque presented ten minutes later is refused with "already paid — here's the receipt". After a while, old cheque numbers are archived out of the book (TTL).
What problem these features solve
| Feature | Solves | Limits and sharp edges (checked 2026-11-18) |
|---|---|---|
| Condition expressions | Lost updates, overdrafts, duplicate inserts on one item | Single item; failure raises ConditionalCheckFailedException and still consumes write capacity |
ReturnValuesOnConditionCheckFailure | Seeing why a condition failed without an extra read | Returns the item as it was (ALL_OLD) |
TransactWriteItems | All-or-nothing changes across items and tables | Up to 100 actions, 4 MB total, one account and Region; two units per item; cannot touch the same item twice; not across global table Regions |
ClientRequestToken | Safe SDK-level retry of a whole transaction | Valid for 10 minutes after the request finishes; same token with different parameters → IdempotentParameterMismatchException |
| DynamoDB Streams | Reacting to changes (audit, projections, outbox) | 24-hour retention; ordered per item; records of one transaction can be interleaved with others; consumer must be idempotent |
| TTL | Expiring data without paying for deletes | Deletes typically happen within a few days of expiry, not at the exact second; expired items stay readable until deleted |
Microsoft / Azure comparison
| You know (Microsoft/Azure) | AWS equivalent | What's different |
|---|---|---|
UPDATE … WHERE Balance >= @amt in SQL Server | UpdateItem with ConditionExpression = "balance >= :amt" | Same idea, one item at a time |
rowversion / EF Core concurrency token | Version attribute + ConditionExpression = "version = :expected" | You implement it; DynamoDBContext can do it with [DynamoDBVersion] |
Unique index on IdempotencyKey | Item keyed by the idempotency key + attribute_not_exists(PK) | The key itself is the uniqueness constraint |
TransactionScope / BEGIN TRAN over several tables | TransactWriteItems | Bounded size; no reads inside except ConditionCheck; no interactive transaction |
| Cosmos DB change feed | DynamoDB Streams (or Kinesis Data Streams for DynamoDB) | 24 h retention; Lambda event source mapping does the polling |
| Cosmos DB TTL | DynamoDB TTL | Cosmos deletes near expiry; DynamoDB within a few days — always filter |
| SQL Server Change Data Capture | DynamoDB Streams | Streams are on by a table setting, not a job |
Architecture
Security-boundary view
The API cannot write to the audit table, and the audit Lambda cannot change balances. That separation of duties is what makes the audit trail credible.
The patterns
Pattern 1 — balance guard on a single item
UpdateItem Key: PK=ACC#ACC-0000000001, SK=PROFILE
UpdateExpression: SET balance = balance - :amt, version = version + :one
ConditionExpression: balance >= :amt AND #st = :openDynamoDB evaluates the condition and applies the update atomically. Concurrent debits are serialised per item, so the lost update from lesson 9 cannot happen.
Pattern 2 — optimistic locking for read-modify-write
When you need to read the item, run business logic in C#, then write (for example, recalculating an overdraft limit), store a version number and write with ConditionExpression = "version = :expected". If someone else changed it, you get ConditionalCheckFailedException: re-read and retry.
Pattern 3 — idempotency record
| Attribute | Value |
|---|---|
PK | IDEM#<customer_id>#<idempotency-key> (scope keys to the caller so one client cannot collide with another) |
SK | IDEM |
request_hash | SHA-256 of the canonical request body |
status | COMPLETED (or IN_PROGRESS for long workflows) |
response | The JSON you returned the first time |
expires_at | Unix epoch seconds, e.g. now + 24 hours — the TTL attribute |
Rules:
- Same key, same hash → replay the stored response.
- Same key, different hash → reject with 422 (the client reused a key for a different payment — a bug or an attack).
- New key → process and store.
Because expires_at is only enforced eventually, the read path must also treat an item whose expires_at is in the past as absent — or simply accept that duplicates are blocked for at least 24 hours, which is the safe direction.
Pattern 4 — transfer in one transaction
Five actions, one call:
PutIDEM#…withattribute_not_exists(PK).Updatedebit account:balance >= :amt AND #st = :open.Updatecredit account:attribute_exists(PK) AND #st = :open.Putdebit transaction itemTX#date#id-Dwithattribute_not_exists(PK).Putcredit transaction itemTX#date#id-C.
If any condition fails, nothing is written, and TransactionCanceledException.CancellationReasons tells you which action failed and why, in the same order as your actions.
Pattern 5 — idempotent stream consumer
Lambda reads stream batches at least once. A retry after a partial failure redelivers records. Make the consumer idempotent by writing audit events keyed by the stream record's SequenceNumber (unique per shard) with attribute_not_exists(PK), and return partial batch failures so only the failed record and later ones are retried.
Step-by-step implementation
Start DynamoDB Local and create the tables
Reuse hl-m06-banking from the previous lesson, and add an audit table:
docker run --name hl-m06-ddb -p 8000:8000 -d amazon/dynamodb-local:latest -jar DynamoDBLocal.jar -sharedDb -inMemory
aws dynamodb create-table --endpoint-url http://localhost:8000 --region us-east-1 \
--table-name hl-m06-banking \
--attribute-definitions AttributeName=PK,AttributeType=S AttributeName=SK,AttributeType=S \
--key-schema AttributeName=PK,KeyType=HASH AttributeName=SK,KeyType=RANGE \
--billing-mode PAY_PER_REQUEST \
--stream-specification StreamEnabled=true,StreamViewType=NEW_AND_OLD_IMAGES
aws dynamodb create-table --endpoint-url http://localhost:8000 --region us-east-1 \
--table-name hl-m06-audit \
--attribute-definitions AttributeName=PK,AttributeType=S \
--key-schema AttributeName=PK,KeyType=HASH \
--billing-mode PAY_PER_REQUESTEnable TTL on the idempotency attribute
aws dynamodb update-time-to-live --endpoint-url http://localhost:8000 --region us-east-1 \
--table-name hl-m06-banking --time-to-live-specification Enabled=true,AttributeName=expires_atTTL applies only to items that have a Number attribute expires_at holding epoch seconds. Accounts and transactions without it are never expired.
Run the transfer demo
Build the console app in Complete code. It seeds two accounts, sends the same transfer three times with the same key (one posts, two replay), sends a reused key with a different amount (rejected), and fires 20 concurrent transfers that would overdraw the account (only the affordable ones post).
Deploy the Streams Lambda (optional, AWS)
Use the CDK stack in Infrastructure as Code to create the table with a stream, the audit table and the .NET 10 Lambda with an event source mapping. Run the demo against AWS (DDB_LOCAL=false), then check the audit table. Destroy the stack afterwards.
AWS Console instructions
- Streams: Console → DynamoDB → Tables → hl-m06-banking → Exports and streams → DynamoDB stream details → Turn on, view type New and old images.
- Trigger: on the same tab, Trigger → Create trigger, choose the audit Lambda, batch size 100, and enable Report batch item failures in the Lambda's event source mapping (Lambda → Functions → hl-m06-audit → Configuration → Triggers → Edit). Also set an On-failure destination (an SQS queue) and Retry attempts (for example 5) so a poison record does not block the shard for 24 hours.
- TTL: Additional settings → Time to Live (TTL) → Turn on, attribute name
expires_at. Use Run preview to see which items would expire. - Metrics: Monitor tab →
ConditionalCheckFailedRequests,TransactionConflict,TimeToLiveDeletedItemCount; and on the Lambda,IteratorAge(how far behind the stream consumer is).
AWS CLI
# Conditional debit: succeeds only if funds are sufficient and the account is open.
aws dynamodb update-item --table-name hl-m06-banking \
--key '{"PK":{"S":"ACC#ACC-0000000001"},"SK":{"S":"PROFILE"}}' \
--update-expression "SET balance = balance - :amt, version = version + :one" \
--condition-expression "balance >= :amt AND #st = :open" \
--expression-attribute-names '{"#st":"status"}' \
--expression-attribute-values '{":amt":{"N":"250.00"},":one":{"N":"1"},":open":{"S":"open"}}' \
--return-values-on-condition-check-failure ALL_OLD \
--profile learn-sandbox --region us-east-1
# Turn on a stream with new and old images.
aws dynamodb update-table --table-name hl-m06-banking \
--stream-specification StreamEnabled=true,StreamViewType=NEW_AND_OLD_IMAGES \
--profile learn-sandbox --region us-east-1
# Event source mapping with partial batch failure reporting and a DLQ.
aws lambda create-event-source-mapping --function-name hl-m06-audit \
--event-source-arn "$STREAM_ARN" --starting-position LATEST --batch-size 100 \
--function-response-types ReportBatchItemFailures \
--maximum-retry-attempts 5 --bisect-batch-on-function-error \
--destination-config "{\"OnFailure\":{\"Destination\":\"$DLQ_ARN\"}}" \
--profile learn-sandbox --region us-east-1
# Turn on TTL.
aws dynamodb update-time-to-live --table-name hl-m06-banking \
--time-to-live-specification Enabled=true,AttributeName=expires_at \
--profile learn-sandbox --region us-east-1Infrastructure as Code
using Amazon.CDK;
using Amazon.CDK.AWS.DynamoDB;
using Attribute = Amazon.CDK.AWS.DynamoDB.Attribute; // avoids CS0104 with System.Attribute when ImplicitUsings is on
using Amazon.CDK.AWS.Lambda;
using Amazon.CDK.AWS.Lambda.EventSources;
using Amazon.CDK.AWS.Logs;
using Amazon.CDK.AWS.SQS;
using Constructs;
namespace Harbourline.Infra;
public sealed class CorrectnessStack : Stack
{
public CorrectnessStack(Construct scope, string id, IStackProps? props = null) : base(scope, id, props)
{
var banking = new TableV2(this, "Banking", new TablePropsV2
{
TableName = "hl-m06-banking",
PartitionKey = new Attribute { Name = "PK", Type = AttributeType.STRING },
SortKey = new Attribute { Name = "SK", Type = AttributeType.STRING },
Billing = Billing.OnDemand(),
DynamoStream = StreamViewType.NEW_AND_OLD_IMAGES,
TimeToLiveAttribute = "expires_at",
RemovalPolicy = RemovalPolicy.DESTROY // lab only
});
var audit = new TableV2(this, "Audit", new TablePropsV2
{
TableName = "hl-m06-audit",
PartitionKey = new Attribute { Name = "PK", Type = AttributeType.STRING },
Billing = Billing.OnDemand(),
RemovalPolicy = RemovalPolicy.DESTROY
});
var dlq = new Queue(this, "AuditDlq", new QueueProps
{
QueueName = "hl-m06-audit-dlq",
Encryption = QueueEncryption.SQS_MANAGED,
EnforceSSL = true,
RetentionPeriod = Duration.Days(14)
});
var fn = new Function(this, "AuditFn", new FunctionProps
{
FunctionName = "hl-m06-audit",
Runtime = Runtime.DOTNET_10, // ⚠ VERIFY the constant name in your Amazon.CDK.Lib version
Architecture = Architecture.ARM_64,
Handler = "Harbourline.Audit.StreamHandler::Harbourline.Audit.StreamHandler.Function::HandleAsync",
Code = Code.FromAsset("../src/Harbourline.Audit.StreamHandler/bin/Release/net10.0/publish"),
MemorySize = 512,
Timeout = Duration.Seconds(30),
Environment = new Dictionary<string, string> { ["AUDIT_TABLE"] = audit.TableName },
LogGroup = new LogGroup(this, "AuditLogs", new LogGroupProps
{
LogGroupName = "/aws/lambda/hl-m06-audit",
Retention = RetentionDays.ONE_MONTH,
RemovalPolicy = RemovalPolicy.DESTROY
})
});
fn.AddEventSource(new DynamoEventSource(banking, new DynamoEventSourceProps
{
StartingPosition = StartingPosition.LATEST,
BatchSize = 100,
BisectBatchOnError = true,
RetryAttempts = 5,
ReportBatchItemFailures = true,
OnFailure = new SqsDlq(dlq)
}));
// Least privilege: the function may only add audit items, never update or delete them.
audit.Grant(fn, "dynamodb:PutItem");
Tags.SetTag("Project", "cea-lab");
Tags.SetTag("Lab", "m06-ddb-correctness");
Tags.SetTag("Environment", "learn");
}
}DynamoEventSource grants the function permission to read the stream automatically.
Complete code
The transfer service
using System.Globalization;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using Amazon.DynamoDBv2;
using Amazon.DynamoDBv2.Model;
namespace Harbourline.Payments.DynamoDb;
public sealed record TransferRequest(string CustomerId, string FromAccountId, string ToAccountId, decimal Amount, string Currency, string Reference);
public sealed record TransferResult(string TransferId, string Status, decimal Amount, string Currency, DateTimeOffset BookedAt);
public enum TransferOutcome { Posted, Replayed, InsufficientFunds, AccountNotOpen, KeyReusedWithDifferentRequest, Conflict }
public sealed class TransferService(IAmazonDynamoDB ddb, TimeProvider clock, string tableName = "hl-m06-banking")
{
private static readonly CultureInfo Inv = CultureInfo.InvariantCulture;
private static readonly TimeSpan IdempotencyWindow = TimeSpan.FromHours(24);
public async Task<(TransferOutcome Outcome, TransferResult? Result)> TransferAsync(TransferRequest req, string idempotencyKey, CancellationToken ct)
{
if (req.Amount <= 0) throw new ArgumentOutOfRangeException(nameof(req), "Amount must be positive");
if (req.FromAccountId == req.ToAccountId) throw new ArgumentException("Cannot transfer to the same account");
var now = clock.GetUtcNow();
var transferId = Guid.CreateVersion7().ToString();
var result = new TransferResult(transferId, "booked", req.Amount, req.Currency, now);
var idemPk = $"IDEM#{req.CustomerId}#{idempotencyKey}";
var requestHash = Hash(req);
var amt = new AttributeValue { N = req.Amount.ToString(Inv) };
var date = now.ToString("yyyy-MM-dd", Inv);
var actions = new List<TransactWriteItem>
{
// 0: claim the idempotency key
new() { Put = new Put
{
TableName = tableName,
Item = new()
{
["PK"] = S(idemPk), ["SK"] = S("IDEM"),
["request_hash"] = S(requestHash),
["status"] = S("COMPLETED"),
["response"] = S(JsonSerializer.Serialize(result)),
["expires_at"] = new AttributeValue { N = now.Add(IdempotencyWindow).ToUnixTimeSeconds().ToString(Inv) }
},
ConditionExpression = "attribute_not_exists(PK)",
ReturnValuesOnConditionCheckFailure = ReturnValuesOnConditionCheckFailure.ALL_OLD
}},
// 1: debit with balance and status guard
new() { Update = new Update
{
TableName = tableName,
Key = Key($"ACC#{req.FromAccountId}", "PROFILE"),
UpdateExpression = "SET balance = balance - :amt, version = version + :one",
ConditionExpression = "balance >= :amt AND #st = :open AND currency = :ccy",
ExpressionAttributeNames = new() { ["#st"] = "status" },
ExpressionAttributeValues = new() { [":amt"] = amt, [":one"] = new() { N = "1" }, [":open"] = S("open"), [":ccy"] = S(req.Currency) },
ReturnValuesOnConditionCheckFailure = ReturnValuesOnConditionCheckFailure.ALL_OLD
}},
// 2: credit, only if the target exists and is open
new() { Update = new Update
{
TableName = tableName,
Key = Key($"ACC#{req.ToAccountId}", "PROFILE"),
UpdateExpression = "SET balance = balance + :amt, version = version + :one",
ConditionExpression = "attribute_exists(PK) AND #st = :open AND currency = :ccy",
ExpressionAttributeNames = new() { ["#st"] = "status" },
ExpressionAttributeValues = new() { [":amt"] = amt, [":one"] = new() { N = "1" }, [":open"] = S("open"), [":ccy"] = S(req.Currency) },
ReturnValuesOnConditionCheckFailure = ReturnValuesOnConditionCheckFailure.ALL_OLD
}},
// 3 and 4: the two ledger lines
new() { Put = LedgerLine(req.FromAccountId, date, transferId, "D", -req.Amount, req, now) },
new() { Put = LedgerLine(req.ToAccountId, date, transferId, "C", req.Amount, req, now) }
};
try
{
await ddb.TransactWriteItemsAsync(new TransactWriteItemsRequest
{
// No ClientRequestToken set: the SDK generates one per call, so its own automatic
// retries of this exact request are no-ops within DynamoDB's 10-minute window.
TransactItems = actions
}, ct);
return (TransferOutcome.Posted, result);
}
catch (TransactionCanceledException ex)
{
var reasons = ex.CancellationReasons ?? [];
string Code(int i) => i < reasons.Count ? reasons[i].Code ?? "None" : "None";
if (Code(0) == "ConditionalCheckFailed")
{
// The key exists: compare request hashes, then replay or reject.
var existing = reasons[0].Item ?? await ReadIdempotencyAsync(idemPk, ct);
if (existing is null) return (TransferOutcome.Conflict, null);
if (existing["request_hash"].S != requestHash) return (TransferOutcome.KeyReusedWithDifferentRequest, null);
return (TransferOutcome.Replayed, JsonSerializer.Deserialize<TransferResult>(existing["response"].S));
}
if (Code(1) == "ConditionalCheckFailed")
{
var old = reasons[1].Item;
var isOpen = old is not null && old.TryGetValue("status", out var st) && st.S == "open";
return (isOpen ? TransferOutcome.InsufficientFunds : TransferOutcome.AccountNotOpen, null);
}
if (Code(2) == "ConditionalCheckFailed") return (TransferOutcome.AccountNotOpen, null);
// "TransactionConflict": another transaction touched one of these items at the same time. Caller may retry.
return (TransferOutcome.Conflict, null);
}
}
private async Task<Dictionary<string, AttributeValue>?> ReadIdempotencyAsync(string pk, CancellationToken ct)
{
var resp = await ddb.GetItemAsync(new GetItemRequest { TableName = tableName, Key = Key(pk, "IDEM"), ConsistentRead = true }, ct);
return resp.Item is { Count: > 0 } ? resp.Item : null;
}
private Put LedgerLine(string accountId, string date, string transferId, string side, decimal amount, TransferRequest req, DateTimeOffset now) => new()
{
TableName = tableName,
Item = new()
{
["PK"] = S($"ACC#{accountId}"),
["SK"] = S($"TX#{date}#{transferId}-{side}"),
["entity"] = S("transaction"),
["transaction_id"] = S($"{transferId}-{side}"),
["booking_date"] = S(date),
["amount"] = new AttributeValue { N = amount.ToString(Inv) },
["currency"] = S(req.Currency),
["type"] = S(side == "D" ? "transfer_out" : "transfer_in"),
["channel"] = S("mobile"),
["reference"] = S(req.Reference),
["status"] = S("booked"),
["created_at"] = S(now.ToString("O", Inv))
},
ConditionExpression = "attribute_not_exists(PK)"
};
// Canonical hash of the business content of the request (not headers or timestamps).
private static string Hash(TransferRequest r)
{
var canonical = string.Join('|', r.CustomerId, r.FromAccountId, r.ToAccountId, r.Amount.ToString("0.00", Inv), r.Currency, r.Reference);
return Convert.ToHexString(SHA256.HashData(Encoding.UTF8.GetBytes(canonical)));
}
private static AttributeValue S(string v) => new() { S = v };
private static Dictionary<string, AttributeValue> Key(string pk, string sk) => new() { ["PK"] = S(pk), ["SK"] = S(sk) };
}The demo
using System.Globalization;
using Amazon;
using Amazon.DynamoDBv2;
using Amazon.DynamoDBv2.Model;
using Amazon.Runtime;
using Harbourline.Payments.DynamoDb;
var local = !string.Equals(Environment.GetEnvironmentVariable("DDB_LOCAL"), "false", StringComparison.OrdinalIgnoreCase);
IAmazonDynamoDB ddb = local
? new AmazonDynamoDBClient(new BasicAWSCredentials("local", "local"),
new AmazonDynamoDBConfig { ServiceURL = "http://localhost:8000", AuthenticationRegion = "us-east-1" })
: new AmazonDynamoDBClient(RegionEndpoint.USEast1);
await SeedAccountAsync("ACC-0000000001", 500.00m);
await SeedAccountAsync("ACC-0000000002", 0.00m);
var svc = new TransferService(ddb, TimeProvider.System);
var ct = CancellationToken.None;
var req = new TransferRequest("CUS-00012345", "ACC-0000000001", "ACC-0000000002", 250.00m, "GBP", "Rent November");
// Same key three times: one Posted, two Replayed with the SAME transfer id.
for (var i = 1; i <= 3; i++)
{
var (outcome, result) = await svc.TransferAsync(req, "7f4e2a1c-rent-nov", ct);
Console.WriteLine($"Attempt {i}: {outcome} {result?.TransferId}");
}
// Same key, different amount: rejected.
var (reuse, _) = await svc.TransferAsync(req with { Amount = 999.00m }, "7f4e2a1c-rent-nov", ct);
Console.WriteLine($"Reused key with different body: {reuse}");
// 20 concurrent £20 transfers with distinct keys against the remaining £250: at most 12 can post.
var outcomes = await Task.WhenAll(Enumerable.Range(1, 20).Select(i =>
svc.TransferAsync(req with { Amount = 20.00m, Reference = $"Split {i}" }, $"split-{i:D2}", ct)));
Console.WriteLine(string.Join(", ", outcomes.GroupBy(o => o.Outcome).Select(g => $"{g.Key}={g.Count()}")));
var from = await ddb.GetItemAsync(new GetItemRequest
{
TableName = "hl-m06-banking",
Key = new() { ["PK"] = new() { S = "ACC#ACC-0000000001" }, ["SK"] = new() { S = "PROFILE" } },
ConsistentRead = true
});
Console.WriteLine($"Final balance ACC-0000000001: {from.Item["balance"].N}");
async Task SeedAccountAsync(string id, decimal balance) =>
await ddb.PutItemAsync(new PutItemRequest
{
TableName = "hl-m06-banking",
Item = new()
{
["PK"] = new() { S = $"ACC#{id}" }, ["SK"] = new() { S = "PROFILE" },
["account_id"] = new() { S = id }, ["currency"] = new() { S = "GBP" },
["status"] = new() { S = "open" }, ["version"] = new() { N = "0" },
["balance"] = new() { N = balance.ToString(CultureInfo.InvariantCulture) }
}
});Example output:
Attempt 1: Posted 019a3d0b-…
Attempt 2: Replayed 019a3d0b-…
Attempt 3: Replayed 019a3d0b-…
Reused key with different body: KeyReusedWithDifferentRequest
Posted=12, InsufficientFunds=6, Conflict=2
Final balance ACC-0000000001: 10The exact split between Posted, InsufficientFunds and Conflict varies run to run: concurrent transactions on the same item can cancel each other with TransactionConflict, so fewer than 12 may post. The final balance is always 500 − 250 − 20 × (number posted) — never negative and never inconsistent. In an API you would retry Conflict a few times with backoff, using the same idempotency key.
The audit stream consumer
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles>
<AWSProjectType>Lambda</AWSProjectType>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Amazon.Lambda.Core" Version="3.4.0" />
<PackageReference Include="Amazon.Lambda.DynamoDBEvents" Version="4.0.0" />
<PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" Version="3.0.1" />
<PackageReference Include="AWSSDK.DynamoDBv2" Version="4.1.0.2" />
</ItemGroup>
</Project>using Amazon.DynamoDBv2;
using Amazon.DynamoDBv2.Model;
using Amazon.Lambda.Core;
using Amazon.Lambda.DynamoDBEvents;
[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]
namespace Harbourline.Audit.StreamHandler;
public sealed class Function
{
private readonly IAmazonDynamoDB _ddb;
private readonly string _auditTable;
// Lambda calls the parameterless constructor once per execution environment.
public Function() : this(new AmazonDynamoDBClient(), Environment.GetEnvironmentVariable("AUDIT_TABLE") ?? "hl-m06-audit") { }
public Function(IAmazonDynamoDB ddb, string auditTable) { _ddb = ddb; _auditTable = auditTable; }
public async Task<StreamsEventResponse> HandleAsync(DynamoDBEvent evt, ILambdaContext context)
{
var failures = new List<StreamsEventResponse.BatchItemFailure>();
foreach (var record in evt.Records)
{
try
{
var keys = record.Dynamodb.Keys;
var pk = keys["PK"].S;
// Only balance-bearing account items are audited here; ledger lines are audited by the ledger itself.
if (!pk.StartsWith("ACC#", StringComparison.Ordinal) || keys["SK"].S != "PROFILE") continue;
var oldBalance = record.Dynamodb.OldImage?.GetValueOrDefault("balance")?.N;
var newBalance = record.Dynamodb.NewImage?.GetValueOrDefault("balance")?.N;
var byService = record.UserIdentity?.PrincipalId == "dynamodb.amazonaws.com"; // TTL deletes
await _ddb.PutItemAsync(new PutItemRequest
{
TableName = _auditTable,
Item = new()
{
// The stream sequence number is unique per shard: a natural idempotency key for the consumer.
["PK"] = new() { S = $"EVT#{record.Dynamodb.SequenceNumber}" },
["occurred_at"] = new() { S = DateTime.SpecifyKind(record.Dynamodb.ApproximateCreationDateTime, DateTimeKind.Utc).ToString("O") },
["actor"] = new() { S = byService ? "dynamodb-ttl" : "payments-api" },
["action"] = new() { S = record.EventName },
["resource"] = new() { S = pk },
["old_balance"] = new() { S = oldBalance ?? "-" },
["new_balance"] = new() { S = newBalance ?? "-" },
["outcome"] = new() { S = "recorded" },
["correlation_id"] = new() { S = record.EventID }
},
ConditionExpression = "attribute_not_exists(PK)"
});
}
catch (ConditionalCheckFailedException)
{
// Already recorded by an earlier delivery of the same record: success.
}
catch (Exception ex)
{
context.Logger.LogError($"Audit write failed for {record.EventID}: {ex.GetType().Name}");
failures.Add(new StreamsEventResponse.BatchItemFailure { ItemIdentifier = record.Dynamodb.SequenceNumber });
break; // stream order matters: stop at the first failure, Lambda retries from here
}
}
return new StreamsEventResponse { BatchItemFailures = failures };
}
}Code explained line by line
TimeProvider clock— .NET 8+ abstraction for time; tests can inject a fake clock to test idempotency expiry.idemPk = $"IDEM#{CustomerId}#{key}"— scopes keys per customer, so two customers who happen to generate the same key never collide, and one cannot probe another's keys.Hash(req)— hashes the business content only, formatted with the invariant culture so250and250.00hash identically.- Action 0,
attribute_not_exists(PK)— the idempotency claim. If it fails, the whole transaction is cancelled: no money moves. ReturnValuesOnConditionCheckFailure = ALL_OLD— when the condition fails, the cancellation reason carries the existing item, so the replay needs no extra read in most cases.- Action 1,
balance >= :amt AND #st = :open AND currency = :ccy— the guard: funds, account state and currency match.#stis needed becausestatusis a reserved word. - Action 2,
attribute_exists(PK)on the credit side — without it,UpdateItemwould happily create a new account item for a mistyped account ID. - Actions 3 and 4 — the ledger lines are written inside the same transaction, so the balance and the ledger can never disagree.
- No explicit
ClientRequestToken— the SDK fills one in per call, so if the SDK retries after a network error, DynamoDB recognises the token for 10 minutes and does not re-apply. Do not derive it from the idempotency key yourself: a client retry builds a new request (new transfer ID, new timestamp), and the same token with different parameters raisesIdempotentParameterMismatchException. The 24-hour idempotency record handles client retries. catch (TransactionCanceledException ex)andCancellationReasons— one reason per action, in order.CodeisNone,ConditionalCheckFailed,TransactionConflict,ItemCollectionSizeLimitExceeded,ThrottlingErrororValidationError.- Replay path — compare stored and current hashes. Equal → return the stored response (same transfer ID). Different →
KeyReusedWithDifferentRequest, which the API maps to HTTP 422 as a ProblemDetails response. TransactionConflict→Conflict— a concurrent transaction touched the same item. Safe to retry with the same key.- In the Lambda,
record.Dynamodb.SequenceNumberas the audit item key plusattribute_not_exists(PK)— redelivered records are detected and skipped. UserIdentity.PrincipalId == "dynamodb.amazonaws.com"— TTL deletions appear in the stream as service deletes, distinguishable from user deletes. Useful evidence for "why did this record disappear?"failures.Add(...)thenbreak— withReportBatchItemFailures, Lambda retries from the first reported sequence number. Breaking preserves per-item ordering.- The audit table role has only
dynamodb:PutItem— audit records cannot be modified by the code that writes them.
Security considerations
⚠ Learning shortcut — not for production
The lab writes audit records to a table in the same account with RemovalPolicy.DESTROY and default encryption. In production, send audit events to a separate audit account (or an S3 bucket with Object Lock in compliance mode), encrypt with a customer managed KMS key whose policy excludes the application team, and retain according to your records policy.
- Never trust the client's key alone. Bind it to the authenticated customer (part of the PK) and compare request hashes.
- Rate-limit key creation per customer at the API layer so an attacker cannot fill the table with keys.
- Responses stored in the idempotency record can contain personal data. Store the minimum needed to replay (IDs, status, amounts) and let TTL remove it. Document the retention in your records of processing.
- Streams contain full images. With
NEW_AND_OLD_IMAGES, every consumer sees every attribute. Grantdynamodb:GetRecordsonly to roles that need the data, or useKEYS_ONLYand re-read selectively. - TTL is not a GDPR erasure mechanism with a deadline. Deletion happens within a few days; for erasure requests with a deadline, delete explicitly and record that you did.
- Do not log balances or IBANs in Lambda logs. The handler logs only the event ID and exception type.
Banking use case
Harbourline applies these patterns to the Payments Service:
| Endpoint / process | Pattern |
|---|---|
POST /payments, POST /transfers | Idempotency record + TransactWriteItems |
PATCH /accounts/{id}/overdraft-limit | Optimistic locking with version |
| Card authorisation holds | Item with expires_at = auth expiry; TTL cleans up abandoned holds; a scheduled job releases holds explicitly at the exact time (TTL is not punctual) |
| Audit trail | Streams → Lambda → audit table / S3 Object Lock |
| Open Banking consent tokens | TTL as a backstop; the API checks expires_at on every read |
Common mistakes
- Using
BatchWriteItemfor a transfer → it looks like "one call" → batches are not atomic; useTransactWriteItems. - Relying on
ClientRequestTokenas the idempotency mechanism → it only lasts 10 minutes and only for identical calls → keep your own idempotency record. - Forgetting
attribute_existson the credit side →UpdateItemupserts → typo'd accounts get created with a positive balance. - Treating TTL as exact → items may live days past expiry → filter on
expires_atin reads that care, and use explicit deletes or Step Functions/EventBridge Scheduler for time-critical actions. - Non-idempotent stream consumers → duplicate audit lines and double notifications → key consumer writes by sequence number or event ID.
- No DLQ on the event source mapping → a poison record blocks the shard and records age out after 24 hours → set retries, bisect and an on-failure destination; alarm on
IteratorAge. - Hot idempotency partition → using a constant PK like
IDEMwith the key in SK → put the key in the PK.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
TransactionCanceledException with reason ConditionalCheckFailed at index 0 | Duplicate idempotency key | Expected: replay or reject |
Reason TransactionConflict | Concurrent transaction on the same item | Retry with backoff; reduce contention (fewer items per transaction) |
ValidationException: Transaction request cannot include multiple operations on one item | Same key twice in the transaction (for example, from and to accounts equal) | Validate inputs before building the transaction |
IdempotentParameterMismatchException | Same ClientRequestToken reused with different parameters within 10 minutes | Let the SDK generate the token; handle client retries with your own idempotency record |
CancellationReasons is empty | Older SDK or exception thrown before DynamoDB evaluated actions | Upgrade to SDK v4; check the exception message |
Lambda IteratorAge keeps rising | Consumer failing repeatedly or too slow | Check logs and the DLQ; raise batch size or parallelization factor |
| Items not expiring | TTL disabled, attribute not a Number, or value in milliseconds | Store epoch seconds as N; enable TTL on that attribute name |
| Expired idempotency record still blocks a retry after 24 h | TTL has not deleted it yet | Expected and safe; if you need exactness, check expires_at in code |
Cost and cleanup
Local runs are free. In AWS:
- Transactions consume two write units per item: the five-action transfer costs 10 write request units ≈ USD 0.00000625 on-demand. A failed transaction still consumes capacity.
- Streams: the first 2.5 million stream read requests per month are free; GetRecords calls made by Lambda triggers are not charged; Lambda invocations are within the free tier for a lab.
- TTL deletes are free (no write units) in the Region where they occur.
- SQS DLQ and CloudWatch Logs: cents at lab volume; log retention set to one month.
# Local
docker rm -f hl-m06-ddb
# AWS (CDK)
cdk destroy CorrectnessStack --profile learn-sandbox
# or Terraform
terraform destroy
aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=cea-lab \
--query "ResourceTagMappingList[].ResourceARN" --profile learn-sandbox --region us-east-1Exercises
- 🟢 Guided: Run the demo. Then change the debit condition to remove the currency check and send a EUR transfer from a GBP account. Observe what happens, then restore the check.
- 🟡 Challenge: Add an
IN_PROGRESSstate for a long-running payment that calls an external sanctions-screening service: claim the key asIN_PROGRESS, call the service, then complete. A concurrent duplicate must get 409 Conflict while in progress, not a second screening call. Handle a crashed worker that leavesIN_PROGRESSforever. - 🔴 Live problem: The operations team reports that last night 214 card-authorisation holds were still visible in the app 30 hours after they should have expired. The developer says "TTL is broken". Investigate and fix.
Assignment
Package the transfer service as a reusable Harbourline.Idempotency library with an interface IIdempotencyStore, a DynamoDB implementation, and xUnit v3 tests against DynamoDB Local (Testcontainers) covering: first call, replay, key reuse with different body, concurrent duplicates (exactly one Posted), and expiry with a fake TimeProvider.
Solutions
Without the currency check, the transaction succeeds: £/€ amounts are subtracted and added as plain numbers, so the GBP account loses 250 and the EUR account gains 250 — a silent FX error. With the check restored, action 1 fails with ConditionalCheckFailed. Map that case to a clear error by inspecting the returned ALL_OLD item (currency differs).
Why: conditions encode business invariants at the database. Every invariant you leave to application code is one bug away from being violated.
- Claim:
PutItemIDEM#…withstatus = IN_PROGRESS,locked_until = now + 2 min, conditionattribute_not_exists(PK) OR (#st = :inprog AND locked_until < :now)— the second half lets a new worker take over a crashed one. - On
ConditionalCheckFailedException, read the item:COMPLETED→ replay;IN_PROGRESSand lock still valid → return 409 withRetry-After. - Call the screening service with the payment ID as its idempotency key, so a takeover does not double-screen.
- Complete with
TransactWriteItems: updateIDEM#…toCOMPLETEDwith condition#st = :inprog AND owner = :me, plus the money movement.
Why: the lock expiry handles crashes; the owner check stops a slow first worker from overwriting a second worker's result. AWS Lambda Powertools for .NET has an Idempotency utility (AWS.Lambda.Powertools.Idempotency) that implements this pattern with DynamoDB if you prefer not to build it.
TTL is working as designed: expired items are deleted typically within a few days, not at the expiry time. The app read holds without checking expires_at.
Fix:
- Immediate: filter holds in the query with
FilterExpression = "expires_at > :now"(and in the API mapping) so expired holds are never shown. - Correctness: release holds explicitly at the exact time with an EventBridge Scheduler one-time schedule per hold, or a scheduled sweep using a sparse GSI on
hold_expires_bucket(e.g. per 5-minute bucket). Keep TTL as the free garbage collector. - Evidence: check CloudWatch
TimeToLiveDeletedItemCountand the stream (service deletes) to confirm TTL ran later. Write a short incident note: "TTL is cleanup, not scheduling."
Knowledge check
Answer: 10. Transactions perform two underlying writes per item (prepare and commit). The capacity is consumed even if the transaction is cancelled.
Answer: It is only honoured for 10 minutes after the request finishes and only for an identical request. Clients may retry hours later (offline phone). An application idempotency record with a longer window and request-hash comparison is required.
Answer: The function returns record 37's sequence number as the failure. Lambda checkpoints records 1–36 and retries from 37. Records 38–100 are redelivered too, so processing must be idempotent.
Answer: The value looks like a date tens of thousands of years in the future, so items never expire. TTL needs epoch seconds as a Number.
Answer: No. Each item can appear only once per transaction. Put the condition on the Update itself.
Interview questions
- "How do you implement idempotent payment APIs?" — Client-supplied key scoped to the caller, stored with a conditional write together with a hash of the request and the response, in the same atomic operation as the money movement; replay on match, reject on mismatch, expire after a defined window.
- "What are DynamoDB transactions' limits?" — Up to 100 actions and 4 MB, one Region and account, each item once, double capacity cost, serializable against single-item operations but not across
Query/Scanas a unit, not cross-Region in global tables. - "How would you build an audit trail from DynamoDB?" — Streams with
NEW_AND_OLD_IMAGESto an idempotent Lambda writing to a separate, append-only store with restricted permissions, DLQ andIteratorAgealarms; for long retention, S3 with Object Lock. - "Is TTL suitable for deleting data on a deadline?" — No; it is best-effort within days. Use explicit deletes for deadlines and TTL for cost cleanup.
Certification practice
An API writes payment records to DynamoDB. Clients sometimes retry after timeouts, creating duplicate payments. Each request includes a unique client-generated ID. What is the simplest reliable fix?
A. Use BatchWriteItem so writes are grouped.
B. Use PutItem with ConditionExpression attribute_not_exists(PK) where PK includes the client ID.
C. Enable DynamoDB Streams and delete duplicates in a Lambda function.
D. Switch to strongly consistent reads before each write.
Answer: B. The conditional put atomically rejects the second write. Batches are not conditional or atomic (A). Cleaning up afterwards means the duplicate existed and may have triggered side effects (C). Read-then-write still races (D).
A Lambda function processes a DynamoDB stream. One malformed record causes the function to fail repeatedly, and new records are delayed for hours. What should the architect configure? (Choose two.)
A. Increase the stream retention period to 7 days.
B. Configure maximum retry attempts, bisect batch on error, and an on-failure destination.
C. Enable ReportBatchItemFailures and return the failing record's sequence number.
D. Change the stream view type to KEYS_ONLY.
E. Reserve concurrency for the function.
Answer: B and C. Bounded retries with bisect isolate the bad record and send it to a DLQ; partial batch responses avoid reprocessing successful records. Stream retention is fixed at 24 hours (A). View type does not fix poison records (D). Concurrency does not help a single shard stuck on one record (E).
A team stores session tokens in DynamoDB with a TTL attribute. A compliance reviewer requires that tokens are unusable immediately after expiry. What should the team do?
A. Nothing; TTL deletes items at the expiry second. B. Reduce the TTL value by one day to compensate. C. Check the expiry attribute in the application (or a filter expression) on every read, and keep TTL for cleanup. D. Use a Scan every minute to delete expired items.
Answer: C. TTL deletion is asynchronous, typically within a few days, so reads must enforce expiry. Shortening TTL changes the business rule without guaranteeing timing (B). A minute-by-minute Scan is expensive and still racy (D). A is false.
Completion checklist
- I can explain the difference between a conditional write, a transaction and
ClientRequestToken - I ran the transfer demo and saw Posted, Replayed and KeyReusedWithDifferentRequest outcomes
- I can decode
CancellationReasonsto a business error - I can describe how the stream consumer stays idempotent and ordered
- I know why TTL is cleanup, not scheduling
- I removed the local container and any AWS stack
Key takeaways
- Condition expressions are DynamoDB's integrity constraints: balance guards, existence checks and optimistic locking, all atomic.
- Idempotency = caller-scoped key + request hash + stored response, claimed in the same transaction as the money movement.
TransactWriteItemsgives all-or-nothing across up to 100 items; readCancellationReasonsin order; retryTransactionConflict.- Streams feed audit and projections; consumers must be idempotent, ordered and protected with retries, bisect, DLQ and
IteratorAgealarms. - TTL deletes for free but not on time — enforce expiry in reads.
Further reading
Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.
Amazon DynamoDB fundamentals
Model Harbourline customers, accounts and transactions in a single DynamoDB table: choose partition and sort keys from access patterns, pick a capacity mode, add a global secondary index, and query it from .NET 10 with the AWS SDK v4 against DynamoDB Local.
Amazon ElastiCache (Valkey/Redis OSS)
Use ElastiCache for Valkey safely in a bank: cache-aside with .NET HybridCache, distributed session state, atomic rate-limit counters with a Lua script, choosing serverless or node-based caches, securing them with TLS, RBAC or IAM authentication — and knowing what must never be cached.