Skip to Content
Cloud Engineering Academy
6. Databases & Data Architecture

DynamoDB for banking correctness

Make DynamoDB safe for money: conditional writes for balance guards and optimistic locking, an idempotency-key store that rejects duplicate payments, TransactWriteItems for transfers, Streams with an idempotent .NET 10 Lambda for audit events, and TTL for expiring records.

advanced⏱ 2 h 30 min🟢 FreeDVA-C03SAA-C03DEA-C01

Assumes: .NET 10 LTS · AWS SDK for .NET v4 (AWSSDK.DynamoDBv2 4.1.0.2) · Amazon.Lambda.DynamoDBEvents 4.0.0 · Lambda runtime dotnet10 · DynamoDB Local · Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Use condition expressions to guard balances and to implement optimistic locking with a version attribute.
  • Build an idempotency-key store that guarantees a payment request is processed at most once, even when the client, load balancer or Lambda retries.
  • Post a transfer with TransactWriteItems: debit, credit, two ledger items and the idempotency record, all or nothing — and decode TransactionCanceledException reasons.
  • Consume DynamoDB Streams with a .NET 10 Lambda that writes audit events, handles partial batch failures and is itself idempotent.
  • Use TTL to expire idempotency records and short-lived data, and explain exactly what TTL does and does not guarantee.

Prerequisites

Why this matters in banking

At 12:03 a Harbourline customer taps Send £250 to their landlord. The mobile network drops after the request leaves the phone. The app does what every good app does: it retries. The API gateway also has a retry policy. And the Lambda behind it timed out after writing the payment but before returning. Without idempotency, the landlord gets £750 and the customer gets an overdraft fee.

Duplicate money movement is not an edge case — at scale it happens every day. Regulators expect firms to prevent and correct it, and the UK's payment schemes expect each instruction to be processed once. DynamoDB gives you the building blocks to make this impossible by construction: atomic conditional writes, multi-item transactions, a change stream and automatic expiry. This lesson assembles them into patterns you can reuse in every payment-like endpoint.

The idea in plain language

  • Conditional write: "Write this item only if this condition is true at the moment of writing." DynamoDB checks and writes atomically. There is no gap for another request to sneak in.
  • Idempotency key: a unique ID the client sends with a request (Idempotency-Key header). The server records "I've seen this key" with a conditional write the first time. A repeat with the same key gets the original result, not a second payment.
  • Transaction: up to 100 actions on up to 100 different items that all succeed or all fail.
  • Stream: an ordered log of every change to items in a table, kept for 24 hours, that a Lambda function can read.
  • TTL (time to live): an attribute holding an expiry time; DynamoDB deletes the item some time after it passes, for free.

Analogy: An idempotency key is the cheque number. The bank's clerk keeps a book of cheque numbers already paid. Before paying, they check and write the number in one motion. A photocopy of the same cheque presented ten minutes later is refused with "already paid — here's the receipt". After a while, old cheque numbers are archived out of the book (TTL).

What problem these features solve

FeatureSolvesLimits and sharp edges (checked 2026-11-18)
Condition expressionsLost updates, overdrafts, duplicate inserts on one itemSingle item; failure raises ConditionalCheckFailedException and still consumes write capacity
ReturnValuesOnConditionCheckFailureSeeing why a condition failed without an extra readReturns the item as it was (ALL_OLD)
TransactWriteItemsAll-or-nothing changes across items and tablesUp to 100 actions, 4 MB total, one account and Region; two units per item; cannot touch the same item twice; not across global table Regions
ClientRequestTokenSafe SDK-level retry of a whole transactionValid for 10 minutes after the request finishes; same token with different parameters → IdempotentParameterMismatchException
DynamoDB StreamsReacting to changes (audit, projections, outbox)24-hour retention; ordered per item; records of one transaction can be interleaved with others; consumer must be idempotent
TTLExpiring data without paying for deletesDeletes typically happen within a few days of expiry, not at the exact second; expired items stay readable until deleted

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
UPDATE … WHERE Balance >= @amt in SQL ServerUpdateItem with ConditionExpression = "balance >= :amt"Same idea, one item at a time
rowversion / EF Core concurrency tokenVersion attribute + ConditionExpression = "version = :expected"You implement it; DynamoDBContext can do it with [DynamoDBVersion]
Unique index on IdempotencyKeyItem keyed by the idempotency key + attribute_not_exists(PK)The key itself is the uniqueness constraint
TransactionScope / BEGIN TRAN over several tablesTransactWriteItemsBounded size; no reads inside except ConditionCheck; no interactive transaction
Cosmos DB change feedDynamoDB Streams (or Kinesis Data Streams for DynamoDB)24 h retention; Lambda event source mapping does the polling
Cosmos DB TTLDynamoDB TTLCosmos deletes near expiry; DynamoDB within a few days — always filter
SQL Server Change Data CaptureDynamoDB StreamsStreams are on by a table setting, not a job

Architecture

Security-boundary view

The API cannot write to the audit table, and the audit Lambda cannot change balances. That separation of duties is what makes the audit trail credible.

The patterns

Pattern 1 — balance guard on a single item

UpdateItem  Key: PK=ACC#ACC-0000000001, SK=PROFILE
  UpdateExpression:    SET balance = balance - :amt, version = version + :one
  ConditionExpression: balance >= :amt AND #st = :open

DynamoDB evaluates the condition and applies the update atomically. Concurrent debits are serialised per item, so the lost update from lesson 9 cannot happen.

Pattern 2 — optimistic locking for read-modify-write

When you need to read the item, run business logic in C#, then write (for example, recalculating an overdraft limit), store a version number and write with ConditionExpression = "version = :expected". If someone else changed it, you get ConditionalCheckFailedException: re-read and retry.

Pattern 3 — idempotency record

AttributeValue
PKIDEM#<customer_id>#<idempotency-key> (scope keys to the caller so one client cannot collide with another)
SKIDEM
request_hashSHA-256 of the canonical request body
statusCOMPLETED (or IN_PROGRESS for long workflows)
responseThe JSON you returned the first time
expires_atUnix epoch seconds, e.g. now + 24 hours — the TTL attribute

Rules:

  1. Same key, same hash → replay the stored response.
  2. Same key, different hash → reject with 422 (the client reused a key for a different payment — a bug or an attack).
  3. New key → process and store.

Because expires_at is only enforced eventually, the read path must also treat an item whose expires_at is in the past as absent — or simply accept that duplicates are blocked for at least 24 hours, which is the safe direction.

Pattern 4 — transfer in one transaction

Five actions, one call:

  1. Put IDEM#… with attribute_not_exists(PK).
  2. Update debit account: balance >= :amt AND #st = :open.
  3. Update credit account: attribute_exists(PK) AND #st = :open.
  4. Put debit transaction item TX#date#id-D with attribute_not_exists(PK).
  5. Put credit transaction item TX#date#id-C.

If any condition fails, nothing is written, and TransactionCanceledException.CancellationReasons tells you which action failed and why, in the same order as your actions.

Pattern 5 — idempotent stream consumer

Lambda reads stream batches at least once. A retry after a partial failure redelivers records. Make the consumer idempotent by writing audit events keyed by the stream record's SequenceNumber (unique per shard) with attribute_not_exists(PK), and return partial batch failures so only the failed record and later ones are retried.

Step-by-step implementation

Start DynamoDB Local and create the tables

Reuse hl-m06-banking from the previous lesson, and add an audit table:

docker run --name hl-m06-ddb -p 8000:8000 -d amazon/dynamodb-local:latest -jar DynamoDBLocal.jar -sharedDb -inMemory

aws dynamodb create-table --endpoint-url http://localhost:8000 --region us-east-1 \
  --table-name hl-m06-banking \
  --attribute-definitions AttributeName=PK,AttributeType=S AttributeName=SK,AttributeType=S \
  --key-schema AttributeName=PK,KeyType=HASH AttributeName=SK,KeyType=RANGE \
  --billing-mode PAY_PER_REQUEST \
  --stream-specification StreamEnabled=true,StreamViewType=NEW_AND_OLD_IMAGES

aws dynamodb create-table --endpoint-url http://localhost:8000 --region us-east-1 \
  --table-name hl-m06-audit \
  --attribute-definitions AttributeName=PK,AttributeType=S \
  --key-schema AttributeName=PK,KeyType=HASH \
  --billing-mode PAY_PER_REQUEST

Enable TTL on the idempotency attribute

aws dynamodb update-time-to-live --endpoint-url http://localhost:8000 --region us-east-1 \
  --table-name hl-m06-banking --time-to-live-specification Enabled=true,AttributeName=expires_at

TTL applies only to items that have a Number attribute expires_at holding epoch seconds. Accounts and transactions without it are never expired.

Run the transfer demo

Build the console app in Complete code. It seeds two accounts, sends the same transfer three times with the same key (one posts, two replay), sends a reused key with a different amount (rejected), and fires 20 concurrent transfers that would overdraw the account (only the affordable ones post).

Deploy the Streams Lambda (optional, AWS)

Use the CDK stack in Infrastructure as Code to create the table with a stream, the audit table and the .NET 10 Lambda with an event source mapping. Run the demo against AWS (DDB_LOCAL=false), then check the audit table. Destroy the stack afterwards.

AWS Console instructions

  • Streams: Console → DynamoDB → Tables → hl-m06-banking → Exports and streams → DynamoDB stream details → Turn on, view type New and old images.
  • Trigger: on the same tab, Trigger → Create trigger, choose the audit Lambda, batch size 100, and enable Report batch item failures in the Lambda's event source mapping (Lambda → Functions → hl-m06-audit → Configuration → Triggers → Edit). Also set an On-failure destination (an SQS queue) and Retry attempts (for example 5) so a poison record does not block the shard for 24 hours.
  • TTL: Additional settings → Time to Live (TTL) → Turn on, attribute name expires_at. Use Run preview to see which items would expire.
  • Metrics: Monitor tab → ConditionalCheckFailedRequests, TransactionConflict, TimeToLiveDeletedItemCount; and on the Lambda, IteratorAge (how far behind the stream consumer is).

AWS CLI

# Conditional debit: succeeds only if funds are sufficient and the account is open.
aws dynamodb update-item --table-name hl-m06-banking \
  --key '{"PK":{"S":"ACC#ACC-0000000001"},"SK":{"S":"PROFILE"}}' \
  --update-expression "SET balance = balance - :amt, version = version + :one" \
  --condition-expression "balance >= :amt AND #st = :open" \
  --expression-attribute-names '{"#st":"status"}' \
  --expression-attribute-values '{":amt":{"N":"250.00"},":one":{"N":"1"},":open":{"S":"open"}}' \
  --return-values-on-condition-check-failure ALL_OLD \
  --profile learn-sandbox --region us-east-1

# Turn on a stream with new and old images.
aws dynamodb update-table --table-name hl-m06-banking \
  --stream-specification StreamEnabled=true,StreamViewType=NEW_AND_OLD_IMAGES \
  --profile learn-sandbox --region us-east-1

# Event source mapping with partial batch failure reporting and a DLQ.
aws lambda create-event-source-mapping --function-name hl-m06-audit \
  --event-source-arn "$STREAM_ARN" --starting-position LATEST --batch-size 100 \
  --function-response-types ReportBatchItemFailures \
  --maximum-retry-attempts 5 --bisect-batch-on-function-error \
  --destination-config "{\"OnFailure\":{\"Destination\":\"$DLQ_ARN\"}}" \
  --profile learn-sandbox --region us-east-1

# Turn on TTL.
aws dynamodb update-time-to-live --table-name hl-m06-banking \
  --time-to-live-specification Enabled=true,AttributeName=expires_at \
  --profile learn-sandbox --region us-east-1

Infrastructure as Code

CorrectnessStack.cs
using Amazon.CDK;
using Amazon.CDK.AWS.DynamoDB;
using Attribute = Amazon.CDK.AWS.DynamoDB.Attribute; // avoids CS0104 with System.Attribute when ImplicitUsings is on
using Amazon.CDK.AWS.Lambda;
using Amazon.CDK.AWS.Lambda.EventSources;
using Amazon.CDK.AWS.Logs;
using Amazon.CDK.AWS.SQS;
using Constructs;

namespace Harbourline.Infra;

public sealed class CorrectnessStack : Stack
{
    public CorrectnessStack(Construct scope, string id, IStackProps? props = null) : base(scope, id, props)
    {
        var banking = new TableV2(this, "Banking", new TablePropsV2
        {
            TableName = "hl-m06-banking",
            PartitionKey = new Attribute { Name = "PK", Type = AttributeType.STRING },
            SortKey = new Attribute { Name = "SK", Type = AttributeType.STRING },
            Billing = Billing.OnDemand(),
            DynamoStream = StreamViewType.NEW_AND_OLD_IMAGES,
            TimeToLiveAttribute = "expires_at",
            RemovalPolicy = RemovalPolicy.DESTROY // lab only
        });

        var audit = new TableV2(this, "Audit", new TablePropsV2
        {
            TableName = "hl-m06-audit",
            PartitionKey = new Attribute { Name = "PK", Type = AttributeType.STRING },
            Billing = Billing.OnDemand(),
            RemovalPolicy = RemovalPolicy.DESTROY
        });

        var dlq = new Queue(this, "AuditDlq", new QueueProps
        {
            QueueName = "hl-m06-audit-dlq",
            Encryption = QueueEncryption.SQS_MANAGED,
            EnforceSSL = true,
            RetentionPeriod = Duration.Days(14)
        });

        var fn = new Function(this, "AuditFn", new FunctionProps
        {
            FunctionName = "hl-m06-audit",
            Runtime = Runtime.DOTNET_10, // ⚠ VERIFY the constant name in your Amazon.CDK.Lib version
            Architecture = Architecture.ARM_64,
            Handler = "Harbourline.Audit.StreamHandler::Harbourline.Audit.StreamHandler.Function::HandleAsync",
            Code = Code.FromAsset("../src/Harbourline.Audit.StreamHandler/bin/Release/net10.0/publish"),
            MemorySize = 512,
            Timeout = Duration.Seconds(30),
            Environment = new Dictionary<string, string> { ["AUDIT_TABLE"] = audit.TableName },
            LogGroup = new LogGroup(this, "AuditLogs", new LogGroupProps
            {
                LogGroupName = "/aws/lambda/hl-m06-audit",
                Retention = RetentionDays.ONE_MONTH,
                RemovalPolicy = RemovalPolicy.DESTROY
            })
        });

        fn.AddEventSource(new DynamoEventSource(banking, new DynamoEventSourceProps
        {
            StartingPosition = StartingPosition.LATEST,
            BatchSize = 100,
            BisectBatchOnError = true,
            RetryAttempts = 5,
            ReportBatchItemFailures = true,
            OnFailure = new SqsDlq(dlq)
        }));

        // Least privilege: the function may only add audit items, never update or delete them.
        audit.Grant(fn, "dynamodb:PutItem");

        Tags.SetTag("Project", "cea-lab");
        Tags.SetTag("Lab", "m06-ddb-correctness");
        Tags.SetTag("Environment", "learn");
    }
}

DynamoEventSource grants the function permission to read the stream automatically.

Complete code

The transfer service

TransferService.cs
using System.Globalization;
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using Amazon.DynamoDBv2;
using Amazon.DynamoDBv2.Model;

namespace Harbourline.Payments.DynamoDb;

public sealed record TransferRequest(string CustomerId, string FromAccountId, string ToAccountId, decimal Amount, string Currency, string Reference);
public sealed record TransferResult(string TransferId, string Status, decimal Amount, string Currency, DateTimeOffset BookedAt);

public enum TransferOutcome { Posted, Replayed, InsufficientFunds, AccountNotOpen, KeyReusedWithDifferentRequest, Conflict }

public sealed class TransferService(IAmazonDynamoDB ddb, TimeProvider clock, string tableName = "hl-m06-banking")
{
    private static readonly CultureInfo Inv = CultureInfo.InvariantCulture;
    private static readonly TimeSpan IdempotencyWindow = TimeSpan.FromHours(24);

    public async Task<(TransferOutcome Outcome, TransferResult? Result)> TransferAsync(TransferRequest req, string idempotencyKey, CancellationToken ct)
    {
        if (req.Amount <= 0) throw new ArgumentOutOfRangeException(nameof(req), "Amount must be positive");
        if (req.FromAccountId == req.ToAccountId) throw new ArgumentException("Cannot transfer to the same account");

        var now = clock.GetUtcNow();
        var transferId = Guid.CreateVersion7().ToString();
        var result = new TransferResult(transferId, "booked", req.Amount, req.Currency, now);
        var idemPk = $"IDEM#{req.CustomerId}#{idempotencyKey}";
        var requestHash = Hash(req);
        var amt = new AttributeValue { N = req.Amount.ToString(Inv) };
        var date = now.ToString("yyyy-MM-dd", Inv);

        var actions = new List<TransactWriteItem>
        {
            // 0: claim the idempotency key
            new() { Put = new Put
            {
                TableName = tableName,
                Item = new()
                {
                    ["PK"] = S(idemPk), ["SK"] = S("IDEM"),
                    ["request_hash"] = S(requestHash),
                    ["status"] = S("COMPLETED"),
                    ["response"] = S(JsonSerializer.Serialize(result)),
                    ["expires_at"] = new AttributeValue { N = now.Add(IdempotencyWindow).ToUnixTimeSeconds().ToString(Inv) }
                },
                ConditionExpression = "attribute_not_exists(PK)",
                ReturnValuesOnConditionCheckFailure = ReturnValuesOnConditionCheckFailure.ALL_OLD
            }},
            // 1: debit with balance and status guard
            new() { Update = new Update
            {
                TableName = tableName,
                Key = Key($"ACC#{req.FromAccountId}", "PROFILE"),
                UpdateExpression = "SET balance = balance - :amt, version = version + :one",
                ConditionExpression = "balance >= :amt AND #st = :open AND currency = :ccy",
                ExpressionAttributeNames = new() { ["#st"] = "status" },
                ExpressionAttributeValues = new() { [":amt"] = amt, [":one"] = new() { N = "1" }, [":open"] = S("open"), [":ccy"] = S(req.Currency) },
                ReturnValuesOnConditionCheckFailure = ReturnValuesOnConditionCheckFailure.ALL_OLD
            }},
            // 2: credit, only if the target exists and is open
            new() { Update = new Update
            {
                TableName = tableName,
                Key = Key($"ACC#{req.ToAccountId}", "PROFILE"),
                UpdateExpression = "SET balance = balance + :amt, version = version + :one",
                ConditionExpression = "attribute_exists(PK) AND #st = :open AND currency = :ccy",
                ExpressionAttributeNames = new() { ["#st"] = "status" },
                ExpressionAttributeValues = new() { [":amt"] = amt, [":one"] = new() { N = "1" }, [":open"] = S("open"), [":ccy"] = S(req.Currency) },
                ReturnValuesOnConditionCheckFailure = ReturnValuesOnConditionCheckFailure.ALL_OLD
            }},
            // 3 and 4: the two ledger lines
            new() { Put = LedgerLine(req.FromAccountId, date, transferId, "D", -req.Amount, req, now) },
            new() { Put = LedgerLine(req.ToAccountId, date, transferId, "C", req.Amount, req, now) }
        };

        try
        {
            await ddb.TransactWriteItemsAsync(new TransactWriteItemsRequest
            {
                // No ClientRequestToken set: the SDK generates one per call, so its own automatic
                // retries of this exact request are no-ops within DynamoDB's 10-minute window.
                TransactItems = actions
            }, ct);
            return (TransferOutcome.Posted, result);
        }
        catch (TransactionCanceledException ex)
        {
            var reasons = ex.CancellationReasons ?? [];
            string Code(int i) => i < reasons.Count ? reasons[i].Code ?? "None" : "None";

            if (Code(0) == "ConditionalCheckFailed")
            {
                // The key exists: compare request hashes, then replay or reject.
                var existing = reasons[0].Item ?? await ReadIdempotencyAsync(idemPk, ct);
                if (existing is null) return (TransferOutcome.Conflict, null);
                if (existing["request_hash"].S != requestHash) return (TransferOutcome.KeyReusedWithDifferentRequest, null);
                return (TransferOutcome.Replayed, JsonSerializer.Deserialize<TransferResult>(existing["response"].S));
            }
            if (Code(1) == "ConditionalCheckFailed")
            {
                var old = reasons[1].Item;
                var isOpen = old is not null && old.TryGetValue("status", out var st) && st.S == "open";
                return (isOpen ? TransferOutcome.InsufficientFunds : TransferOutcome.AccountNotOpen, null);
            }
            if (Code(2) == "ConditionalCheckFailed") return (TransferOutcome.AccountNotOpen, null);

            // "TransactionConflict": another transaction touched one of these items at the same time. Caller may retry.
            return (TransferOutcome.Conflict, null);
        }
    }

    private async Task<Dictionary<string, AttributeValue>?> ReadIdempotencyAsync(string pk, CancellationToken ct)
    {
        var resp = await ddb.GetItemAsync(new GetItemRequest { TableName = tableName, Key = Key(pk, "IDEM"), ConsistentRead = true }, ct);
        return resp.Item is { Count: > 0 } ? resp.Item : null;
    }

    private Put LedgerLine(string accountId, string date, string transferId, string side, decimal amount, TransferRequest req, DateTimeOffset now) => new()
    {
        TableName = tableName,
        Item = new()
        {
            ["PK"] = S($"ACC#{accountId}"),
            ["SK"] = S($"TX#{date}#{transferId}-{side}"),
            ["entity"] = S("transaction"),
            ["transaction_id"] = S($"{transferId}-{side}"),
            ["booking_date"] = S(date),
            ["amount"] = new AttributeValue { N = amount.ToString(Inv) },
            ["currency"] = S(req.Currency),
            ["type"] = S(side == "D" ? "transfer_out" : "transfer_in"),
            ["channel"] = S("mobile"),
            ["reference"] = S(req.Reference),
            ["status"] = S("booked"),
            ["created_at"] = S(now.ToString("O", Inv))
        },
        ConditionExpression = "attribute_not_exists(PK)"
    };

    // Canonical hash of the business content of the request (not headers or timestamps).
    private static string Hash(TransferRequest r)
    {
        var canonical = string.Join('|', r.CustomerId, r.FromAccountId, r.ToAccountId, r.Amount.ToString("0.00", Inv), r.Currency, r.Reference);
        return Convert.ToHexString(SHA256.HashData(Encoding.UTF8.GetBytes(canonical)));
    }

    private static AttributeValue S(string v) => new() { S = v };
    private static Dictionary<string, AttributeValue> Key(string pk, string sk) => new() { ["PK"] = S(pk), ["SK"] = S(sk) };
}

The demo

Program.cs
using System.Globalization;
using Amazon;
using Amazon.DynamoDBv2;
using Amazon.DynamoDBv2.Model;
using Amazon.Runtime;
using Harbourline.Payments.DynamoDb;

var local = !string.Equals(Environment.GetEnvironmentVariable("DDB_LOCAL"), "false", StringComparison.OrdinalIgnoreCase);
IAmazonDynamoDB ddb = local
    ? new AmazonDynamoDBClient(new BasicAWSCredentials("local", "local"),
        new AmazonDynamoDBConfig { ServiceURL = "http://localhost:8000", AuthenticationRegion = "us-east-1" })
    : new AmazonDynamoDBClient(RegionEndpoint.USEast1);

await SeedAccountAsync("ACC-0000000001", 500.00m);
await SeedAccountAsync("ACC-0000000002", 0.00m);

var svc = new TransferService(ddb, TimeProvider.System);
var ct = CancellationToken.None;
var req = new TransferRequest("CUS-00012345", "ACC-0000000001", "ACC-0000000002", 250.00m, "GBP", "Rent November");

// Same key three times: one Posted, two Replayed with the SAME transfer id.
for (var i = 1; i <= 3; i++)
{
    var (outcome, result) = await svc.TransferAsync(req, "7f4e2a1c-rent-nov", ct);
    Console.WriteLine($"Attempt {i}: {outcome} {result?.TransferId}");
}

// Same key, different amount: rejected.
var (reuse, _) = await svc.TransferAsync(req with { Amount = 999.00m }, "7f4e2a1c-rent-nov", ct);
Console.WriteLine($"Reused key with different body: {reuse}");

// 20 concurrent £20 transfers with distinct keys against the remaining £250: at most 12 can post.
var outcomes = await Task.WhenAll(Enumerable.Range(1, 20).Select(i =>
    svc.TransferAsync(req with { Amount = 20.00m, Reference = $"Split {i}" }, $"split-{i:D2}", ct)));
Console.WriteLine(string.Join(", ", outcomes.GroupBy(o => o.Outcome).Select(g => $"{g.Key}={g.Count()}")));

var from = await ddb.GetItemAsync(new GetItemRequest
{
    TableName = "hl-m06-banking",
    Key = new() { ["PK"] = new() { S = "ACC#ACC-0000000001" }, ["SK"] = new() { S = "PROFILE" } },
    ConsistentRead = true
});
Console.WriteLine($"Final balance ACC-0000000001: {from.Item["balance"].N}");

async Task SeedAccountAsync(string id, decimal balance) =>
    await ddb.PutItemAsync(new PutItemRequest
    {
        TableName = "hl-m06-banking",
        Item = new()
        {
            ["PK"] = new() { S = $"ACC#{id}" }, ["SK"] = new() { S = "PROFILE" },
            ["account_id"] = new() { S = id }, ["currency"] = new() { S = "GBP" },
            ["status"] = new() { S = "open" }, ["version"] = new() { N = "0" },
            ["balance"] = new() { N = balance.ToString(CultureInfo.InvariantCulture) }
        }
    });

Example output:

Attempt 1: Posted 019a3d0b-…
Attempt 2: Replayed 019a3d0b-…
Attempt 3: Replayed 019a3d0b-…
Reused key with different body: KeyReusedWithDifferentRequest
Posted=12, InsufficientFunds=6, Conflict=2
Final balance ACC-0000000001: 10

The exact split between Posted, InsufficientFunds and Conflict varies run to run: concurrent transactions on the same item can cancel each other with TransactionConflict, so fewer than 12 may post. The final balance is always 500 − 250 − 20 × (number posted) — never negative and never inconsistent. In an API you would retry Conflict a few times with backoff, using the same idempotency key.

The audit stream consumer

Harbourline.Audit.StreamHandler.csproj
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net10.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
    <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
    <GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles>
    <AWSProjectType>Lambda</AWSProjectType>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Amazon.Lambda.Core" Version="3.4.0" />
    <PackageReference Include="Amazon.Lambda.DynamoDBEvents" Version="4.0.0" />
    <PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" Version="3.0.1" />
    <PackageReference Include="AWSSDK.DynamoDBv2" Version="4.1.0.2" />
  </ItemGroup>
</Project>
Function.cs
using Amazon.DynamoDBv2;
using Amazon.DynamoDBv2.Model;
using Amazon.Lambda.Core;
using Amazon.Lambda.DynamoDBEvents;

[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]

namespace Harbourline.Audit.StreamHandler;

public sealed class Function
{
    private readonly IAmazonDynamoDB _ddb;
    private readonly string _auditTable;

    // Lambda calls the parameterless constructor once per execution environment.
    public Function() : this(new AmazonDynamoDBClient(), Environment.GetEnvironmentVariable("AUDIT_TABLE") ?? "hl-m06-audit") { }

    public Function(IAmazonDynamoDB ddb, string auditTable) { _ddb = ddb; _auditTable = auditTable; }

    public async Task<StreamsEventResponse> HandleAsync(DynamoDBEvent evt, ILambdaContext context)
    {
        var failures = new List<StreamsEventResponse.BatchItemFailure>();

        foreach (var record in evt.Records)
        {
            try
            {
                var keys = record.Dynamodb.Keys;
                var pk = keys["PK"].S;
                // Only balance-bearing account items are audited here; ledger lines are audited by the ledger itself.
                if (!pk.StartsWith("ACC#", StringComparison.Ordinal) || keys["SK"].S != "PROFILE") continue;

                var oldBalance = record.Dynamodb.OldImage?.GetValueOrDefault("balance")?.N;
                var newBalance = record.Dynamodb.NewImage?.GetValueOrDefault("balance")?.N;
                var byService = record.UserIdentity?.PrincipalId == "dynamodb.amazonaws.com"; // TTL deletes

                await _ddb.PutItemAsync(new PutItemRequest
                {
                    TableName = _auditTable,
                    Item = new()
                    {
                        // The stream sequence number is unique per shard: a natural idempotency key for the consumer.
                        ["PK"] = new() { S = $"EVT#{record.Dynamodb.SequenceNumber}" },
                        ["occurred_at"] = new() { S = DateTime.SpecifyKind(record.Dynamodb.ApproximateCreationDateTime, DateTimeKind.Utc).ToString("O") },
                        ["actor"] = new() { S = byService ? "dynamodb-ttl" : "payments-api" },
                        ["action"] = new() { S = record.EventName },
                        ["resource"] = new() { S = pk },
                        ["old_balance"] = new() { S = oldBalance ?? "-" },
                        ["new_balance"] = new() { S = newBalance ?? "-" },
                        ["outcome"] = new() { S = "recorded" },
                        ["correlation_id"] = new() { S = record.EventID }
                    },
                    ConditionExpression = "attribute_not_exists(PK)"
                });
            }
            catch (ConditionalCheckFailedException)
            {
                // Already recorded by an earlier delivery of the same record: success.
            }
            catch (Exception ex)
            {
                context.Logger.LogError($"Audit write failed for {record.EventID}: {ex.GetType().Name}");
                failures.Add(new StreamsEventResponse.BatchItemFailure { ItemIdentifier = record.Dynamodb.SequenceNumber });
                break; // stream order matters: stop at the first failure, Lambda retries from here
            }
        }

        return new StreamsEventResponse { BatchItemFailures = failures };
    }
}

Code explained line by line

  1. TimeProvider clock — .NET 8+ abstraction for time; tests can inject a fake clock to test idempotency expiry.
  2. idemPk = $"IDEM#{CustomerId}#{key}" — scopes keys per customer, so two customers who happen to generate the same key never collide, and one cannot probe another's keys.
  3. Hash(req) — hashes the business content only, formatted with the invariant culture so 250 and 250.00 hash identically.
  4. Action 0, attribute_not_exists(PK) — the idempotency claim. If it fails, the whole transaction is cancelled: no money moves.
  5. ReturnValuesOnConditionCheckFailure = ALL_OLD — when the condition fails, the cancellation reason carries the existing item, so the replay needs no extra read in most cases.
  6. Action 1, balance >= :amt AND #st = :open AND currency = :ccy — the guard: funds, account state and currency match. #st is needed because status is a reserved word.
  7. Action 2, attribute_exists(PK) on the credit side — without it, UpdateItem would happily create a new account item for a mistyped account ID.
  8. Actions 3 and 4 — the ledger lines are written inside the same transaction, so the balance and the ledger can never disagree.
  9. No explicit ClientRequestToken — the SDK fills one in per call, so if the SDK retries after a network error, DynamoDB recognises the token for 10 minutes and does not re-apply. Do not derive it from the idempotency key yourself: a client retry builds a new request (new transfer ID, new timestamp), and the same token with different parameters raises IdempotentParameterMismatchException. The 24-hour idempotency record handles client retries.
  10. catch (TransactionCanceledException ex) and CancellationReasons — one reason per action, in order. Code is None, ConditionalCheckFailed, TransactionConflict, ItemCollectionSizeLimitExceeded, ThrottlingError or ValidationError.
  11. Replay path — compare stored and current hashes. Equal → return the stored response (same transfer ID). Different → KeyReusedWithDifferentRequest, which the API maps to HTTP 422 as a ProblemDetails response.
  12. TransactionConflict → Conflict — a concurrent transaction touched the same item. Safe to retry with the same key.
  13. In the Lambda, record.Dynamodb.SequenceNumber as the audit item key plus attribute_not_exists(PK) — redelivered records are detected and skipped.
  14. UserIdentity.PrincipalId == "dynamodb.amazonaws.com" — TTL deletions appear in the stream as service deletes, distinguishable from user deletes. Useful evidence for "why did this record disappear?"
  15. failures.Add(...) then break — with ReportBatchItemFailures, Lambda retries from the first reported sequence number. Breaking preserves per-item ordering.
  16. The audit table role has only dynamodb:PutItem — audit records cannot be modified by the code that writes them.

Security considerations

⚠ Learning shortcut — not for production

The lab writes audit records to a table in the same account with RemovalPolicy.DESTROY and default encryption. In production, send audit events to a separate audit account (or an S3 bucket with Object Lock in compliance mode), encrypt with a customer managed KMS key whose policy excludes the application team, and retain according to your records policy.

  • Never trust the client's key alone. Bind it to the authenticated customer (part of the PK) and compare request hashes.
  • Rate-limit key creation per customer at the API layer so an attacker cannot fill the table with keys.
  • Responses stored in the idempotency record can contain personal data. Store the minimum needed to replay (IDs, status, amounts) and let TTL remove it. Document the retention in your records of processing.
  • Streams contain full images. With NEW_AND_OLD_IMAGES, every consumer sees every attribute. Grant dynamodb:GetRecords only to roles that need the data, or use KEYS_ONLY and re-read selectively.
  • TTL is not a GDPR erasure mechanism with a deadline. Deletion happens within a few days; for erasure requests with a deadline, delete explicitly and record that you did.
  • Do not log balances or IBANs in Lambda logs. The handler logs only the event ID and exception type.

Banking use case

Harbourline applies these patterns to the Payments Service:

Endpoint / processPattern
POST /payments, POST /transfersIdempotency record + TransactWriteItems
PATCH /accounts/{id}/overdraft-limitOptimistic locking with version
Card authorisation holdsItem with expires_at = auth expiry; TTL cleans up abandoned holds; a scheduled job releases holds explicitly at the exact time (TTL is not punctual)
Audit trailStreams → Lambda → audit table / S3 Object Lock
Open Banking consent tokensTTL as a backstop; the API checks expires_at on every read

Common mistakes

  • Using BatchWriteItem for a transfer → it looks like "one call" → batches are not atomic; use TransactWriteItems.
  • Relying on ClientRequestToken as the idempotency mechanism → it only lasts 10 minutes and only for identical calls → keep your own idempotency record.
  • Forgetting attribute_exists on the credit side → UpdateItem upserts → typo'd accounts get created with a positive balance.
  • Treating TTL as exact → items may live days past expiry → filter on expires_at in reads that care, and use explicit deletes or Step Functions/EventBridge Scheduler for time-critical actions.
  • Non-idempotent stream consumers → duplicate audit lines and double notifications → key consumer writes by sequence number or event ID.
  • No DLQ on the event source mapping → a poison record blocks the shard and records age out after 24 hours → set retries, bisect and an on-failure destination; alarm on IteratorAge.
  • Hot idempotency partition → using a constant PK like IDEM with the key in SK → put the key in the PK.

Troubleshooting

SymptomLikely causeFix
TransactionCanceledException with reason ConditionalCheckFailed at index 0Duplicate idempotency keyExpected: replay or reject
Reason TransactionConflictConcurrent transaction on the same itemRetry with backoff; reduce contention (fewer items per transaction)
ValidationException: Transaction request cannot include multiple operations on one itemSame key twice in the transaction (for example, from and to accounts equal)Validate inputs before building the transaction
IdempotentParameterMismatchExceptionSame ClientRequestToken reused with different parameters within 10 minutesLet the SDK generate the token; handle client retries with your own idempotency record
CancellationReasons is emptyOlder SDK or exception thrown before DynamoDB evaluated actionsUpgrade to SDK v4; check the exception message
Lambda IteratorAge keeps risingConsumer failing repeatedly or too slowCheck logs and the DLQ; raise batch size or parallelization factor
Items not expiringTTL disabled, attribute not a Number, or value in millisecondsStore epoch seconds as N; enable TTL on that attribute name
Expired idempotency record still blocks a retry after 24 hTTL has not deleted it yetExpected and safe; if you need exactness, check expires_at in code

Cost and cleanup

Local runs are free. In AWS:

  • Transactions consume two write units per item: the five-action transfer costs 10 write request units ≈ USD 0.00000625 on-demand. A failed transaction still consumes capacity.
  • Streams: the first 2.5 million stream read requests per month are free; GetRecords calls made by Lambda triggers are not charged; Lambda invocations are within the free tier for a lab.
  • TTL deletes are free (no write units) in the Region where they occur.
  • SQS DLQ and CloudWatch Logs: cents at lab volume; log retention set to one month.
# Local
docker rm -f hl-m06-ddb

# AWS (CDK)
cdk destroy CorrectnessStack --profile learn-sandbox
# or Terraform
terraform destroy

aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=cea-lab \
  --query "ResourceTagMappingList[].ResourceARN" --profile learn-sandbox --region us-east-1

Exercises

  • 🟢 Guided: Run the demo. Then change the debit condition to remove the currency check and send a EUR transfer from a GBP account. Observe what happens, then restore the check.
  • 🟡 Challenge: Add an IN_PROGRESS state for a long-running payment that calls an external sanctions-screening service: claim the key as IN_PROGRESS, call the service, then complete. A concurrent duplicate must get 409 Conflict while in progress, not a second screening call. Handle a crashed worker that leaves IN_PROGRESS forever.
  • 🔴 Live problem: The operations team reports that last night 214 card-authorisation holds were still visible in the app 30 hours after they should have expired. The developer says "TTL is broken". Investigate and fix.

Assignment

Package the transfer service as a reusable Harbourline.Idempotency library with an interface IIdempotencyStore, a DynamoDB implementation, and xUnit v3 tests against DynamoDB Local (Testcontainers) covering: first call, replay, key reuse with different body, concurrent duplicates (exactly one Posted), and expiry with a fake TimeProvider.

Solutions

Knowledge check

Interview questions

  • "How do you implement idempotent payment APIs?" — Client-supplied key scoped to the caller, stored with a conditional write together with a hash of the request and the response, in the same atomic operation as the money movement; replay on match, reject on mismatch, expire after a defined window.
  • "What are DynamoDB transactions' limits?" — Up to 100 actions and 4 MB, one Region and account, each item once, double capacity cost, serializable against single-item operations but not across Query/Scan as a unit, not cross-Region in global tables.
  • "How would you build an audit trail from DynamoDB?" — Streams with NEW_AND_OLD_IMAGES to an idempotent Lambda writing to a separate, append-only store with restricted permissions, DLQ and IteratorAge alarms; for long retention, S3 with Object Lock.
  • "Is TTL suitable for deleting data on a deadline?" — No; it is best-effort within days. Use explicit deletes for deadlines and TTL for cost cleanup.

Certification practice

Completion checklist

  • I can explain the difference between a conditional write, a transaction and ClientRequestToken
  • I ran the transfer demo and saw Posted, Replayed and KeyReusedWithDifferentRequest outcomes
  • I can decode CancellationReasons to a business error
  • I can describe how the stream consumer stays idempotent and ordered
  • I know why TTL is cleanup, not scheduling
  • I removed the local container and any AWS stack

Key takeaways

  • Condition expressions are DynamoDB's integrity constraints: balance guards, existence checks and optimistic locking, all atomic.
  • Idempotency = caller-scoped key + request hash + stored response, claimed in the same transaction as the money movement.
  • TransactWriteItems gives all-or-nothing across up to 100 items; read CancellationReasons in order; retry TransactionConflict.
  • Streams feed audit and projections; consumers must be idempotent, ordered and protected with retries, bisect, DLQ and IteratorAge alarms.
  • TTL deletes for free but not on time — enforce expiry in reads.

Further reading

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page