Skip to Content
Cloud Engineering Academy
14. Advanced Banking Architecture

Event-driven banking architecture

Design domain events, per-domain EventBridge buses, schema governance and safe replay for a bank, keep personal data out of events, and check event contracts and duplicate delivery with a runnable C# validator.

advanced⏱ 2 h 30 min🟢 FreeSAP-C03SAA-C03DVA-C03

Assumes: .NET 10 LTS (file-based apps) · AWS SDK for .NET v4 · Amazon.CDK.Lib 2.277.0 · Terraform AWS provider v6 · Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Distinguish a domain event, a command and an integration event, and say which belongs on a shared bus.
  • Design a bus topology (one bus per domain, a central audit bus, cross-account access) and justify it.
  • Define an event envelope and a schema-governance process with versioning and compatibility rules.
  • Keep personal data out of events, and explain how that supports erasure, retention and replay.
  • Choose between EventBridge, SNS, SQS, Kinesis Data Streams and Amazon MSK for a given requirement.
  • Make consumers safe under duplicates, reordering and replay, and run a C# envelope validator with a duplicate filter.

Prerequisites

Why this matters in banking

A bank is a network of domains: customers, accounts, payments, cards, fraud, notifications, reporting. When a payment settles, many parties care: the ledger, the customer's app, the fraud team's analytics, the data lake, the regulatory-reporting pipeline. If the payments service calls each one directly, every new consumer means changing payments, and every consumer outage slows payments down.

Events invert that. Payments publishes "payment settled" once. Interested domains subscribe. But events also create new risks that a bank must design out:

  • A careless event carries a customer's email address, and now personal data is copied into six systems and an archive that cannot be erased.
  • Producers change a field name and break a consumer that nobody knew existed.
  • A replay re-sends 10,000 old events and customers receive 10,000 duplicate texts.
  • Two events arrive in the wrong order and an account appears closed before it was opened.

An event-driven architecture in banking is therefore as much about contracts and governance as it is about services.

Educational content, not legal advice

How personal data in events interacts with erasure rights, retention duties and record-keeping obligations is a legal and data-protection question ⚠ VERIFY with your data-protection officer and legal team. This lesson shows engineering patterns that reduce the problem; it does not claim that any design satisfies UK GDPR or any other regime.

The idea in plain language

An event is a fact about something that already happened, named in the past tense: payment.settled, account.frozen. It is immutable. A command is a request: settle this payment. Commands go to one owner; events go to anyone who is interested.

KindExampleAudienceWhere it travels
CommandReserveFundsOne serviceDirect call, queue
Domain eventPaymentSettledInside the owning domainInternal bus or stream
Integration eventpayments.payment.settled v1Other domainsShared bus, with a published schema

Analogy: a branch notice board versus a phone call. A command is a phone call to a named person. An event is a dated notice pinned to the board: "Account ACC-... was frozen at 10:02". Anyone may read it, nobody can edit it, and the board has rules about what may be written on a public notice (never a customer's home address).

Notification events versus state-carrying events

StylePayloadProsCons
Notification (thin)Identifiers and type, such as paymentId, customerIdSmall, little personal data, always current when the consumer looks upConsumer must call back, so the producer sees extra load
Event-carried state (fat)Many fieldsConsumers independent of the producerMore copied data, harder to retire fields, higher privacy risk

Banking default: thin events plus a small set of non-personal facts (amount, currency, status, timestamps). Consumers that need more fetch it through an authorised API.

What problem the AWS services solve

NeedServiceHonest limits
Route events by content to many targetsAmazon EventBridge event bus and rulesAt-least-once delivery, no ordering guarantee; payload size limits (about 256 KB per entry historically; some limits were raised recently ⚠ VERIFY)
Cross-account and cross-Region routingEventBridge bus targets and bus resource policiesPolicy design matters; one misconfigured policy opens a bus
Contract definitionEventBridge Schema Registry (JSON Schema or OpenAPI)A registry stores schemas; it does not enforce them at publish time unless you add a check
ReplayEventBridge archives and replaysReplays re-trigger every matching rule, including side-effect consumers
Fan-out to queues and topicsSQS, SNSSNS adds pub/sub with filtering; SQS gives a durable buffer per consumer
Ordered, high-throughput streamsKinesis Data Streams, Amazon MSKShard or partition management; ordering within a shard or partition only
Point-to-point source to targetEventBridge PipesGood for filtering and enrichment between two services

Choosing the transport

RequirementLikely choice
Many independent consumers, content routing, SaaS and AWS integrationsEventBridge bus
One consumer, work queue, back-pressure, retriesSQS
Simple topic fan-out with filtering and mobile or email deliverySNS
Ordered per key, replay from an offset, very high volumeKinesis Data Streams or MSK
Kafka-compatible tooling already in useAmazon MSK

Many banks use EventBridge for business events and Kinesis or MSK for high-volume streams such as card authorisation telemetry (real-time processing).

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
Azure Event GridEventBridgeBoth route events by filters; EventBridge adds archives, replay and a schema registry
Azure Service Bus topics and subscriptionsSNS with SQS subscriptionsPub/sub with durable per-consumer queues
Azure Event HubsKinesis Data Streams, MSKPartitioned, ordered per partition, replay by offset
Azure Schema RegistryEventBridge Schema Registry, AWS Glue Schema RegistryGlue Schema Registry targets Kafka/Kinesis serialisation formats
MassTransit or NServiceBus message contractsShared contract library plus schema registrySame discipline; AWS gives you a hosted catalogue
CloudEvents specSame spec idea; EventBridge has its own envelopeMap between them deliberately

Architecture

Security-boundary view

A producer can publish only to its own bus and only with its own source. Consumers get only the events their rules select, delivered into queues they own.

The 15-section design

Sections 1 to 3: requirements

SectionContent
1 BusinessLet domains react to each other without direct coupling; give the data and reporting teams one governed feed; make onboarding a new consumer a matter of days, not a change request to the producer
2 FunctionalF1 Publish integration events from each domain reliably (outbox). F2 Route by type to consumers. F3 Version and publish schemas. F4 Archive and replay for recovery. F5 Reject events that break the contract in CI. F6 Provide an audit copy of all events
3 Non-functionalEnd-to-end p99 delivery under 2 seconds for business events. 99.95% publish availability. At-least-once delivery with consumer idempotency. Throughput 500 events per second sustained, 3,000 burst (⚠ check quotas). Archive 30 days. Audit copy retained 7 years ⚠ VERIFY. No personal data in event payloads

Section 5: service selection

ChoiceWhyRejected alternativeTrade-off
One EventBridge bus per domainIsolated policies, clear ownership, per-domain quotas and archivesOne bus for the bankMore buses to manage; far less blast radius
Central audit bus in a log-archive accountOne tamper-resistant copyEach domain keeps its own history onlyAnother hop and bus to run
SQS per consumer behind rulesBuffering, DLQs, back-pressureLambda targets directlySlightly more resources; much better failure behaviour
Outbox for publishingAtomic with the business transactionPublish after commit in codeRelay component to run (outbox pattern)
Kinesis for high-volume telemetryOrdered per key, replay by positionEventBridge for everythingCapacity planning, separate from the business bus

Sections 6 and 7: network and security

  • Network. EventBridge, SQS and SNS are regional service endpoints. Private subnets reach them through interface VPC endpoints (VPC endpoints).
  • Bus policies. Allow events:PutEvents only for principals in your organisation (aws:PrincipalOrgID) and, where possible, restrict the source field with the events:source condition key ⚠ VERIFY the key and syntax.
  • Encryption. Customer-managed KMS keys for buses and archives where supported ⚠ VERIFY; SQS with KMS or SQS-managed encryption.
  • No personal data in events. Use customerId, accountId and paymentId as opaque IDs; amounts and currencies are fine. A consumer needing a name or address calls an authorised API.
  • Schema rules in CI. Block a pull request if an event schema change is breaking.
  • Audit. CloudTrail records bus management calls; the audit bus keeps the event stream itself (audit and compliance architecture).

Sections 8 to 10: API, data and flow

Envelope. Every integration event uses the same envelope:

FieldMeaning
eventIdGlobally unique (UUID v7); the deduplication key
typedomain.entity.event, for example payments.payment.settled
schemaVersionInteger; consumers declare which versions they handle
sourceProducing domain, for example harbourline.payments
occurredAtWhen the fact happened (UTC)
correlationIdTies the event to the originating request
dataNon-personal facts and opaque IDs

Schema governance.

ChangeAllowed without a new version?
Add an optional fieldYes (backward compatible)
Add a new enum valueOnly if consumers are told to tolerate unknown values; otherwise treat as breaking
Remove or rename a fieldNo: new schemaVersion; run old and new in parallel until consumers move
Change a field's type or meaningNo: new version
Add a required fieldNo: new version

Flow. (1) Service commits state and an outbox row. (2) Relay publishes to the domain bus. (3) Rules deliver to consumer queues. (4) Consumers validate, deduplicate by eventId, process, and delete the message. (5) The archive and audit bus keep copies.

Section 11: failure scenarios

FailureDetectionAutomatic responseImpactManual step
Duplicate deliveryeventId already processedSkipNoneNone
Out-of-order eventsoccurredAt or version older than stored stateIgnore stale or hold brieflyNoneNone
Consumer downQueue age alarmEvents buffer in SQSDelayed reactionFix consumer
Poison eventRepeated failureMove to DLQOne event unprocessedFix and redrive
Breaking schema changeCI contract test failsBlock mergeNone in productionVersion the schema
Replay causes side effectsReplay plannedConsumers are idempotent; side-effect consumers filter on replay-name or a flagAvoidedPlan replay with owners
Rule target throttled or unreachableFailed invocation metricRetries per target retry policy, then target DLQDelayInvestigate
Wrong bus policyAccess Analyzer findingNone; detectionPossible exposureFix policy

Sections 12 to 15

  • Scalability. Add consumers without touching producers. Watch EventBridge PutEvents and invocation quotas in your Region ⚠ VERIFY, and use Kinesis or MSK for sustained very high rates.
  • Cost. EventBridge bills per event published and per event replayed or archived ⚠ VERIFY; SQS per request; archive storage per GB. Estimate events per day by type, number of rules and targets. Thin events lower cost as well as risk.
  • Implementation. Phase 1 envelope and schema repo. Phase 2 one domain bus with outbox relay. Phase 3 two consumers behind SQS. Phase 4 archive, replay rehearsal, audit bus. Phase 5 contract tests in every producer's pipeline.
  • Compliance touchpoints.
AreaWhere it touchesReviewing teamEvidence
Data protection and erasureWhat fields events may carry; archive retentionData protection officerField catalogue, archive policy ⚠ VERIFY
Record keepingAudit bus retentionCompliance, legalRetention schedule
Operational resilienceDependency of important services on the busOperational resilience leadScenario tests
Third-party riskAny SaaS event sources or targetsThird-party riskAssessment
Access governanceBus policies, who may publishSecurityPolicy review

Step-by-step implementation

Define the envelope and the forbidden-fields list

Agree the seven envelope fields and a list of banned payload names (email, phone, names, dates of birth, IBAN). Put both in a shared repository that producers and consumers reference.

Create one bus per domain with an archive

Name them hl-payments, hl-accounts and so on. Enable an archive with a retention you can justify. Set the bus policy for your organisation.

Publish through an outbox

Write the event to an outbox table in the same transaction as the state change. A relay reads and calls PutEvents, and marks rows sent. Never publish directly after commit and hope.

Put an SQS queue between each rule and its consumer

Give every queue a DLQ and an age alarm. The consumer validates the envelope, checks eventId against a dedupe table, processes, then deletes the message.

Add schema checks to CI

Validate sample events against the registry schemas and fail on breaking changes. Keep the schema repository the source of truth, and publish to the registry from the pipeline.

Rehearse a replay

Replay one hour of archive events into a test consumer and confirm that idempotent consumers do nothing new and side-effect consumers are protected.

AWS Console instructions

  • Console → Amazon EventBridge → Buses → Event buses → Create event bus. Name it, set the resource policy, optionally enable Archive.
  • EventBridge → Rules → Create rule on that bus. Use an event pattern with source and detail-type, choose an SQS target, and configure Dead-letter queue and Retry policy under additional settings.
  • EventBridge → Schema registry → Registries / Discovered schemas to browse schemas, view versions and download code bindings where available.
  • EventBridge → Archives and Replays to start a replay for a time range.

AWS CLI

# Create a domain bus.
aws events create-event-bus --name hl-m14-payments \
  --tags Key=Project,Value=cea-lab Key=Lab,Value=m14-l14 Key=Environment,Value=learn Key=Owner,Value=learner \
  --profile learn-sandbox --region us-east-1

# Archive everything from the payments source for 30 days.
aws events create-archive --archive-name hl-m14-payments-archive \
  --event-source-arn arn:aws:events:us-east-1:111122223333:event-bus/hl-m14-payments \
  --retention-days 30 \
  --event-pattern '{"source":["harbourline.payments"]}' \
  --profile learn-sandbox --region us-east-1

# Publish one event (note: Detail is a JSON string; no personal data).
aws events put-events --entries '[{
  "EventBusName":"hl-m14-payments",
  "Source":"harbourline.payments",
  "DetailType":"payments.payment.settled",
  "Detail":"{\"eventId\":\"0198a1b2-0000-7000-8000-000000000001\",\"schemaVersion\":1,\"correlationId\":\"corr-123\",\"data\":{\"paymentId\":\"PAY-0198a1b2\",\"amount\":\"25.00\",\"currency\":\"GBP\"}}"
}]' --profile learn-sandbox --region us-east-1

# Replay a time range from the archive back to the bus.
aws events start-replay --replay-name hl-m14-replay-1 \
  --event-source-arn arn:aws:events:us-east-1:111122223333:archive/hl-m14-payments-archive \
  --destination Arn=arn:aws:events:us-east-1:111122223333:event-bus/hl-m14-payments \
  --event-start-time 2026-11-18T09:00:00Z --event-end-time 2026-11-18T10:00:00Z \
  --profile learn-sandbox --region us-east-1

Account IDs are fictional. In a real event, eventId, type, source and occurredAt sit in the envelope your producers add; EventBridge itself supplies source, detail-type and time, so decide which fields you duplicate in detail ⚠ VERIFY your convention.

Infrastructure as Code

DomainEventBusStack.cs
using Amazon.CDK;
using Amazon.CDK.AWS.EventSchemas;
using Amazon.CDK.AWS.Events;
using Amazon.CDK.AWS.Events.Targets;
using Amazon.CDK.AWS.SQS;
using Constructs;
using EventBus = Amazon.CDK.AWS.Events.EventBus;
using EventBusProps = Amazon.CDK.AWS.Events.EventBusProps;

namespace Harbourline.Architecture.Events;

public class DomainEventBusStack : Stack
{
    public DomainEventBusStack(Construct scope, string id, IStackProps? props = null) : base(scope, id, props)
    {
        var bus = new EventBus(this, "PaymentsBus", new EventBusProps { EventBusName = "hl-m14-payments" });

        bus.Archive("PaymentsArchive", new BaseArchiveProps
        {
            ArchiveName = "hl-m14-payments-archive",
            Description = "30-day archive for replay",
            EventPattern = new EventPattern { Source = ["harbourline.payments"] },
            Retention = Duration.Days(30),
        });

        var dlq = new Queue(this, "LedgerConsumerDlq", new QueueProps
        {
            Encryption = QueueEncryption.SQS_MANAGED,
            EnforceSSL = true,
            RetentionPeriod = Duration.Days(14),
        });
        var ledgerQueue = new Queue(this, "LedgerConsumerQueue", new QueueProps
        {
            Encryption = QueueEncryption.SQS_MANAGED,
            EnforceSSL = true,
            VisibilityTimeout = Duration.Seconds(60),
            DeadLetterQueue = new DeadLetterQueue { MaxReceiveCount = 5, Queue = dlq },
        });
        var targetDlq = new Queue(this, "RuleTargetDlq", new QueueProps
        {
            Encryption = QueueEncryption.SQS_MANAGED,
            EnforceSSL = true,
            RetentionPeriod = Duration.Days(14),
        });

        new Rule(this, "PaymentSettledToLedger", new RuleProps
        {
            EventBus = bus,
            EventPattern = new EventPattern
            {
                Source = ["harbourline.payments"],
                DetailType = ["payments.payment.settled"],
            },
            Targets =
            [
                new SqsQueue(ledgerQueue, new SqsQueueProps
                {
                    DeadLetterQueue = targetDlq,
                    RetryAttempts = 10,
                    MaxEventAge = Duration.Hours(2),
                }),
            ],
        });

        // Contract: the schema registry holds the published shape of the event.
        var registry = new CfnRegistry(this, "PaymentsRegistry", new CfnRegistryProps
        {
            RegistryName = "hl-m14-payments",
            Description = "Integration event contracts for the payments domain",
        });
        new CfnSchema(this, "PaymentSettledV1", new CfnSchemaProps
        {
            RegistryName = registry.RegistryName!,
            SchemaName = "payments.payment.settled",
            Type = "JSONSchemaDraft4",
            Content = """
            {
              "$schema": "http://json-schema.org/draft-04/schema#",
              "type": "object",
              "required": ["eventId", "schemaVersion", "correlationId", "data"],
              "properties": {
                "eventId": { "type": "string" },
                "schemaVersion": { "type": "integer", "enum": [1] },
                "correlationId": { "type": "string" },
                "data": {
                  "type": "object",
                  "required": ["paymentId", "amount", "currency"],
                  "properties": {
                    "paymentId": { "type": "string" },
                    "amount": { "type": "string" },
                    "currency": { "type": "string" }
                  },
                  "additionalProperties": false
                }
              }
            }
            """,
        });

        Amazon.CDK.Tags.Of(this).Add("Project", "cea-lab");
        Amazon.CDK.Tags.Of(this).Add("Lab", "m14-l14");
        Amazon.CDK.Tags.Of(this).Add("Environment", "learn");
        Amazon.CDK.Tags.Of(this).Add("Owner", "learner");
    }
}

The two using aliases resolve a name clash: EventBus exists in both Amazon.CDK.AWS.Events and Amazon.CDK.AWS.Events.Targets. additionalProperties: false on data is how the contract keeps surprise fields (such as an email) out. The schema registry itself does not reject publishes; enforce the schema in CI and in consumers.

Complete code

The validator checks the producer contract (required fields, naming pattern, supported version, no personal-data fields) and the consumer discipline (skip duplicates). Run it before relying on any consumer to be correct.

EventEnvelope.cs
// Run with: dotnet run EventEnvelope.cs
// A producer-side contract check and a consumer-side duplicate filter for domain events.
using System.Text.Json;
using System.Text.RegularExpressions;

var validator = new EnvelopeValidator(supportedVersions: [1, 2]);
var dedupe = new Deduplicator();

string good = """
{"eventId":"0198a1b2-0000-7000-8000-000000000001","type":"payments.payment.settled","schemaVersion":1,
 "source":"harbourline.payments","occurredAt":"2026-11-18T10:00:00Z","correlationId":"corr-123",
 "data":{"paymentId":"PAY-0198a1b2","customerId":"CUS-00012345","amount":"25.00","currency":"GBP"}}
""";
string leaksPii = good.Replace("\"customerId\":\"CUS-00012345\"", "\"email\":\"jo@example.com\"")
                      .Replace("0000000001", "0000000002");
string futureVersion = good.Replace("\"schemaVersion\":1", "\"schemaVersion\":9").Replace("0000000001", "0000000003");

foreach (var (label, json) in new[] { ("good", good), ("leaks PII", leaksPii), ("future version", futureVersion), ("good again", good) })
{
    var problems = validator.Validate(json, out var id);
    var verdict = problems.Count > 0 ? "REJECT " + string.Join("; ", problems)
                : dedupe.FirstTime(id!) ? "PROCESS" : "DUPLICATE (skip)";
    Console.WriteLine($"{label,-15} -> {verdict}");
}
return 0;

public sealed class Deduplicator
{
    // Production: DynamoDB conditional put on eventId with a TTL longer than the maximum redelivery window.
    private readonly HashSet<string> _seen = [];
    public bool FirstTime(string eventId) => _seen.Add(eventId);
}

public sealed partial class EnvelopeValidator(int[] supportedVersions)
{
    private static readonly string[] Required =
        ["eventId", "type", "schemaVersion", "source", "occurredAt", "correlationId", "data"];

    private static readonly HashSet<string> ForbiddenInPayload = new(StringComparer.OrdinalIgnoreCase)
        { "email", "phone", "first_name", "firstName", "last_name", "lastName", "date_of_birth", "dateOfBirth", "iban", "pan" };

    [GeneratedRegex(@"^[a-z]+(\.[a-z]+){2,}$")]
    private static partial Regex TypePattern();

    public List<string> Validate(string json, out string? eventId)
    {
        var problems = new List<string>();
        eventId = null;
        if (json.Length > 200_000) { problems.Add("envelope too large: store a reference, not the payload"); return problems; }

        using var doc = JsonDocument.Parse(json);
        var root = doc.RootElement;
        foreach (var name in Required)
            if (!root.TryGetProperty(name, out _)) problems.Add($"missing {name}");
        if (problems.Count > 0) return problems;

        eventId = root.GetProperty("eventId").GetString();
        if (!TypePattern().IsMatch(root.GetProperty("type").GetString() ?? ""))
            problems.Add("type must look like domain.entity.event");
        if (!supportedVersions.Contains(root.GetProperty("schemaVersion").GetInt32()))
            problems.Add("unsupported schemaVersion");
        if (root.GetProperty("source").GetString()?.StartsWith("harbourline.", StringComparison.Ordinal) != true)
            problems.Add("source must start with harbourline.");
        foreach (var bad in FindForbidden(root.GetProperty("data")))
            problems.Add($"personal-data field '{bad}' must not travel in events (use customerId)");
        return problems;
    }

    private static IEnumerable<string> FindForbidden(JsonElement element)
    {
        if (element.ValueKind == JsonValueKind.Object)
            foreach (var p in element.EnumerateObject())
            {
                if (ForbiddenInPayload.Contains(p.Name)) yield return p.Name;
                foreach (var inner in FindForbidden(p.Value)) yield return inner;
            }
        else if (element.ValueKind == JsonValueKind.Array)
            foreach (var item in element.EnumerateArray())
                foreach (var inner in FindForbidden(item)) yield return inner;
    }
}

Expected output:

good            -> PROCESS
leaks PII       -> REJECT personal-data field 'email' must not travel in events (use customerId)
future version  -> REJECT unsupported schemaVersion
good again      -> DUPLICATE (skip)

Code explained line by line

  1. The sample events are C# raw string literals ("""..."""), so JSON needs no escaping. Replace creates the two bad variants; the digit replacement gives each a fresh eventId so duplicate detection does not hide the contract errors.
  2. EnvelopeValidator(int[] supportedVersions) takes the versions the consumer can handle. A consumer that has not been upgraded for version 3 rejects it explicitly instead of misreading it.
  3. Required lists the seven envelope fields. Missing fields are reported all at once, and the method returns early because later checks assume they exist.
  4. The [GeneratedRegex] attribute compiles the pattern at build time. ^[a-z]+(\.[a-z]+){2,}$ demands at least three lower-case dot-separated words, such as payments.payment.settled.
  5. ForbiddenInPayload is a case-insensitive set of names that signal personal data. FindForbidden walks the JSON recursively with yield return, so nested objects and arrays are covered.
  6. Deduplicator.FirstTime relies on HashSet.Add returning false for an existing item. In production this is a DynamoDB conditional write with a TTL that outlasts the longest redelivery or replay window.
  7. The final loop shows the order of decisions: contract first (reject), then duplicates (skip), then processing. Rejected events go to the DLQ for a human; duplicates are silently dropped.
  8. The size check (200,000 characters) guards the parser and encodes the principle that events carry references, not documents.

Security considerations

⚠ Learning shortcut — not for production

The lab uses SQS-managed encryption, an open-to-account bus policy and a 30-day archive. Production uses customer-managed keys where supported, organisation-scoped bus policies with source conditions, an archive retention agreed with data protection and legal, and a schema-governance process owned by a named team. Never add personal data to events "just for now": the archive and every consumer's copy will outlive the shortcut.

  • Least privilege. Producers may put events only to their own bus; consumers read only their own queue.
  • Do not trust event content. An event is input. Validate it, even from an internal producer.
  • Protect the archive. It is a copy of everything on the bus. Encrypt it, restrict replays, and log who starts one.
  • Guard replays. A replay is a privileged operation: require approval and a runbook.
  • Detect exposure. Use IAM Access Analyzer to find buses or queues accessible from outside the organisation (Access Analyzer).

Banking use case

Harbourline's payments domain publishes payments.payment.settled v1 with paymentId, customerId, amount and currency. The ledger consumer posts the entry. The notifications consumer looks up the customer's contact preferences through an API and sends an SMS. The data lake consumer writes the event to Parquet for reporting. When the fraud team asks for a new feed, they add a rule and a queue in their own account without changing payments. A month later payments wants to add scheme: it is an optional field, so it ships in v1. When payments needs to split amount into amount and fee, it publishes v2 alongside v1 for 90 days, tracks which consumers still read v1, and then retires it.

Common mistakes

  • Treating events as an API with no owner. Every schema needs a named owning team and a deprecation policy.
  • Putting personal data in the payload. It spreads into archives, queues and lakes beyond erasure's reach.
  • Assuming ordering. EventBridge does not guarantee order; design consumers around versions or timestamps, or use Kinesis for ordered streams.
  • No consumer idempotency. At-least-once delivery means duplicates are normal.
  • Replaying into side-effect consumers. Customers get old notifications again.
  • One shared bus for the whole bank. One policy mistake exposes everything.

Troubleshooting

SymptomLikely causeFix
Rule matches nothingPattern case or field mismatch (detail-type vs DetailType)Test the pattern with the console test tool
Events arrive but SQS is emptyQueue policy missing for events.amazonaws.comAdd the policy with aws:SourceArn condition
PutEvents returns FailedEntryCount greater than zeroEntry too large, bad bus name or throttlingInspect each entry's ErrorCode; reduce size
Replay does nothingTime range outside archive or pattern filter excludes eventsCheck archive state and range
Consumer processes the same event twiceNo dedupe store, or TTL shorter than redelivery windowAdd conditional-put dedupe with sufficient TTL
CI contract test fails after a producer changeBreaking schema changeAdd an optional field instead, or publish a new version

Cost and cleanup

The C# validator is free. The CDK or Terraform stack creates an event bus, an archive, three queues and a registry: effectively free at lab volumes (a few events). Archives bill by stored GB and replays by event ⚠ VERIFY; keep the lab to a handful of events and delete everything afterwards.

# Delete the replay and archive before the bus, then the bus and queues, or use the IaC destroy.
aws events delete-archive --archive-name hl-m14-payments-archive --profile learn-sandbox --region us-east-1
cdk destroy --profile learn-sandbox
terraform destroy

# Confirm nothing tagged for the course remains.
aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=cea-lab \
  --profile learn-sandbox --region us-east-1

Exercises

  • 🟢 Guided: run the validator and add a fifth event with type set to PaymentSettled. Predict the verdict.
  • 🟡 Challenge: write a compatibility checker that takes two JSON Schemas for payments.payment.settled (v1 and v1.1) and reports whether the change is backward compatible according to the table in this lesson.
  • 🔴 Live problem: after an incident, operations replays six hours of the payments archive. Three hundred customers receive duplicate "payment sent" texts. Explain why it happened, what the notifications consumer should have done, and the controls you would add around replay.

Assignment

Add architecture/event-backbone/ to your portfolio: the 15-section document, an event catalogue for three domains with owners and schema versions, the envelope validator with unit tests, and an ADR "one bus per domain with an outbox and SQS per consumer".

Solutions

Knowledge check

Interview questions

  • How do you stop an event system becoming a tangle? One bus per domain, published schemas with owners and versions, contract tests in CI, and a catalogue of who consumes what.
  • How do you handle schema evolution? Optional additions are compatible; anything else is a new version run in parallel until consumers move.
  • How do you make a consumer safe? Validate, deduplicate by eventId, tolerate reordering, and send failures to a DLQ.
  • EventBridge or Kinesis? EventBridge for routed business events; Kinesis or MSK for ordered, high-volume streams with offset replay.
  • How do you handle replay safely? Idempotent consumers, flagged replays, approval and a runbook, and a test replay into a non-production consumer.

Certification practice

Completion checklist

  • I can explain event versus command and thin versus fat events
  • I ran the validator and understand each verdict
  • I wrote the schema compatibility rules for my domain
  • My design has a replay procedure that protects side-effect consumers
  • I ran cleanup and confirmed nothing tagged Project=cea-lab remains

Key takeaways

  • Integration events are public contracts: versioned, owned and free of personal data.
  • One bus per domain limits blast radius; an SQS queue per consumer provides buffering and DLQs.
  • Delivery is at-least-once with no ordering guarantee, so every consumer is idempotent.
  • Replay is powerful and dangerous; plan it, flag it and approve it.
  • Erasure, retention and audit questions need data-protection and legal review.

Further reading

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page