Skip to Content
Cloud Engineering Academy
8. Messaging, Events & Microservices

Asynchronous communication and event-driven architecture

Tell queues, topics, event buses, streams and workflows apart, decide which AWS service fits a Harbourline integration with a written decision tree, and publish your first well-formed domain event from .NET 10.

intermediateโฑ 2 h ๐ŸŸข FreeDVA-C03SAA-C03DEA-C01

Assumes: .NET 10 LTS ยท AWS SDK for .NET v4 (AWSSDK.EventBridge 4.1.0.2, AWSSDK.SQS 4.1.0.2) ยท AWS CLI v2 ยท Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Explain the difference between synchronous and asynchronous communication, and name the three kinds of coupling (time, location, format) that asynchronous messaging removes.
  • Tell apart a command, an event and a document message, and decide which one a given Harbourline interaction should be.
  • Describe the five shapes of asynchronous integration on AWS โ€” queue, topic, event bus, stream and workflow โ€” and map each to its AWS service (SQS, SNS, EventBridge, Kinesis/MSK, Step Functions), plus Amazon MQ for legacy protocols.
  • Walk the "which service when?" decision tree for a new integration and defend the answer in a design review.
  • Design a domain event envelope that keeps personal data out of the payload, and publish it to an EventBridge bus from .NET 10 with the AWS SDK v4.

Prerequisites

  • Module 3: .NET on AWS โ€” Lambda, Worker Services, registering SDK clients with AddAWSService<T>()
  • Module 6: Databases โ€” transactions and consistency vocabulary
  • Module 8 overview โ€” the module map and the notification-system project you are working towards
  • Tools: .NET 10 SDK, AWS CLI v2 with the learn-sandbox SSO profile (only for the optional AWS steps)

Why this matters in banking

It is the last Friday of the month at Harbourline Bank. Salaries land between 00:30 and 02:00, and the mobile app sends a push notification for every credit. Today the Payments Service calls the Notifications Service over HTTP for each posted transaction, waits for a 200, and only then marks the posting as complete.

At 01:10 the SMS provider behind the Notifications Service slows down. Calls that took 80 ms now take eight seconds. Payments threads pile up waiting for replies, the connection pool runs dry, and the posting engine โ€” the part that actually moves money โ€” starts timing out. A cosmetic feature (a "you've been paid" text) has taken down a critical one (crediting salaries). The incident review records one root cause: temporal coupling. Payments could only finish its work if Notifications was healthy at the same moment.

Asynchronous messaging is how you break that dependency. Payments records the posting, emits a TransactionPosted event and moves on. Notifications reads the event when it can. If the SMS provider is down for an hour, events wait safely in a queue and drain afterwards. Nobody's salary is late.

But asynchronous design brings its own failure modes, and they matter more in a bank than almost anywhere else: a message delivered twice can mean a double debit, a message processed out of order can show a customer a balance that never existed, and a malformed message can block a whole queue. This module teaches the patterns that make asynchronous systems safe. This first lesson gives you the vocabulary and the decision framework for everything that follows.

The idea in plain language

Synchronous communication is a phone call. You dial, the other person must pick up right now, and you wait on the line until they answer. If they are busy, you are stuck.

Asynchronous communication is a letter or a voicemail. You leave the message somewhere durable and get on with your day. The recipient deals with it when they can. You do not know exactly when, and you may not get a reply at all โ€” but you are not blocked.

In software, "somewhere durable" is a piece of middleware: a queue, a topic, an event bus or a stream. The sender (producer) hands the message to the middleware, which stores it redundantly and delivers it to one or more receivers (consumers).

Asynchronous messaging removes three kinds of coupling โ€” dependencies that make two systems fail together:

CouplingMeaningHow messaging removes it
TimeBoth systems must be up at the same momentThe message waits in durable storage until the consumer is ready
LocationThe sender must know the receiver's addressThe sender only knows the queue, topic or bus; consumers can move, scale or be replaced
RateThe sender can only go as fast as the receiverThe queue absorbs bursts; consumers process at their own pace (load levelling)

What messaging does not remove is format coupling: both sides must still agree on what a message looks like. That is why event schemas and versioning matter (you meet the EventBridge schema registry in the EventBridge lesson).

Analogy: Harbourline's branch post room. Tellers drop internal mail into labelled trays (queues). A circular for "all branch managers" goes on the noticeboard (a topic) and every manager reads their own copy. The head-office sorting clerk reads every envelope and forwards it by rules โ€” "anything about mortgages goes to Leeds" โ€” (an event bus). The general ledger is a bound book where every posting is written in order and auditors can reread any page (a stream). And a mortgage application follows a checklist that a case officer walks step by step, chasing each department (a workflow).

Commands, events and documents

Not every message means the same thing. Getting the type right shapes the whole design.

KindMeaningTenseWho decides what happens?Harbourline example
Command"Please do this"ImperativeThe sender expects a specific receiver to actSendSmsNotification, SubmitPaymentToScheme
Event"This happened"PastEach subscriber decides for itself; the sender does not care who listensTransactionPosted, PaymentSettled, CardFrozen
Document"Here is some data"NeutralThe receiver decidesA daily exchange-rate file; a statement batch

A command usually has one intended handler, so it belongs on a queue. An event can have many listeners โ€” Notifications, Fraud, the Data Platform โ€” so it belongs on a topic, bus or stream. A common mistake is to publish "events" that are really commands in disguise (SendWelcomeEmailEvent), which couples the producer back to a specific consumer.

Delivery semantics in one paragraph

Every AWS messaging service in this module delivers at least once: under failure, retries or network glitches, a consumer can see the same message twice. Some services (SQS FIFO, SNS FIFO) deduplicate within a window, and some (SQS FIFO, Kinesis per shard, MSK per partition) preserve order within a key. None of them can make your side effects happen exactly once โ€” that is your job, using idempotent consumers. You build those properly in the idempotency lesson (see the module overview, lesson 11) and in the module project.

What problem these AWS services solve

AWS offers several managed services because the five integration shapes behave very differently. Here is each one in plain words, with its honest limitations.

ShapeAWS serviceWhat it doesDeliveryOrderingRetention / replaySharp edges
Queue (point-to-point, pull)Amazon SQSStores messages until one consumer receives and deletes eachAt least once (standard); deduplicated within 5 minutes (FIFO)None (standard); per message group (FIFO)60 s to 14 days, default 4 days; no replay after deleteConsumers must delete; visibility timeout must exceed processing time
Topic (pub/sub, push)Amazon SNSCopies each message to every subscriber (SQS, Lambda, HTTPS, email, SMS)At least once, with delivery retriesNone (standard); per group (FIFO topics)No storage โ€” if a subscriber is down after retries, the message goes to a subscription DLQ or is lostSubscribers must be reachable; use SNS โ†’ SQS for durability
Event bus (content-based routing, push)Amazon EventBridgeMatches events against rules and sends to up to 5 targets per ruleAt least once; retries for up to 24 hours by defaultNoneOptional archive and replayHigher latency than SNS (typically around half a second) โš  VERIFY; no ordering
Stream (append-only log, pull)Kinesis Data Streams, Amazon MSK (Kafka)Stores an ordered log per shard/partition; many consumers read at their own positionAt least oncePer shard / partition keyKinesis: 24 hours default, up to 365 days; MSK: configurableYou pay per shard-hour or broker-hour even when idle (except on-demand/serverless options)
Workflow (orchestration)AWS Step FunctionsRuns a state machine that calls services in order, with retries, branches and compensationStandard: exactly-once step execution; Express: at least onceDefined by the state machineStandard runs up to 1 year; Express up to 5 minutesNot a transport โ€” it coordinates calls; state size limits
Legacy brokerAmazon MQ (ActiveMQ, RabbitMQ)Managed broker speaking JMS, AMQP, MQTT, STOMP, OpenWireBroker semanticsPer queueBroker storageYou pay per broker-hour; you still size and patch-window a broker

Two things in this table deserve emphasis for a bank:

  1. Only streams and EventBridge archives let you replay history. SQS deletes messages once processed. If the Data Platform team asks "send me every TransactionPosted from last Tuesday again", a queue cannot help; a Kinesis stream (within retention) or an EventBridge archive can.
  2. Only FIFO queues/topics, Kinesis and MSK preserve order โ€” and only per key. Global ordering across a whole bank does not exist at scale. You will design for "ordered per account" instead (lesson 10 in the overview).

Payload sizes in 2026

Payload limits rose recently, and older blog posts and practice exams still quote 256 KB everywhere.

ServiceMaximum message / event size (checked 2026-11-18)
SQS1 MiB (1,048,576 bytes), billed in 64 KB chunks
EventBridge PutEvents1 MB per request (raised from 256 KB in January 2026)
Lambda asynchronous invocation1 MB (raised from 256 KB in October 2025)
SNS256 KB โš  VERIFY โ€” check the SNS quotas page; a 1 MiB option has been reported but not confirmed in the documentation at time of writing

Bigger is not better. A notification event should carry identifiers, not documents. If you need a large payload, store it in S3 and send a pointer (the "claim check" pattern).

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
Synchronous HttpClient call between servicesAPI Gateway / ALB โ†’ serviceSame idea; still couples time and rate
MSMQ private queueAmazon SQS standard queueNo server; unlimited backlog; pull over HTTPS
Azure Service Bus queue (sessions)SQS FIFO queue (message groups)MessageGroupId plays the role of SessionId; visibility timeout replaces peek-lock
Azure Service Bus topic + subscriptions with SQL filtersSNS topic + filter policies (JSON)SNS pushes; it does not store. Fan out to SQS for durability
Azure Event GridAmazon EventBridgeEventBridge rules use JSON event patterns; built-in archive/replay, Pipes and Scheduler
Azure Event HubsKinesis Data Streams / Amazon MSKShards (Kinesis) or partitions (MSK) instead of Event Hubs partitions; MSK is real Apache Kafka
Azure Durable Functions / Logic AppsAWS Step FunctionsState machines defined in Amazon States Language (JSON) or the visual Workflow Studio
IBM MQ / on-premises ActiveMQAmazon MQManaged ActiveMQ or RabbitMQ only โ€” not IBM MQ
MassTransit / NServiceBusAWS.Messaging for .NET (lesson 15)AWS's own lightweight framework; MassTransit also has an SQS/SNS transport
IHostedService background worker polling a Service Bus queue.NET Worker Service polling SQS, or Lambda with an SQS event source mappingLambda polling is managed for you; a Worker gives you more control

The biggest mental shift from Service Bus: Service Bus is one product that does queues and topics with rich broker features; AWS splits those jobs across SQS (store), SNS (fan-out) and EventBridge (route) and you compose them. The classic AWS combination "SNS topic โ†’ several SQS queues" is the equivalent of a Service Bus topic with subscriptions.

Architecture

Before: a synchronous chain

Every box in the chain must be healthy at the same time, and the slowest one sets the response time.

After: event-driven

Payments now depends on exactly one thing besides its ledger: the EventBridge API. Fraud and Notifications evolve and fail independently. This is the shape of the module project in the notification system walkthrough.

Security-boundary view

Every hop is an authenticated AWS API call: there are no shared connection strings. Each queue's resource policy says who may send to it (for example, "only this SNS topic"), and each consumer's IAM role says which queue it may read. Encryption at rest applies to SQS (SSE-SQS by default on new queues, or SSE-KMS), SNS (KMS) and EventBridge (AWS owned key by default; customer managed keys are supported for buses โš  VERIFY scope). Payload content is visible to anyone who can read the queue, so keep personal data out.

The "which service when?" decision tree

This is the heart of the lesson. Walk it top to bottom for any new integration. It is deliberately opinionated: it gives the default answer a design review would accept, not the only possible one.

How to read the leaves:

  • Synchronous API. Balance enquiry from the app, confirmation of payee, card authorisation โ€” the user is waiting. Keep it synchronous, but protect it with timeouts and circuit breakers (lesson 9).
  • Step Functions. "Validate โ†’ sanctions screen โ†’ reserve funds โ†’ submit to scheme โ†’ wait for settlement โ†’ notify, and undo the reservation if anything fails." That is a workflow, not a message. The Step Functions lesson covers it.
  • Amazon MQ. A card processor that only speaks JMS over OpenWire. The Amazon MQ lesson shows how to keep it and bridge to native services.
  • Kinesis / MSK. Card-authorisation telemetry at thousands of records per second, read by fraud scoring, the data lake and a real-time dashboard, each at its own pace, with replay after a model change. See Kinesis and MSK.
  • SQS standard / FIFO. "Generate this statement PDF", "send this SMS". One kind of worker does the job. FIFO when order per customer matters or you want SQS to drop duplicate sends. See SQS in depth.
  • EventBridge. TransactionPosted is interesting to Notifications, Fraud, Rewards and the Data Platform, each wanting a different subset. Rules on content keep producers ignorant of consumers. See EventBridge.
  • SNS. Very high-volume, low-latency fan-out of the same message to known subscriber types, especially "one event โ†’ N SQS queues". See SNS.

The last box matters: EventBridge and SNS push. If a consumer is slow or briefly unavailable, push delivery retries but eventually gives up. Putting an SQS queue between the topic or bus and the consumer turns push into durable pull, gives you a dead-letter queue, and lets the consumer control its own concurrency. In banking, nearly every consumer gets its own queue.

Tie-breakers when two answers look right

DilemmaLean towardsBecause
SNS vs EventBridge for fan-outEventBridge by default; SNS when you need very high throughput, SMS/email/mobile push endpoints or FIFO fan-outEventBridge's content filtering, archive/replay and schema registry pay off in a bank; SNS has lower latency and FIFO topics
SQS vs Kinesis for a high-volume feedSQS when each record is an independent job; Kinesis when consumers need order per key or replaySQS scales consumers per message; Kinesis scales per shard
Kinesis vs MSKKinesis unless you need Kafka APIs, Kafka Connect, long retention economics or existing Kafka skillsKinesis is simpler and fully serverless; MSK is a Kafka cluster (or MSK Serverless)
Step Functions vs choreography with eventsStep Functions when one team owns the end-to-end process and needs visibility; choreography when domains are independentLesson 12 compares them in depth
Amazon MQ vs SQS for a new systemSQS/SNSAmazon MQ exists for protocol compatibility, not as the default for new builds

Designing a good event

A well-designed event is the contract between teams. Harbourline's integration standard uses this envelope (the same shape EventBridge itself uses for detail):

TransactionPosted.json
{
  "eventId": "01937a4e-5c1b-7c4e-9a51-2f0f6d2c8b11",
  "eventType": "TransactionPosted",
  "eventVersion": "1.0",
  "occurredAt": "2026-11-27T01:12:44.512Z",
  "correlationId": "c0b9f4a2-1d7e-4c37-8a7f-6a0e1b2c3d4e",
  "producer": "harbourline.core-banking",
  "data": {
    "transactionId": "01937a4e-5b90-7f21-8c3d-9e8f7a6b5c4d",
    "accountId": "ACC-0000123456",
    "customerId": "CUS-00012345",
    "amount": 2450.00,
    "currency": "GBP",
    "type": "transfer_in",
    "channel": "batch",
    "bookingDate": "2026-11-27"
  }
}

Rules behind it:

  1. Past tense, business language. TransactionPosted, not TxnTableInsert.
  2. A unique eventId (UUID v7, time-ordered) so every consumer can deduplicate.
  3. occurredAt from the producer's clock, not the broker's, so consumers can detect late or out-of-order events.
  4. eventVersion, so you can add fields (minor version) or make breaking changes (new major version, run both in parallel).
  5. Identifiers, not personal data. No name, email, phone or full IBAN. The Notifications Service looks up contact details from the customer service using customerId. This keeps personal data out of queues, archives and logs, where retention and access are harder to control โ€” and it matters because SNS Message Data Protection is closed to new customers (see the module overview).
  6. Money as a decimal with a currency code. Serialise with System.Text.Json and a decimal; never double.
  7. A correlationId carried from the original API request through every hop, so one customer complaint can be traced end to end.

Thin vs fat events. A thin event ("transaction X posted, go and look it up") minimises data exposure but makes every consumer call back to the producer. A fat event carries everything a consumer needs. Harbourline's compromise, above, is "fat on business facts, thin on personal data": amount, currency and type are in; who the customer is stays behind an API.

Step-by-step implementation

In this lesson you will classify Harbourline's integrations with the decision tree, then publish one real event to EventBridge and read it back through a queue. The full notification system comes later, in the project walkthrough.

Classify the integrations on paper

Take these six Harbourline flows and run each through the decision tree. Write the answer and the one-line reason. (Model answers are in the Solutions section.)

  1. Mobile app shows the current balance.
  2. A posted transaction triggers an SMS and an email.
  3. A SEPA Instant payment: validate, screen, reserve funds, submit, await confirmation, compensate on failure.
  4. Card-authorisation telemetry at 3,000 records per second, read by fraud scoring and the data lake.
  5. The legacy treasury system sends FX deals over JMS and cannot be changed this year.
  6. A nightly job asks for 40,000 PDF statements to be generated.

Create a bus, a queue and a rule (optional, free tier)

Use the CLI commands in the AWS CLI section. You create an EventBridge custom bus hl-m08-events, an SQS queue hl-m08-l1-tap with a policy that lets only that rule send to it, and a rule that matches TransactionPosted events. This is a "tap" queue that lets you see events arriving.

Publish an event from .NET

Run the console app in Complete code. It builds the envelope above and calls PutEvents. It prints the event ID that EventBridge assigned and fails loudly if the entry was rejected.

Read it from the queue

Receive from the tap queue with the CLI. You will see EventBridge's own envelope (version, id, detail-type, source, account, time, region, resources, detail) wrapped around your JSON. Notice that your data.amount arrives intact as a JSON number.

Tear down

Run the cleanup commands. Nothing in this lesson bills by the hour, but leaving resources behind is a bad habit and an audit finding.

AWS Console instructions

You can do the optional AWS steps in the console instead of the CLI:

  1. Console โ†’ Amazon EventBridge โ†’ Buses โ†’ Event buses โ†’ Create event bus. Name hl-m08-events. Leave archive off for now. Add tags Project=cea-lab, Lab=m08-l1, Environment=learn, Owner=<you>. Create.
  2. Console โ†’ Amazon SQS โ†’ Create queue. Type Standard, name hl-m08-l1-tap. Under Encryption, keep server-side encryption enabled (SSE-SQS). Add the same tags. Create.
  3. Console โ†’ Amazon EventBridge โ†’ Buses โ†’ Rules โ†’ Create rule. Event bus hl-m08-events, name hl-m08-l1-tap-rule, rule type Rule with an event pattern. Choose Custom pattern (JSON editor) and paste the pattern from the CLI section. Target: SQS queue โ†’ hl-m08-l1-tap. The console offers to update the queue's access policy for you; accept it. Create.
  4. To send a test event without code: EventBridge โ†’ Event buses โ†’ hl-m08-events โ†’ Send events. Source harbourline.core-banking, detail type TransactionPosted, and paste the data-bearing JSON as the detail.
  5. SQS โ†’ hl-m08-l1-tap โ†’ Send and receive messages โ†’ Poll for messages to see it arrive.

Console layouts change; if a label differs, look for the same idea under a nearby heading.

AWS CLI

# Variables used by every command below
export AWS_PROFILE=learn-sandbox
export AWS_REGION=us-east-1
ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
TAGS='Key=Project,Value=cea-lab Key=Lab,Value=m08-l1 Key=Environment,Value=learn Key=Owner,Value=you'

# 1. Custom event bus. Keep your events off the account's default bus.
aws events create-event-bus --name hl-m08-events \
  --tags $TAGS

# 2. A standard queue to "tap" the bus. SSE-SQS is on by default for new queues;
#    we set it explicitly so the intent is visible in code review.
QUEUE_URL=$(aws sqs create-queue --queue-name hl-m08-l1-tap \
  --attributes SqsManagedSseEnabled=true,MessageRetentionPeriod=3600 \
  --tags Project=cea-lab,Lab=m08-l1,Environment=learn,Owner=you \
  --query QueueUrl --output text)
QUEUE_ARN=$(aws sqs get-queue-attributes --queue-url "$QUEUE_URL" \
  --attribute-names QueueArn --query Attributes.QueueArn --output text)

# 3. A rule that matches TransactionPosted events from core banking
aws events put-rule --name hl-m08-l1-tap-rule \
  --event-bus-name hl-m08-events \
  --event-pattern '{"source":["harbourline.core-banking"],"detail-type":["TransactionPosted"]}' \
  --tags $TAGS
RULE_ARN=$(aws events describe-rule --name hl-m08-l1-tap-rule \
  --event-bus-name hl-m08-events --query Arn --output text)

# 4. Queue policy: only this rule (via the EventBridge service) may send
cat > tap-policy.json <<EOF
{
  "Policy": "{\"Version\":\"2012-10-17\",\"Statement\":[{\"Sid\":\"AllowRuleOnly\",\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"events.amazonaws.com\"},\"Action\":\"sqs:SendMessage\",\"Resource\":\"$QUEUE_ARN\",\"Condition\":{\"ArnEquals\":{\"aws:SourceArn\":\"$RULE_ARN\"}}}]}"
}
EOF
aws sqs set-queue-attributes --queue-url "$QUEUE_URL" --attributes file://tap-policy.json

# 5. Point the rule at the queue
aws events put-targets --rule hl-m08-l1-tap-rule --event-bus-name hl-m08-events \
  --targets "Id=tap,Arn=$QUEUE_ARN"

# 6. Send a test event from the CLI (the .NET app does the same thing)
aws events put-events --entries '[{"EventBusName":"hl-m08-events","Source":"harbourline.core-banking","DetailType":"TransactionPosted","Detail":"{\"eventId\":\"cli-test-1\",\"eventVersion\":\"1.0\",\"data\":{\"accountId\":\"ACC-0000123456\",\"amount\":12.50,\"currency\":\"GBP\"}}"}]'

# 7. Read it back with long polling (waits up to 10 s)
aws sqs receive-message --queue-url "$QUEUE_URL" --wait-time-seconds 10 \
  --max-number-of-messages 10

Things to notice:

  • put-events returns FailedEntryCount. A 200 response does not mean every entry was accepted โ€” always check that count. The .NET code below does.
  • The queue policy uses aws:SourceArn so that no other EventBridge rule โ€” in your account or anyone else's โ€” can write into the queue. This is the "confused deputy" protection you met in Module 5.

Infrastructure as Code

The same three resources in CDK (C#) and Terraform. The module project's infra/cdk builds the full version.

TapStack.cs
using Amazon.CDK;
using Amazon.CDK.AWS.Events;
using Amazon.CDK.AWS.Events.Targets;
using Amazon.CDK.AWS.SQS;
using Constructs;

namespace Harbourline.Notifications.Infra;

public sealed class TapStack : Stack
{
    public TapStack(Construct scope, string id, IStackProps? props = null) : base(scope, id, props)
    {
        // Custom bus: producers publish here, never to the default bus.
        var bus = new EventBus(this, "Bus", new EventBusProps { EventBusName = "hl-m08-events" });

        // Tap queue, encrypted with SQS-managed keys, short retention for a lab.
        var tap = new Queue(this, "Tap", new QueueProps
        {
            QueueName = "hl-m08-l1-tap",
            Encryption = QueueEncryption.SQS_MANAGED,
            EnforceSSL = true,
            RetentionPeriod = Duration.Hours(1),
        });

        // Rule matching TransactionPosted from core banking.
        var rule = new Rule(this, "TapRule", new RuleProps
        {
            RuleName = "hl-m08-l1-tap-rule",
            EventBus = bus,
            EventPattern = new EventPattern
            {
                Source = ["harbourline.core-banking"],
                DetailType = ["TransactionPosted"],
            },
        });

        // SqsQueue target also writes a queue policy scoped to this rule's ARN.
        rule.AddTarget(new SqsQueue(tap));

        foreach (var (k, v) in new[] { ("Project", "cea-lab"), ("Lab", "m08-l1"), ("Environment", "learn"), ("Owner", "you") })
            Amazon.CDK.Tags.Of(this).Add(k, v);
    }
}

Terraform still calls EventBridge resources aws_cloudwatch_event_* because EventBridge grew out of CloudWatch Events.

Complete code

A .NET 10 console app that publishes a TransactionPosted event. It runs against AWS with your SSO profile; there is no local EventBridge emulator in this course, so for offline work use the in-memory simulator in the module project (labs/m08-notification-system/).

Harbourline.Events.Publisher.csproj
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
    <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
    <RootNamespace>Harbourline.Events.Publisher</RootNamespace>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="AWSSDK.EventBridge" Version="4.1.0.2" />
  </ItemGroup>
</Project>
Program.cs
using System.Text.Json;
using System.Text.Json.Serialization;
using Amazon;
using Amazon.EventBridge;
using Amazon.EventBridge.Model;

// --- 1. Build the domain event (identifiers only, no personal data) -------------
var evt = new EventEnvelope<TransactionPostedData>(
    EventId: Guid.CreateVersion7(),
    EventType: "TransactionPosted",
    EventVersion: "1.0",
    OccurredAt: DateTimeOffset.UtcNow,
    CorrelationId: Guid.NewGuid(),
    Producer: "harbourline.core-banking",
    Data: new TransactionPostedData(
        TransactionId: Guid.CreateVersion7(),
        AccountId: "ACC-0000123456",
        CustomerId: "CUS-00012345",
        Amount: 2450.00m,
        Currency: "GBP",
        Type: "transfer_in",
        Channel: "batch",
        BookingDate: DateOnly.FromDateTime(DateTime.UtcNow)));

var json = JsonSerializer.Serialize(evt, EventJson.Options);

// --- 2. Create the client (credentials from the learn-sandbox SSO profile) ------
var busName = Environment.GetEnvironmentVariable("HL_EVENT_BUS") ?? "hl-m08-events";
using var client = new AmazonEventBridgeClient(RegionEndpoint.USEast1);

// --- 3. Publish ---------------------------------------------------------------
var response = await client.PutEventsAsync(new PutEventsRequest
{
    Entries =
    [
        new PutEventsRequestEntry
        {
            EventBusName = busName,
            Source = evt.Producer,
            DetailType = evt.EventType,
            Detail = json,
            Time = evt.OccurredAt.UtcDateTime,
        },
    ],
});

// --- 4. Check per-entry results: HTTP 200 can still contain failures ------------
var entry = response.Entries?.FirstOrDefault();
if ((response.FailedEntryCount ?? 0) > 0 || entry is null || entry.ErrorCode is not null)
{
    Console.Error.WriteLine($"Rejected: {entry?.ErrorCode} {entry?.ErrorMessage}");
    return 1;
}

Console.WriteLine($"Published {evt.EventType} {evt.EventId} as EventBridge id {entry.EventId}");
Console.WriteLine($"Correlation id: {evt.CorrelationId}");
return 0;

// --- Types --------------------------------------------------------------------
public sealed record EventEnvelope<T>(
    Guid EventId,
    string EventType,
    string EventVersion,
    DateTimeOffset OccurredAt,
    Guid CorrelationId,
    string Producer,
    T Data);

public sealed record TransactionPostedData(
    Guid TransactionId,
    string AccountId,
    string CustomerId,
    decimal Amount,
    string Currency,
    string Type,
    string Channel,
    DateOnly BookingDate);

public static class EventJson
{
    public static readonly JsonSerializerOptions Options = new(JsonSerializerDefaults.Web)
    {
        DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
    };
}

Run it:

aws sso login --profile learn-sandbox
AWS_PROFILE=learn-sandbox dotnet run

Expected output (IDs differ):

Published TransactionPosted 01937a4e-5c1b-7c4e-9a51-2f0f6d2c8b11 as EventBridge id 5f6c0b2e-0d4a-8a1c-3e6b-2a9c7d0e4f11
Correlation id: c0b9f4a2-1d7e-4c37-8a7f-6a0e1b2c3d4e

Code explained line by line

  1. EventEnvelope<TransactionPostedData> โ€” a generic record so every Harbourline event shares the same outer fields. Consumers can read eventType and eventVersion before deciding how to parse data.
  2. Guid.CreateVersion7() โ€” new in .NET 9, a time-ordered UUID v7. It matches the transaction_id convention in CONVENTIONS.md and makes IDs sort roughly by creation time, which helps when debugging.
  3. OccurredAt: DateTimeOffset.UtcNow โ€” the producer's clock. Always UTC.
  4. Amount: 2450.00m โ€” a decimal literal. System.Text.Json writes it as a JSON number without binary floating-point error.
  5. JsonSerializerDefaults.Web โ€” camelCase property names, matching the JSON convention in CONVENTIONS.md.
  6. HL_EVENT_BUS environment variable โ€” the bus name is configuration, not code, so the same binary can publish to a test bus.
  7. new AmazonEventBridgeClient(RegionEndpoint.USEast1) โ€” the SDK finds credentials through the default chain; AWS_PROFILE=learn-sandbox makes it use your SSO session. In an ASP.NET Core app you would register IAmazonEventBridge with AddAWSService<IAmazonEventBridge>() instead.
  8. Entries = [ ... ] โ€” a C# 12 collection expression. PutEvents accepts several entries per call (batching saves money and latency).
  9. Source, DetailType, Detail โ€” the three fields rules match on. Source must not start with aws.; that prefix is reserved for AWS services.
  10. Time โ€” sets the event's time field to the business time instead of the moment EventBridge received it.
  11. response.FailedEntryCount ?? 0 โ€” in SDK v4, value-type properties are nullable (int?) and collections default to null, so the code guards both. Forgetting this is the most common v3 โ†’ v4 upgrade bug.
  12. entry.ErrorCode โ€” per-entry failures (throttling, an internal error, a malformed entry) appear here even when the HTTP call succeeded. A production publisher retries only the failed entries, with backoff.
  13. return 1 โ€” a non-zero exit code makes failures visible to scripts and CI.

Security considerations

โš  Learning shortcut โ€” not for production

The console app publishes with your own SSO identity, which can do far more than publish one event. In production the Payments Service runs under a task or function role whose only messaging permission is events:PutEvents on arn:aws:events:us-east-1:<account>:event-bus/hl-m08-events, ideally with a condition on events:source so it can only publish as harbourline.core-banking โš  VERIFY the condition key name for your use.

โš  Learning shortcut โ€” not for production

The tap queue uses SSE-SQS (keys managed by SQS). For queues that may contain customer-related data, banks usually require SSE-KMS with a customer managed key so key use is logged in CloudTrail and access can be revoked by key policy. Note that queue metadata and message attributes are not encrypted by SSE โ€” only the body.

Further points:

  • Least privilege on both sides. Producers get publish rights to one bus or topic; consumers get receive/delete on one queue. Resource policies on queues and topics restrict who may send.
  • No personal data in events. Send identifiers and let consumers fetch details through an authorised API. This keeps queues, archives, DLQs and logs out of scope for most data-subject requests and reduces breach impact. Your data-protection officer still needs to review what is in the payload โ€” account IDs and amounts are personal data under GDPR when linked to a person.
  • Retention is a data-protection decision. SQS keeps messages up to 14 days, EventBridge archives can keep events indefinitely, Kinesis up to a year. Set retention deliberately and record why.
  • TLS everywhere. Add aws:SecureTransport conditions to queue and topic policies (CDK's EnforceSSL = true does this).
  • Audit. CloudTrail records management calls (create queue, put rule) by default; data-plane calls like SendMessage and PutEvents are data events you opt into โš  VERIFY which messaging data events CloudTrail supports.

Banking use case

Harbourline's architecture board adopted these rules for its event-driven estate (fictional, but typical):

FlowPatternServiceWhy
Card authorisationSynchronousAPI (ALB โ†’ ECS)Merchant terminal waits for approve/decline in under a second
TransactionPostedEvent, many consumersEventBridge โ†’ per-consumer SQSNotifications, Fraud, Rewards and Data Platform each filter differently; archive for replay
Customer notificationsCommand per channelSNS โ†’ SQS โ†’ LambdaFan-out by channel with filter policies; DLQ per channel
Outbound paymentWorkflow with compensationStep Functions StandardMust survive hours waiting for scheme confirmation; auditable history
Card telemetry for fraud modelsStreamKinesis Data StreamsOrdered per card; ML team replays after retraining
FX deals from treasuryLegacy JMSAmazon MQ (ActiveMQ) bridged to EventBridgeTreasury system cannot change this year
Statement generationWork queueSQS standard + Fargate workers40,000 independent jobs; scale workers on queue depth

Note how many flows end in SQS. Queues are the shock absorbers of the estate.

Common mistakes

  • Making everything asynchronous. A balance check that goes through a queue and back is slower, harder to debug and no more reliable. โ†’ Use the first question of the decision tree honestly.
  • Using SNS or EventBridge to deliver straight to a fragile consumer. Push delivery retries, then gives up. โ†’ Put an SQS queue (with a DLQ) in front of each consumer.
  • Assuming exactly-once. Every service here can deliver twice. โ†’ Make every consumer idempotent from day one.
  • Assuming order on a standard queue or on EventBridge. Neither preserves order. โ†’ Use FIFO, Kinesis or MSK with a sensible key, or design consumers to tolerate disorder (use occurredAt and version numbers).
  • Putting personal data in events "for convenience". It then lives in queues, DLQs, archives and logs. โ†’ Identifiers only.
  • Events that are commands in disguise. SendSmsEvent on a bus couples the producer to one consumer. โ†’ Publish facts (TransactionPosted); let Notifications decide to send an SMS.
  • Ignoring FailedEntryCount on PutEvents (or Failed on SQS batch sends). โ†’ Check partial failures and retry only those entries.
  • One giant shared queue for every message type. A poison message of one type blocks or delays all others; IAM cannot separate them. โ†’ One queue per consumer and purpose.

Troubleshooting

SymptomLikely causeFix
PutEvents returns 200 but the tap queue stays emptyRule pattern does not match (detail-type vs DetailType, case, or a typo in source)Compare the pattern with the event; use EventBridge โ†’ Rules โ†’ Edit โ†’ Test pattern with a sample event
Rule matches (metric MatchedEvents rises) but queue still empty; FailedInvocations risesQueue policy does not allow events.amazonaws.com from this ruleFix the policy's aws:SourceArn; check the rule ARN includes the bus name
FailedInvocations with a KMS-encrypted queueEventBridge cannot use the queue's customer managed keyAllow events.amazonaws.com kms:GenerateDataKey and kms:Decrypt in the key policy
AccessDeniedException on PutEventsYour role lacks events:PutEvents on the bus ARNGrant it on the specific bus
ExpiredTokenException / SSO session expiredSSO login timed outaws sso login --profile learn-sandbox
NullReferenceException reading response.EntriesSDK v4 collections default to nullUse ?. and ?? [], as in the sample
Events arrive minutes lateTarget throttling and retries, or a consumer backlogCheck ThrottledRules, the queue's ApproximateAgeOfOldestMessage, and consumer concurrency

Cost and cleanup

Cost: free at lab volumes. Custom-bus events published with PutEvents cost about $1.00 per million (each 64 KB chunk counts as one event) โš  VERIFY current price; SQS includes 1 million requests per month free, permanently. A few dozen test events cost nothing measurable. None of these resources bills by the hour.

export AWS_PROFILE=learn-sandbox AWS_REGION=us-east-1
# Remove the target before the rule, and the rule before the bus
aws events remove-targets --rule hl-m08-l1-tap-rule --event-bus-name hl-m08-events --ids tap
aws events delete-rule --name hl-m08-l1-tap-rule --event-bus-name hl-m08-events
aws events delete-event-bus --name hl-m08-events
aws sqs delete-queue --queue-url "$(aws sqs get-queue-url --queue-name hl-m08-l1-tap --query QueueUrl --output text)"
rm -f tap-policy.json

# Confirm nothing tagged for this lab remains
aws resourcegroupstaggingapi get-resources --tag-filters Key=Lab,Values=m08-l1 \
  --query 'ResourceTagMappingList[].ResourceARN'

The last command should print []. (The tagging API can lag by a few minutes after deletion.) If you plan to go straight on to the module project, you may keep hl-m08-events; the project's CDK stack creates its own resources and expects the name to be free, so delete it before deploying the project.

Exercises

  • ๐ŸŸข Guided: Complete the six-flow classification in Step 1. For each, write the leaf of the decision tree you reached and the deciding question.
  • ๐ŸŸก Challenge: Extend the console app to publish a batch of 10 TransactionPosted events in one PutEvents call, deliberately make one entry invalid (for example, an empty Detail), and retry only the failed entries with exponential backoff, at most three times.
  • ๐Ÿ”ด Live problem: A ticket from the Rewards team: "We subscribed a Lambda function directly to the TransactionPosted SNS topic. During last night's deployment the function was failing for 40 minutes and we lost about 6,000 events. Fraud, who use a queue, lost none. Can messaging be made reliable?" Write the reply: explain why the two teams saw different outcomes, what you would change, and how to recover the lost 6,000 events given that EventBridge has an archive on the bus.

Assignment

Write a one-page integration decision record (ADR) for Harbourline's "account frozen" event: who produces it, who consumes it (at least Notifications, Cards and Fraud), which service carries it and why (cite the decision tree), the event envelope with a sample payload containing no personal data, retention, and the IAM boundaries. Commit it to your portfolio repository as docs/adr/0001-account-frozen-event.md.

Solutions

Knowledge check

Interview questions

  • "When would you not use messaging between two services?" โ€” When the caller needs the result to continue (balance check, authorisation), when latency budgets are tight and the dependency is reliable, or when the added operational complexity outweighs the decoupling. I'd keep it synchronous but add timeouts, retries with jitter and a circuit breaker.
  • "Explain the difference between SNS and EventBridge." โ€” Both are push-based pub/sub. SNS is a high-throughput fan-out of the same message to subscribers, with attribute or payload filter policies and FIFO topics. EventBridge routes events by matching rules on content, integrates SaaS and AWS service events, and adds a schema registry, archive/replay, Pipes and cross-account buses. I default to EventBridge for domain events and use SNS for very high-volume fan-out or SMS/email/mobile endpoints.
  • "How do you stop a slow downstream system from taking down your API?" โ€” Remove temporal coupling: record the fact, publish an event, return. The downstream reads from its own queue at its own pace. Then bound what remains synchronous with timeouts and bulkheads.
  • "What would you put in an event payload in a bank?" โ€” A unique event ID, type, version, occurrence time, correlation ID and business facts needed for routing and processing. Identifiers instead of personal data; consumers fetch details through authorised APIs. Money as decimal plus currency.
  • "Can you get exactly-once processing on AWS?" โ€” Not end to end from the transport alone. Services give at-least-once delivery, some with deduplication windows. Exactly-once effects come from idempotent consumers, typically a conditional write on an idempotency key in DynamoDB.

Certification practice

Completion checklist

  • I can explain time, location, rate and format coupling to a colleague
  • I can say whether a message is a command, an event or a document
  • I can walk the decision tree for a new integration and justify the leaf
  • I designed an event envelope with no personal data
  • (Optional) I published an event with .NET and saw it in the tap queue
  • I ran the cleanup commands and the tag check returned []

Key takeaways

  • Asynchronous messaging removes time, location and rate coupling; it does not remove format coupling or the need for idempotency.
  • Queue (SQS), topic (SNS), bus (EventBridge), stream (Kinesis/MSK) and workflow (Step Functions) are different tools; Amazon MQ exists for protocol compatibility.
  • Walk the decision tree: synchronous need first, then workflow, legacy protocol, replay/order, and finally one consumer vs many.
  • In banking, put an SQS queue with a DLQ in front of almost every consumer, and keep personal data out of events.
  • Limits have moved: SQS and EventBridge now accept payloads up to 1 MiB / 1 MB; check before trusting old material.

Further reading

Independent learning resource โ€” not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page