Skip to Content
Cloud Engineering Academy
8. Messaging, Events & Microservices

Module 8: Messaging, Events & Microservices

Learn how AWS services pass messages and events between systems reliably: SQS, SNS, EventBridge, Step Functions, Kinesis, MSK and Amazon MQ. You apply retries, dead-letter queues, idempotency, sagas and the outbox pattern in .NET, and build a banking notification system that survives duplicates and failures.

DVA-C03SAA-C03DEA-C01

Assumes: .NET 10 LTS · AWS SDK for .NET v4 · AWS.Messaging 1.6.0 · Powertools for AWS Lambda (.NET) 3.2.x · Region us-east-1. Verified against official sources on 2026-11-18.

Learn how AWS services pass messages and events between systems reliably: SQS, SNS, EventBridge, Step Functions, Kinesis, MSK and Amazon MQ. You apply retries, dead-letter queues, idempotency, sagas and the outbox pattern in .NET, and build a banking notification system that survives duplicates and failures.

Estimated timeAbout 22 hours (lessons 13 h · hands-on exercises 5 h · project 4 h)
Priority for youHigh — priority 2 (system integration)
LevelIntermediate → advanced
CertificationsDVA-C03 (primary) · SAA-C03 · DEA-C01
Cost profile🟢 SQS, SNS, EventBridge, Lambda, Step Functions at lab volumes · 🟡 Kinesis · 🔴 Amazon MQ, MSK

Why this matters in banking

Payments, card authorisations and customer notifications cross many systems, and every hop can fail, time out or deliver twice. A queue consumer that is not idempotent posts the same debit twice; a missing dead-letter queue lets one malformed message block a whole payment run; an unordered event stream shows a customer a balance that never existed. Asynchronous design is how banks absorb peaks (salary day, Black Friday) and isolate failures, but only if duplicates, ordering and compensation are designed in from the start. This module gives you those patterns with .NET code you can reuse.

From .NET/Azure to AWS

You know (Microsoft/Azure)AWS equivalentWhat's different
MSMQ / Service Bus queuesAmazon SQS (standard or FIFO)No broker to run; FIFO gives ordering per message group; visibility timeout replaces peek-lock
Service Bus topics + subscriptionsAmazon SNS with subscription filter policiesSNS pushes to SQS, Lambda, HTTP, email; typical pattern is SNS → SQS fan-out
Azure Event GridAmazon EventBridgeRules match on event content; archive/replay and Pipes built in
Durable FunctionsAWS Step FunctionsJSON/visual state machines; Standard (long-running) vs Express (high-volume)
Event HubsKinesis Data Streams / Amazon MSKShards vs Kafka partitions; replay by position
IBM MQ / ActiveMQ on-premisesAmazon MQManaged ActiveMQ or RabbitMQ for JMS/AMQP legacy integration
MassTransit / NServiceBusAWS.Messaging frameworkAWS's .NET library for publishing and handling SQS/SNS/EventBridge messages
Service Bus duplicate detectionSQS FIFO deduplication + your own idempotency storeWindow-limited dedup; durable idempotency lives in DynamoDB

Learning objectives

By the end of this module you can:

  • Choose between SQS, SNS, EventBridge, Kinesis, MSK and Amazon MQ for a given integration and justify the choice.
  • Configure SQS standard and FIFO queues with suitable visibility timeouts, long polling, batching, dead-letter queues and redrive.
  • Build SNS fan-out with filter policies and EventBridge rules with schemas, archive/replay and Pipes.
  • Implement retries with exponential backoff and jitter, and make every consumer idempotent with a DynamoDB idempotency table or Powertools.
  • Explain at-least-once vs exactly-once delivery and event ordering, and what each means for a ledger.
  • Model a payment as a saga with compensation in Step Functions, and publish events reliably with the outbox pattern in EF Core.
  • Design microservice boundaries (bounded contexts, domain events) and service discovery with Cloud Map or ECS Service Connect.

Prerequisites

How the pieces fit

Lessons

The order follows the dependencies between lessons: the building blocks first (queues, topics, buses), then the reliability toolkit (retries, delivery guarantees, idempotency), then workflows and streaming, then the architecture patterns that tie them together.

  1. Asynchronous communication and event-driven architecture — queues vs pub/sub vs event buses vs streams, with a "which service when?" decision tree.
  2. Amazon SQS in depth — standard vs FIFO, visibility timeout, long polling, batching, message groups and the .NET consumer loop.
  3. Dead-letter queues and redrive — poison messages, maxReceiveCount, partial batch responses, redrive to source and alarms on DLQ depth.
  4. Amazon SNS: fan-out and message filtering — topics, fan-out to SQS and Lambda, filter policies, delivery retries and FIFO topics.
  5. Amazon EventBridge: buses, rules, schemas and Pipes — buses, rules, the schema registry, archive and replay, Pipes and the enhanced custom event bus.
  6. Retry strategies — exponential backoff with jitter, SDK retry modes and avoiding retry storms.
  7. Delivery guarantees and ordering — at-least-once vs exactly-once, FIFO groups, sequence numbers and out-of-order events.
  8. Idempotency in .NET with DynamoDB and Powertools — idempotency keys, conditional writes, TTL and the Powertools for AWS Lambda (.NET) idempotency utility.
  9. Distributed transactions and the saga pattern — orchestration vs choreography, compensation and a Step Functions payment saga.
  10. AWS Step Functions: workflows, retries and callbacks — Standard vs Express workflows, retries and catch, task-token callbacks, Map and Distributed Map.
  11. The outbox pattern with EF Core — writing state and events atomically, relaying to EventBridge and handling relay failures.
  12. Kinesis Data Streams and Amazon MSK — streams vs queues, shards and partitions, ordering and replay.
  13. Amazon MQ: bridging legacy JMS and AMQP systems — when MQ-style systems must stay, and bridging them to native services.
  14. Microservices communication and service discovery — bounded contexts, queries vs commands vs events, safe synchronous calls and discovery with Cloud Map and ECS Service Connect.
  15. AWS.Messaging for .NET — publishers, handlers, the message envelope and hosting handlers in a Worker Service or Lambda.

Hands-on code and module project

Module 8 has no separate lab walkthrough pages. The hands-on work sits inside the lessons, backed by two code folders in the repository. Both run their tests without an AWS account.

FolderWhat it holdsUsed by
labs/m08-notification-system/The module project: EventBridge → SNS → SQS → Lambda notification system with idempotent handlers, backoff, a DLQ, redrive, failure-injection scripts and RUNBOOK.md (41 tests)Module project, plus the SQS, DLQ and EventBridge lessons
labs/m08-messaging-services/Runnable, tested code for the service lessons: SNS publisher and filter-policy checks, EventBridge publisher, Step Functions definition and client, Kinesis shard routing and producers, Amazon MQ translator and bridge (30 tests)SNS, EventBridge, Step Functions, Kinesis and MSK, Amazon MQ

Hands-on topics, each inside the named lesson with its own cost label:

  • SQS standard vs FIFO from .NET: send, receive, batch, change visibility, see deduplication — Amazon SQS in depth.
  • Poison-message drill: a handler that rejects bad input, a DLQ that catches it, a redrive after the fix — Dead-letter queues and redrive.
  • SNS → SQS fan-out with filter policies, then an EventBridge rule with archive and replay — SNS and EventBridge.
  • Payment saga with compensation in Step Functions — Saga pattern and Step Functions.
  • Outbox pattern: EF Core writes the payment and the event in one transaction and a relay publishes it — Outbox pattern (tests run on SQLite).
  • Kinesis ordering and replay (🟡 deploy, test and delete within an hour) — Kinesis and MSK.

Module project — banking transaction notification system. Using .NET, SQS, SNS, EventBridge and Lambda, notify fictional customers when a transaction posts. Demonstrate reliable processing, duplicate prevention (idempotency table), retries with backoff, a poison message landing in a DLQ, redrive after a fix, and recovery after a downstream outage. It includes a failure-injection script and a short runbook.

When you finish, keep the Module 8 cheat sheet handy for revision.

Cost profile

PartLabelFree alternative
SQS, SNS, EventBridge, Lambda, Step Functions at lab volumes🟢 freeWithin the always-free allowances for small test volumes ⚠ VERIFY current allowances; AWS Emulator or SAM local for offline runs
DynamoDB idempotency table🟢 freeDynamoDB Local or the always-free table allowance
Kinesis Data Streams🟡 creditsShard-hours bill while the stream exists — deploy-and-destroy within an hour
Amazon MQ broker🔴 paidDesign-only, or ActiveMQ/RabbitMQ in Docker locally
Amazon MSK🔴 paidDesign-only, or Kafka in Docker locally
ECS Service Connect / Cloud Map demo on Fargate🟡 creditsDeploy-and-destroy; or design-only with the Module 3 Fargate stack

Certification alignment

ExamDomains (weight)What this module covers
DVA-C03Development (30%) · Troubleshoot (22%)SQS/SNS/EventBridge APIs, Lambda event sources, Step Functions, DLQs and retries
SAA-C03Resilient (26%)Decoupling with queues and events, loosely coupled architectures
DEA-C01Ingestion (34%)Kinesis and MSK streaming ingestion concepts

The domain labels are short forms; see each exam page for the official wording.

Service availability notes

Services that changed — verified 2026-11-17

Amazon SNS Message Data Protection is in maintenance (announced 2026-03-31) and closed to new customers. Do not rely on it to detect or mask personal data in messages; keep PII out of event payloads (send identifiers and let consumers fetch details) and mask in your own code.

SQS, SNS, EventBridge, Step Functions, Kinesis, MSK and Amazon MQ are all fully available.

Design note for banks

Notification content (balances, partial account numbers) is still personal data. Which channels may carry which data, and for how long messages may be retained in queues and archives, needs review by your data-protection and security teams.

Key AWS services

Amazon SQS · Amazon SNS · Amazon EventBridge (rules, schema registry, archive/replay, Pipes, Scheduler) · AWS Step Functions · Amazon Kinesis Data Streams · Amazon MSK · Amazon MQ · AWS Lambda · Amazon DynamoDB · AWS Cloud Map · Amazon ECS Service Connect · AWS.Messaging for .NET · Powertools for AWS Lambda (.NET)

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page