AWS.Messaging for .NET: publishers, handlers and envelopes
Publish and handle SQS, SNS and EventBridge messages with the AWS Message Processing Framework for .NET: typed publishers, IMessageHandler classes, the CloudEvents envelope, middleware and error handlers, hosted in a Worker Service or Lambda, with idempotency added on top.
Assumes: .NET 10 LTS · AWS.Messaging 1.6.0 · AWS.Messaging.Lambda 1.0.3 · AWSSDK.SQS 4.1.0.2 · Amazon.Lambda.Annotations 2.4.2 · xUnit v3 4.0.3 · Amazon.CDK.Lib 2.277.0 · Terraform AWS provider v6 · Region us-east-1. Verified against official sources on 2026-11-18.
Learning objectives
By the end of this lesson you can:
- Explain what the AWS Message Processing Framework for .NET (
AWS.Messaging) does for you and what it deliberately leaves to you (idempotency, outbox, sagas). - Register publishers for SQS, SNS and EventBridge and publish a typed message, including to a FIFO queue with a group ID and deduplication ID.
- Write an
IMessageHandler<T>that returnsMessageProcessStatus.Success()orFailed(), and explain how that decision maps to SQS delete, retry and the dead-letter queue. - Read the CloudEvents-style envelope the framework writes, and map message type identifiers to handlers so publisher and consumer need not share .NET types.
- Host the same handler in a Worker Service (SQS poller) or in Lambda (
AWS.Messaging.Lambda). - Add middleware and an error handler, make the handler idempotent with the module project's DynamoDB store, and unit-test it without AWS.
Prerequisites
- Amazon SQS in depth, Dead-letter queues and redrive and Retry strategies: visibility timeout,
maxReceiveCountand why failed messages return to the queue. - Idempotency in .NET: the claim, do, complete pattern used by the handler in this lesson.
- Module 3: .NET on AWS: Worker Services, dependency injection and Lambda basics.
- Tools: .NET 10 SDK (or Docker with the
mcr.microsoft.com/dotnet/sdk:10.0image), AWS CLI v2 withlearn-sandboxfor the optional AWS part. The handler tests need no AWS account.
Why this matters in banking
The notification worker in Retry strategies is about 150 lines of hand-written code: a long-polling loop, JSON parsing, concurrency control, a visibility-timeout heartbeat for slow messages, deletion on success, and careful shutdown. Each of those is a place for a bug. A missing heartbeat means a slow SMS provider makes SQS hand the same message to a second worker. A missing try/catch around the loop means one malformed message kills the process and the whole backlog stops. A team that writes this loop six times for six services ends up with six slightly different loops, and six different incident reports.
The AWS Message Processing Framework for .NET moves that plumbing into a library with defaults chosen by AWS. You write the one part that is your business: the handler. It is the closest AWS equivalent to the parts of MassTransit or NServiceBus a typical banking team uses on RabbitMQ or Azure Service Bus: typed messages, handlers, middleware and a hosted poller.
It does not make a payment handler safe. Delivery is still at least once, so the same handler can run twice for the same payment. This lesson shows the framework doing its job, and then shows where your idempotency code goes, because that is the part that stops a customer being notified, or debited, twice.
Design note for banks
A message envelope carries metadata (type, source, time, an ID) around your data. Treat the whole message as personal data if the payload can identify a customer. By default the framework does not log message payloads; leave that default in place in production, and decide with your data-protection team what may appear in logs, queues and archives.
The idea in plain language
AWS.Messaging is a small framework built on the standard .NET dependency injection container and generic host. It has four moving parts:
- Message types: plain C# records or classes. They are the contract.
- Publishers: you register "messages of type X go to this SQS queue / SNS topic / EventBridge bus". At runtime you inject
IMessagePublisherand callPublishAsync(message). The framework serialises your object into an envelope and sends it. - Handlers: classes implementing
IMessageHandler<T>. You register "messages with this type identifier are handled by this class". - Pollers and processors: for SQS in a long-running process, a background service polls the queue, runs your handler and deletes the message when the handler says it succeeded. In Lambda, a processor class does the equivalent for the batch Lambda hands you.
Analogy: Think of the bank's internal post room. Publishers hand over a typed document; the post room puts it in a standard envelope with a sender, a document type and a timestamp, and delivers it to the right tray. In each team, a clerk (the handler) opens only the envelopes marked with a type they know how to process, does the work, and tells the post room "done" (so the envelope is destroyed) or "can't do it now" (so it goes back in the tray). The post room does not know what is inside, and it does not check whether the clerk already did this job yesterday. That check is the clerk's responsibility.
What the envelope looks like
The framework wraps every message in a CloudEvents JSON envelope and puts that JSON into the service-specific field: the SQS MessageBody, the SNS Message, or the EventBridge Detail. The documented fields:
| Field | Meaning | Default | Override |
|---|---|---|---|
id | Unique message ID | A new GUID | IMessageIdGenerator in DI |
source | Which system sent it | The Lambda function name, the ECS cluster and task, the EC2 instance ID, or /aws/messaging as fallback | AddMessageSource or AddMessageSourceSuffix |
specversion | CloudEvents version | 1.0 | n/a |
type | Routes the message to a handler | Full .NET type name | The messageTypeIdentifier argument on AddSQSPublisher, AddSNSPublisher, AddEventBridgePublisher and AddMessageHandler |
time | When it was created, UTC | Now | IDateTimeHandler in DI |
data | Your serialised .NET object | n/a | ConfigureSerializationOptions, AddSerializationCallback |
The type field is the important one. If publisher and consumer are in different repositories (and they should be), the consumer does not have the publisher's .NET class. Use a stable string identifier such as hl.payment.settled.v1 on both sides. The consumer defines its own record with the fields it needs, and System.Text.Json ignores the rest.
What problem the AWS service solves
The framework solves the plumbing problems that every SQS consumer in .NET has to solve:
- Long polling in a loop, with a configurable number of concurrent handlers.
- Visibility timeout management: while a handler runs, the framework extends the message's visibility timeout so a slow handler does not make the message reappear for another worker.
- Deserialising JSON into typed messages and finding the right handler.
- Deleting messages on success and leaving them on failure so SQS retries and eventually uses the DLQ.
- Capped exponential backoff between polling errors.
- Publishing to SQS, SNS and EventBridge through one
IMessagePublisherabstraction, with FIFO options and SQS batch sending. - OpenTelemetry integration through a separate package (
AWS.Messaging.Telemetry.OpenTelemetry).
What it does not solve, and what the rest of this module teaches:
| Not included | Why it matters | Where to look |
|---|---|---|
| Idempotent handling | SQS standard, SNS and EventBridge deliver at least once | Idempotency in .NET |
| A transactional outbox | Publishing after a database commit can still lose the message | The outbox pattern with EF Core |
| Sagas and orchestration | Multi-service consistency | Distributed transactions and the saga pattern |
| Request/reply, scheduling, routing slips | It is a message-processing framework, not a service bus | Step Functions, EventBridge Scheduler |
| Transports other than SQS, SNS, EventBridge | By design | Amazon MQ or MSK clients for other brokers |
Other things to know before you adopt it:
- It is an AWS open-source project (Apache-2.0 licence; repository
aws/aws-dotnet-messaging). Check the project's release notes and issue tracker for maturity and support commitments before you standardise on it in a regulated environment ⚠ VERIFY. - Messages published with the framework use its envelope. A handler registered with the default envelope mode expects that envelope. For messages that arrive from other sources (a raw EventBridge event, a raw S3 notification), use a poller with the envelope mode turned off. That mode supports one message type per queue; see the code below.
- Handlers are matched by type identifier. A typo means "no handler registered" and the message ends up failing until it reaches the DLQ.
Microsoft / Azure comparison
| You know (Microsoft/Azure) | AWS.Messaging equivalent | What's different |
|---|---|---|
MassTransit IConsumer<T> / NServiceBus IHandleMessages<T> | IMessageHandler<T> with HandleAsync(MessageEnvelope<T>, CancellationToken) | You return a status (Success/Failed) instead of completing by not throwing |
IBus.Publish / ISendEndpoint.Send | IMessagePublisher.PublishAsync, or ISQSPublisher, ISNSPublisher, IEventBridgePublisher | The generic publisher routes by message type; the service-specific ones expose service options (FIFO IDs, batch) |
ServiceBusProcessor (Azure.Messaging.ServiceBus) | AddSQSPoller (a BackgroundService) | Concurrency and visibility extension are poller options |
CompleteMessageAsync / AbandonMessageAsync | MessageProcessStatus.Success() / Failed() | A failed message stays on the queue and is redelivered after the visibility timeout |
| Auto lock renewal | Visibility timeout extension (VisibilityTimeoutExtensionThreshold, heartbeat) | Built in; same purpose |
| MassTransit consume/send filters, NServiceBus behaviours | IHandlerMiddleware, registered with AddHandlerMiddleware<T>() | Run in registration order, around the handler |
MassTransit retry policies / UseMessageRetry | IMessageErrorHandler returning Retry, Failed or Success, plus SQS redelivery | In-process retry (new DI scope) is separate from SQS-level redelivery |
Message type URN (urn:message:...) | type field of the envelope; messageTypeIdentifier argument | Use stable strings, not .NET names, across repositories |
| MassTransit outbox / inbox | Not provided | Your EF Core outbox and idempotency store |
| Service Bus dead-letter sub-queue | A separate SQS dead-letter queue configured by redrive policy | The framework doesn't manage it |
Architecture
Security-boundary view. Two IAM roles, two narrow policies. The publisher role may events:PutEvents on one bus and sqs:SendMessage on one queue. The worker role may sqs:ReceiveMessage, DeleteMessage, ChangeMessageVisibility and GetQueueAttributes on one queue, plus dynamodb:PutItem and DeleteItem on the idempotency table. The framework needs ChangeMessageVisibility to extend the visibility timeout, so a policy that allows only receive and delete will fail at runtime ⚠ VERIFY against the framework's current IAM guidance. The framework does not log message payloads unless you call EnableDataMessageLogging(). Do not enable that in production.
Step-by-step implementation
Define the message contracts
A message is a plain C# type. Give it a stable identifier you will not change (hl.notification.send.v1). Put the version in the identifier, as the previous lesson recommends.
Register publishers
In the Payments service, call AddAWSMessageBus and register where each message type goes: a bus for events (AddEventBridgePublisher), a queue for commands (AddSQSPublisher). Inject IMessagePublisher or the service-specific interface.
Write the handler
Implement IMessageHandler<SendNotification>. The handler claims an idempotency key, sends, records completion, and returns Success(). For a temporary downstream failure it releases the claim and returns Failed() so SQS redelivers.
Host it
Register the poller (AddSQSPoller) and the handler (AddMessageHandler) in a Worker Service, or register the Lambda message processor in an AWS Lambda function.
Add middleware and an error handler
Middleware wraps every handler call, which is the right place for correlation IDs. An error handler decides what happens when a handler throws: retry now in a fresh DI scope, or give up and leave the message on the queue.
Unit-test the handler
A handler is a class with a method. Construct it with fakes, call HandleAsync with an envelope, and assert on the returned status and the side effects. No AWS account is needed.
Deploy and look at the real envelope
Create the queue and bus, publish a message, and read it with the CLI at a visibility timeout of zero so you see exactly what the framework wrote.
AWS Console instructions
- Console → Amazon SQS → Queues → Create queue. Choose Standard, name it
hl-m08-msg-notify, set Visibility timeout to 180 seconds, and keep Server-side encryption on. Under Dead-letter queue, enable it, pickhl-m08-msg-notify-dlq(create it first) and set Maximum receives to 5. - Console → Amazon EventBridge → Event buses → Create event bus named
hl-m08-msg-events. Then Rules → Create rule on that bus with an event pattern matchingdetail-typeequal tohl.payment.settled.v1, and the queue as target. - After publishing, open the queue and choose Send and receive messages → Poll for messages. Open one message and read its Body: this is the envelope. (Polling from the console counts as a receive and increases the receive count; do not leave a poll open on a queue you want to keep calm.)
- Console → Amazon SQS →
hl-m08-msg-notify-dlqto confirm that nothing arrives there while your handler returnsSuccess(). Make one message fail on purpose (see the Exercises) and watch it arrive after five receives.
AWS CLI
# Create the DLQ, then the source queue that points at it.
aws sqs create-queue --queue-name hl-m08-msg-notify-dlq --profile learn-sandbox \
--attributes MessageRetentionPeriod=1209600
DLQ_ARN=$(aws sqs get-queue-attributes --profile learn-sandbox \
--queue-url "$(aws sqs get-queue-url --queue-name hl-m08-msg-notify-dlq --profile learn-sandbox --query QueueUrl --output text)" \
--attribute-names QueueArn --query Attributes.QueueArn --output text)
aws sqs create-queue --queue-name hl-m08-msg-notify --profile learn-sandbox \
--attributes "VisibilityTimeout=180,RedrivePolicy={\"deadLetterTargetArn\":\"$DLQ_ARN\",\"maxReceiveCount\":\"5\"}"
# Peek at one message WITHOUT hiding it (visibility timeout 0), to see the real envelope.
QUEUE_URL=$(aws sqs get-queue-url --queue-name hl-m08-msg-notify --profile learn-sandbox --query QueueUrl --output text)
aws sqs receive-message --queue-url "$QUEUE_URL" --visibility-timeout 0 \
--max-number-of-messages 1 --attribute-names All --profile learn-sandbox
# Depth of the queue and the DLQ.
aws sqs get-queue-attributes --queue-url "$QUEUE_URL" --profile learn-sandbox \
--attribute-names ApproximateNumberOfMessages ApproximateNumberOfMessagesNotVisibleThe envelope you read back should match the shape in the table above (id, source, specversion, type, time, data). The documentation shows data as the serialised message; whether the framework nests it as a JSON string or an object in the version you run is easiest to settle by looking at one real message ⚠ VERIFY.
Infrastructure as Code
The same resources in both tools: the queue with its DLQ and a redrive policy, an event bus with a rule that routes hl.payment.settled.v1 events to the queue, the idempotency table, and two least-privilege policies. The CDK version also includes the optional Lambda hosting (note ReportBatchItemFailures = true: the Lambda code returns an SQSBatchResponse).
using Amazon.CDK;
using Amazon.CDK.AWS.DynamoDB;
using Amazon.CDK.AWS.Events;
using Amazon.CDK.AWS.IAM;
using Amazon.CDK.AWS.Lambda;
using Amazon.CDK.AWS.Lambda.EventSources;
using Amazon.CDK.AWS.SQS;
using Constructs;
using EventTargets = Amazon.CDK.AWS.Events.Targets;
var app = new App();
_ = new MessagingFrameworkStack(app, "HlM08-AwsMessaging", new StackProps());
app.Synth();
public sealed class MessagingFrameworkStack : Stack
{
public MessagingFrameworkStack(Construct scope, string id, IStackProps props) : base(scope, id, props)
{
Amazon.CDK.Tags.Of(this).Add("Project", "cea-lab");
Amazon.CDK.Tags.Of(this).Add("Lab", "m08-aws-messaging");
Amazon.CDK.Tags.Of(this).Add("Environment", "learn");
var dlq = new Queue(this, "NotifyDlq", new QueueProps
{
QueueName = "hl-m08-msg-notify-dlq",
RetentionPeriod = Duration.Days(14),
Encryption = QueueEncryption.SQS_MANAGED,
EnforceSSL = true,
});
var queue = new Queue(this, "NotifyQueue", new QueueProps
{
QueueName = "hl-m08-msg-notify",
// The worker's poller uses VisibilityTimeout = 120 seconds and extends it while a handler runs.
// For Lambda the rule of thumb is at least 6 x the function timeout.
VisibilityTimeout = Duration.Seconds(180),
Encryption = QueueEncryption.SQS_MANAGED,
EnforceSSL = true,
DeadLetterQueue = new DeadLetterQueue { Queue = dlq, MaxReceiveCount = 5 },
});
var bus = new EventBus(this, "Bus", new EventBusProps { EventBusName = "hl-m08-msg-events" });
// Route PaymentSettled events (published by AWS.Messaging's EventBridge publisher) to the queue.
// The framework puts its message type identifier in detail-type.
var rule = new Rule(this, "PaymentSettledRule", new RuleProps
{
EventBus = bus,
EventPattern = new EventPattern { DetailType = ["hl.payment.settled.v1"] },
});
rule.AddTarget(new EventTargets.SqsQueue(queue));
var table = new Table(this, "IdempotencyTable", new TableProps
{
TableName = "hl-m08-msg-idempotency",
PartitionKey = new Amazon.CDK.AWS.DynamoDB.Attribute { Name = "id", Type = AttributeType.STRING },
BillingMode = BillingMode.PAY_PER_REQUEST,
TimeToLiveAttribute = "expiresAt",
RemovalPolicy = RemovalPolicy.DESTROY, // lab only
});
// Least-privilege policy for the WORKER that polls the queue (attach to the ECS task role).
var workerPolicy = new ManagedPolicy(this, "WorkerPolicy", new ManagedPolicyProps
{
ManagedPolicyName = "hl-m08-msg-worker",
Statements =
[
new PolicyStatement(new PolicyStatementProps
{
Actions = ["sqs:ReceiveMessage", "sqs:DeleteMessage", "sqs:ChangeMessageVisibility", "sqs:GetQueueAttributes"],
Resources = [queue.QueueArn],
}),
new PolicyStatement(new PolicyStatementProps
{
Actions = ["dynamodb:PutItem", "dynamodb:DeleteItem"],
Resources = [table.TableArn],
}),
],
});
// Least-privilege policy for a PUBLISHER (attach to the Payments task role).
_ = new ManagedPolicy(this, "PublisherPolicy", new ManagedPolicyProps
{
ManagedPolicyName = "hl-m08-msg-publisher",
Statements =
[
new PolicyStatement(new PolicyStatementProps
{
Actions = ["events:PutEvents"],
Resources = [bus.EventBusArn],
}),
],
});
// Optional: the SAME handler class hosted in Lambda instead of a worker.
// The event source mapping must report batch item failures, because the code returns SQSBatchResponse.
var function = new Function(this, "NotifyFunction", new FunctionProps
{
FunctionName = "hl-m08-msg-notify-handler",
Runtime = new Runtime("dotnet10", RuntimeFamily.DOTNET_CORE),
Architecture = Architecture.ARM_64,
Handler = "Harbourline.Notifications.Lambda::Harbourline.Notifications.Lambda.Function_FunctionHandler_Generated::FunctionHandler",
Code = Code.FromAsset("publish/lambda"),
MemorySize = 512,
Timeout = Duration.Seconds(30),
Environment = new Dictionary<string, string> { ["IDEMPOTENCY_TABLE"] = table.TableName },
});
table.GrantReadWriteData(function);
function.AddEventSource(new SqsEventSource(queue, new SqsEventSourceProps
{
BatchSize = 10,
ReportBatchItemFailures = true,
MaxConcurrency = 5,
}));
_ = new CfnOutput(this, "QueueUrl", new CfnOutputProps { Value = queue.QueueUrl });
_ = new CfnOutput(this, "WorkerPolicyArn", new CfnOutputProps { Value = workerPolicy.ManagedPolicyArn });
}
}The CDK code compiled and the Terraform validated (v6 AWS provider) on 2026-11-18; neither was deployed. The idempotency table and the DLQ policy are the same ideas as in the module project. The CDK SqsQueue rule target adds the queue permission for EventBridge for you; Terraform needs the explicit queue policy shown.
Complete code
Everything below compiled against AWS.Messaging 1.6.0 and AWS.Messaging.Lambda 1.0.3 on .NET 10 on 2026-11-18, and the two tests passed. The handler uses IIdempotencyStore, DynamoDbIdempotencyStore, INotificationSender and FakeNotificationSender from the module project's Harbourline.Notifications.Core library (lab README). The solution layout is Contracts (optional), Publisher, Worker, Lambda and Tests, with one project reference from each to Harbourline.Notifications.Core.
The consumer: handler, middleware, error handler
namespace Harbourline.Notifications.Worker;
/// <summary>A command: send one notification. The consumer owns this type; the publisher may define its own.</summary>
public sealed record SendNotification
{
public required string TransactionId { get; init; }
public required string CustomerId { get; init; }
public required string Channel { get; init; }
public required string Text { get; init; }
}using AWS.Messaging;
using Harbourline.Notifications.Core.Idempotency;
using Harbourline.Notifications.Core.Processing;
using Harbourline.Notifications.Core.Resilience;
namespace Harbourline.Notifications.Worker;
public sealed class SendNotificationHandler(
IIdempotencyStore store,
INotificationSender sender,
ILogger<SendNotificationHandler> logger) : IMessageHandler<SendNotification>
{
public async Task<MessageProcessStatus> HandleAsync(
MessageEnvelope<SendNotification> envelope, CancellationToken token = default)
{
var command = envelope.Message;
// The key comes from the BUSINESS identity, not from envelope.Id: a publisher retry gets a new envelope ID
// for the same transaction, and that retry is exactly the duplicate we must stop.
var key = $"notify#{command.Channel}#{command.TransactionId}";
var claim = await store.TryClaimAsync(key, TimeSpan.FromSeconds(60), token);
if (claim == ClaimResult.AlreadyCompleted)
{
return MessageProcessStatus.Success(); // duplicate: nothing to do, delete the message
}
if (claim == ClaimResult.InProgress)
{
return MessageProcessStatus.Failed(); // another worker has it: let SQS redeliver later
}
try
{
await sender.SendAsync(new Notification(command.Channel, command.CustomerId, command.Text, key), token);
}
catch (TransientDownstreamException)
{
await store.ReleaseAsync(key, token); // give the claim back so the retry can take it
return MessageProcessStatus.Failed(); // message stays on the queue
}
await store.CompleteAsync(key, TimeSpan.FromDays(7), token);
logger.LogInformation("Sent {Channel} notification {Key} (message {MessageId})",
command.Channel, key, envelope.Id);
return MessageProcessStatus.Success();
}
}
/// <summary>Runs around every handler call. Here: puts the message ID on every log line written inside the handler.</summary>
public sealed class CorrelationMiddleware(ILogger<CorrelationMiddleware> logger) : IHandlerMiddleware
{
public async Task<MessageProcessStatus> InvokeAsync<T>(
MessageEnvelope<T> envelope, RequestDelegate next, CancellationToken token = default)
{
using var scope = logger.BeginScope(new Dictionary<string, object> { ["messageId"] = envelope.Id });
return await next();
}
}
/// <summary>Decides what happens when a handler THROWS: retry now (new DI scope) up to 3 attempts, then leave it for SQS.</summary>
public sealed class PoisonAwareErrorHandler(ILogger<PoisonAwareErrorHandler> logger) : IMessageErrorHandler
{
public ValueTask<MessageErrorHandlerResponse> OnHandleError<T>(
MessageEnvelope<T> envelope, Exception exception, int attempt, CancellationToken token = default)
{
logger.LogWarning("Handler failed for {MessageId} on attempt {Attempt}: {Type}",
envelope.Id, attempt, exception.GetType().Name);
return ValueTask.FromResult(attempt < 3 ? MessageErrorHandlerResponse.Retry : MessageErrorHandlerResponse.Failed);
}
}using Amazon.DynamoDBv2;
using AWS.Messaging;
using Harbourline.Notifications.Core.Idempotency;
using Harbourline.Notifications.Core.Processing;
using Harbourline.Notifications.Worker;
var builder = Host.CreateApplicationBuilder(args);
builder.Services.AddSingleton<IAmazonDynamoDB>(_ => new AmazonDynamoDBClient());
builder.Services.AddSingleton<IIdempotencyStore>(sp => new DynamoDbIdempotencyStore(
sp.GetRequiredService<IAmazonDynamoDB>(), builder.Configuration["IdempotencyTable"]!, TimeProvider.System));
builder.Services.AddSingleton<INotificationSender, FakeNotificationSender>();
builder.Services.AddAWSMessageBus(bus =>
{
bus.AddSQSPoller(builder.Configuration["Messaging:NotifyQueueUrl"]!, options =>
{
options.MaxNumberOfConcurrentMessages = 5; // five handlers at once on this worker
options.WaitTimeSeconds = 20; // long polling
options.VisibilityTimeout = 120; // hide a received message for 2 minutes...
options.VisibilityTimeoutExtensionThreshold = 10; // ...and extend it when 10 seconds remain
});
// The string is the message TYPE IDENTIFIER: it must equal the "type" the publisher writes.
bus.AddMessageHandler<SendNotificationHandler, SendNotification>("hl.notification.send.v1");
bus.AddHandlerMiddleware<CorrelationMiddleware>();
bus.AddMessageErrorHandler<PoisonAwareErrorHandler>();
bus.AddMessageSource("/harbourline/notifications-worker");
bus.ConfigureBackoffPolicy(policy => policy.UseCappedExponentialBackoff());
});
await builder.Build().RunAsync();<Project Sdk="Microsoft.NET.Sdk.Worker">
<ItemGroup>
<PackageReference Include="AWS.Messaging" Version="1.6.0" />
<PackageReference Include="AWSSDK.DynamoDBv2" Version="4.1.0.2" />
<ProjectReference Include="..\Harbourline.Notifications.Core\Harbourline.Notifications.Core.csproj" />
</ItemGroup>
</Project>The publisher
using AWS.Messaging;
using AWS.Messaging.Publishers.SQS;
using Harbourline.Payments.Messages;
var builder = WebApplication.CreateBuilder(args);
builder.Services.AddAWSMessageBus(bus =>
{
// Events go to an EventBridge bus (the framework accepts the bus ARN).
bus.AddEventBridgePublisher<PaymentSettled>(
builder.Configuration["Messaging:EventBusArn"]!, "hl.payment.settled.v1");
// Commands go to one SQS queue. This one is a FIFO queue, so a group ID is required when publishing.
bus.AddSQSPublisher<SendNotification>(
builder.Configuration["Messaging:NotifyFifoQueueUrl"]!, "hl.notification.send.v1");
bus.AddMessageSource("/harbourline/payments-api");
});
var app = builder.Build();
// Generic publisher: the framework looks up the destination from the message's .NET type.
app.MapPost("/payments/{paymentId}/settle", async (string paymentId, IMessagePublisher publisher, CancellationToken ct) =>
{
var settled = new PaymentSettled(paymentId, "ACC-0000000001", "CUS-00000001", 1250.00m, "GBP", DateTimeOffset.UtcNow);
var response = await publisher.PublishAsync(settled, ct);
return Results.Accepted(value: new { messageId = response.MessageId });
});
// Service-specific publisher: exposes SQS options such as FIFO group and deduplication IDs.
app.MapPost("/notify/{transactionId}", async (string transactionId, ISQSPublisher sqs, CancellationToken ct) =>
{
var command = new SendNotification(transactionId, "CUS-00000001", "sms", "Harbourline: credit of GBP 12.50 on account ending 0001.");
var response = await sqs.SendAsync(command, new SQSOptions
{
MessageGroupId = "CUS-00000001", // ordering scope: one customer
MessageDeduplicationId = $"notify-sms-{transactionId}", // 5-minute SQS FIFO dedup window
}, ct);
return Results.Accepted(value: new { messageId = response.MessageId });
});
app.Run();
namespace Harbourline.Payments.Messages
{
public sealed record PaymentSettled(string PaymentId, string AccountId, string CustomerId, decimal Amount, string Currency, DateTimeOffset SettledAt);
public sealed record SendNotification(string TransactionId, string CustomerId, string Channel, string Text);
}Notice that the publisher's SendNotification and the consumer's SendNotification are different .NET types in different namespaces. They agree on the string hl.notification.send.v1 and on the JSON property names, and nothing else. That is what lets the two teams deploy separately.
Hosting the same handler in Lambda
With the framework's Lambda package, a function hosts the same IMessageHandler<T> classes. The Lambda polls the queue (an event source mapping); the framework deserialises each record, calls the handler and builds the SQSBatchResponse.
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<AssemblyName>Harbourline.Notifications.Lambda</AssemblyName>
<AWSProjectType>Lambda</AWSProjectType>
<GenerateRuntimeConfigurationFiles>true</GenerateRuntimeConfigurationFiles>
<OutputType>Library</OutputType>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Amazon.Lambda.Annotations" Version="2.4.2" />
<PackageReference Include="Amazon.Lambda.Core" Version="3.4.0" />
<PackageReference Include="Amazon.Lambda.SQSEvents" Version="3.0.1" />
<PackageReference Include="Amazon.Lambda.Serialization.SystemTextJson" Version="3.0.1" />
<PackageReference Include="AWS.Messaging.Lambda" Version="1.0.3" />
<PackageReference Include="Microsoft.Extensions.Hosting" Version="10.0.15" />
<PackageReference Include="AWSSDK.DynamoDBv2" Version="4.1.0.2" />
<ProjectReference Include="..\Harbourline.Notifications.Core\Harbourline.Notifications.Core.csproj" />
<ProjectReference Include="..\Worker\Worker.csproj" />
</ItemGroup>
</Project>using Amazon.DynamoDBv2;
using Amazon.Lambda.Annotations;
using Amazon.Lambda.Core;
using Amazon.Lambda.SQSEvents;
using AWS.Messaging;
using AWS.Messaging.Lambda;
using Harbourline.Notifications.Core.Idempotency;
using Harbourline.Notifications.Core.Processing;
using Harbourline.Notifications.Worker;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Hosting;
[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]
namespace Harbourline.Notifications.Lambda;
[LambdaStartup]
public class Startup
{
public HostApplicationBuilder ConfigureHostBuilder()
{
var hostBuilder = new HostApplicationBuilder();
hostBuilder.Services.AddSingleton<IAmazonDynamoDB>(_ => new AmazonDynamoDBClient());
hostBuilder.Services.AddSingleton<IIdempotencyStore>(sp => new DynamoDbIdempotencyStore(
sp.GetRequiredService<IAmazonDynamoDB>(),
Environment.GetEnvironmentVariable("IDEMPOTENCY_TABLE")!, TimeProvider.System));
hostBuilder.Services.AddSingleton<INotificationSender, FakeNotificationSender>();
hostBuilder.Services.AddAWSMessageBus(bus =>
{
bus.AddMessageHandler<SendNotificationHandler, SendNotification>("hl.notification.send.v1");
bus.AddLambdaMessageProcessor(options =>
{
options.MaxNumberOfConcurrentMessages = 4;
});
});
return hostBuilder;
}
}
public class Function
{
[LambdaFunction]
public async Task<SQSBatchResponse> FunctionHandler(
[FromServices] ILambdaMessaging messaging, SQSEvent sqsEvent, ILambdaContext context)
{
// Returns only the failed message IDs, so Lambda re-queues just those.
// Requires FunctionResponseTypes = ReportBatchItemFailures on the event source mapping (see the IaC above).
return await messaging.ProcessLambdaEventWithBatchResponseAsync(sqsEvent, context);
}
}The Lambda Annotations source generator produces the entry point Harbourline.Notifications.Lambda::Harbourline.Notifications.Lambda.Function_FunctionHandler_Generated::FunctionHandler, which is the handler string used in the CDK code. Build the package with dotnet publish -c Release -r linux-arm64 --self-contained false -o publish/lambda.
Consuming raw EventBridge or SNS events without the envelope
The module project's Lambda functions receive raw events that EventBridge delivered through SNS, not messages published by this framework. For such a queue, turn envelope handling off. The poller then deserialises the body directly into one type, so each such queue holds one message type:
using System.Text.Json;
using System.Text.Json.Serialization;
using AWS.Messaging;
using AWS.Messaging.Configuration;
public sealed record RawTransactionPosted
{
[JsonPropertyName("detail-type")] public string? DetailType { get; init; }
[JsonPropertyName("detail")] public JsonElement Detail { get; init; }
}
public sealed class RawTransactionPostedHandler : IMessageHandler<RawTransactionPosted>
{
public Task<MessageProcessStatus> HandleAsync(MessageEnvelope<RawTransactionPosted> envelope, CancellationToken token = default)
=> Task.FromResult(MessageProcessStatus.Success());
}
// In AddAWSMessageBus(...):
// bus.AddSQSPoller<RawTransactionPosted>(rawQueueUrl, messageEnvelopeMode: MessageEnvelopeMode.NotSupported);
// bus.AddMessageHandler<RawTransactionPostedHandler, RawTransactionPosted>();Unit tests
using AWS.Messaging;
using Harbourline.Notifications.Core.Idempotency;
using Harbourline.Notifications.Core.Processing;
using Harbourline.Notifications.Worker;
using Microsoft.Extensions.Logging.Abstractions;
namespace Harbourline.Notifications.Tests;
public sealed class SendNotificationHandlerTests
{
private static MessageEnvelope<SendNotification> Envelope(string transactionId = "T-1") => new()
{
Id = Guid.NewGuid().ToString(),
Source = new Uri("/test", UriKind.Relative),
TimeStamp = DateTimeOffset.UtcNow,
Message = new SendNotification
{
TransactionId = transactionId,
CustomerId = "CUS-00000001",
Channel = "sms",
Text = "Harbourline: credit of GBP 12.50 on account ending 0001.",
},
};
private static (SendNotificationHandler Handler, FakeNotificationSender Sender) Build()
{
var sender = new FakeNotificationSender();
var store = new InMemoryIdempotencyStore(TimeProvider.System);
return (new SendNotificationHandler(store, sender, NullLogger<SendNotificationHandler>.Instance), sender);
}
[Fact]
public async Task Duplicate_delivery_sends_once_and_both_deliveries_succeed()
{
var (handler, sender) = Build();
var first = await handler.HandleAsync(Envelope("T-1"), TestContext.Current.CancellationToken);
var second = await handler.HandleAsync(Envelope("T-1"), TestContext.Current.CancellationToken); // same transaction, new envelope id
Assert.Equal(MessageProcessStatus.Success(), first);
Assert.Equal(MessageProcessStatus.Success(), second);
Assert.Single(sender.Sent);
}
[Fact]
public async Task Provider_outage_returns_Failed_and_the_next_delivery_can_succeed()
{
var (handler, sender) = Build();
sender.OutageActive = true;
var failed = await handler.HandleAsync(Envelope("T-2"), TestContext.Current.CancellationToken);
sender.OutageActive = false;
var retried = await handler.HandleAsync(Envelope("T-2"), TestContext.Current.CancellationToken);
Assert.Equal(MessageProcessStatus.Failed(), failed);
Assert.Equal(MessageProcessStatus.Success(), retried);
Assert.Single(sender.Sent);
}
}Run them with dotnet test --project Tests (xUnit v3 on Microsoft.Testing.Platform). Expected: total: 2, failed: 0, succeeded: 2. The test project is an executable (OutputType Exe) that references xunit.v3 4.0.3 and Microsoft.Extensions.Logging.Abstractions, as in the module project.
Code explained line by line
AddAWSMessageBus(bus => { ... }): the single entry point. Everything (publishers, pollers, handlers, middleware) is registered on theMessageBusBuilderand ends up in the DI container.AddEventBridgePublisher<PaymentSettled>(arn, "hl.payment.settled.v1"): "when someone publishes aPaymentSettled, send it to this bus with type identifierhl.payment.settled.v1". The README example passes the bus ARN.AddSQSPublisher<SendNotification>(queueUrl, "hl.notification.send.v1"): the same for a queue. A message type is tied to one destination; to send the same type to two destinations you use the service-specific publisher interfaces.IMessagePublisher.PublishAsync(settled, ct): looks up the destination by the object's type, builds the envelope, sends it, and returns a response with aMessageId. For SQS,MessageIdis the SQS message ID.SQSOptions { MessageGroupId, MessageDeduplicationId }: required for FIFO queues. The group ID is the ordering scope (all messages for one customer are processed in order). The deduplication ID makes a publisher retry harmless within SQS's five-minute window (Delivery guarantees).AddSQSPoller(url, options => ...): registers aBackgroundServicethat long-polls the queue.MaxNumberOfConcurrentMessages(default 10) bounds concurrency on this worker;WaitTimeSeconds(default 20) is the long-poll duration.VisibilityTimeout,VisibilityTimeoutExtensionThreshold: the poller receives messages with this visibility timeout (default 30 s) and, when a message being handled is within the threshold of reappearing (default 5 s), extends it by anotherVisibilityTimeout. A handler that takes 90 seconds therefore does not cause a duplicate delivery.AddMessageHandler<SendNotificationHandler, SendNotification>("hl.notification.send.v1"): maps the type identifier to the handler and the message type. Handlers are created per message in a DI scope, so they can take scoped dependencies such as aDbContext.HandleAsync(MessageEnvelope<SendNotification> envelope, ...):envelope.Messageis your object;envelope.Id,Source,TimeStampand other metadata come from the CloudEvents envelope.var key = $"notify#{command.Channel}#{command.TransactionId}": the idempotency key is built from business fields. Usingenvelope.Idwould be wrong: every publish creates a new ID, so a publisher retry would look like a new message.TryClaimAsync(...): a conditionalPutItemin DynamoDB.AlreadyCompletedmeans this notification was sent; the handler returnsSuccess()so the framework deletes the duplicate.InProgressmeans another worker is mid-send; returningFailed()leaves the message to be retried after the visibility timeout.catch (TransientDownstreamException)→ReleaseAsync→Failed(): a short, expected failure (the provider is down). The handler gives the claim back so the next delivery can take it, and returns a status instead of throwing. Using the status for expected problems and exceptions for unexpected ones keeps your logs and error handler meaningful.CompleteAsync(key, 7 days): records success. If the process dies between the send and this call, the claim expires and the notification can be sent twice; that is the accepted window for a side effect that has no transaction (Idempotency in .NET).IHandlerMiddleware.InvokeAsync<T>(envelope, next, token): wraps the handler. Callawait next()to continue the pipeline and return its status. Middleware is registered as a singleton by default.IMessageErrorHandler.OnHandleError<T>(envelope, exception, attempt, token): called when a handler throws.Retryruns the middleware and handler again in a new DI scope;Failedleaves the message on the queue for SQS to redeliver;Successdeletes it. ReturningSuccessafter an exception discards the message, so use it only for a deliberate "drop and log" decision.ConfigureBackoffPolicy(policy => policy.UseCappedExponentialBackoff()): governs the poller's own backoff when polling itself errors (for example, throttling or a permissions problem). It is not a per-message retry policy.AddLambdaMessageProcessor(...)andProcessLambdaEventWithBatchResponseAsync(sqsEvent, context): the Lambda equivalent of the poller. The function returns the SQS batch response so only failed messages return to the queue. WithoutReportBatchItemFailureson the event source mapping, Lambda ignores the response body and retries the whole batch on an error, so set it.MessageEnvelopeMode.NotSupported: tells the poller that these messages carry no framework envelope, so it deserialises the body straight into the one registered type.- The test constructs
MessageEnvelope<SendNotification>with an object initialiser. It callsHandleAsynctwice with different envelope IDs and the same transaction ID, which is exactly what a publisher retry looks like, and asserts that one notification went out.
Security considerations
⚠ Learning shortcut — not for production
The sample uses FakeNotificationSender (it never contacts an SMS provider), a placeholder source name, and a queue protected only by SQS-managed encryption. Production: a real provider behind an adapter with its own secret in AWS Secrets Manager, customer managed KMS keys if your policy requires them (the queue, DynamoDB table and event bus each need key-policy changes), private networking through VPC endpoints, and alarms on queue age and DLQ depth.
- Least privilege per role. The worker needs
sqs:ReceiveMessage,sqs:DeleteMessage,sqs:ChangeMessageVisibilityandsqs:GetQueueAttributeson one queue; the publisher needsevents:PutEventsorsqs:SendMessageon one target. Do not usesqs:*orResource: "*". - Do not log payloads. The framework does not log message data by default and offers
EnableDataMessageLogging()for debugging. Treat that call as a production incident if it ships; messages hold personal data. - Validate on the consumer side. A message from a queue is untrusted input. Validate it, as the module project's processor does (currency code, ID formats), and return
Failed()or throw for messages that can never succeed so they reach the DLQ. Do not silently returnSuccess()for bad data. - Type identifiers are routing, not authorisation. Anyone who can send to the queue can claim any
type. Restrictsqs:SendMessageto the intended publisher roles, and use queue policies withaws:SourceArnconditions as in the Terraform above. - Poison messages. Keep
maxReceiveCountand the DLQ. A handler that always throws will otherwise spin on the same message until retention expires. - Compliance. What may appear in a message body, how long queues and archives may retain it, and which Regions may hold it are decisions for your legal, data-protection and security teams. Nothing here is a compliance claim.
Banking use case
Harbourline's Payments service publishes PaymentSettled to the event bus. A rule routes it to the Notifications queue; the worker's handler turns it into an SMS and an e-mail request. Because the handler is a class with one dependency list, the team can run it in a Worker Service on Fargate during business hours (where a steady flow justifies a running container) and in Lambda for a lower-volume region, with the same handler code and the same idempotency table.
What the framework contributed: no hand-written polling loop; correct visibility extension for the slow e-mail provider; one place (the middleware) to attach a correlation ID that appears in every log line. What the team still wrote: the idempotency key, the claim and release logic, the decision of which failures are Failed() and which are poison, the alarm on DLQ depth, and the runbook.
Common mistakes
- Assuming the framework deduplicates. It does not. Duplicates arrive from SQS standard queues, EventBridge and publisher retries. Add the idempotency claim in the handler.
- Using
envelope.Idas the idempotency key. It changes on every publish. Key on the business identity. - A typo in the type identifier. No handler matches, so the message fails until it lands in the DLQ. Keep identifiers in a shared constants file or an integration test.
- Throwing for an expected failure. A downstream outage becomes a stream of exceptions and error-handler retries. Return
Failed()for expected, retryable conditions. - Returning
Success()for bad data. The message is deleted and the problem is invisible. Fail it so it reaches the DLQ, or log and alert deliberately. - Forgetting
ReportBatchItemFailuresfor Lambda. The function returns a batch response, but Lambda ignores it, and a single bad message makes the whole batch return to the queue. - Visibility timeout shorter than the handler, with no extension. Rare with the framework (it extends), but a Lambda event source mapping has no such heartbeat: set the queue visibility timeout to at least six times the function timeout.
- Publishing with the framework and consuming with a raw consumer (or the reverse). The consumer sees the envelope as part of the body, or the poller expects an envelope that is not there. Decide the envelope mode per queue.
- Calling
PublishAsyncafterSaveChangesAsyncand calling it reliable. The process can die between them. Use the outbox. - Enabling data-message logging "just for a minute". Personal data lands in log storage. Use test data and remove it before release.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| Messages arrive but the handler never runs; they reach the DLQ | Type identifier on the consumer does not match the envelope type | Peek at a message (--visibility-timeout 0), compare type, and fix AddMessageHandler's identifier |
AccessDenied on ChangeMessageVisibility in worker logs | Worker policy lacks that action | Add sqs:ChangeMessageVisibility on the queue |
| Same message handled by two workers | Visibility timeout expired and the extension is not reaching SQS (permissions or a very long pause) | Fix the permission; check the handler is not blocked on a long synchronous call; raise VisibilityTimeout |
Handler gets null or default fields | Property names differ between publisher and consumer types | Align JSON property names or set the same ConfigureSerializationOptions on both sides |
| The whole Lambda batch retries after one bad message | ReportBatchItemFailures not set on the event source mapping | Set FunctionResponseTypes to ReportBatchItemFailures (CDK: ReportBatchItemFailures = true) |
| FIFO publish fails with a missing group ID error | MessageGroupId not set | Pass SQSOptions { MessageGroupId = ... } through ISQSPublisher |
| Raw events fail to deserialise | Poller is in envelope mode but the body is a raw EventBridge event | Use MessageEnvelopeMode.NotSupported on a queue that holds one raw message type |
| Test fails to compile with xUnit1051 | Missing cancellation token in an async test call | Pass TestContext.Current.CancellationToken |
| Nothing logged for the message data | Intended: payload logging is off by default | Log identifiers and outcomes; do not enable data logging in production |
Cost and cleanup
Cost label: 🟢 free. The handler tests run offline. Deploying the stack at lab volumes (a few dozen messages) stays inside the always-free allowances of SQS, EventBridge custom events, DynamoDB on-demand and Lambda at this volume ⚠ VERIFY current allowances; the worst case is a few cents if you leave a poller running for a day (long polling makes a small number of SQS requests per hour).
Teardown:
# CDK
cd infra/cdk && cdk destroy HlM08-AwsMessaging --profile learn-sandbox
# Terraform
cd infra/terraform && terraform destroy -auto-approve
# CLI route, if you created the queues by hand
aws sqs delete-queue --profile learn-sandbox \
--queue-url "$(aws sqs get-queue-url --queue-name hl-m08-msg-notify --profile learn-sandbox --query QueueUrl --output text)"
aws sqs delete-queue --profile learn-sandbox \
--queue-url "$(aws sqs get-queue-url --queue-name hl-m08-msg-notify-dlq --profile learn-sandbox --query QueueUrl --output text)"
# Confirm nothing tagged for this lab remains (tag data can lag a few minutes)
aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=cea-lab Key=Lab,Values=m08-aws-messaging \
--query "ResourceTagMappingList[].ResourceARN" --output text --profile learn-sandboxQueues created by hand are not tagged; delete them by name. A deleted SQS queue name cannot be reused for 60 seconds ⚠ VERIFY.
Exercises
- 🟢 Guided: Run the two tests. Then add a third that sends the same transaction on the
emailchannel and asserts that both channels send once each (the idempotency key includes the channel). Finally, break the key on purpose (remove the channel) and watch the test fail. - 🟡 Challenge: Add a validation step: a message whose
Textis empty can never succeed. Make the handler throwArgumentException, register an error handler that returnsFailedimmediately forArgumentException(no in-process retries) andRetryfor transient exceptions, and write unit tests for the error handler's decisions. - 🔴 Live problem: After a deployment, the Notifications queue backs up and the DLQ fills with messages that look fine. The handler logs
No handler found for type 'Harbourline.Payments.Messages.PaymentSettled'. Payments upgraded the publisher library this morning. Find the cause, explain why the queue still accepted the messages, fix it for both teams without a coordinated deployment, and redrive the DLQ safely.
Assignment
Build a small portfolio repository aws-messaging-handlers with the publisher, the worker handler (using the idempotency store from the module project) and the tests. Add a README that explains: the type identifier convention, what the handler returns for each failure class (duplicate, in progress, transient, poison), and the IAM policies each role needs.
Solutions
Add a test that calls the handler for T-3 on the sms channel and again on the email channel (a second envelope with Channel = "email"). Both succeed and sender.Sent has two entries, because the keys are notify#sms#T-3 and notify#email#T-3. If you drop the channel from the key, the second call is treated as a duplicate and only one notification is sent, so the assertion Assert.Equal(2, sender.Sent.Count) fails. The lesson: the idempotency key must include every dimension that makes the side effect distinct: here, transaction and channel.
In the handler, throw ArgumentException when Text is empty. The error handler checks exception is ArgumentException and returns MessageErrorHandlerResponse.Failed, otherwise attempt < 3 ? Retry : Failed. Retrying an ArgumentException can never help, so failing immediately saves three pointless attempts and moves the message toward the DLQ faster (the DLQ still needs maxReceiveCount receives, so the message reappears until then). Test the decisions by calling OnHandleError directly with a MessageEnvelope<SendNotification>, an exception and an attempt number, and assert on the returned response.
The handler on the consumer is registered under the type identifier hl.payment.settled.v1, but the new publisher build registered the event without an explicit identifier, so the framework used the default, the full .NET type name (Harbourline.Payments.Messages.PaymentSettled). SQS accepts any body, so the messages were delivered and then failed in the handler lookup. Fix: on the publisher, pass "hl.payment.settled.v1" explicitly in AddEventBridgePublisher. To avoid a coordinated deployment, the consumer can additionally register the .NET-name identifier for the same handler for a transition period (a second AddMessageHandler call). Once the publisher is fixed and the DLQ cause is gone, redrive the DLQ to the source queue at a low rate (aws sqs start-message-move-task); the idempotency store makes a redrive safe. Prevent recurrence with a test that asserts both sides use the same constant, and a contract test on the identifier.
Knowledge check
Answer: It stays on the queue and becomes visible again after its visibility timeout, so it is redelivered. After maxReceiveCount receives, SQS moves it to the DLQ if one is configured. Success() deletes it. The framework does not move messages to the DLQ itself; that is the queue's redrive policy.
Answer: No. Delivery from SQS standard queues, EventBridge and SNS is at least once, and a publisher can retry. You must make the handler idempotent, for example with a DynamoDB conditional write keyed on the business identity.
Answer: The default is the full .NET type name, which couples publisher and consumer to the same class and namespace. A stable string lets each side define its own type, carries a version, and survives refactoring.
Answer: Inside the data field of a CloudEvents-style JSON envelope. That envelope JSON is the SQS MessageBody, the SNS Message or the EventBridge Detail.
Answer: ReportBatchItemFailures on the event source mapping. Without it, Lambda does not read the batch response, and an error retries the entire batch.
Answer: For a queue that receives raw events not published by this framework (for example a raw EventBridge or S3 event). The poller deserialises the body straight into the one registered type, so each such queue carries one message type.
Interview questions
- "What does AWS.Messaging give you that a hand-written SQS loop does not?" — Long polling with bounded concurrency, visibility-timeout extension, typed deserialisation and handler lookup, delete-on-success, backoff on polling errors, and one publishing API for SQS, SNS and EventBridge. It does not give idempotency, an outbox or orchestration.
- "How does a handler signal success or failure?" — By returning
MessageProcessStatus.Success()(the framework deletes the message) orFailed()(it stays for redelivery and eventually the DLQ). Unexpected exceptions go to the error handler, which can retry in a new DI scope, fail or drop. - "How would you make the handler safe against duplicates?" — A conditional write on a durable store keyed by a business identity (transaction plus channel), with an in-progress state and expiry so a crashed worker's claim can be taken over, and a completed state kept longer than the maximum redelivery gap.
- "Worker Service or Lambda for the consumer?" — Lambda for spiky or low volume (no idle cost, built-in scaling, partial batch responses); a worker on Fargate for a steady flow, long handlers or special runtime needs. The handler class can be the same.
- "How do you evolve a message?" — Add optional fields freely (tolerant readers); for a breaking change publish a new type identifier (
.v2), run both, migrate consumers and then retire the old identifier.
Certification practice
A Lambda function processes batches of 10 messages from an SQS standard queue. When one message fails, all 10 are processed again, causing duplicate side effects for the nine that succeeded. The function returns a response listing the failed message IDs. What must the developer do so only the failed message is retried?
A. Reduce the batch size to 1 and increase the function timeout.
B. Enable ReportBatchItemFailures in the event source mapping's function response types.
C. Change the queue to FIFO and delete messages manually.
D. Configure a longer visibility timeout on the queue.
Answer: B. Lambda only honours the returned batch item failures when the event source mapping has ReportBatchItemFailures enabled. A reduces the symptom but costs throughput. C is unnecessary, and manual deletion adds complexity. D affects how soon retries happen, not which messages are retried.
A worker polls an SQS standard queue. A message takes 90 seconds to process, but the queue's visibility timeout is 30 seconds, and the worker does not change it. Customers receive two notifications. What is the BEST fix?
A. Switch to short polling. B. Extend the visibility timeout while processing (or set it above the maximum processing time) and make the handler idempotent. C. Delete the message before processing starts. D. Increase the queue's maximum message size.
Answer: B. The message became visible again while the first worker was still processing, so a second worker received it. Extending the timeout fixes the cause; idempotency covers the unavoidable remaining duplicates. A changes polling cost, not duplicates. C risks losing the message if the worker crashes. D is irrelevant.
An application publishes events to one Amazon EventBridge bus. Which IAM statement is MOST appropriate for its task role?
A. events:* on *.
B. events:PutEvents on the ARN of that event bus.
C. events:PutEvents on *.
D. sqs:SendMessage on *.
Answer: B. Allow only the action needed, on only the resource needed. A and C are broader than necessary, and D is for a different service.
Completion checklist
- I can explain what the framework handles and what it leaves to me
- I can say how
Success(),Failed()and a thrown exception each affect the SQS message - I ran the handler tests and watched the duplicate test pass
- I can explain why the idempotency key uses business fields, not the envelope ID
- I know when to use the Lambda hosting and why
ReportBatchItemFailuresmatters - I ran the cleanup and confirmed no queues, buses or tables remain
Key takeaways
AWS.Messagingmoves plumbing (polling, visibility extension, serialisation, delete-on-success, publishing) into a library, so your code is the handler.- A stable string type identifier routes messages and decouples publisher and consumer types; put the version in it.
- Handlers return
Success()orFailed(); exceptions go to the error handler; the DLQ comes from the queue's redrive policy. - You still own idempotency, the outbox and sagas. Key the idempotency record on the business identity, never on the envelope ID.
- The same handler can run in a Worker Service (poller) or in Lambda (message processor with batch item failures).
Further reading
- AWS Message Processing Framework for .NET (SDK developer guide)
- Consume messages with the framework
- Customize the framework (envelope, middleware, serialisation)
- Security for the framework
- aws/aws-dotnet-messaging on GitHub
- Back to the module overview · Previous: Microservices communication and service discovery · Module project: Banking transaction notification system
Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.
Microservices communication and service discovery
Draw bounded contexts for a bank, choose between a query, a command and an event for every interaction, make synchronous calls safe with timeouts and retries, and let services find each other with Amazon ECS Service Connect and AWS Cloud Map.
Module project: banking transaction notification system
Build, break and repair a notification system: EventBridge to SNS to SQS to Lambda in .NET, with idempotent handlers, retries with backoff, a poison message in a DLQ, redrive after a fix and recovery from a downstream outage, using the m08-notification-system lab.