Amazon MQ: bridging legacy JMS and AMQP systems
Know when a managed ActiveMQ or RabbitMQ broker is the right tool: keep a JMS/AMQP system running on Amazon MQ, bridge it to SQS, SNS and EventBridge with a Lambda event source mapping and a translator, handle poison messages, and plan the migration, with .NET 10 clients, CDK and Terraform.
Assumes: .NET 10 LTS · AWS SDK for .NET v4 · Amazon.Lambda.MQEvents 3.0.0 · Apache.NMS.ActiveMQ 2.3.0 · RabbitMQ.Client 7.2.2 · Amazon.CDK.Lib 2.277.0 · Terraform AWS provider v6 · Region us-east-1. Verified against official sources on 2026-11-18.
Learning objectives
By the end of this lesson you can:
- Explain what Amazon MQ is (managed Apache ActiveMQ and RabbitMQ), which protocols it speaks, and why it exists next to SQS and SNS.
- Decide, for a given system, whether to keep a broker (Amazon MQ), bridge it to native services, or replace it, using a clear decision rule.
- Describe the deployment modes and what they mean for availability, maintenance windows and cost.
- Bridge an ActiveMQ queue to EventBridge with a Lambda event source mapping, a translator (anti-corruption layer), idempotency and a quarantine queue for poison messages, and know that mapping's restrictions.
- Write a .NET 10 producer for ActiveMQ (OpenWire/NMS) and a consumer for RabbitMQ (AMQP 0-9-1) with correct acknowledgement behaviour.
- Secure a broker: private networking, TLS, broker users in Secrets Manager, logging, and the limits of IAM.
- Cost a broker honestly, and use a free local alternative (Docker) for development.
Prerequisites
- Asynchronous communication and event-driven architecture and Amazon SQS in depth
- Amazon EventBridge: buses, rules and Pipes: Pipes can read an MQ broker
- Idempotency in .NET and Delivery guarantees and ordering
- Module 2: Networking: private subnets and security groups (a broker lives in your VPC)
- Tools: .NET 10 SDK or Docker, AWS CLI v2 with the
learn-sandboxSSO profile. A deployed broker is optional and costs money
Why this matters in banking
Harbourline's card processor, a vendor package installed in 2011, publishes settlement files as JMS text messages over OpenWire to an ActiveMQ broker in a data centre. The same broker feeds a reconciliation job and a regulatory reporting extract. The vendor will not change the integration, the contract runs for three more years, and the reconciliation job's authors have left.
You cannot point that processor at SQS: SQS has no JMS provider, no OpenWire and no AMQP. Rewriting the processor's side is not on offer. But the rest of the bank wants those settlements as events on the new platform, and the data centre is closing next year.
Amazon MQ is the bridge for exactly this. You move the broker into AWS unchanged (same protocols, same clients, same queue names), so the legacy applications keep working. Then you extract the events into native services at the edge, and migrate consumers one at a time until the broker has no users. This is the strangler pattern with a managed broker as the first stone.
The traps are specific: the bridge delivers at least once and one bad message can block a queue, a single-instance broker goes offline during its weekly maintenance window, brokers are billed by the hour whether or not a message flows, and IAM does not authenticate legacy clients to the broker.
The idea in plain language
A message broker is a server that holds queues and topics. Producers send; the broker stores; consumers receive and acknowledge. It is the same idea as SQS and SNS, with an older, richer contract:
- JMS (Java Message Service) is an API. A JMS client talks to the broker through a wire protocol. For ActiveMQ the native wire protocol is OpenWire; AMQP, STOMP, MQTT and WebSocket are also available. The .NET sibling of JMS is NMS.
- RabbitMQ speaks AMQP 0-9-1 natively, with exchanges, bindings and queues, and other protocols through plugins.
- Clients keep a connection to the broker, and acknowledge messages explicitly. Delivery can be persistent (survives a broker restart) or not.
Analogy: SQS is the post room's pigeonholes: public, simple, run by the building, with its own rules. A message broker is the old pneumatic-tube network between the branch and head office. Some systems were built to use its tubes and its brass canisters, and they work. You cannot swap the tube for a pigeonhole without rebuilding the machine at each end. Amazon MQ is a contractor who keeps the tube network running in the new building, so you can move the machines one at a time.
What Amazon MQ solves
Amazon MQ is a managed message broker service for Apache ActiveMQ and RabbitMQ. AWS provisions the broker instances, storage and multi-AZ failover, patches the engine in a maintenance window, and publishes metrics and logs. You keep the broker's native protocols and clients, so migration of existing applications is mostly a connection-string change.
It exists for compatibility, not for new design. For new applications on AWS, SQS, SNS and EventBridge are simpler, cheaper and have no broker to size.
Engines, versions and deployment modes (checked 2026-11-18)
| ActiveMQ | RabbitMQ | |
|---|---|---|
| Versions | 5.19 (recommended) and 5.18; older versions have reached end of support and are upgraded automatically | RabbitMQ 4.3 and 4.2 (only on mq.m7g instances); RabbitMQ 3.13 on mq.t3, mq.m5 and mq.m7g |
| Deployment modes | Single instance (one broker, one AZ), active/standby (a standby in a second AZ takes over) | Single instance, cluster (three-node, across AZs) |
| Protocols | OpenWire (JMS), AMQP, STOMP, MQTT, WebSocket | AMQP 0-9-1, and others through plugins ⚠ VERIFY the supported protocol list |
| Not supported | RabbitMQ streams (creating one causes data loss) and structured JSON logging | |
| Storage | EFS (durability optimised) or EBS (throughput optimised) | EBS |
Use aws mq describe-broker-engine-types --engine-type ACTIVEMQ (or RABBITMQ) for the current list in your Region, and describe-broker-instance-options for instance types.
Limits (checked 2026-11-18)
| Setting | Value |
|---|---|
| Brokers per Region | 200 |
| ActiveMQ wire-level connections per broker | 300 on mq.*.micro, 2,000 on larger instances, per protocol |
| RabbitMQ connections | Per node, by instance type, not overridable |
| ActiveMQ storage | 20 GB on micro, 200 GB on mq.m5.* |
| Users per ActiveMQ broker (simple auth) | 250; passwords 12 to 250 characters with at least 4 unique characters |
| Security groups per broker | 5 |
| CloudWatch destinations monitored | The first 1,000 (ActiveMQ) or 500 (RabbitMQ) |
| Management API throttling | Burst 100, rate 15 per second per account. Does not throttle messaging |
Honest limitations
- You pay by the hour, always. A broker has no free idle state.
- Single-instance is not highly available, and it is unavailable during the weekly maintenance window. Lambda cannot consume from it then.
- IAM does not authenticate clients to the broker. Clients use broker-level users (ActiveMQ simple authentication or LDAP; RabbitMQ users). IAM only controls the management API.
- You are still operating a broker's semantics: persistence modes, redelivery policies, dead-letter queues, consumer prefetch, virtual topics.
- Throughput is bounded by the instance, not elastic.
- Version churn. Engine versions reach end of support; AWS upgrades brokers automatically in a maintenance window, so test your clients against the next version.
When to keep, bridge or replace
| Option | When | Cost | Risk |
|---|---|---|---|
| Replace the broker | You own both ends and can change them | Project effort | Highest effort, lowest long-term cost |
| Keep on Amazon MQ | A vendor or frozen system speaks JMS/AMQP/MQTT | Hourly broker cost | Compatibility is high; you run a broker's semantics |
| Bridge to native | The rest of the platform wants events, but the producer cannot change | Broker + a small Lambda | Duplicates and poison messages must be designed for |
| Self-managed on EC2/ECS | Only when you need an engine or a plugin Amazon MQ forbids | Compute plus operations | You patch, back up and fail over |
Microsoft / Azure comparison
| You know (Microsoft/Azure) | AWS equivalent | What's different |
|---|---|---|
| MSMQ, IBM MQ, on-premises ActiveMQ | Amazon MQ | The "lift the broker" route. IBM MQ has no Amazon MQ engine: that is a different migration |
| Azure Service Bus (AMQP 1.0, JMS 2.0 on Premium) | Amazon MQ for JMS/AMQP clients; SQS and SNS for native | Service Bus is a service with its own SDK that also speaks AMQP 1.0. Amazon MQ is a broker with the broker's protocols |
| NServiceBus, MassTransit on RabbitMQ | RabbitMQ on Amazon MQ with the same libraries | RabbitMQ.Client 7.x is async-first. MassTransit and NServiceBus connect unchanged; some use features (streams) that Amazon MQ does not support |
Service Bus peek-lock and Complete/Abandon | Acknowledge and Nack/redelivery in ActiveMQ; BasicAck/BasicNack in RabbitMQ | Acknowledgement modes are per session or channel, not per message lock |
| Service Bus dead-letter queue | ActiveMQ dead-letter queue (after the redelivery policy), RabbitMQ dead-letter exchange | You configure it; with the Lambda mapping it is not automatic (see below) |
| Service Bus topics and subscriptions | ActiveMQ virtual topics, RabbitMQ exchanges | Virtual topics behave as a topic to publishers and as queues to consumers (the form Lambda can read) |
| Azure VM running a broker | Amazon MQ (managed) vs EC2 (self-managed) | AWS patches and fails over for you, in a maintenance window you choose |
Bridging to native services
There are two managed ways to read from an Amazon MQ broker without running a consumer process.
Lambda event source mapping
Lambda polls the broker queue, batches messages, and invokes your function synchronously. For an ActiveMQ queue it consumes over OpenWire (JMS TextMessage and BytesMessage), for RabbitMQ over AMQP 0-9-1. The documented restrictions matter for the design:
| Restriction | Consequence |
|---|---|
| Concurrency: 5 concurrent executions per ActiveMQ mapping, 1 per RabbitMQ mapping | Throughput is capped. Support can raise it per mapping |
| At-least-once | Duplicates happen. Make the function idempotent |
| Same account only | No cross-account consumption |
Authentication: broker username and password from Secrets Manager (BASIC_AUTH). ActiveMQ simple authentication only; RabbitMQ PLAIN only | Create a dedicated, least-privilege broker user |
| Queues only (no topics) | Use ActiveMQ virtual topics to expose a topic as queues |
| Topology: one single-instance or active/standby broker per ActiveMQ mapping; one single-instance or cluster per RabbitMQ mapping | |
| A failed batch is retried whole until success or message expiry | One poison message can block the queue. There is no on-failure destination |
| Network: a private broker needs the function in the VPC (and an outbound path to Secrets Manager and Lambda service endpoints) | VPC endpoints or a NAT gateway |
| Maintenance window: a single-instance broker is unavailable, so nothing is consumed | Use standby for production |
| Max function timeout 14 minutes for the mapping |
The event carries eventSource (aws:mq for ActiveMQ, aws:rmq for RabbitMQ), the broker ARN, and messages (ActiveMQ) or rmqMessagesByQueue (RabbitMQ). Each ActiveMQ message has messageID, messageType, destination, redelivered, properties, and the body base64-encoded in data.
EventBridge Pipes
A Pipe can use an Amazon MQ broker as its source, filter, enrich and deliver to a target (an event bus, Step Functions, SQS and others) with no function code. Choose the mapping when you need custom translation logic (as in this lesson); choose a Pipe when a filter and a field mapping are enough ⚠ VERIFY Pipe batching and failure options for MQ sources before relying on them.
The pattern that works
- Translate the legacy message into a typed event in an anti-corruption layer. The legacy format stays inside one class.
- Key the event with a stable idempotency key (the broker message ID, or a business key in the body).
- Publish to EventBridge with retry of failed entries (the EventBridge lesson).
- Quarantine a message that cannot be translated into an encrypted SQS queue, with the broker message ID and the reason, and carry on. Alarm on that queue's depth.
- Let the batch fail (throw) only when the publish to EventBridge fails, which is a transient problem worth retrying.
Reliability inside the broker
| Concern | ActiveMQ | RabbitMQ |
|---|---|---|
| Survive a broker restart | Persistent delivery mode, durable destinations | Durable queues and persistent messages |
| Acknowledgement | Session acknowledgement mode (auto, client, transacted). With NMS: AcknowledgementMode | BasicAck after the work is done; autoAck: false |
| Poison messages | Redelivery policy, then the dead-letter queue (by default a shared ActiveMQ.DLQ; configure an individual DLQ per queue) ⚠ VERIFY the defaults for your version | BasicNack with requeue: false to a dead-letter exchange |
| Back-pressure | Producer flow control, memory and store limits | Prefetch (BasicQos) limits unacknowledged messages per consumer |
| High availability | Active/standby with shared storage | Quorum queues replicate across a cluster; classic mirrored queues are gone in RabbitMQ 4 ⚠ VERIFY |
Architecture
Security-boundary view
Step-by-step implementation
Decide: do you need a deployed broker?
For learning, you do not. Run ActiveMQ and RabbitMQ in Docker (the Cost and cleanup section has a Compose file), write the translator and the clients, and prove them with unit tests. Deploy a real broker only if you have a budget alarm and will delete it the same day.
Write the translator first
The legacy message is STL|S-1001|ACC-42|125.50|GBP. Put its parsing in one class with unit tests: valid input, missing fields, wrong type, non-positive amount, a non-numeric amount. Everything else depends on this being right and on malformed input failing loudly.
Build the bridge function
The function decodes each message, translates it, collects typed events, and publishes with the retry-only-failures publisher. A message that cannot be translated goes to the quarantine queue and is not published.
Deploy the broker and the mapping (optional, paid)
Create a private single-instance ActiveMQ broker mq.t3.micro with a generated password in Secrets Manager, in two or more subnets only if you choose active/standby. Create the event source mapping for the queue settlements with BASIC_AUTH. The function must be in the same VPC.
Send a good message and a bad one
Use the NMS publisher from a machine inside the VPC (a Cloud9 or a bastion) or through the ActiveMQ web console's Send page. Send STL|S-1001|ACC-42|125.50|GBP and garbage. Confirm one event on the bus and one message in the quarantine queue.
Break it: duplicates
Make the function throw once after publishing. Lambda retries the whole batch, so the same settlement is published twice with the same idempotency key. Show that a consumer using the key from the previous lessons ignores the second.
Delete the broker
A broker bills every hour. Run the cleanup and confirm in the console that its state is DELETION_IN_PROGRESS and then gone.
AWS Console instructions
Console → Amazon MQ → Brokers → Create brokers. Choose the engine (Apache ActiveMQ or RabbitMQ), the deployment mode (Single-instance broker for learning; Active/standby or Cluster for production), then Broker name and Broker instance type (mq.t3.micro is the cheapest for ActiveMQ). Set Username and a Password (12 to 250 characters). Under Additional settings choose Private access, your VPC, subnet(s) and security group, leave Encryption and Logs (general and audit) enabled, pick a Maintenance window you can live with, and review the estimated cost before Create. Creation takes about 15 to 25 minutes ⚠ VERIFY.
On the broker page: Connections shows the protocol endpoints (OpenWire over TLS on 61617, AMQP over TLS on 5671, MQTT over TLS on 8883, STOMP on 61614, WebSocket on 61619, web console on 8162 ⚠ VERIFY the port list for your engine version). The ActiveMQ web console lets you browse queues, send test messages and see the dead-letter queue. For RabbitMQ, the RabbitMQ web console is the equivalent.
Console → Lambda → your function → Add trigger → MQ: pick the broker, the queue name(s), the Secrets Manager secret for authentication, and batch size.
AWS CLI
export AWS_PROFILE=learn-sandbox AWS_REGION=us-east-1
# What can I create here? Engines, versions and instance types.
aws mq describe-broker-engine-types --engine-type ACTIVEMQ \
--query 'BrokerEngineTypes[].EngineVersions[].Name'
aws mq describe-broker-instance-options --engine-type ACTIVEMQ --host-instance-type mq.t3.micro \
--query 'BrokerInstanceOptions[].SupportedEngineVersions'
# Store the broker user's password in Secrets Manager first (the CDK and Terraform do this for you).
aws secretsmanager create-secret --name hl-m08gap/mq-bridge \
--secret-string "{\"username\":\"svc-bridge\",\"password\":\"$(openssl rand -base64 24 | tr -d '/+=')\"}" \
--tags Key=Project,Value=cea-lab Key=Lab,Value=m08-messaging-services
# Create a private single-instance broker (about 15 to 25 minutes). Costs money by the hour.
aws mq create-broker --broker-name hl-m08gap-legacy --engine-type ACTIVEMQ --engine-version 5.19 \
--host-instance-type mq.t3.micro --deployment-mode SINGLE_INSTANCE --no-publicly-accessible \
--auto-minor-version-upgrade --subnet-ids "$SUBNET_ID" --security-groups "$SG_ID" \
--logs General=true,Audit=true \
--users "Username=svc-bridge,Password=$BROKER_PASSWORD" \
--tags Project=cea-lab,Lab=m08-messaging-services
aws mq describe-broker --broker-id "$BROKER_ID" --query '{state:BrokerState,endpoints:BrokerInstances[].Endpoints}'
# Wire the bridge: Lambda polls the queue "settlements" with the secret as credentials.
aws lambda create-event-source-mapping --function-name hl-m08gap-mq-bridge \
--event-source-arn "$BROKER_ARN" --queues settlements --batch-size 10 \
--source-access-configurations "Type=BASIC_AUTH,URI=$SECRET_ARN"
aws lambda list-event-source-mappings --function-name hl-m08gap-mq-bridge \
--query 'EventSourceMappings[].{state:State,reason:StateTransitionReason}'
# Reboot (for a config change) and delete.
aws mq reboot-broker --broker-id "$BROKER_ID"
aws mq delete-broker --broker-id "$BROKER_ID"Infrastructure as Code
The stack looks up the default VPC, creates a security group that allows OpenWire over TLS (61617) only from inside the VPC, generates the broker credentials in Secrets Manager, creates the private single-instance broker with logs on, an encrypted quarantine queue, and the event source mapping with BASIC_AUTH. Brokers have no high-level (L2) construct, so this uses the L1 CfnBroker. The secret is passed as a dynamic reference: the password never appears in the template.
using Amazon.CDK;
using Amazon.CDK.AWS.EC2;
using Amazon.CDK.AWS.Lambda;
using Amazon.CDK.AWS.AmazonMQ;
using Amazon.CDK.AWS.SecretsManager;
using Amazon.CDK.AWS.SQS;
using Constructs;
namespace Harbourline.Messaging.Infra;
/// <summary>
/// A private, single-instance Amazon MQ for ActiveMQ broker for the legacy integration lab. SINGLE_INSTANCE is the
/// cheap, non-resilient choice for learning: production uses ACTIVE_STANDBY_MULTI_AZ and two subnets.
/// The broker password lives in Secrets Manager and is injected as a dynamic reference, never written to the template.
/// </summary>
public sealed class BrokerStack : Stack
{
public BrokerStack(Construct scope, string id, IStackProps? props = null) : base(scope, id, props)
{
var vpc = Vpc.FromLookup(this, "Vpc", new VpcLookupOptions { IsDefault = true });
var brokerSg = new SecurityGroup(this, "BrokerSg", new SecurityGroupProps
{
Vpc = vpc,
Description = "Amazon MQ broker: OpenWire over TLS from inside the VPC only",
AllowAllOutbound = false,
});
brokerSg.AddIngressRule(Peer.Ipv4(vpc.VpcCidrBlock), Port.Tcp(61617), "OpenWire+TLS from the VPC");
var credentials = new Secret(this, "BrokerCredentials", new SecretProps
{
SecretName = "hl-m08gap/mq-bridge",
GenerateSecretString = new SecretStringGenerator
{
SecretStringTemplate = """{ "username": "svc-bridge" }""",
GenerateStringKey = "password",
PasswordLength = 24,
ExcludePunctuation = true,
},
});
// Messages the bridge cannot translate land here, encrypted, instead of blocking the broker queue.
_ = new Queue(this, "Quarantine", new QueueProps
{
QueueName = "hl-m08gap-mq-quarantine",
RetentionPeriod = Duration.Days(14),
Encryption = QueueEncryption.SQS_MANAGED,
EnforceSSL = true,
});
var broker = new CfnBroker(this, "LegacyBroker", new CfnBrokerProps
{
BrokerName = "hl-m08gap-legacy",
EngineType = "ACTIVEMQ",
EngineVersion = "5.19", // check the current list: aws mq describe-broker-engine-types --engine-type ACTIVEMQ
HostInstanceType = "mq.t3.micro",
DeploymentMode = "SINGLE_INSTANCE",
PubliclyAccessible = false,
AutoMinorVersionUpgrade = true,
SubnetIds = [vpc.PrivateSubnets.Count > 0 ? vpc.PrivateSubnets[0].SubnetId : vpc.PublicSubnets[0].SubnetId],
SecurityGroups = [brokerSg.SecurityGroupId],
Logs = new CfnBroker.LogListProperty { General = true, Audit = true },
Users =
[
new CfnBroker.UserProperty
{
Username = credentials.SecretValueFromJson("username").UnsafeUnwrap(),
Password = credentials.SecretValueFromJson("password").UnsafeUnwrap(),
},
],
});
// The bridge: Lambda polls the broker queue "settlements" and invokes the function with batches.
// Credentials come from Secrets Manager (BASIC_AUTH). The function must run in the same VPC and account.
_ = new CfnEventSourceMapping(this, "SettlementsMapping", new CfnEventSourceMappingProps
{
FunctionName = "hl-m08gap-mq-bridge",
EventSourceArn = broker.AttrArn,
Queues = ["settlements"],
BatchSize = 10,
Enabled = true,
SourceAccessConfigurations =
[
new CfnEventSourceMapping.SourceAccessConfigurationProperty { Type = "BASIC_AUTH", Uri = credentials.SecretArn },
],
});
}
}The Lambda's execution role needs the permissions that let the mapping read the broker description and the secret, and create network interfaces in your VPC (mq:DescribeBroker, secretsmanager:GetSecretValue on the one secret, and the EC2 network-interface actions). AWS publishes a managed policy for this; check its current name and contents before using it ⚠ VERIFY.
Complete code
Compiled and unit-tested (see labs/m08-messaging-services/). Packages:
dotnet add package Amazon.Lambda.Core --version 3.4.0
dotnet add package Amazon.Lambda.MQEvents --version 3.0.0
dotnet add package AWSSDK.EventBridge --version 4.1.0.2
dotnet add package AWSSDK.SQS --version 4.1.0.2
dotnet add package Apache.NMS.ActiveMQ --version 2.3.0
dotnet add package RabbitMQ.Client --version 7.2.2The translator (anti-corruption layer)
using System.Globalization;
using System.Text;
namespace Harbourline.Messaging.Samples.Mq;
/// <summary>The modern event we publish to EventBridge. IdempotencyKey is derived from the broker's message id.</summary>
public sealed record CardSettlement(string IdempotencyKey, string SettlementId, string AccountId, decimal Amount, string Currency);
/// <summary>
/// Anti-corruption layer between a legacy card processor and the platform. The processor publishes a pipe-delimited
/// text message such as "STL|S-1001|ACC-42|125.50|GBP" to an MQ queue. We translate it into a typed event and keep
/// the legacy format (and its quirks) out of every other service.
/// </summary>
public static class LegacyMessageTranslator
{
public static CardSettlement Translate(string brokerMessageId, string base64Data)
{
// Lambda's MQ event source delivers the body base64 encoded.
var text = Encoding.UTF8.GetString(Convert.FromBase64String(base64Data));
return TranslateText(brokerMessageId, text);
}
public static CardSettlement TranslateText(string brokerMessageId, string text)
{
var parts = text.Split('|');
if (parts.Length != 5 || parts[0] != "STL")
throw new FormatException("Not a settlement message: expected STL|id|account|amount|currency.");
if (!decimal.TryParse(parts[3], NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture, out var amount) || amount <= 0)
throw new FormatException("Settlement amount must be a positive decimal.");
return new CardSettlement(
// The broker id is unique per message on that broker, and a redelivered message keeps it:
// downstream consumers dedupe on this value.
IdempotencyKey: brokerMessageId,
SettlementId: parts[1],
AccountId: parts[2],
Amount: amount,
Currency: parts[4]);
}
}The bridge function
using Amazon.EventBridge;
using Amazon.Lambda.MQEvents;
using Amazon.Lambda.Core;
using Amazon.SQS;
using Amazon.SQS.Model;
using Harbourline.Messaging.Samples.EventBridge;
namespace Harbourline.Messaging.Samples.Mq;
/// <summary>
/// Bridges an Amazon MQ (ActiveMQ) queue to an EventBridge bus. Lambda's event source mapping polls the broker and
/// invokes this function with a batch. If the function throws, Lambda retries the WHOLE batch, so a settlement can be
/// published twice: consumers dedupe on IdempotencyKey (the broker message id).
/// An MQ event source mapping has no dead-letter destination, and one untranslatable message would block the queue
/// if we kept throwing. So bad messages go to an encrypted SQS quarantine queue (kept for investigation, alarmed on
/// depth) and the batch carries on.
/// </summary>
public sealed class MqToEventBridgeFunction(IAmazonEventBridge events, IAmazonSQS sqs, string busName, string quarantineQueueUrl)
{
public MqToEventBridgeFunction()
: this(
new AmazonEventBridgeClient(),
new AmazonSQSClient(),
Environment.GetEnvironmentVariable("EVENT_BUS_NAME") ?? "default",
Environment.GetEnvironmentVariable("QUARANTINE_QUEUE_URL") ?? throw new InvalidOperationException("QUARANTINE_QUEUE_URL is not set.")) { }
public async Task HandleAsync(ActiveMQEvent mqEvent, ILambdaContext context)
{
var settlements = new List<object>();
foreach (var message in mqEvent.Messages)
{
try
{
settlements.Add(LegacyMessageTranslator.Translate(message.MessageId, message.Data));
}
catch (FormatException ex)
{
// Keep the body (still base64) for the investigation, but log only the broker id: bodies may hold account data.
context.Logger.LogError($"Quarantined untranslatable message {message.MessageId}");
await sqs.SendMessageAsync(new SendMessageRequest
{
QueueUrl = quarantineQueueUrl,
MessageBody = message.Data,
MessageAttributes = new Dictionary<string, MessageAttributeValue>
{
["brokerMessageId"] = new() { DataType = "String", StringValue = message.MessageId },
["reason"] = new() { DataType = "String", StringValue = ex.Message },
},
});
}
}
var publisher = new TransactionEventPublisher(events, busName);
var failed = await publisher.PutAsync("CardSettlementReceived", settlements);
if (failed.Count > 0)
throw new InvalidOperationException($"{failed.Count} settlements could not be published; the batch will be retried.");
}
}The clients
using Apache.NMS;
using Apache.NMS.ActiveMQ;
namespace Harbourline.Messaging.Samples.Mq;
/// <summary>
/// Publishes a persistent text message to an Amazon MQ for ActiveMQ queue over OpenWire with TLS (what a JMS-style
/// client does). NMS is the .NET sibling of JMS: IConnection -> ISession -> IMessageProducer.
/// </summary>
public static class ActiveMqPublisher
{
public static void Send(string brokerUrl, string user, string password, string queue, string body)
{
// brokerUrl example: failover:(ssl://b-1234.mq.us-east-1.on.aws:61617)
// Credentials come from Secrets Manager in real code, never from source or appsettings.
var factory = new ConnectionFactory(brokerUrl);
using var connection = factory.CreateConnection(user, password);
connection.Start();
using var session = connection.CreateSession(AcknowledgementMode.AutoAcknowledge);
using var producer = session.CreateProducer(session.GetQueue(queue));
producer.DeliveryMode = MsgDeliveryMode.Persistent; // survive a broker restart
var message = session.CreateTextMessage(body);
message.NMSCorrelationID = Guid.NewGuid().ToString("N");
producer.Send(message);
}
}using System.Text;
using RabbitMQ.Client;
using RabbitMQ.Client.Events;
namespace Harbourline.Messaging.Samples.Mq;
/// <summary>
/// A RabbitMQ consumer (client library 7.x is async-first). Manual acknowledgements and a prefetch limit are the
/// two settings that decide whether a crash loses a message: with autoAck:true a message is gone the moment it is
/// delivered; with manual ack it is redelivered if the consumer dies before BasicAck.
/// </summary>
public static class RabbitMqConsumer
{
public static async Task RunAsync(Uri amqpsUri, string queue, Func<string, Task> handleAsync, CancellationToken ct)
{
// amqpsUri example: amqps://user:password@b-1234.mq.us-east-1.on.aws:5671 (secret from Secrets Manager)
var factory = new ConnectionFactory { Uri = amqpsUri };
await using var connection = await factory.CreateConnectionAsync(ct);
await using var channel = await connection.CreateChannelAsync(cancellationToken: ct);
await channel.QueueDeclareAsync(queue, durable: true, exclusive: false, autoDelete: false, cancellationToken: ct);
await channel.BasicQosAsync(prefetchSize: 0, prefetchCount: 10, global: false, ct); // at most 10 unacknowledged messages
var consumer = new AsyncEventingBasicConsumer(channel);
consumer.ReceivedAsync += async (_, ea) =>
{
try
{
await handleAsync(Encoding.UTF8.GetString(ea.Body.Span));
await channel.BasicAckAsync(ea.DeliveryTag, multiple: false, ct);
}
catch (Exception)
{
// requeue:false sends the message to the queue's dead-letter exchange (if configured) instead of looping forever.
await channel.BasicNackAsync(ea.DeliveryTag, multiple: false, requeue: false, ct);
}
};
await channel.BasicConsumeAsync(queue, autoAck: false, consumer, ct);
await Task.Delay(Timeout.Infinite, ct).ContinueWith(_ => { }, TaskScheduler.Default);
}
}The tests pin the translator and the quarantine behaviour:
[Theory]
[InlineData("STL|S-1|ACC-1|12.00")] // missing field
[InlineData("XXX|S-1|ACC-1|12.00|GBP")] // wrong message type
[InlineData("STL|S-1|ACC-1|-5|GBP")] // not positive
[InlineData("STL|S-1|ACC-1|twelve|GBP")] // not a number
public void Malformed_messages_are_rejected(string text) =>
Assert.Throws<FormatException>(() => LegacyMessageTranslator.TranslateText("ID:1", text));
[Fact]
public async Task Bad_messages_are_quarantined_and_good_ones_are_published()
{
// One valid and one garbage message in the same batch.
await function.HandleAsync(new ActiveMQEvent { Messages = [Message("ID:1", "STL|S-1|ACC-1|10|GBP"), Message("ID:2", "garbage")] }, context);
await sqs.Received(1).SendMessageAsync(
Arg.Is<SendMessageRequest>(r => r.MessageAttributes["brokerMessageId"].StringValue == "ID:2"),
Arg.Any<CancellationToken>());
await events.Received(1).PutEventsAsync(
Arg.Is<PutEventsRequest>(r => r.Entries.Count == 1 && r.Entries[0].DetailType == "CardSettlementReceived"),
Arg.Any<CancellationToken>());
}Code explained line by line
LegacyMessageTranslator.Translatedecodes the Lambda event's base64 body to UTF-8 text, then callsTranslateText. Splitting the two keeps the parsing testable without base64.TranslateTextinsists on exactly five fields and theSTLtype marker, and parses the amount withCultureInfo.InvariantCultureandNumberStyles.AllowDecimalPoint(digits and a dot only: no thousands separators, no currency symbols). A legacy system's amount format must never depend on the server's locale. A non-positive amount is rejected: a settlement of zero or less is a bug in the sender, not a valid event.- The
IdempotencyKeyis the broker message ID. A redelivered message keeps its ID, so downstream consumers can deduplicate on it. If the body carries its own business key (S-1001) prefer that as well. MqToEventBridgeFunctionhas a parameterless constructor for the Lambda runtime (real clients and environment variables) and a dependency-injecting one for tests.QUARANTINE_QUEUE_URLis required: the function fails fast at start-up if it is missing.- In the loop, a
FormatExceptiontriggers the quarantine path: the function logs only the broker message ID (not the body, which may contain account data), and sends the still-base64 body and the reason to SQS. It then continues with the rest of the batch. - Good messages are collected, then published in one go with
TransactionEventPublisher.PutAsync, which retries only the entries EventBridge rejected. - If entries remain unsent, the function throws. Lambda retries the whole batch, so settlements already published are published again with the same idempotency key. That is acceptable only because consumers are idempotent.
ActiveMqPublisher.Sendopens an NMS connection (ConnectionFactory,CreateConnection,Start), a session withAutoAcknowledge, a queue producer withMsgDeliveryMode.Persistent(survives a broker restart) and sends a text message with a correlation ID. Credentials are parameters: real code reads them from Secrets Manager.RabbitMqConsumer.RunAsyncdeclares a durable queue, limits prefetch to 10 unacknowledged messages, and handles each message with manual acknowledgement:BasicAckAsyncafter success,BasicNackAsync(requeue: false)on failure, which dead-letters the message instead of looping forever.autoAck: falseis the line that prevents message loss on a crash.
Security considerations
⚠ Learning shortcut — not for production
The lab uses a single-instance broker with a generated broker user and the default encryption settings. Production uses active/standby (ActiveMQ) or a cluster (RabbitMQ) across Availability Zones, a customer managed KMS key for encryption at rest where policy requires it, a separate broker user per client (the processor can only send to settlements; the bridge can only read it) with ActiveMQ authorisation entries, credentials rotated in Secrets Manager, and a maintenance window agreed with the business.
⚠ Learning shortcut — not for production
You would test with the ActiveMQ web console, which is a powerful admin tool. In production do not expose the console beyond a management network, restrict it to named admins, and treat access to it as access to every message in the broker.
- Private by default.
PubliclyAccessible = false, private subnets, a security group that allows only the protocol port you use (61617 for OpenWire over TLS) from the VPN and the Lambda security group. Never open the web console to the internet. - TLS everywhere. Use the TLS endpoints (
ssl://for OpenWire,amqps://for AMQP). Legacy clients that only speak plaintext need a client upgrade, or a network design that keeps plaintext off shared segments: record this as a risk. - IAM does not authenticate clients. Broker users are the control. Use one user per application, least privilege per destination, and keep credentials in Secrets Manager with rotation. A shared "admin" user in a connection string is the most common finding.
- The Lambda role.
secretsmanager:GetSecretValueon the one secret,events:PutEventson the one bus,sqs:SendMessageon the quarantine queue, and the network-interface permissions. Nothing else. - Logging. Enable general and audit logs to CloudWatch Logs and set retention. Logs can contain destination names and, with verbose settings, message content: review before enabling.
- Quarantine is personal data. The quarantine queue holds raw message bodies. Encrypt it, give it a retention period, alarm on depth, and restrict who can read it.
- Data in transit from a data centre. Use Direct Connect or Site-to-Site VPN, plus TLS: do not rely on the network alone.
- Compliance. This lesson does not make any system compliant with GDPR, UK GDPR or PCI DSS. Message content, retention in queues and dead-letter queues, and the vendor's own controls need legal, data-protection and security review.
Banking use case
Harbourline's card settlement bridge:
| Element | Decision |
|---|---|
| Broker | Amazon MQ for ActiveMQ, active/standby in two AZs, EFS storage, private subnets, TLS only |
| Producer | The vendor's processor over a Site-to-Site VPN, broker user svc-processor allowed to send to settlements only |
| Bridge | Lambda event source mapping, batch of 10, in the VPC, role scoped to one secret, one bus, one queue |
| Translator | One class, unit-tested against the vendor's message examples and every malformed shape |
| Idempotency | Broker message ID plus settlementId; consumers deduplicate in DynamoDB |
| Poison messages | Quarantine queue with a depth alarm and a runbook, so the processor's queue never blocks |
| Alarms | Broker CpuUtilization, store usage and ConnectionCount; queue depth on settlements; quarantine depth; Lambda errors and iterator age |
| Exit plan | Consumers move to the EventBridge events one by one; when settlements has no remaining consumers except the bridge, the vendor integration is renegotiated and the broker retired |
Common mistakes
- Choosing Amazon MQ for a new application. It adds an hourly cost and a broker to size. → Use SQS, SNS or EventBridge unless a JMS/AMQP dependency forces it.
- A single-instance broker in production. An outage or maintenance window stops the integration. → Active/standby or cluster.
- Ignoring the mapping's restrictions. Five concurrent invocations for ActiveMQ, one for RabbitMQ, queues only, same account. → Design throughput and topology around them.
- A function that throws on a bad message. The whole batch retries forever and the queue stalls. → Quarantine untranslatable messages.
- Putting message bodies in logs. → Log the broker message ID only.
autoAck: trueor auto-acknowledge with slow work. A crash loses the message. → Manual acknowledgement after processing, with prefetch.- Shared broker credentials in config files. → Secrets Manager, one user per client, rotation.
- Forgetting the maintenance window. → Agree it, alarm on the broker state, and use standby for anything important.
- Assuming IAM secures the broker. → Broker users, TLS and network controls do.
- Leaving a broker running after a lab. → The cleanup script, tags and a budget alarm.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
Event source mapping stuck in Creating or Problem | Function not in the VPC, no route to Secrets Manager, security group blocks the port, wrong secret format | Check StateTransitionReason; add VPC endpoints or a NAT gateway; secret needs username and password keys |
| Nothing consumed from a single-instance broker | Weekly maintenance window | Check the broker state and window; move to standby |
| Batch retries repeatedly | Function throws (poison message, publish failure) | Look at the function logs by broker message ID; quarantine bad messages |
| Throughput stuck at a low rate | ActiveMQ mapping limited to 5 concurrent invocations (RabbitMQ 1) | Larger batches, a faster function, or ask Support for a higher limit per mapping |
| Duplicates downstream | At-least-once delivery and batch retries | Idempotency key on every consumer |
ConnectionTimeout from a client | Security group, route, wrong port or protocol, public endpoint on a private broker | Check the endpoint and port for the protocol; test from inside the VPC |
User is not authenticated | Wrong broker user, or the user lacks authorisation on the destination | Check the broker user and the ActiveMQ authorisation entries |
| Messages disappear after a broker restart | Non-persistent delivery or non-durable destination | MsgDeliveryMode.Persistent; durable queues |
| Queue grows, consumers idle | Messages in flight but consumers stuck, or prefetch too high | Check consumer counts, unacknowledged messages and prefetch |
| Broker creation fails | Quota, instance type not available for the engine, subnet or security group in another VPC | Use describe-broker-instance-options; check VPC and subnet IDs |
| RabbitMQ data lost after enabling a stream | Streams are not supported on Amazon MQ | Use quorum queues instead |
| CDK deploy shows the password in plain text | A literal was used | Use the Secrets Manager dynamic reference as in the stack |
Cost and cleanup
Cost: paid. A broker is billed per hour, running or idle, and there is no meaningful free tier for new accounts today (new customers receive Free Tier credits, which may apply ⚠ VERIFY on the Amazon MQ pricing page). The pricing page's worked examples show an mq.m5.large ActiveMQ active/standby broker at about USD 0.576 per hour, RabbitMQ mq.m7g.large at about USD 0.27 per broker-hour (three brokers in a cluster), and storage at about USD 0.10 per GB-month for EBS (about USD 0.30 for EFS) ⚠ VERIFY. The mq.t3.micro single-instance broker used in the lab is the cheapest option, but its rate is not in those examples: check the pricing page before you create it. As a rule of thumb, a small broker kept for a month costs tens of dollars and a production active/standby pair hundreds. Deploy, test and delete in under an hour, or do not deploy.
The free alternative is local Docker and it is the recommended way to learn this lesson. This Compose file runs both brokers. Default credentials are for local use only.
services:
activemq:
image: apache/activemq-classic:latest # pin a version tag in real use
ports:
- "61616:61616" # OpenWire (no TLS locally)
- "8161:8161" # web console
rabbitmq:
image: rabbitmq:4-management # pin a version tag in real use
ports:
- "5672:5672" # AMQP 0-9-1
- "15672:15672" # management UIdocker compose up -d # then open http://localhost:8161 and http://localhost:15672
docker compose down -v # remove containers and volumesThe image tags and default console credentials were not run in the authoring environment ⚠ VERIFY them against the image documentation. Local brokers use plaintext and default users, so never reuse their settings anywhere real.
export AWS_PROFILE=learn-sandbox AWS_REGION=us-east-1
# Remove the mapping, then the broker (deletion takes several minutes), then the supporting resources.
for uuid in $(aws lambda list-event-source-mappings --function-name hl-m08gap-mq-bridge \
--query 'EventSourceMappings[].UUID' --output text 2>/dev/null); do
aws lambda delete-event-source-mapping --uuid "$uuid"
done
BROKER_ID=$(aws mq list-brokers --query "BrokerSummaries[?BrokerName=='hl-m08gap-legacy'].BrokerId" --output text)
[ -n "$BROKER_ID" ] && aws mq delete-broker --broker-id "$BROKER_ID"
aws secretsmanager delete-secret --secret-id hl-m08gap/mq-bridge --force-delete-without-recovery
url=$(aws sqs get-queue-url --queue-name hl-m08gap-mq-quarantine --query QueueUrl --output text 2>/dev/null) \
&& aws sqs delete-queue --queue-url "$url"
# If you deployed with IaC: cdk destroy HlM08Gap-Broker | terraform destroy
# Confirm nothing is still billing.
aws mq list-brokers --query 'BrokerSummaries[].{name:BrokerName,state:BrokerState}'
aws resourcegroupstaggingapi get-resources --tag-filters Key=Lab,Values=m08-messaging-services \
--query 'ResourceTagMappingList[].ResourceARN'Exercises
- 🟢 Guided: Run ActiveMQ locally with Docker Compose. Send
STL|S-1001|ACC-42|125.50|GBPwithActiveMqPublisher(usetcp://localhost:61616and the console's credentials) and see it in the web console. Then run the translator unit tests and add two more malformed cases of your own. - 🟡 Challenge: Make the RabbitMQ consumer production-shaped: declare a dead-letter exchange and queue, bind the main queue to it with
x-dead-letter-exchange, and prove that a handler that throws moves the message to the dead-letter queue after one failure. Then add a small retry (two redeliveries) using a delay queue or a header counter, and explain whyrequeue: truewould be wrong. - 🔴 Live problem: Ticket at 03:10: "The settlements queue has 40,000 messages and is growing, but the bridge Lambda shows no invocations since 02:00 and no errors. Reconciliation is blocked. The vendor says they are sending normally." Work out the likely causes in order, how you confirm each, and what you do at 03:30 to get settlements flowing without losing or duplicating any.
Assignment
Add mq-migration-plan.md to your portfolio repository for one legacy JMS or AMQP integration. Include: the current system and protocol, the decision (keep, bridge, replace) with the reasons, the broker design (engine, mode, instance, storage, subnets), network and TLS design, broker users and permissions per client, the bridge design with translator contract, idempotency key, poison-message policy and alarms, the mapping restrictions that affect your throughput, a cost estimate for 12 months of the broker with and without standby, and a phased exit plan with the criteria to retire the broker.
Solutions
With the default local ActiveMQ settings, the console is at http://localhost:8161 and OpenWire at tcp://localhost:61616; use the credentials from the image's documentation. After Send, the queue settlements shows one pending message that you can browse. The translator tests pass; for your own cases try an empty string (split gives one part, so FormatException) and an amount with a thousands separator such as 1,250.00 (the translator allows only digits and a decimal point, so it is rejected: decide with the vendor whether that format can ever occur, because rejecting it quarantines a real settlement while accepting it risks misreading a European 1.250,00).
Why: the point of the exercise is that the translator is the one place the legacy format lives, so every odd shape the vendor can produce becomes a test, not an incident.
Declare the dead-letter exchange settlements.dlx and queue settlements.dead, then declare the main queue with the argument x-dead-letter-exchange: settlements.dlx. When the handler throws, BasicNackAsync(deliveryTag, multiple: false, requeue: false) rejects the message and the broker routes it through the dead-letter exchange to settlements.dead, where you can inspect it and republish after a fix.
requeue: true would put the message straight back at the head of the queue, the consumer would receive it again immediately, fail again, and loop at full speed (a hot loop that burns CPU and hides the failure). A bounded retry needs a delay: publish the failed message to a delay queue with a per-message TTL and a dead-letter exchange back to the main queue, and count attempts in a header so that after two it goes to settlements.dead instead.
Read the signals. A growing queue, no invocations and no errors means the mapping is not consuming, not that the function is failing.
- The broker is in its maintenance window or restarting. A single-instance broker is unavailable then. Check the broker state and the maintenance window time. (If so the queue would not be growing unless the vendor reaches the broker: so check connectivity too.)
- The event source mapping is in a bad state. Check
StateTransitionReasonfor the mapping: the secret was rotated and the mapping still holds the old credentials, the function lost its VPC access, a security group changed, or a subnet lost its route to Secrets Manager or Lambda endpoints. - The mapping is disabled or hit a limit. ActiveMQ mappings are capped at five concurrent invocations: if each invocation takes minutes (slow EventBridge publish with retries), throughput can be too low rather than zero.
- The queue is not the one the mapping reads (a renamed queue or virtual topic consumer queue).
At 03:30: fix the mapping's access (credentials, VPC route), then re-enable it. The 40,000 messages are still on the broker, so nothing is lost; consumption restarts from the head. Expect duplicates if some batches were partly processed before the failure, which is why downstream consumers dedupe on the broker message ID. Watch Lambda errors and the quarantine queue while the backlog drains, and raise batch size or ask Support for more concurrency if the drain is too slow.
Prevent: an alarm on the mapping state and on queue depth, an alarm on Lambda invocations falling to zero while depth is above zero, secret rotation that updates the mapping, and standby for the broker.
Knowledge check
Answer: Amazon MQ (for ActiveMQ). It keeps the broker's wire protocols and clients. SQS has no JMS or OpenWire support.
Answer: If the function throws, Lambda retries the whole batch until it succeeds or the message expires, and the mapping has no on-failure destination. Catch translation failures in the function, quarantine the message (for example in an encrypted SQS queue) and carry on, and throw only for transient failures such as a failed publish.
Answer: Five concurrent execution environments per ActiveMQ mapping, and one for RabbitMQ, by default. Support can raise it per mapping.
Answer: No. Clients use broker-level users (ActiveMQ simple authentication or LDAP, RabbitMQ users). IAM controls only the management API.
Answer: During the weekly maintenance window, and during any outage, because a single-instance broker has no standby. Use active/standby (ActiveMQ) or a cluster (RabbitMQ) for production.
Answer: Streams. Amazon MQ does not support them, and creating one causes data loss. Use quorum queues instead.
Interview questions
- "A vendor system needs JMS and cannot change. What do you do?" — Keep the broker contract with Amazon MQ for ActiveMQ so the vendor's client works unchanged, then bridge the edge to native events: a translator in a Lambda mapping or a Pipe, an idempotency key, and a quarantine queue for poison messages. Migrate consumers one by one, then retire the broker when no legacy consumer remains.
- "When would you not use Amazon MQ?" — For a new application with no JMS/AMQP/MQTT/STOMP dependency. SQS, SNS and EventBridge have no broker to size, no hourly floor and are simpler to run.
- "How do you make a broker highly available?" — ActiveMQ active/standby across two AZs, or a RabbitMQ cluster across three; persistent messages and durable destinations; and a maintenance window that suits the business.
- "How do you secure Amazon MQ?" — Private subnets and security groups, TLS endpoints, a broker user per client with least privilege, credentials in Secrets Manager with rotation, encryption at rest, audit logging, and tight control of the web console.
- "What are the main risks of the Lambda bridge?" — At-least-once delivery (idempotent consumers), a poison message stalling the queue (quarantine), the low concurrency cap, VPC connectivity to Secrets Manager and Lambda, and single-instance maintenance windows.
Certification practice
A company is migrating an on-premises application that uses Apache ActiveMQ and the JMS API to AWS. The company wants to minimise code changes and avoid managing the broker's servers. Which service should the solutions architect recommend?
A. Amazon SQS standard queues. B. Amazon MQ. C. Amazon SNS. D. Amazon Kinesis Data Streams.
Answer: B. Amazon MQ is a managed ActiveMQ/RabbitMQ service that keeps the JMS API and wire protocols. A, C and D use proprietary APIs and would need code changes.
A critical workload uses an Amazon MQ for ActiveMQ broker. The broker must survive the failure of a single Availability Zone. What should the architect configure?
A. A single-instance broker with a larger instance type. B. An active/standby broker deployment across two Availability Zones. C. A second single-instance broker in the same AZ. D. SQS dead-letter queues.
Answer: B. Active/standby places a standby in another AZ that takes over on failure. A and C stay in one AZ. D handles failed messages, not broker failure.
A developer needs to process messages from an Amazon MQ queue without running a consumer server. What is the MOST appropriate approach?
A. Poll the broker from a scheduled script on an EC2 instance. B. Configure an AWS Lambda event source mapping for the broker queue, with credentials stored in AWS Secrets Manager. C. Configure an SNS subscription on the broker. D. Use S3 event notifications.
Answer: B. Lambda can poll an Amazon MQ queue through an event source mapping using BASIC_AUTH credentials from Secrets Manager. A needs a server. C and D are not integrations of Amazon MQ.
An AWS Lambda function consuming an Amazon MQ queue fails on one malformed message. The same batch is invoked again repeatedly and no other messages are processed. What should the developer do?
A. Increase the function memory. B. Catch the translation error in the function, move the bad message to a quarantine queue, and continue with the rest of the batch. C. Reduce the batch size to 1 only. D. Switch the broker to single-instance.
Answer: B. The mapping retries the whole batch while the function throws, and has no on-failure destination, so the function must handle poison messages itself. A and D do not address it; C reduces the blast radius but still blocks on the bad message.
Completion checklist
- I can explain when to keep a broker, bridge it or replace it
- I can list the Lambda-to-MQ mapping restrictions that affect throughput and failure handling
- I wrote a translator with unit tests for every malformed shape
- I ran ActiveMQ and RabbitMQ locally in Docker (or deployed and deleted a broker the same day)
- I can explain why IAM does not authenticate broker clients
- I know what an Amazon MQ broker costs per hour and how I checked it
- I ran the cleanup and the checks returned nothing
Key takeaways
- Amazon MQ is for compatibility. Keep the JMS/AMQP/MQTT/STOMP contract of a system you cannot change; use SQS, SNS and EventBridge for new work.
- Bridge at the edge: a translator class, an idempotency key from the broker message ID, a retry-only-failures publisher and a quarantine queue, because the Lambda mapping delivers at least once and has no failure destination.
- Mapping limits: 5 concurrent invocations for ActiveMQ, 1 for RabbitMQ, queues only, same account, credentials from Secrets Manager.
- Brokers are billed by the hour and single-instance means maintenance-window downtime. Use active/standby or a cluster for anything important and delete lab brokers the same day.
- IAM does not authenticate broker clients: secure with private networking, TLS, per-client broker users and Secrets Manager.
Further reading
Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.
Kinesis Data Streams and Amazon MSK for event streaming
Use an ordered, replayable log instead of a queue: Kinesis Data Streams shards, partition keys, capacity modes, consumers and poison records, then Amazon MSK (Kafka) with IAM authentication and a .NET producer, and a clear rule for choosing between streams, queues and buses.
Microservices communication and service discovery
Draw bounded contexts for a bank, choose between a query, a command and an event for every interaction, make synchronous calls safe with timeouts and retries, and let services find each other with Amazon ECS Service Connect and AWS Cloud Map.