Skip to Content
Cloud Engineering Academy
8. Messaging, Events & Microservices

Amazon MQ: bridging legacy JMS and AMQP systems

Know when a managed ActiveMQ or RabbitMQ broker is the right tool: keep a JMS/AMQP system running on Amazon MQ, bridge it to SQS, SNS and EventBridge with a Lambda event source mapping and a translator, handle poison messages, and plan the migration, with .NET 10 clients, CDK and Terraform.

intermediate⏱ 2 h 30 min🔴 Paid — optionalSAA-C03DVA-C03

Assumes: .NET 10 LTS · AWS SDK for .NET v4 · Amazon.Lambda.MQEvents 3.0.0 · Apache.NMS.ActiveMQ 2.3.0 · RabbitMQ.Client 7.2.2 · Amazon.CDK.Lib 2.277.0 · Terraform AWS provider v6 · Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Explain what Amazon MQ is (managed Apache ActiveMQ and RabbitMQ), which protocols it speaks, and why it exists next to SQS and SNS.
  • Decide, for a given system, whether to keep a broker (Amazon MQ), bridge it to native services, or replace it, using a clear decision rule.
  • Describe the deployment modes and what they mean for availability, maintenance windows and cost.
  • Bridge an ActiveMQ queue to EventBridge with a Lambda event source mapping, a translator (anti-corruption layer), idempotency and a quarantine queue for poison messages, and know that mapping's restrictions.
  • Write a .NET 10 producer for ActiveMQ (OpenWire/NMS) and a consumer for RabbitMQ (AMQP 0-9-1) with correct acknowledgement behaviour.
  • Secure a broker: private networking, TLS, broker users in Secrets Manager, logging, and the limits of IAM.
  • Cost a broker honestly, and use a free local alternative (Docker) for development.

Prerequisites

Why this matters in banking

Harbourline's card processor, a vendor package installed in 2011, publishes settlement files as JMS text messages over OpenWire to an ActiveMQ broker in a data centre. The same broker feeds a reconciliation job and a regulatory reporting extract. The vendor will not change the integration, the contract runs for three more years, and the reconciliation job's authors have left.

You cannot point that processor at SQS: SQS has no JMS provider, no OpenWire and no AMQP. Rewriting the processor's side is not on offer. But the rest of the bank wants those settlements as events on the new platform, and the data centre is closing next year.

Amazon MQ is the bridge for exactly this. You move the broker into AWS unchanged (same protocols, same clients, same queue names), so the legacy applications keep working. Then you extract the events into native services at the edge, and migrate consumers one at a time until the broker has no users. This is the strangler pattern with a managed broker as the first stone.

The traps are specific: the bridge delivers at least once and one bad message can block a queue, a single-instance broker goes offline during its weekly maintenance window, brokers are billed by the hour whether or not a message flows, and IAM does not authenticate legacy clients to the broker.

The idea in plain language

A message broker is a server that holds queues and topics. Producers send; the broker stores; consumers receive and acknowledge. It is the same idea as SQS and SNS, with an older, richer contract:

  • JMS (Java Message Service) is an API. A JMS client talks to the broker through a wire protocol. For ActiveMQ the native wire protocol is OpenWire; AMQP, STOMP, MQTT and WebSocket are also available. The .NET sibling of JMS is NMS.
  • RabbitMQ speaks AMQP 0-9-1 natively, with exchanges, bindings and queues, and other protocols through plugins.
  • Clients keep a connection to the broker, and acknowledge messages explicitly. Delivery can be persistent (survives a broker restart) or not.

Analogy: SQS is the post room's pigeonholes: public, simple, run by the building, with its own rules. A message broker is the old pneumatic-tube network between the branch and head office. Some systems were built to use its tubes and its brass canisters, and they work. You cannot swap the tube for a pigeonhole without rebuilding the machine at each end. Amazon MQ is a contractor who keeps the tube network running in the new building, so you can move the machines one at a time.

What Amazon MQ solves

Amazon MQ is a managed message broker service for Apache ActiveMQ and RabbitMQ. AWS provisions the broker instances, storage and multi-AZ failover, patches the engine in a maintenance window, and publishes metrics and logs. You keep the broker's native protocols and clients, so migration of existing applications is mostly a connection-string change.

It exists for compatibility, not for new design. For new applications on AWS, SQS, SNS and EventBridge are simpler, cheaper and have no broker to size.

Engines, versions and deployment modes (checked 2026-11-18)

ActiveMQRabbitMQ
Versions5.19 (recommended) and 5.18; older versions have reached end of support and are upgraded automaticallyRabbitMQ 4.3 and 4.2 (only on mq.m7g instances); RabbitMQ 3.13 on mq.t3, mq.m5 and mq.m7g
Deployment modesSingle instance (one broker, one AZ), active/standby (a standby in a second AZ takes over)Single instance, cluster (three-node, across AZs)
ProtocolsOpenWire (JMS), AMQP, STOMP, MQTT, WebSocketAMQP 0-9-1, and others through plugins ⚠ VERIFY the supported protocol list
Not supportedRabbitMQ streams (creating one causes data loss) and structured JSON logging
StorageEFS (durability optimised) or EBS (throughput optimised)EBS

Use aws mq describe-broker-engine-types --engine-type ACTIVEMQ (or RABBITMQ) for the current list in your Region, and describe-broker-instance-options for instance types.

Limits (checked 2026-11-18)

SettingValue
Brokers per Region200
ActiveMQ wire-level connections per broker300 on mq.*.micro, 2,000 on larger instances, per protocol
RabbitMQ connectionsPer node, by instance type, not overridable
ActiveMQ storage20 GB on micro, 200 GB on mq.m5.*
Users per ActiveMQ broker (simple auth)250; passwords 12 to 250 characters with at least 4 unique characters
Security groups per broker5
CloudWatch destinations monitoredThe first 1,000 (ActiveMQ) or 500 (RabbitMQ)
Management API throttlingBurst 100, rate 15 per second per account. Does not throttle messaging

Honest limitations

  • You pay by the hour, always. A broker has no free idle state.
  • Single-instance is not highly available, and it is unavailable during the weekly maintenance window. Lambda cannot consume from it then.
  • IAM does not authenticate clients to the broker. Clients use broker-level users (ActiveMQ simple authentication or LDAP; RabbitMQ users). IAM only controls the management API.
  • You are still operating a broker's semantics: persistence modes, redelivery policies, dead-letter queues, consumer prefetch, virtual topics.
  • Throughput is bounded by the instance, not elastic.
  • Version churn. Engine versions reach end of support; AWS upgrades brokers automatically in a maintenance window, so test your clients against the next version.

When to keep, bridge or replace

OptionWhenCostRisk
Replace the brokerYou own both ends and can change themProject effortHighest effort, lowest long-term cost
Keep on Amazon MQA vendor or frozen system speaks JMS/AMQP/MQTTHourly broker costCompatibility is high; you run a broker's semantics
Bridge to nativeThe rest of the platform wants events, but the producer cannot changeBroker + a small LambdaDuplicates and poison messages must be designed for
Self-managed on EC2/ECSOnly when you need an engine or a plugin Amazon MQ forbidsCompute plus operationsYou patch, back up and fail over

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
MSMQ, IBM MQ, on-premises ActiveMQAmazon MQThe "lift the broker" route. IBM MQ has no Amazon MQ engine: that is a different migration
Azure Service Bus (AMQP 1.0, JMS 2.0 on Premium)Amazon MQ for JMS/AMQP clients; SQS and SNS for nativeService Bus is a service with its own SDK that also speaks AMQP 1.0. Amazon MQ is a broker with the broker's protocols
NServiceBus, MassTransit on RabbitMQRabbitMQ on Amazon MQ with the same librariesRabbitMQ.Client 7.x is async-first. MassTransit and NServiceBus connect unchanged; some use features (streams) that Amazon MQ does not support
Service Bus peek-lock and Complete/AbandonAcknowledge and Nack/redelivery in ActiveMQ; BasicAck/BasicNack in RabbitMQAcknowledgement modes are per session or channel, not per message lock
Service Bus dead-letter queueActiveMQ dead-letter queue (after the redelivery policy), RabbitMQ dead-letter exchangeYou configure it; with the Lambda mapping it is not automatic (see below)
Service Bus topics and subscriptionsActiveMQ virtual topics, RabbitMQ exchangesVirtual topics behave as a topic to publishers and as queues to consumers (the form Lambda can read)
Azure VM running a brokerAmazon MQ (managed) vs EC2 (self-managed)AWS patches and fails over for you, in a maintenance window you choose

Bridging to native services

There are two managed ways to read from an Amazon MQ broker without running a consumer process.

Lambda event source mapping

Lambda polls the broker queue, batches messages, and invokes your function synchronously. For an ActiveMQ queue it consumes over OpenWire (JMS TextMessage and BytesMessage), for RabbitMQ over AMQP 0-9-1. The documented restrictions matter for the design:

RestrictionConsequence
Concurrency: 5 concurrent executions per ActiveMQ mapping, 1 per RabbitMQ mappingThroughput is capped. Support can raise it per mapping
At-least-onceDuplicates happen. Make the function idempotent
Same account onlyNo cross-account consumption
Authentication: broker username and password from Secrets Manager (BASIC_AUTH). ActiveMQ simple authentication only; RabbitMQ PLAIN onlyCreate a dedicated, least-privilege broker user
Queues only (no topics)Use ActiveMQ virtual topics to expose a topic as queues
Topology: one single-instance or active/standby broker per ActiveMQ mapping; one single-instance or cluster per RabbitMQ mapping
A failed batch is retried whole until success or message expiryOne poison message can block the queue. There is no on-failure destination
Network: a private broker needs the function in the VPC (and an outbound path to Secrets Manager and Lambda service endpoints)VPC endpoints or a NAT gateway
Maintenance window: a single-instance broker is unavailable, so nothing is consumedUse standby for production
Max function timeout 14 minutes for the mapping

The event carries eventSource (aws:mq for ActiveMQ, aws:rmq for RabbitMQ), the broker ARN, and messages (ActiveMQ) or rmqMessagesByQueue (RabbitMQ). Each ActiveMQ message has messageID, messageType, destination, redelivered, properties, and the body base64-encoded in data.

EventBridge Pipes

A Pipe can use an Amazon MQ broker as its source, filter, enrich and deliver to a target (an event bus, Step Functions, SQS and others) with no function code. Choose the mapping when you need custom translation logic (as in this lesson); choose a Pipe when a filter and a field mapping are enough ⚠ VERIFY Pipe batching and failure options for MQ sources before relying on them.

The pattern that works

  1. Translate the legacy message into a typed event in an anti-corruption layer. The legacy format stays inside one class.
  2. Key the event with a stable idempotency key (the broker message ID, or a business key in the body).
  3. Publish to EventBridge with retry of failed entries (the EventBridge lesson).
  4. Quarantine a message that cannot be translated into an encrypted SQS queue, with the broker message ID and the reason, and carry on. Alarm on that queue's depth.
  5. Let the batch fail (throw) only when the publish to EventBridge fails, which is a transient problem worth retrying.

Reliability inside the broker

ConcernActiveMQRabbitMQ
Survive a broker restartPersistent delivery mode, durable destinationsDurable queues and persistent messages
AcknowledgementSession acknowledgement mode (auto, client, transacted). With NMS: AcknowledgementModeBasicAck after the work is done; autoAck: false
Poison messagesRedelivery policy, then the dead-letter queue (by default a shared ActiveMQ.DLQ; configure an individual DLQ per queue) ⚠ VERIFY the defaults for your versionBasicNack with requeue: false to a dead-letter exchange
Back-pressureProducer flow control, memory and store limitsPrefetch (BasicQos) limits unacknowledged messages per consumer
High availabilityActive/standby with shared storageQuorum queues replicate across a cluster; classic mirrored queues are gone in RabbitMQ 4 ⚠ VERIFY

Architecture

Security-boundary view

Step-by-step implementation

Decide: do you need a deployed broker?

For learning, you do not. Run ActiveMQ and RabbitMQ in Docker (the Cost and cleanup section has a Compose file), write the translator and the clients, and prove them with unit tests. Deploy a real broker only if you have a budget alarm and will delete it the same day.

Write the translator first

The legacy message is STL|S-1001|ACC-42|125.50|GBP. Put its parsing in one class with unit tests: valid input, missing fields, wrong type, non-positive amount, a non-numeric amount. Everything else depends on this being right and on malformed input failing loudly.

Build the bridge function

The function decodes each message, translates it, collects typed events, and publishes with the retry-only-failures publisher. A message that cannot be translated goes to the quarantine queue and is not published.

Deploy the broker and the mapping (optional, paid)

Create a private single-instance ActiveMQ broker mq.t3.micro with a generated password in Secrets Manager, in two or more subnets only if you choose active/standby. Create the event source mapping for the queue settlements with BASIC_AUTH. The function must be in the same VPC.

Send a good message and a bad one

Use the NMS publisher from a machine inside the VPC (a Cloud9 or a bastion) or through the ActiveMQ web console's Send page. Send STL|S-1001|ACC-42|125.50|GBP and garbage. Confirm one event on the bus and one message in the quarantine queue.

Break it: duplicates

Make the function throw once after publishing. Lambda retries the whole batch, so the same settlement is published twice with the same idempotency key. Show that a consumer using the key from the previous lessons ignores the second.

Delete the broker

A broker bills every hour. Run the cleanup and confirm in the console that its state is DELETION_IN_PROGRESS and then gone.

AWS Console instructions

Console → Amazon MQ → Brokers → Create brokers. Choose the engine (Apache ActiveMQ or RabbitMQ), the deployment mode (Single-instance broker for learning; Active/standby or Cluster for production), then Broker name and Broker instance type (mq.t3.micro is the cheapest for ActiveMQ). Set Username and a Password (12 to 250 characters). Under Additional settings choose Private access, your VPC, subnet(s) and security group, leave Encryption and Logs (general and audit) enabled, pick a Maintenance window you can live with, and review the estimated cost before Create. Creation takes about 15 to 25 minutes ⚠ VERIFY.

On the broker page: Connections shows the protocol endpoints (OpenWire over TLS on 61617, AMQP over TLS on 5671, MQTT over TLS on 8883, STOMP on 61614, WebSocket on 61619, web console on 8162 ⚠ VERIFY the port list for your engine version). The ActiveMQ web console lets you browse queues, send test messages and see the dead-letter queue. For RabbitMQ, the RabbitMQ web console is the equivalent.

Console → Lambda → your function → Add trigger → MQ: pick the broker, the queue name(s), the Secrets Manager secret for authentication, and batch size.

AWS CLI

export AWS_PROFILE=learn-sandbox AWS_REGION=us-east-1

# What can I create here? Engines, versions and instance types.
aws mq describe-broker-engine-types --engine-type ACTIVEMQ \
  --query 'BrokerEngineTypes[].EngineVersions[].Name'
aws mq describe-broker-instance-options --engine-type ACTIVEMQ --host-instance-type mq.t3.micro \
  --query 'BrokerInstanceOptions[].SupportedEngineVersions'

# Store the broker user's password in Secrets Manager first (the CDK and Terraform do this for you).
aws secretsmanager create-secret --name hl-m08gap/mq-bridge \
  --secret-string "{\"username\":\"svc-bridge\",\"password\":\"$(openssl rand -base64 24 | tr -d '/+=')\"}" \
  --tags Key=Project,Value=cea-lab Key=Lab,Value=m08-messaging-services

# Create a private single-instance broker (about 15 to 25 minutes). Costs money by the hour.
aws mq create-broker --broker-name hl-m08gap-legacy --engine-type ACTIVEMQ --engine-version 5.19 \
  --host-instance-type mq.t3.micro --deployment-mode SINGLE_INSTANCE --no-publicly-accessible \
  --auto-minor-version-upgrade --subnet-ids "$SUBNET_ID" --security-groups "$SG_ID" \
  --logs General=true,Audit=true \
  --users "Username=svc-bridge,Password=$BROKER_PASSWORD" \
  --tags Project=cea-lab,Lab=m08-messaging-services
aws mq describe-broker --broker-id "$BROKER_ID" --query '{state:BrokerState,endpoints:BrokerInstances[].Endpoints}'

# Wire the bridge: Lambda polls the queue "settlements" with the secret as credentials.
aws lambda create-event-source-mapping --function-name hl-m08gap-mq-bridge \
  --event-source-arn "$BROKER_ARN" --queues settlements --batch-size 10 \
  --source-access-configurations "Type=BASIC_AUTH,URI=$SECRET_ARN"
aws lambda list-event-source-mappings --function-name hl-m08gap-mq-bridge \
  --query 'EventSourceMappings[].{state:State,reason:StateTransitionReason}'

# Reboot (for a config change) and delete.
aws mq reboot-broker --broker-id "$BROKER_ID"
aws mq delete-broker --broker-id "$BROKER_ID"

Infrastructure as Code

The stack looks up the default VPC, creates a security group that allows OpenWire over TLS (61617) only from inside the VPC, generates the broker credentials in Secrets Manager, creates the private single-instance broker with logs on, an encrypted quarantine queue, and the event source mapping with BASIC_AUTH. Brokers have no high-level (L2) construct, so this uses the L1 CfnBroker. The secret is passed as a dynamic reference: the password never appears in the template.

BrokerStack.cs
using Amazon.CDK;
using Amazon.CDK.AWS.EC2;
using Amazon.CDK.AWS.Lambda;
using Amazon.CDK.AWS.AmazonMQ;
using Amazon.CDK.AWS.SecretsManager;
using Amazon.CDK.AWS.SQS;
using Constructs;

namespace Harbourline.Messaging.Infra;

/// <summary>
/// A private, single-instance Amazon MQ for ActiveMQ broker for the legacy integration lab. SINGLE_INSTANCE is the
/// cheap, non-resilient choice for learning: production uses ACTIVE_STANDBY_MULTI_AZ and two subnets.
/// The broker password lives in Secrets Manager and is injected as a dynamic reference, never written to the template.
/// </summary>
public sealed class BrokerStack : Stack
{
    public BrokerStack(Construct scope, string id, IStackProps? props = null) : base(scope, id, props)
    {
        var vpc = Vpc.FromLookup(this, "Vpc", new VpcLookupOptions { IsDefault = true });

        var brokerSg = new SecurityGroup(this, "BrokerSg", new SecurityGroupProps
        {
            Vpc = vpc,
            Description = "Amazon MQ broker: OpenWire over TLS from inside the VPC only",
            AllowAllOutbound = false,
        });
        brokerSg.AddIngressRule(Peer.Ipv4(vpc.VpcCidrBlock), Port.Tcp(61617), "OpenWire+TLS from the VPC");

        var credentials = new Secret(this, "BrokerCredentials", new SecretProps
        {
            SecretName = "hl-m08gap/mq-bridge",
            GenerateSecretString = new SecretStringGenerator
            {
                SecretStringTemplate = """{ "username": "svc-bridge" }""",
                GenerateStringKey = "password",
                PasswordLength = 24,
                ExcludePunctuation = true,
            },
        });

        // Messages the bridge cannot translate land here, encrypted, instead of blocking the broker queue.
        _ = new Queue(this, "Quarantine", new QueueProps
        {
            QueueName = "hl-m08gap-mq-quarantine",
            RetentionPeriod = Duration.Days(14),
            Encryption = QueueEncryption.SQS_MANAGED,
            EnforceSSL = true,
        });

        var broker = new CfnBroker(this, "LegacyBroker", new CfnBrokerProps
        {
            BrokerName = "hl-m08gap-legacy",
            EngineType = "ACTIVEMQ",
            EngineVersion = "5.19", // check the current list: aws mq describe-broker-engine-types --engine-type ACTIVEMQ
            HostInstanceType = "mq.t3.micro",
            DeploymentMode = "SINGLE_INSTANCE",
            PubliclyAccessible = false,
            AutoMinorVersionUpgrade = true,
            SubnetIds = [vpc.PrivateSubnets.Count > 0 ? vpc.PrivateSubnets[0].SubnetId : vpc.PublicSubnets[0].SubnetId],
            SecurityGroups = [brokerSg.SecurityGroupId],
            Logs = new CfnBroker.LogListProperty { General = true, Audit = true },
            Users =
            [
                new CfnBroker.UserProperty
                {
                    Username = credentials.SecretValueFromJson("username").UnsafeUnwrap(),
                    Password = credentials.SecretValueFromJson("password").UnsafeUnwrap(),
                },
            ],
        });

        // The bridge: Lambda polls the broker queue "settlements" and invokes the function with batches.
        // Credentials come from Secrets Manager (BASIC_AUTH). The function must run in the same VPC and account.
        _ = new CfnEventSourceMapping(this, "SettlementsMapping", new CfnEventSourceMappingProps
        {
            FunctionName = "hl-m08gap-mq-bridge",
            EventSourceArn = broker.AttrArn,
            Queues = ["settlements"],
            BatchSize = 10,
            Enabled = true,
            SourceAccessConfigurations =
            [
                new CfnEventSourceMapping.SourceAccessConfigurationProperty { Type = "BASIC_AUTH", Uri = credentials.SecretArn },
            ],
        });
    }
}

The Lambda's execution role needs the permissions that let the mapping read the broker description and the secret, and create network interfaces in your VPC (mq:DescribeBroker, secretsmanager:GetSecretValue on the one secret, and the EC2 network-interface actions). AWS publishes a managed policy for this; check its current name and contents before using it ⚠ VERIFY.

Complete code

Compiled and unit-tested (see labs/m08-messaging-services/). Packages:

dotnet add package Amazon.Lambda.Core --version 3.4.0
dotnet add package Amazon.Lambda.MQEvents --version 3.0.0
dotnet add package AWSSDK.EventBridge --version 4.1.0.2
dotnet add package AWSSDK.SQS --version 4.1.0.2
dotnet add package Apache.NMS.ActiveMQ --version 2.3.0
dotnet add package RabbitMQ.Client --version 7.2.2

The translator (anti-corruption layer)

LegacyMessageTranslator.cs
using System.Globalization;
using System.Text;

namespace Harbourline.Messaging.Samples.Mq;

/// <summary>The modern event we publish to EventBridge. IdempotencyKey is derived from the broker's message id.</summary>
public sealed record CardSettlement(string IdempotencyKey, string SettlementId, string AccountId, decimal Amount, string Currency);

/// <summary>
/// Anti-corruption layer between a legacy card processor and the platform. The processor publishes a pipe-delimited
/// text message such as "STL|S-1001|ACC-42|125.50|GBP" to an MQ queue. We translate it into a typed event and keep
/// the legacy format (and its quirks) out of every other service.
/// </summary>
public static class LegacyMessageTranslator
{
    public static CardSettlement Translate(string brokerMessageId, string base64Data)
    {
        // Lambda's MQ event source delivers the body base64 encoded.
        var text = Encoding.UTF8.GetString(Convert.FromBase64String(base64Data));
        return TranslateText(brokerMessageId, text);
    }

    public static CardSettlement TranslateText(string brokerMessageId, string text)
    {
        var parts = text.Split('|');
        if (parts.Length != 5 || parts[0] != "STL")
            throw new FormatException("Not a settlement message: expected STL|id|account|amount|currency.");

        if (!decimal.TryParse(parts[3], NumberStyles.AllowDecimalPoint, CultureInfo.InvariantCulture, out var amount) || amount <= 0)
            throw new FormatException("Settlement amount must be a positive decimal.");

        return new CardSettlement(
            // The broker id is unique per message on that broker, and a redelivered message keeps it:
            // downstream consumers dedupe on this value.
            IdempotencyKey: brokerMessageId,
            SettlementId: parts[1],
            AccountId: parts[2],
            Amount: amount,
            Currency: parts[4]);
    }
}

The bridge function

MqToEventBridgeFunction.cs
using Amazon.EventBridge;
using Amazon.Lambda.MQEvents;
using Amazon.Lambda.Core;
using Amazon.SQS;
using Amazon.SQS.Model;
using Harbourline.Messaging.Samples.EventBridge;

namespace Harbourline.Messaging.Samples.Mq;

/// <summary>
/// Bridges an Amazon MQ (ActiveMQ) queue to an EventBridge bus. Lambda's event source mapping polls the broker and
/// invokes this function with a batch. If the function throws, Lambda retries the WHOLE batch, so a settlement can be
/// published twice: consumers dedupe on IdempotencyKey (the broker message id).
/// An MQ event source mapping has no dead-letter destination, and one untranslatable message would block the queue
/// if we kept throwing. So bad messages go to an encrypted SQS quarantine queue (kept for investigation, alarmed on
/// depth) and the batch carries on.
/// </summary>
public sealed class MqToEventBridgeFunction(IAmazonEventBridge events, IAmazonSQS sqs, string busName, string quarantineQueueUrl)
{
    public MqToEventBridgeFunction()
        : this(
            new AmazonEventBridgeClient(),
            new AmazonSQSClient(),
            Environment.GetEnvironmentVariable("EVENT_BUS_NAME") ?? "default",
            Environment.GetEnvironmentVariable("QUARANTINE_QUEUE_URL") ?? throw new InvalidOperationException("QUARANTINE_QUEUE_URL is not set.")) { }

    public async Task HandleAsync(ActiveMQEvent mqEvent, ILambdaContext context)
    {
        var settlements = new List<object>();

        foreach (var message in mqEvent.Messages)
        {
            try
            {
                settlements.Add(LegacyMessageTranslator.Translate(message.MessageId, message.Data));
            }
            catch (FormatException ex)
            {
                // Keep the body (still base64) for the investigation, but log only the broker id: bodies may hold account data.
                context.Logger.LogError($"Quarantined untranslatable message {message.MessageId}");
                await sqs.SendMessageAsync(new SendMessageRequest
                {
                    QueueUrl = quarantineQueueUrl,
                    MessageBody = message.Data,
                    MessageAttributes = new Dictionary<string, MessageAttributeValue>
                    {
                        ["brokerMessageId"] = new() { DataType = "String", StringValue = message.MessageId },
                        ["reason"] = new() { DataType = "String", StringValue = ex.Message },
                    },
                });
            }
        }

        var publisher = new TransactionEventPublisher(events, busName);
        var failed = await publisher.PutAsync("CardSettlementReceived", settlements);

        if (failed.Count > 0)
            throw new InvalidOperationException($"{failed.Count} settlements could not be published; the batch will be retried.");
    }
}

The clients

ActiveMqPublisher.cs
using Apache.NMS;
using Apache.NMS.ActiveMQ;

namespace Harbourline.Messaging.Samples.Mq;

/// <summary>
/// Publishes a persistent text message to an Amazon MQ for ActiveMQ queue over OpenWire with TLS (what a JMS-style
/// client does). NMS is the .NET sibling of JMS: IConnection -> ISession -> IMessageProducer.
/// </summary>
public static class ActiveMqPublisher
{
    public static void Send(string brokerUrl, string user, string password, string queue, string body)
    {
        // brokerUrl example: failover:(ssl://b-1234.mq.us-east-1.on.aws:61617)
        // Credentials come from Secrets Manager in real code, never from source or appsettings.
        var factory = new ConnectionFactory(brokerUrl);
        using var connection = factory.CreateConnection(user, password);
        connection.Start();

        using var session = connection.CreateSession(AcknowledgementMode.AutoAcknowledge);
        using var producer = session.CreateProducer(session.GetQueue(queue));
        producer.DeliveryMode = MsgDeliveryMode.Persistent;   // survive a broker restart

        var message = session.CreateTextMessage(body);
        message.NMSCorrelationID = Guid.NewGuid().ToString("N");
        producer.Send(message);
    }
}
RabbitMqConsumer.cs
using System.Text;
using RabbitMQ.Client;
using RabbitMQ.Client.Events;

namespace Harbourline.Messaging.Samples.Mq;

/// <summary>
/// A RabbitMQ consumer (client library 7.x is async-first). Manual acknowledgements and a prefetch limit are the
/// two settings that decide whether a crash loses a message: with autoAck:true a message is gone the moment it is
/// delivered; with manual ack it is redelivered if the consumer dies before BasicAck.
/// </summary>
public static class RabbitMqConsumer
{
    public static async Task RunAsync(Uri amqpsUri, string queue, Func<string, Task> handleAsync, CancellationToken ct)
    {
        // amqpsUri example: amqps://user:password@b-1234.mq.us-east-1.on.aws:5671 (secret from Secrets Manager)
        var factory = new ConnectionFactory { Uri = amqpsUri };

        await using var connection = await factory.CreateConnectionAsync(ct);
        await using var channel = await connection.CreateChannelAsync(cancellationToken: ct);

        await channel.QueueDeclareAsync(queue, durable: true, exclusive: false, autoDelete: false, cancellationToken: ct);
        await channel.BasicQosAsync(prefetchSize: 0, prefetchCount: 10, global: false, ct); // at most 10 unacknowledged messages

        var consumer = new AsyncEventingBasicConsumer(channel);
        consumer.ReceivedAsync += async (_, ea) =>
        {
            try
            {
                await handleAsync(Encoding.UTF8.GetString(ea.Body.Span));
                await channel.BasicAckAsync(ea.DeliveryTag, multiple: false, ct);
            }
            catch (Exception)
            {
                // requeue:false sends the message to the queue's dead-letter exchange (if configured) instead of looping forever.
                await channel.BasicNackAsync(ea.DeliveryTag, multiple: false, requeue: false, ct);
            }
        };

        await channel.BasicConsumeAsync(queue, autoAck: false, consumer, ct);
        await Task.Delay(Timeout.Infinite, ct).ContinueWith(_ => { }, TaskScheduler.Default);
    }
}

The tests pin the translator and the quarantine behaviour:

StreamingAndMqTests.cs (excerpt)
[Theory]
[InlineData("STL|S-1|ACC-1|12.00")]            // missing field
[InlineData("XXX|S-1|ACC-1|12.00|GBP")]        // wrong message type
[InlineData("STL|S-1|ACC-1|-5|GBP")]           // not positive
[InlineData("STL|S-1|ACC-1|twelve|GBP")]       // not a number
public void Malformed_messages_are_rejected(string text) =>
    Assert.Throws<FormatException>(() => LegacyMessageTranslator.TranslateText("ID:1", text));

[Fact]
public async Task Bad_messages_are_quarantined_and_good_ones_are_published()
{
    // One valid and one garbage message in the same batch.
    await function.HandleAsync(new ActiveMQEvent { Messages = [Message("ID:1", "STL|S-1|ACC-1|10|GBP"), Message("ID:2", "garbage")] }, context);

    await sqs.Received(1).SendMessageAsync(
        Arg.Is<SendMessageRequest>(r => r.MessageAttributes["brokerMessageId"].StringValue == "ID:2"),
        Arg.Any<CancellationToken>());
    await events.Received(1).PutEventsAsync(
        Arg.Is<PutEventsRequest>(r => r.Entries.Count == 1 && r.Entries[0].DetailType == "CardSettlementReceived"),
        Arg.Any<CancellationToken>());
}

Code explained line by line

  1. LegacyMessageTranslator.Translate decodes the Lambda event's base64 body to UTF-8 text, then calls TranslateText. Splitting the two keeps the parsing testable without base64.
  2. TranslateText insists on exactly five fields and the STL type marker, and parses the amount with CultureInfo.InvariantCulture and NumberStyles.AllowDecimalPoint (digits and a dot only: no thousands separators, no currency symbols). A legacy system's amount format must never depend on the server's locale. A non-positive amount is rejected: a settlement of zero or less is a bug in the sender, not a valid event.
  3. The IdempotencyKey is the broker message ID. A redelivered message keeps its ID, so downstream consumers can deduplicate on it. If the body carries its own business key (S-1001) prefer that as well.
  4. MqToEventBridgeFunction has a parameterless constructor for the Lambda runtime (real clients and environment variables) and a dependency-injecting one for tests. QUARANTINE_QUEUE_URL is required: the function fails fast at start-up if it is missing.
  5. In the loop, a FormatException triggers the quarantine path: the function logs only the broker message ID (not the body, which may contain account data), and sends the still-base64 body and the reason to SQS. It then continues with the rest of the batch.
  6. Good messages are collected, then published in one go with TransactionEventPublisher.PutAsync, which retries only the entries EventBridge rejected.
  7. If entries remain unsent, the function throws. Lambda retries the whole batch, so settlements already published are published again with the same idempotency key. That is acceptable only because consumers are idempotent.
  8. ActiveMqPublisher.Send opens an NMS connection (ConnectionFactory, CreateConnection, Start), a session with AutoAcknowledge, a queue producer with MsgDeliveryMode.Persistent (survives a broker restart) and sends a text message with a correlation ID. Credentials are parameters: real code reads them from Secrets Manager.
  9. RabbitMqConsumer.RunAsync declares a durable queue, limits prefetch to 10 unacknowledged messages, and handles each message with manual acknowledgement: BasicAckAsync after success, BasicNackAsync(requeue: false) on failure, which dead-letters the message instead of looping forever. autoAck: false is the line that prevents message loss on a crash.

Security considerations

⚠ Learning shortcut — not for production

The lab uses a single-instance broker with a generated broker user and the default encryption settings. Production uses active/standby (ActiveMQ) or a cluster (RabbitMQ) across Availability Zones, a customer managed KMS key for encryption at rest where policy requires it, a separate broker user per client (the processor can only send to settlements; the bridge can only read it) with ActiveMQ authorisation entries, credentials rotated in Secrets Manager, and a maintenance window agreed with the business.

⚠ Learning shortcut — not for production

You would test with the ActiveMQ web console, which is a powerful admin tool. In production do not expose the console beyond a management network, restrict it to named admins, and treat access to it as access to every message in the broker.

  • Private by default. PubliclyAccessible = false, private subnets, a security group that allows only the protocol port you use (61617 for OpenWire over TLS) from the VPN and the Lambda security group. Never open the web console to the internet.
  • TLS everywhere. Use the TLS endpoints (ssl:// for OpenWire, amqps:// for AMQP). Legacy clients that only speak plaintext need a client upgrade, or a network design that keeps plaintext off shared segments: record this as a risk.
  • IAM does not authenticate clients. Broker users are the control. Use one user per application, least privilege per destination, and keep credentials in Secrets Manager with rotation. A shared "admin" user in a connection string is the most common finding.
  • The Lambda role. secretsmanager:GetSecretValue on the one secret, events:PutEvents on the one bus, sqs:SendMessage on the quarantine queue, and the network-interface permissions. Nothing else.
  • Logging. Enable general and audit logs to CloudWatch Logs and set retention. Logs can contain destination names and, with verbose settings, message content: review before enabling.
  • Quarantine is personal data. The quarantine queue holds raw message bodies. Encrypt it, give it a retention period, alarm on depth, and restrict who can read it.
  • Data in transit from a data centre. Use Direct Connect or Site-to-Site VPN, plus TLS: do not rely on the network alone.
  • Compliance. This lesson does not make any system compliant with GDPR, UK GDPR or PCI DSS. Message content, retention in queues and dead-letter queues, and the vendor's own controls need legal, data-protection and security review.

Banking use case

Harbourline's card settlement bridge:

ElementDecision
BrokerAmazon MQ for ActiveMQ, active/standby in two AZs, EFS storage, private subnets, TLS only
ProducerThe vendor's processor over a Site-to-Site VPN, broker user svc-processor allowed to send to settlements only
BridgeLambda event source mapping, batch of 10, in the VPC, role scoped to one secret, one bus, one queue
TranslatorOne class, unit-tested against the vendor's message examples and every malformed shape
IdempotencyBroker message ID plus settlementId; consumers deduplicate in DynamoDB
Poison messagesQuarantine queue with a depth alarm and a runbook, so the processor's queue never blocks
AlarmsBroker CpuUtilization, store usage and ConnectionCount; queue depth on settlements; quarantine depth; Lambda errors and iterator age
Exit planConsumers move to the EventBridge events one by one; when settlements has no remaining consumers except the bridge, the vendor integration is renegotiated and the broker retired

Common mistakes

  • Choosing Amazon MQ for a new application. It adds an hourly cost and a broker to size. → Use SQS, SNS or EventBridge unless a JMS/AMQP dependency forces it.
  • A single-instance broker in production. An outage or maintenance window stops the integration. → Active/standby or cluster.
  • Ignoring the mapping's restrictions. Five concurrent invocations for ActiveMQ, one for RabbitMQ, queues only, same account. → Design throughput and topology around them.
  • A function that throws on a bad message. The whole batch retries forever and the queue stalls. → Quarantine untranslatable messages.
  • Putting message bodies in logs. → Log the broker message ID only.
  • autoAck: true or auto-acknowledge with slow work. A crash loses the message. → Manual acknowledgement after processing, with prefetch.
  • Shared broker credentials in config files. → Secrets Manager, one user per client, rotation.
  • Forgetting the maintenance window. → Agree it, alarm on the broker state, and use standby for anything important.
  • Assuming IAM secures the broker. → Broker users, TLS and network controls do.
  • Leaving a broker running after a lab. → The cleanup script, tags and a budget alarm.

Troubleshooting

SymptomLikely causeFix
Event source mapping stuck in Creating or ProblemFunction not in the VPC, no route to Secrets Manager, security group blocks the port, wrong secret formatCheck StateTransitionReason; add VPC endpoints or a NAT gateway; secret needs username and password keys
Nothing consumed from a single-instance brokerWeekly maintenance windowCheck the broker state and window; move to standby
Batch retries repeatedlyFunction throws (poison message, publish failure)Look at the function logs by broker message ID; quarantine bad messages
Throughput stuck at a low rateActiveMQ mapping limited to 5 concurrent invocations (RabbitMQ 1)Larger batches, a faster function, or ask Support for a higher limit per mapping
Duplicates downstreamAt-least-once delivery and batch retriesIdempotency key on every consumer
ConnectionTimeout from a clientSecurity group, route, wrong port or protocol, public endpoint on a private brokerCheck the endpoint and port for the protocol; test from inside the VPC
User is not authenticatedWrong broker user, or the user lacks authorisation on the destinationCheck the broker user and the ActiveMQ authorisation entries
Messages disappear after a broker restartNon-persistent delivery or non-durable destinationMsgDeliveryMode.Persistent; durable queues
Queue grows, consumers idleMessages in flight but consumers stuck, or prefetch too highCheck consumer counts, unacknowledged messages and prefetch
Broker creation failsQuota, instance type not available for the engine, subnet or security group in another VPCUse describe-broker-instance-options; check VPC and subnet IDs
RabbitMQ data lost after enabling a streamStreams are not supported on Amazon MQUse quorum queues instead
CDK deploy shows the password in plain textA literal was usedUse the Secrets Manager dynamic reference as in the stack

Cost and cleanup

Cost: paid. A broker is billed per hour, running or idle, and there is no meaningful free tier for new accounts today (new customers receive Free Tier credits, which may apply ⚠ VERIFY on the Amazon MQ pricing page). The pricing page's worked examples show an mq.m5.large ActiveMQ active/standby broker at about USD 0.576 per hour, RabbitMQ mq.m7g.large at about USD 0.27 per broker-hour (three brokers in a cluster), and storage at about USD 0.10 per GB-month for EBS (about USD 0.30 for EFS) ⚠ VERIFY. The mq.t3.micro single-instance broker used in the lab is the cheapest option, but its rate is not in those examples: check the pricing page before you create it. As a rule of thumb, a small broker kept for a month costs tens of dollars and a production active/standby pair hundreds. Deploy, test and delete in under an hour, or do not deploy.

The free alternative is local Docker and it is the recommended way to learn this lesson. This Compose file runs both brokers. Default credentials are for local use only.

docker-compose.yml
services:
  activemq:
    image: apache/activemq-classic:latest   # pin a version tag in real use
    ports:
      - "61616:61616"   # OpenWire (no TLS locally)
      - "8161:8161"     # web console
  rabbitmq:
    image: rabbitmq:4-management            # pin a version tag in real use
    ports:
      - "5672:5672"     # AMQP 0-9-1
      - "15672:15672"   # management UI
docker compose up -d     # then open http://localhost:8161 and http://localhost:15672
docker compose down -v   # remove containers and volumes

The image tags and default console credentials were not run in the authoring environment ⚠ VERIFY them against the image documentation. Local brokers use plaintext and default users, so never reuse their settings anywhere real.

export AWS_PROFILE=learn-sandbox AWS_REGION=us-east-1

# Remove the mapping, then the broker (deletion takes several minutes), then the supporting resources.
for uuid in $(aws lambda list-event-source-mappings --function-name hl-m08gap-mq-bridge \
    --query 'EventSourceMappings[].UUID' --output text 2>/dev/null); do
  aws lambda delete-event-source-mapping --uuid "$uuid"
done
BROKER_ID=$(aws mq list-brokers --query "BrokerSummaries[?BrokerName=='hl-m08gap-legacy'].BrokerId" --output text)
[ -n "$BROKER_ID" ] && aws mq delete-broker --broker-id "$BROKER_ID"
aws secretsmanager delete-secret --secret-id hl-m08gap/mq-bridge --force-delete-without-recovery
url=$(aws sqs get-queue-url --queue-name hl-m08gap-mq-quarantine --query QueueUrl --output text 2>/dev/null) \
  && aws sqs delete-queue --queue-url "$url"

# If you deployed with IaC: cdk destroy HlM08Gap-Broker   |   terraform destroy

# Confirm nothing is still billing.
aws mq list-brokers --query 'BrokerSummaries[].{name:BrokerName,state:BrokerState}'
aws resourcegroupstaggingapi get-resources --tag-filters Key=Lab,Values=m08-messaging-services \
  --query 'ResourceTagMappingList[].ResourceARN'

Exercises

  • 🟢 Guided: Run ActiveMQ locally with Docker Compose. Send STL|S-1001|ACC-42|125.50|GBP with ActiveMqPublisher (use tcp://localhost:61616 and the console's credentials) and see it in the web console. Then run the translator unit tests and add two more malformed cases of your own.
  • 🟡 Challenge: Make the RabbitMQ consumer production-shaped: declare a dead-letter exchange and queue, bind the main queue to it with x-dead-letter-exchange, and prove that a handler that throws moves the message to the dead-letter queue after one failure. Then add a small retry (two redeliveries) using a delay queue or a header counter, and explain why requeue: true would be wrong.
  • 🔴 Live problem: Ticket at 03:10: "The settlements queue has 40,000 messages and is growing, but the bridge Lambda shows no invocations since 02:00 and no errors. Reconciliation is blocked. The vendor says they are sending normally." Work out the likely causes in order, how you confirm each, and what you do at 03:30 to get settlements flowing without losing or duplicating any.

Assignment

Add mq-migration-plan.md to your portfolio repository for one legacy JMS or AMQP integration. Include: the current system and protocol, the decision (keep, bridge, replace) with the reasons, the broker design (engine, mode, instance, storage, subnets), network and TLS design, broker users and permissions per client, the bridge design with translator contract, idempotency key, poison-message policy and alarms, the mapping restrictions that affect your throughput, a cost estimate for 12 months of the broker with and without standby, and a phased exit plan with the criteria to retire the broker.

Solutions

Knowledge check

Interview questions

  • "A vendor system needs JMS and cannot change. What do you do?" — Keep the broker contract with Amazon MQ for ActiveMQ so the vendor's client works unchanged, then bridge the edge to native events: a translator in a Lambda mapping or a Pipe, an idempotency key, and a quarantine queue for poison messages. Migrate consumers one by one, then retire the broker when no legacy consumer remains.
  • "When would you not use Amazon MQ?" — For a new application with no JMS/AMQP/MQTT/STOMP dependency. SQS, SNS and EventBridge have no broker to size, no hourly floor and are simpler to run.
  • "How do you make a broker highly available?" — ActiveMQ active/standby across two AZs, or a RabbitMQ cluster across three; persistent messages and durable destinations; and a maintenance window that suits the business.
  • "How do you secure Amazon MQ?" — Private subnets and security groups, TLS endpoints, a broker user per client with least privilege, credentials in Secrets Manager with rotation, encryption at rest, audit logging, and tight control of the web console.
  • "What are the main risks of the Lambda bridge?" — At-least-once delivery (idempotent consumers), a poison message stalling the queue (quarantine), the low concurrency cap, VPC connectivity to Secrets Manager and Lambda, and single-instance maintenance windows.

Certification practice

Completion checklist

  • I can explain when to keep a broker, bridge it or replace it
  • I can list the Lambda-to-MQ mapping restrictions that affect throughput and failure handling
  • I wrote a translator with unit tests for every malformed shape
  • I ran ActiveMQ and RabbitMQ locally in Docker (or deployed and deleted a broker the same day)
  • I can explain why IAM does not authenticate broker clients
  • I know what an Amazon MQ broker costs per hour and how I checked it
  • I ran the cleanup and the checks returned nothing

Key takeaways

  • Amazon MQ is for compatibility. Keep the JMS/AMQP/MQTT/STOMP contract of a system you cannot change; use SQS, SNS and EventBridge for new work.
  • Bridge at the edge: a translator class, an idempotency key from the broker message ID, a retry-only-failures publisher and a quarantine queue, because the Lambda mapping delivers at least once and has no failure destination.
  • Mapping limits: 5 concurrent invocations for ActiveMQ, 1 for RabbitMQ, queues only, same account, credentials from Secrets Manager.
  • Brokers are billed by the hour and single-instance means maintenance-window downtime. Use active/standby or a cluster for anything important and delete lab brokers the same day.
  • IAM does not authenticate broker clients: secure with private networking, TLS, per-client broker users and Secrets Manager.

Further reading

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page