Skip to Content
Cloud Engineering Academy
13. Microsoft-to-AWS Migration

Integration side by side: API Management, Service Bus and Event Grid vs AWS

Translate Azure API Management policies, Service Bus queues, topics and sessions, and Event Grid subscriptions into API Gateway, SQS, SNS and EventBridge, keep ordering and duplicate protection intact, and run both messaging systems side by side during a long migration.

intermediate⏱ 1 h 30 min🟢 FreeSAA-C03SAP-C03DVA-C03

Assumes: .NET 10 LTS · AWS SDK for .NET v4 (AWSSDK.SQS, AWSSDK.EventBridge) · Azure.Messaging.ServiceBus (comparison only) · AWS CDK v2 (C#) · Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Map API Management concepts (products, subscriptions, policies, backends) to API Gateway features and say which have no direct twin.
  • Map Service Bus features (queues, topics, sessions, duplicate detection, scheduled messages, dead-letter, peek-lock) to SQS, SNS and EventBridge features.
  • Decide when a Service Bus workload belongs on SQS, SNS, EventBridge, or a combination.
  • Write an SQS FIFO publisher and consumer in C# that preserves per-account ordering and rejects duplicates.
  • Put a messaging abstraction in front of both systems so producers and consumers can move independently during coexistence.

Prerequisites

Why this matters in banking

Payments are a conversation between systems, and integration products carry the rules of that conversation: "process each account's instructions in order", "never accept the same payment reference twice", "if downstream is down, hold the message and tell someone". Harbourline's Azure Service Bus topics carry card events to six subscribers; API Management enforces a per-partner rate limit and validates a JWT before anything reaches the payments API. If either set of rules is lost in translation, two things happen: a partner floods you, or a customer's second payment overtakes their first.

Integration is also where coexistence hurts most. Systems do not migrate in a single night; for months, an Azure producer will feed an AWS consumer, or the reverse. This lesson shows the mapping and the bridge pattern that keeps both running.

The idea in plain language

Integration products fall into three families, and AWS splits the family tree differently from Azure.

  • Front door for APIs. Azure API Management (APIM) and Amazon API Gateway both sit in front of backends, check identity, limit traffic and transform requests.
  • Queues and topics for commands and events. Azure Service Bus is one product with queues (one consumer wins each message) and topics (many subscribers). AWS splits that into SQS (queue), SNS (topic, push to subscribers) and, for event routing by content, EventBridge.
  • Event routing. Azure Event Grid routes events from sources to handlers by subscription. EventBridge does the same with rules that match event patterns.

Analogy: Service Bus is a single post office with numbered pigeonholes (queues) and a noticeboard (topics). AWS is three separate counters: a pigeonhole counter (SQS), a loudspeaker (SNS) and a sorting office that reads the label and forwards by rule (EventBridge). You will walk between counters, but each counter is simpler and scales on its own.

What problem the AWS services solve

ServiceUse it forSharp edges
API Gateway (REST API)Full-featured APIs: usage plans and API keys, request validation, caching, WAF, mapping templatesMore features, more cost per request than the HTTP API
API Gateway (HTTP API)Lower-cost proxy with JWT authorisers and simple routingFewer features (no usage plans or API keys, no request validation) ⚠ VERIFY current feature set
SQS StandardHigh-throughput work queues; at-least-once delivery, best-effort orderDuplicates and reordering are normal; consumers must be idempotent
SQS FIFOOrdered processing per key, exactly-once processing within the deduplication windowOrder is per message group; one stuck message blocks its group; throughput limits apply (use high-throughput mode when needed ⚠ VERIFY current limits)
SNSFan-out to many subscribers (SQS, Lambda, HTTPS) with filter policiesNot a queue: if a subscriber is down, delivery relies on retries or an SQS subscription in front
EventBridgeRouting events by content between your services, AWS services and SaaS; schedule and archive and replayPer-event size and rate limits (about 1 MB payloads ⚠ VERIFY exact values); not for high-volume command queues

Message size. SQS accepts payloads up to 1 MiB since August 2025 (announcement); queues created earlier may keep the older limit until you raise it, so check the MaximumMessageSize attribute. Larger payloads use the claim-check pattern: store the file in S3 and put its key in the message.

Microsoft / Azure comparison

API Management to API Gateway

You know (APIM)AWS equivalentWhat's different
API, operation, backendAPI, resource and method, integration (Lambda, HTTP, VPC link to a private ALB)Private backends use a VPC link; see container APIs and VPC link
Product and subscription keyUsage plan and API key (REST APIs only)API keys identify callers for throttling; they are not a security mechanism
validate-jwt policyJWT authoriser (HTTP API), Cognito authoriser or Lambda authoriser (REST API)Authorisation logic lives in an authoriser resource, not in an inline policy
rate-limit, quotaStage and method throttling and usage plan quotasThrottling is token-bucket per stage, method or plan
set-header, rewrite-uri, set-body transformsParameter mapping (HTTP API) and mapping templates in Velocity Template Language (REST API)Mapping templates are powerful and awkward; push heavy transforms into Lambda
cache-lookupREST API stage cacheCache is a paid, sized feature, per stage
mock-responseMock integrationAvailable on REST APIs
validate-contentRequest validators with JSON Schema models (REST API)Validate in the gateway to reject junk before it costs a Lambda invocation
Developer portalNo built-in equivalent in the classic service; publish an OpenAPI document or use a portal solution ⚠ VERIFY current optionsCheck before promising partners a portal
Named values / Key VaultStage variables, Secrets Manager (not in gateway config)Do not put secrets in stage variables
Consumption and tiered pricingPay per request (plus cache and data transfer)No hourly gateway fee for the managed service

Service Bus to SQS, SNS and EventBridge

You know (Service Bus)AWS equivalentWhat's different
QueueSQS queueAWS polls for you with Lambda, or your code long-polls
Topic and subscriptions with filtersSNS topic with subscription filter policies, each subscriber usually an SQS queueFan-out is SNS to SQS; each subscriber gets its own durable queue
Sessions (ordered, exclusive processing per session ID)SQS FIFO message groups (MessageGroupId)Ordering is per group; consumers can process different groups in parallel
Duplicate detection (by MessageId, configurable window)FIFO deduplication ID (fixed five-minute window) ⚠ VERIFYChoose a stable business ID (payment reference), not a random GUID
Peek-lock, lock duration, Complete/AbandonVisibility timeout; delete on success, let it expire (or change visibility) on failureThere is no explicit "abandon"; a message reappears when the timeout ends
Max delivery count then dead-letterRedrive policy maxReceiveCount to a dead-letter queueDLQ is another queue; set alarms on its depth
Scheduled messagesSQS delay (up to 15 minutes per message) or EventBridge Scheduler for later timesLonger delays need the Scheduler or a Step Functions wait
Time to liveMessage retention (1 minute to 14 days)Same idea; set it deliberately
Transactions across entitiesNone across SQS queuesUse the outbox pattern (outbox with EF Core)
Premium large messages (up to 100 MB)1 MiB payload plus claim-check via S3Plan the pattern for statement-sized payloads

Event Grid to EventBridge

You know (Event Grid)AWS equivalentWhat's different
Topics, event subscriptions, filtersEvent buses, rules with event patterns, targetsA bus is the unit of isolation and policy; rules do the matching
System topics (Blob created, resource events)AWS service events on the default bus (S3, EC2, CloudTrail-based events)Many service events arrive on the default bus automatically
CloudEvents schemaEventBridge envelope (with detail-type and detail); CloudEvents needs mappingPick one schema for your own events and document it
Dead-lettering, retry policyTarget retry policy and DLQ per targetSet both on every important target
ReplayArchive and replay on a busUseful for rebuilding a consumer after a bug

Architecture

Security-boundary view. The gateway is the only public entry. Queues and topics are private: access is granted by resource policies (who may send) and IAM roles (who may receive). EventBridge bus policies control cross-account publishing. Encrypt queues with KMS, and log API access to CloudWatch.

Step-by-step implementation

Catalogue the Azure integration estate

For each API in APIM: policies used, products, rate limits, authentication. For each Service Bus entity: sessions on or off, duplicate detection window, max delivery count, lock duration, TTL, forwarding rules, filters. These are the settings you must reproduce.

Choose the AWS shape per entity

  • Queue with no sessions → SQS Standard (add idempotency).
  • Queue with sessions → SQS FIFO, session ID becomes MessageGroupId.
  • Topic with subscribers → SNS topic to SQS queues (use SNS FIFO when order must reach subscribers).
  • Event routed by content between systems → EventBridge rule.

Introduce a messaging abstraction

Define IInstructionPublisher in your code. Provide an Azure implementation (existing) and an AWS implementation (new). A configuration flag chooses where each producer writes. This lets consumers and producers move in any order, which is the only practical way to migrate a large estate.

Port consumers and make them idempotent

Replace ServiceBusProcessor handlers with an SQS poller or a Lambda event source mapping. Keep the same idempotency key.

Rebuild the API policies

Create the API (import OpenAPI if you have it), attach an authoriser, set throttling and request validation, then run the same partner contract tests against the old and new gateways.

Run in parallel, compare, then cut over

Dual-publish or bridge for a period, compare counts and ordering, and only then switch the producer. The cut-over lesson later in this module covers the reconciliation.

AWS Console instructions

  • SQS FIFO queue: Console → SQS → Create queue → FIFO, name ending .fifo, enable content-based deduplication only if the body uniquely identifies the message, set visibility timeout, enable server-side encryption, and configure a dead-letter queue with maxReceiveCount 3.
  • SNS topic and subscription filter: Console → SNS → Topics → Create topic, then Create subscription (protocol Amazon SQS) and set the subscription filter policy.
  • EventBridge rule: Console → Amazon EventBridge → Rules → Create rule, choose an event pattern, set the target and a DLQ.
  • API Gateway: Console → API Gateway → Create API, choose HTTP API or REST API, then Authorizers, Throttling and Models (for validators).

AWS CLI

# Create a FIFO queue with a dead-letter queue (names must end in .fifo).
aws sqs create-queue --queue-name hl-m13-instructions-dlq.fifo \
  --attributes FifoQueue=true --profile learn-sandbox --region us-east-1

aws sqs create-queue --queue-name hl-m13-instructions.fifo \
  --attributes '{"FifoQueue":"true","VisibilityTimeout":"60","SqsManagedSseEnabled":"true"}' \
  --tags Project=cea-lab,Lab=m13-integration,Environment=learn \
  --profile learn-sandbox --region us-east-1

# Send an ordered, deduplicated message. Group = account, dedup = payment reference.
QUEUE_URL=$(aws sqs get-queue-url --queue-name hl-m13-instructions.fifo --query QueueUrl --output text \
  --profile learn-sandbox --region us-east-1)
aws sqs send-message --queue-url "$QUEUE_URL" --message-body '{"paymentId":"PAY-1","amount":"25.50","currency":"GBP"}' \
  --message-group-id ACC-0000000001 --message-deduplication-id PAY-1 \
  --profile learn-sandbox --region us-east-1

# Put a test event on the default EventBridge bus.
aws events put-events --entries '[{"Source":"harbourline.payments","DetailType":"PaymentSettled","Detail":"{\"paymentId\":\"PAY-1\"}"}]' \
  --profile learn-sandbox --region us-east-1

Infrastructure as Code

IntegrationStack.cs
using Amazon.CDK;
using Amazon.CDK.AWS.Events;
using Amazon.CDK.AWS.Events.Targets;
using Amazon.CDK.AWS.SNS;
using Amazon.CDK.AWS.SNS.Subscriptions;
using Amazon.CDK.AWS.SQS;
using Constructs;

public sealed class IntegrationStack : Stack
{
    public IntegrationStack(Construct scope, string id, IStackProps props) : base(scope, id, props)
    {
        var dlq = new Queue(this, "InstructionsDlq", new QueueProps
        {
            Fifo = true,
            Encryption = QueueEncryption.SQS_MANAGED,
            RetentionPeriod = Duration.Days(14),
        });

        _ = new Queue(this, "Instructions", new QueueProps
        {
            Fifo = true,
            Encryption = QueueEncryption.SQS_MANAGED,
            VisibilityTimeout = Duration.Seconds(60),
            DeadLetterQueue = new DeadLetterQueue { Queue = dlq, MaxReceiveCount = 3 },
        });

        // Fan-out: one topic, a durable queue per subscriber, filtered by event type.
        var topic = new Topic(this, "PaymentEvents");
        var ledgerQueue = new Queue(this, "LedgerQueue", new QueueProps { Encryption = QueueEncryption.SQS_MANAGED });
        // Add a FilterPolicy here when producers publish straight to SNS with message attributes.
        topic.AddSubscription(new SqsSubscription(ledgerQueue));

        // Route content-matched events from the default bus to the topic.
        _ = new Rule(this, "SettledRule", new RuleProps
        {
            EventPattern = new EventPattern { Source = ["harbourline.payments"], DetailType = ["PaymentSettled"] },
            Targets = [new SnsTopic(topic)],
        });

        Amazon.CDK.Tags.Of(this).Add("Project", "cea-lab");
        Amazon.CDK.Tags.Of(this).Add("Lab", "m13-integration");
        Amazon.CDK.Tags.Of(this).Add("Environment", "learn");
    }
}

SNS filter policies match on message attributes by default. The EventBridge rule here already filters by event pattern, so the subscription takes everything it receives. If you later publish directly to SNS, add attributes and a filter policy (or a body-scoped filter policy ⚠ VERIFY the filter policy scope setting).

Complete code

The abstraction, the AWS publisher and a consumer. Packages: AWSSDK.SQS, Microsoft.Extensions.Hosting.

InstructionMessaging.cs
using System.Text.Json;
using Amazon.SQS;
using Amazon.SQS.Model;
using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Logging;

namespace Harbourline.Payments.Messaging;

public sealed record PaymentInstruction(string PaymentId, string AccountId, decimal Amount, string Currency);

/// <summary>What producers depend on. One implementation per messaging system during coexistence.</summary>
public interface IInstructionPublisher
{
    Task PublishAsync(PaymentInstruction instruction, CancellationToken ct);
}

public sealed class SqsFifoInstructionPublisher(IAmazonSQS sqs, string queueUrl) : IInstructionPublisher
{
    public async Task PublishAsync(PaymentInstruction instruction, CancellationToken ct)
    {
        await sqs.SendMessageAsync(new SendMessageRequest
        {
            QueueUrl = queueUrl,
            MessageBody = JsonSerializer.Serialize(instruction),
            // Service Bus SessionId -> MessageGroupId: all instructions for one account stay in order.
            MessageGroupId = instruction.AccountId,
            // Service Bus MessageId + duplicate detection -> a stable business key, not a new GUID.
            MessageDeduplicationId = instruction.PaymentId,
        }, ct);
    }
}

public interface IInstructionHandler
{
    /// <summary>Must be idempotent. Throw to leave the message on the queue for a retry.</summary>
    Task HandleAsync(PaymentInstruction instruction, CancellationToken ct);
}

public sealed class SqsInstructionConsumer(
    IAmazonSQS sqs, string queueUrl, IInstructionHandler handler, ILogger<SqsInstructionConsumer> log) : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        while (!stoppingToken.IsCancellationRequested)
        {
            // Long polling: wait up to 20 seconds, which is cheaper and faster than short polling.
            var response = await sqs.ReceiveMessageAsync(new ReceiveMessageRequest
            {
                QueueUrl = queueUrl,
                MaxNumberOfMessages = 10,
                WaitTimeSeconds = 20,
            }, stoppingToken);

            foreach (var message in response.Messages ?? [])
            {
                try
                {
                    var instruction = JsonSerializer.Deserialize<PaymentInstruction>(message.Body)
                                      ?? throw new JsonException("Empty body");
                    await handler.HandleAsync(instruction, stoppingToken);
                    // Delete = Service Bus Complete. Skip this and the message reappears after the visibility timeout.
                    await sqs.DeleteMessageAsync(queueUrl, message.ReceiptHandle, stoppingToken);
                }
                catch (Exception ex) when (ex is not OperationCanceledException)
                {
                    // No delete: the message returns after the visibility timeout, and after maxReceiveCount it goes to the DLQ.
                    log.LogWarning("messageId={MessageId} failed: {Error}", message.MessageId, ex.GetType().Name);
                }
            }
        }
    }
}

An EventBridge publisher for domain events (package AWSSDK.EventBridge):

SettledEventPublisher.cs
using System.Text.Json;
using Amazon.EventBridge;
using Amazon.EventBridge.Model;

public sealed class SettledEventPublisher(IAmazonEventBridge events)
{
    public async Task PublishAsync(string paymentId, CancellationToken ct)
    {
        var result = await events.PutEventsAsync(new PutEventsRequest
        {
            Entries =
            [
                new PutEventsRequestEntry
                {
                    Source = "harbourline.payments",
                    DetailType = "PaymentSettled",
                    Detail = JsonSerializer.Serialize(new { paymentId }),
                    EventBusName = "default",
                },
            ],
        }, ct);

        // PutEvents can partially fail without throwing; always check FailedEntryCount.
        if (result.FailedEntryCount > 0)
        {
            throw new InvalidOperationException($"EventBridge rejected {result.FailedEntryCount} entries");
        }
    }
}

Code explained line by line

  1. PaymentInstruction is the one contract both messaging systems carry.
  2. IInstructionPublisher is the seam. During migration, a configuration value selects the Service Bus or SQS implementation per producer.
  3. MessageGroupId = AccountId gives per-account ordering, the closest match to a Service Bus session. Different accounts proceed in parallel.
  4. MessageDeduplicationId = PaymentId makes a repeated send of the same payment within the deduplication window a no-op.
  5. WaitTimeSeconds = 20 enables long polling. MaxNumberOfMessages = 10 is the SQS per-call maximum.
  6. After a successful handler call, DeleteMessageAsync acknowledges the message. Any exception skips the delete, so the message becomes visible again after the visibility timeout.
  7. The when filter keeps cancellation from being logged as a failure.
  8. In the EventBridge publisher, FailedEntryCount is checked because PutEvents returns success for the call even when individual entries fail.

Security considerations

⚠ Learning shortcut — not for production

The CLI examples use the default event bus and AWS-managed SQS encryption, and the sample consumer has no health reporting. Production uses a custom event bus with a resource policy limiting publishers, a customer managed KMS key on queues that carry regulated data, a queue policy naming the exact sender role, dead-letter queue alarms that page someone, and a consumer that exports metrics and handles poison messages explicitly.

  • API keys are not authentication. Keep a real authoriser (JWT, Cognito or Lambda) in front of partner APIs; use keys only for identification and throttling.
  • Least-privilege queue access. Producers get sqs:SendMessage on one queue; consumers get sqs:ReceiveMessage, sqs:DeleteMessage and sqs:ChangeMessageVisibility on one queue.
  • Do not log payloads that contain account numbers or personal data; log IDs.
  • Cross-account events. An EventBridge bus policy that allows * principals is a data leak; name accounts or an organisation ID.
  • Compliance. Messaging does not by itself satisfy payment or data-protection regulation. Retention, residency and access review remain the bank's decisions.

Banking use case

Harbourline's card-event topic has six subscribers. Migration order matters: first the least critical subscriber (the marketing analytics consumer) moves to an SQS queue fed by an Azure-to-AWS bridge; then the notification service; the ledger consumer moves last, after dual-run reconciliation proves counts and order match. The partner API moves from APIM to API Gateway with identical JWT claims validation and the same per-partner rate limit, with a contract test suite run against both. Rollback is a DNS or producer-flag change back to the Azure path while the bridge still exists.

Common mistakes

  • Treating SNS as a queue. If a subscriber is offline, retries have limits. Put an SQS queue behind every important subscription.
  • Random deduplication IDs. A new GUID per retry defeats deduplication. Use the business key.
  • One message group for everything. All traffic becomes serial. Choose a key with enough variety (account, not bank).
  • Visibility timeout too short. Processing takes longer than the timeout, so the message is delivered twice.
  • Ignoring FailedEntryCount. EventBridge reports per-entry failure in the response.
  • Trusting API keys for security. Anyone with the key is "authenticated".

Troubleshooting

SymptomLikely causeFix
The queue should either have ContentBasedDeduplication enabled or MessageDeduplicationId providedFIFO send without a dedup IDSupply MessageDeduplicationId or enable content-based deduplication
Messages for one account stop flowingA poison message is blocking its group (it keeps failing)Fix or dead-letter it; check the DLQ and the redrive policy
Duplicate processingVisibility timeout shorter than handler time, or handler not idempotentRaise the timeout; add an idempotency key check
403 from API Gateway on a valid tokenAuthoriser audience or issuer mismatchCompare the token aud and iss claims with the authoriser configuration
EventBridge rule never firesEvent pattern does not match the exact source or detail-typeUse the sandbox tester in the console and compare the event JSON
AccessDenied on SendMessageQueue policy or role missing permissionAdd the principal to the queue policy and sqs:SendMessage to the role

Cost and cleanup

Cost label: free at lesson volumes (SQS, SNS and EventBridge include generous monthly free allowances; API Gateway has a first-year free tier for new accounts only ⚠ VERIFY the current free tier). FIFO queues cost more per request than Standard queues, so delete them after the lab.

aws sqs delete-queue --queue-url "$QUEUE_URL" --profile learn-sandbox --region us-east-1
aws sqs delete-queue --queue-url "$(aws sqs get-queue-url --queue-name hl-m13-instructions-dlq.fifo --query QueueUrl --output text --profile learn-sandbox --region us-east-1)" \
  --profile learn-sandbox --region us-east-1
# CDK route:
cdk destroy IntegrationStack --profile learn-sandbox
# Verify nothing tagged for the course remains:
aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=cea-lab \
  --profile learn-sandbox --region us-east-1 --query "ResourceTagMappingList[].ResourceARN"

Exercises

  • 🟢 Guided: create the FIFO queue, send three messages for two accounts, receive them, and observe that each account's messages arrive in order.
  • 🟡 Challenge: write a unit test with a fake IAmazonSQS (a small hand-written class or a mocking library) proving that SqsFifoInstructionPublisher sets MessageGroupId to the account and MessageDeduplicationId to the payment ID.
  • 🔴 Live problem: a partner reports that payment instructions for one customer are arriving late, but others are fine. The FIFO queue has a DLQ with 40 messages. Describe your investigation, what the blocked-group symptom looks like, and how you would unblock it without losing or reordering instructions.

Assignment

Create a messaging mapping document for three Service Bus entities and two APIM APIs in your estate: for each, the AWS shape, the settings carried across (sessions, dedup window, max delivery count, rate limit, auth) and a coexistence plan. Include a diagram of the bridge. Commit it to your portfolio repository.

Solutions

Knowledge check

Interview questions

  • "How would you migrate a Service Bus topic with six subscribers?" Move subscribers one at a time behind a bridge, least critical first, with an abstraction so producers can switch last, and reconcile counts and order at each step.
  • "FIFO or Standard?" FIFO when per-key order or exactly-once processing within the window matters and throughput fits; otherwise Standard with idempotent consumers.
  • "How does ordering fail on FIFO?" A poison message in a group blocks that group until it is dead-lettered; choose group keys with enough variety and set maxReceiveCount.

Certification practice

Completion checklist

  • I can map sessions, dedup, peek-lock and dead-letter to SQS FIFO features
  • I created a FIFO queue, sent ordered messages and observed per-group order
  • I can explain why API keys are not security
  • I deleted the queues and ran the tag query

Key takeaways

  • Service Bus splits into SQS (queue), SNS (fan-out) and EventBridge (routing).
  • Sessions become message groups, duplicate detection becomes the deduplication ID, peek-lock becomes the visibility timeout.
  • Put a messaging abstraction in front of both systems to migrate producers and consumers independently.
  • Always check FailedEntryCount, set DLQs and alarms, and keep API authentication separate from API keys.

Further reading

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page