Skip to Content
Cloud Engineering Academy
13. Microsoft-to-AWS Migration

Cut-over and rollback planning

Plan the night the bank switches over: choose a cut-over pattern, schedule freeze windows, define go/no-go criteria and rollback triggers before you need them, reconcile data, shift traffic gradually with weighted DNS, and assemble the evidence a change board and a regulator will ask for.

advanced⏱ 2 h 30 min🟢 FreeSAP-C03SAA-C03

Assumes: .NET 10 LTS · AWS CLI v2 · Amazon Route 53 · Amazon CloudWatch · AWS CDK v2 (C#) · Terraform AWS provider v6 · Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Choose a cut-over pattern (big bang, phased, parallel run, weighted traffic shift) and justify it for a banking workload.
  • Write a timeline-based runbook with owners, durations, checkpoints and a clearly marked point of no return.
  • Define go/no-go criteria and rollback triggers as testable rules, and encode them so a script can evaluate them.
  • Plan data reconciliation and the data-divergence problem that makes rollback harder after go-live.
  • Shift traffic gradually with weighted Route 53 records, a low TTL and a health signal, using CDK (C#) or Terraform.
  • Assemble the evidence pack for a change advisory board (CAB) and for regulatory conversations.

Prerequisites

Why this matters in banking

A cut-over is where every earlier decision meets reality at 2 a.m. on a Sunday. Banks have a particular problem: transactions do not stop at the migration boundary. A card authorisation approved in the new environment at 02:10 must still be there if you roll back at 02:40, or a customer is charged for something the old ledger has never heard of. Equally, a rollback plan that was never rehearsed is a hope, not a plan.

Change boards and regulators will ask: who approved this, what was tested, what does "success" mean, what makes you stop, how long can customers be affected, how do you know the data matches, and what happens to money in flight. In the UK, operational resilience expectations include staying within defined impact tolerances for important business services ⚠ VERIFY the current regulatory wording with your compliance team. Your cut-over plan is evidence that you can.

The idea in plain language

A cut-over has five phases:

  1. Prepare. Everything that can be done in advance is done: environments built, data synchronised, tests passed, people briefed.
  2. Freeze. Changes stop. No unrelated deployments, no schema changes, no batch schedule edits.
  3. Switch. Stop writes at the source, drain the last data, run reconciliation, point users at the new system.
  4. Verify and decide. Run business checks. Go or roll back, using criteria agreed beforehand.
  5. Settle. Keep the old system intact for a rollback window, watch closely, then decommission.

Two ideas matter most. The point of no return is the moment after which going back loses or corrupts data (for example, once new transactions exist only on the target). Plan to make it as late as possible, and make the decision to cross it explicit. And decide the rollback triggers while calm, not at 03:00 under pressure.

Analogy: Opening a new branch. You rehearse the opening, count the tills in both branches, and agree in advance that if the card machines are not working by 9:30 you reopen the old branch. You do not shut the old branch's vault until the new one has balanced for a week.

What problem the AWS services solve

The cut-over is mostly process, but several AWS features make a safe, gradual switch possible.

NeedAWS featureHonest limits
Gradual traffic shiftRoute 53 weighted records with a low TTLClients and resolvers can cache beyond the TTL; a shift is not instant or exact
Automatic traffic protectionRoute 53 health checks, ALB target group healthRoute 53 health checks reach public endpoints; for private endpoints use CloudWatch alarm-based checks ⚠ VERIFY
Blue/green at the load balancerALB weighted target groupsWorks for HTTP(S) traffic you route through the ALB
Rehost final sync and rollbackMGN test and cut-over launches; keep the source until finaliseAfter finalising, the replica is gone (see rehost lesson)
Data continuityDMS CDC (and reverse replication if configured)Reverse replication must be built and rehearsed; it is not automatic
Runbook execution and evidenceSystems Manager Automation and Run Command, CloudTrail, CloudWatchIncident Manager and Change Manager are closed or in maintenance; use runbooks with SSM Automation and your ITSM tool ⚠ VERIFY replacements
Observability during the windowCloudWatch dashboards and alarms, X-Ray or OpenTelemetryBuild the dashboard before the night

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
Azure Traffic Manager weighted routingRoute 53 weighted routingSame idea; Route 53 adds latency, failover and geolocation policies
Azure Front Door canary and slot swapALB weighted target groups, CloudFront continuous deploymentSlots do not exist; you run two target groups or two stacks
Azure Migrate cut-over and "complete migration"MGN cut-over launch and finalise cutoverFinalising is irreversible for replication
Azure Site Recovery failover and failbackMGN with reverse planning, or Elastic Disaster Recovery failbackFailback is a design item on AWS too; plan it before go-live
ITIL change records in ServiceNowSame ITSM, plus CloudTrail for AWS-side evidenceAWS CloudTrail events are your technical audit trail
Azure Monitor action groupsCloudWatch alarms, SNS and EventBridgeAlarms drive notifications and, if you choose, automated rollback steps

Architecture

Security-boundary view. During cut-over the number of people with elevated access rises: a break-glass role for the migration team, access to both databases, and the ability to change DNS. Control it: time-boxed role assumption through IAM Identity Center, session recording where possible, CloudTrail logging on, a named approver for each irreversible step, and revocation of the elevated access immediately after the window. Keep two-person rules for the irreversible steps (switch, finalise, decommission).

Step-by-step implementation

Choose the cut-over pattern

PatternHow it worksBest forMain risk
Big bangEverything switches in one windowSmall, tightly coupled systems with a long maintenance windowOne shot; rollback is all or nothing
Phased by function or customer segmentMove a slice (staff first, then 1% of customers, then more)Large user bases, tolerant of two systems side by sideData and routing must handle both worlds
Parallel runBoth systems process the same inputs; compare outputs; the old one stays authoritative until provenCalculation-heavy systems (interest, fees, statements)Double the cost; careful side-effect suppression
Weighted traffic shiftGradually move HTTP traffic with weighted DNS or load-balancer weightsStateless front ends and APIsShared state (sessions, databases) must be consistent

For a core ledger, a bank often combines them: parallel-run calculations, a phased move of segments, and a short big-bang for the write path with a rehearsed rollback.

Pick the window

Avoid month-end, quarter-end, payment scheme peaks, tax deadlines and public holidays. Align with the freeze calendar and with the payment scheme cut-off times. Set the freeze start (for example, 72 hours before) and the end (after the rollback window).

Write the runbook as a timeline

Every step has an ID, an owner, a duration, a command or checklist link, an expected result, and what to do on failure. Mark the point of no return and the rollback deadline in bold. The timeline below is the skeleton.

Define go/no-go criteria and rollback triggers

Write them as yes/no tests. Examples are in the next section and in the code. Agree them with business, risk and operations before the rehearsal, and not during it.

Rehearse at least twice, then run a dress rehearsal

Rehearsal one finds the missing steps. Rehearsal two checks the timings. The dress rehearsal uses the real roles and tools, including the rollback. Time everything, and compare against the available window.

Execute, observe, decide

Follow the runbook. Record decisions, times and evidence as you go. At each checkpoint the incident commander states "go" or "stop" aloud and it is logged.

Hold the rollback window, then settle

Keep the old system intact (read-only or powered off) for the agreed window, watching reconciliation and customer complaints. Only then finalise replication, decommission and release licences.

Sample timeline (core ledger, Saturday window)

TimeStepOwnerCheckpoint
T-72hChange freeze starts; dress rehearsal results approvedRelease managerCAB approval recorded
T-24hReadiness gate: replication lag, test evidence, staffingIncident commanderGo/no-go meeting 1
T-2hLower DNS TTL to 60 s (already done T-48h); confirm dashboardsPlatform leadTTL verified
T-0Stop batch scheduler; stop writes at source; announce maintenanceOperationsWrites stopped
T+10mCDC lag reaches zeroDBALag zero confirmed
T+25mReconciliation: counts, control totals, hashesData leadReconciliation clean
T+30mGo/no-go meeting 2 (last rollback point without data divergence)Incident commanderDecision logged
T+35mSwitch application connection strings and DNS weights to 100% newPlatform leadPoint of no return crossed once new writes land
T+60mSmoke tests, payment test transactions, statement checkQA leadBusiness sign-off
T+60 to T+8hHypercare; alarms armed as rollback triggersOn-callHourly checkpoint
T+14 daysRollback window ends; finalise replication; decommission planProgramme managerCAB closure

AWS Console instructions

  • Route 53 → Hosted zones → the zone → Create record: Routing policy Weighted, a Record ID (set identifier) for each side, Weight, TTL 60, optional Health check. Repeat for old and new.
  • CloudWatch → Dashboards → Create dashboard: error rates, latency percentiles, queue depth, DMS latency, reconciliation counters (as custom metrics).
  • CloudWatch → Alarms → Create alarm: each rollback trigger is an alarm with an SNS action to the incident channel.
  • Systems Manager → Automation: store runbook steps that can be scripted (for example, set DNS weights).
  • MGN → Source servers: the cut-over and finalise actions from the previous lesson (finalise only after the rollback window).

AWS CLI

# Shift traffic: 90% old, 10% new. Run once per step. Changes are described in a JSON batch.
cat > shift-10.json <<'JSON'
{
  "Comment": "Cut-over step: 10 percent to AWS",
  "Changes": [
    { "Action": "UPSERT", "ResourceRecordSet": {
        "Name": "portal.example.com", "Type": "CNAME", "SetIdentifier": "old-dc", "Weight": 90, "TTL": 60,
        "ResourceRecords": [ { "Value": "portal-old.dc.example.com" } ] } },
    { "Action": "UPSERT", "ResourceRecordSet": {
        "Name": "portal.example.com", "Type": "CNAME", "SetIdentifier": "aws", "Weight": 10, "TTL": 60,
        "ResourceRecords": [ { "Value": "hl-portal-123456.us-east-1.elb.amazonaws.com" } ] } }
  ]
}
JSON
aws route53 change-resource-record-sets --hosted-zone-id Z0123456789ABCDEFGHIJ \
  --change-batch file://shift-10.json --profile learn-sandbox

# Rollback is the same command with weights 100 and 0. Keep the file ready and rehearsed.

# A rollback-trigger alarm: ALB 5xx rate above 1 percent for 5 consecutive minutes (metric math omitted for brevity).
aws cloudwatch put-metric-alarm --alarm-name hl-m13-portal-5xx --namespace AWS/ApplicationELB \
  --metric-name HTTPCode_Target_5XX_Count --dimensions Name=LoadBalancer,Value=app/hl-portal/0123456789abcdef \
  --statistic Sum --period 60 --evaluation-periods 5 --threshold 50 \
  --comparison-operator GreaterThanThreshold --treat-missing-data notBreaching \
  --alarm-actions arn:aws:sns:us-east-1:123456789012:hl-m13-cutover \
  --profile learn-sandbox --region us-east-1

example.com names and the IDs are placeholders. A count threshold is a simplification; in practice use a rate with metric math so traffic volume does not distort the trigger.

Infrastructure as Code

Weighted DNS records for a gradual shift. Parameterise the AWS weight so the change is a one-line, reviewed pull request and the pipeline records who approved each step.

CutoverDnsStack.cs
using Amazon.CDK;
using Amazon.CDK.AWS.Route53;
using Constructs;

namespace Harbourline.Migration.Infra;

public sealed class CutoverDnsStack : Stack
{
    public CutoverDnsStack(Construct scope, string id, string hostedZoneId, string zoneName, int awsWeight,
        IStackProps? props = null) : base(scope, id, props)
    {
        if (awsWeight is < 0 or > 100)
            throw new ArgumentOutOfRangeException(nameof(awsWeight), "weight must be 0 to 100");

        // Old estate keeps the remainder so the two weights always total 100.
        _ = new CfnRecordSet(this, "Old", new CfnRecordSetProps
        {
            HostedZoneId = hostedZoneId,
            Name = $"portal.{zoneName}",
            Type = "CNAME",
            SetIdentifier = "old-dc",
            Weight = 100 - awsWeight,
            Ttl = "60",
            ResourceRecords = ["portal-old.dc.example.com"]
        });

        _ = new CfnRecordSet(this, "Aws", new CfnRecordSetProps
        {
            HostedZoneId = hostedZoneId,
            Name = $"portal.{zoneName}",
            Type = "CNAME",
            SetIdentifier = "aws",
            Weight = awsWeight,
            Ttl = "60",
            ResourceRecords = ["hl-portal-123456.us-east-1.elb.amazonaws.com"]
        });

        Amazon.CDK.Tags.Of(this).Add("Project", "cea-lab");
        Amazon.CDK.Tags.Of(this).Add("Lab", "m13-cutover");
        Amazon.CDK.Tags.Of(this).Add("Environment", "learn");
        Amazon.CDK.Tags.Of(this).Add("Owner", "your-name");
    }
}

Run with cdk deploy -c awsWeight=10, read in your Program.cs with app.Node.TryGetContext("awsWeight"). A weight of 0 on the AWS record is the rehearsed rollback.

Complete code

Two small pure-C# components: a go/no-go evaluator (readiness criteria with evidence and waivers) and a rollback monitor (sustained-breach triggers). They are deliberately independent of AWS so you can run them in a rehearsal and attach their output to the change record.

GoNoGo.cs
namespace Harbourline.Migration.Cutover;

public enum CriterionStatus { NotStarted, Passed, Failed, Waived }

public sealed record Criterion(
    string Id,
    string Description,
    bool Mandatory,
    CriterionStatus Status,
    string? EvidenceRef = null,   // link to a test report, a reconciliation output, a ticket
    string? WaivedBy = null);     // named approver, required for a waiver

public enum Decision { Go, GoWithWaivers, NoGo }

public sealed record Assessment(Decision Decision, IReadOnlyList<string> Reasons);

public static class GoNoGoEvaluator
{
    public static Assessment Evaluate(IEnumerable<Criterion> criteria)
    {
        var reasons = new List<string>();
        var waivers = 0;

        foreach (var c in criteria)
        {
            switch (c.Status)
            {
                case CriterionStatus.Passed:
                    // A pass without evidence is an opinion, not a pass.
                    if (string.IsNullOrWhiteSpace(c.EvidenceRef))
                        reasons.Add($"{c.Id}: passed but no evidence reference");
                    break;

                case CriterionStatus.Waived:
                    if (c.Mandatory || string.IsNullOrWhiteSpace(c.WaivedBy) || string.IsNullOrWhiteSpace(c.EvidenceRef))
                        reasons.Add($"{c.Id}: waiver invalid (mandatory criteria cannot be waived; waivers need approver and evidence)");
                    else
                        waivers++;
                    break;

                case CriterionStatus.Failed:
                case CriterionStatus.NotStarted:
                    if (c.Mandatory) reasons.Add($"{c.Id}: mandatory criterion {c.Status}: {c.Description}");
                    else waivers++; // a failed optional criterion is carried as a known issue
                    break;
            }
        }

        if (reasons.Count > 0) return new Assessment(Decision.NoGo, reasons);
        return waivers > 0
            ? new Assessment(Decision.GoWithWaivers, [$"{waivers} known issue(s) accepted with approval"])
            : new Assessment(Decision.Go, ["all criteria passed with evidence"]);
    }
}
RollbackMonitor.cs
namespace Harbourline.Migration.Cutover;

public enum Comparison { GreaterThan, LessThan }

// A trigger fires only when the breach is sustained for the whole window: one bad sample is noise.
public sealed record Trigger(string Metric, Comparison Op, double Threshold, TimeSpan Window, string Action);

public sealed record Sample(string Metric, DateTimeOffset At, double Value);

public sealed record Fired(Trigger Trigger, DateTimeOffset Since);

public static class RollbackMonitor
{
    public static IReadOnlyList<Fired> Evaluate(
        IReadOnlyList<Sample> samples, IEnumerable<Trigger> triggers, DateTimeOffset now)
    {
        var fired = new List<Fired>();

        foreach (var t in triggers)
        {
            var inWindow = samples
                .Where(s => s.Metric == t.Metric && s.At > now - t.Window && s.At <= now)
                .OrderBy(s => s.At)
                .ToList();

            // We need data covering the window. Missing data is treated as a breach of monitoring,
            // not as "all fine": a dead monitor must never look like a healthy system.
            if (inWindow.Count < 2 || inWindow[0].At > now - t.Window + TimeSpan.FromSeconds(90))
            {
                fired.Add(new Fired(t with { Action = "MONITORING GAP: " + t.Action }, now));
                continue;
            }

            var allBreach = inWindow.All(s => t.Op == Comparison.GreaterThan ? s.Value > t.Threshold : s.Value < t.Threshold);
            if (allBreach) fired.Add(new Fired(t, inWindow[0].At));
        }

        return fired;
    }
}
Program.cs
using Harbourline.Migration.Cutover;

// Go/no-go for the core ledger cut-over.
var criteria = new[]
{
    new Criterion("G1", "Reconciliation clean for all 1,826 business dates", true, CriterionStatus.Passed, "evidence/recon-2026-11-21.txt"),
    new Criterion("G2", "Dress rehearsal completed within the window", true, CriterionStatus.Passed, "evidence/rehearsal-2.md"),
    new Criterion("G3", "Rollback rehearsed and timed (under 20 minutes)", true, CriterionStatus.Passed, "evidence/rollback-drill.md"),
    new Criterion("G4", "Replication lag under 60 seconds", true, CriterionStatus.Passed, "cw-dashboard/dms-latency"),
    new Criterion("G5", "On-call staffed, incident commander named", true, CriterionStatus.Passed, "rota/2026-11-22"),
    new Criterion("G6", "Customer notice published", false, CriterionStatus.NotStarted)
};

var assessment = GoNoGoEvaluator.Evaluate(criteria);
Console.WriteLine($"Decision: {assessment.Decision}");
foreach (var r in assessment.Reasons) Console.WriteLine($" - {r}");

// Rollback triggers during hypercare.
var now = DateTimeOffset.Parse("2026-11-22T03:00:00Z");
var triggers = new[]
{
    new Trigger("http_5xx_rate_pct", Comparison.GreaterThan, 1.0, TimeSpan.FromMinutes(5), "Shift DNS weight to 0 for AWS"),
    new Trigger("payment_success_pct", Comparison.LessThan, 99.0, TimeSpan.FromMinutes(10), "Invoke rollback runbook R-3"),
    new Trigger("reconciliation_diffs", Comparison.GreaterThan, 0.0, TimeSpan.FromMinutes(2), "Stop switch; page data lead")
};

// Sample feed: five minutes of 5xx above the limit, healthy payments, reconciliation clean.
var samples = new List<Sample>();
for (var m = 0; m <= 10; m++)
{
    var at = now.AddMinutes(-10 + m);
    samples.Add(new Sample("http_5xx_rate_pct", at, m >= 4 ? 2.5 : 0.2));
    samples.Add(new Sample("payment_success_pct", at, 99.8));
    samples.Add(new Sample("reconciliation_diffs", at, 0));
}

foreach (var f in RollbackMonitor.Evaluate(samples, triggers, now))
    Console.WriteLine($"TRIGGER: {f.Trigger.Metric} since {f.Since:HH:mm} -> {f.Trigger.Action}");

return assessment.Decision == Decision.NoGo ? 2 : 0;

Expected output: Decision: GoWithWaivers (G6 is optional and not started), then one trigger line for http_5xx_rate_pct. The reconciliation_diffs trigger has a two-minute window and the samples are one minute apart, so it has enough data; it does not fire because the value is zero.

Code explained line by line

  1. Criterion carries an EvidenceRef. The evaluator treats a pass without evidence as a problem: the change board needs a link, not an assurance.
  2. In Evaluate, a mandatory criterion that is failed or not started produces NoGo. A mandatory criterion cannot be waived (a waiver on a mandatory item is reported as invalid). This encodes "no one can waive the reconciliation".
  3. A failed non-mandatory criterion is counted as a known issue, so the result becomes GoWithWaivers rather than a silent Go.
  4. Trigger includes a Window: the breach must hold for the entire window, which filters single spikes.
  5. In RollbackMonitor.Evaluate, inWindow takes samples in (now - window, now]. If the data does not cover the window (fewer than two samples or the oldest is too recent), the monitor reports a monitoring gap. Absence of data is not evidence of health.
  6. t with { Action = ... } creates a modified copy of the record, leaving the original trigger unchanged.
  7. allBreach uses All(...): every sample in the window must violate the threshold. The earliest sample's time is reported as the start of the breach, which is what you put in the incident timeline.
  8. The program's return code is 2 for NoGo, so a pipeline stage can block the switch step.

Security considerations

⚠ Learning shortcut — not for production

The lesson's evaluator and monitor are simplified: criteria are in code, the samples are fabricated, and the DNS commands run with a broad profile. In production the criteria live in a controlled document with approvers, the metrics come from CloudWatch (and the source systems), the DNS change is made by an automation role with a narrow policy (only the cut-over records), and every irreversible step requires two named approvers. Break-glass roles are time-boxed, monitored and removed after the window.

  • Separation of duties. The person who approves the go decision should not be the person who executes the switch. See Federation and segregation of duties.
  • Evidence integrity. Store the evidence pack in a versioned bucket with Object Lock where policy requires (S3 protection, Object Lock and Macie). CloudTrail provides the technical record of who changed what.
  • Data in two places. During the rollback window both systems hold production personal data. Apply the same access controls, retention and breach monitoring to the old system until it is decommissioned, and then securely wipe it, with evidence. UK GDPR obligations continue throughout; consult your data-protection officer.
  • Customer communication. A maintenance window and a customer notice are part of the plan. If the migration changes where data is processed, review whether notifications, records of processing or regulator engagement are required. That is a compliance decision, not an engineering one.

Banking use case

Harbourline schedules the ledger cut-over for the third weekend of November, away from month-end. The go/no-go pack contains the reconciliation output for every business date, two rehearsal reports, a timed rollback drill (14 minutes) and the staffing rota. The decision at T+30 minutes is "go" on all five mandatory criteria; the customer-notice criterion is carried as a known issue with an approved plan. Traffic moves 10 percent, 50 percent and 100 percent over three hours with one-hour holds. At 03:10 the 5xx trigger fires on a misconfigured connection pool in one availability zone; the incident commander uses the rehearsed step to set the AWS weight to 50, fixes the pool, and resumes. Because the point of no return was not crossed (writes were still replicating both ways through reverse replication), no data was lost. The post-implementation review adds a pool-size check to the readiness gate.

Common mistakes

  • Rollback as an afterthought. If it is not rehearsed and timed, it does not exist. Rehearse it first, and set a deadline for the last safe moment.
  • Defining success after the event. Criteria written during the window drift toward "it looks fine". Write them in advance, with evidence.
  • Ignoring DNS caching. Some clients ignore short TTLs. Lower the TTL days ahead and expect a tail of stale clients.
  • Data divergence on rollback. Once new transactions exist only on the target, going back means replaying or reversing them. Prepare reverse replication or a replay strategy, and decide who owns it.
  • No freeze. An unrelated deployment during the window makes any failure impossible to attribute.
  • Skipping hypercare. Most problems appear on the first business day, with real peak load. Staff for it.
  • Finalising too early. Removing the replica or the old system before the rollback window ends removes your last way back.

Troubleshooting

SymptomLikely causeFix
Traffic still reaches the old estate after the DNS changeResolver or client caching beyond the TTL; sessions pinnedWait out TTL tails, drain old endpoints, and use load-balancer-level routing where possible
Weighted records give no response for AWSHealth check failing or wrong targetCheck the health check path and security groups; verify the ALB DNS name
Alarm never fires during a rehearsalWrong metric, dimension or missing-data treatmentTest the alarm by forcing the condition; set treat-missing-data deliberately
Reconciliation differs after cut-overIn-flight transactions, late CDC, or timezone boundaryCompare by business date with a cut-off, replay CDC, and check the time zone rule
Rollback restores service but customers see old balancesWrites happened on the target and were not reverse-replicatedExecute the data-divergence procedure: reverse-replicate or replay the target's writes
Go/no-go meeting overrunsCriteria unclear or evidence missingMove criteria and evidence links into the checklist a day earlier; use the evaluator output

Cost and cleanup

Cost: free. The code runs locally. Route 53 hosted zones cost USD 0.50 per month and weighted records cost a small amount per million queries ⚠ VERIFY, so the DNS exercise is optional and cheap; deploy only to a test zone you own and remove it the same day.

# Remove the test records (CDK or Terraform users: destroy the stack instead).
cdk destroy
# terraform destroy

# Delete the test alarm and the rollback batch files.
aws cloudwatch delete-alarms --alarm-names hl-m13-portal-5xx --profile learn-sandbox --region us-east-1
rm -f shift-10.json

# Confirm nothing tagged Project=cea-lab remains.
aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=cea-lab --profile learn-sandbox

Exercises

  • 🟢 Guided: Run the program. Change criterion G1 to Failed and confirm the decision becomes NoGo and the reason names G1. Then change G6 to Waived with a WaivedBy and an EvidenceRef and observe the result.
  • 🟡 Challenge: Add a rule that a Passed criterion whose EvidenceRef does not start with evidence/, cw-dashboard/ or rota/ is reported as unverifiable. Add a trigger for DMS latency (greater than 120 seconds for 5 minutes) to the monitor and test it with fabricated samples.
  • 🔴 Live problem: At T+45 minutes, 200 card authorisations have been approved in AWS, the 5xx rate is 3 percent and rising, and the business asks you to "just roll back". The reverse replication has been running for only 10 minutes. Write the decision note: options (roll back with replay, push forward with a fix, partial traffic shift), the data-divergence risk of each, who must approve, and the first three actions.

Assignment

Write cutover-runbook-and-rollback.md for your fictional bookstore or bank application (this feeds Lab 13.5 and the module assignment): the pattern chosen and why, the timeline table, ten go/no-go criteria with evidence references, five rollback triggers with thresholds and windows, the point of no return, the data-divergence procedure and the evidence-pack contents.

Solutions

Knowledge check

Interview questions

  • What goes into a go/no-go decision? Pre-agreed mandatory criteria with evidence (reconciliation, rehearsal, rollback drill, lag, staffing), named approvers, and a recorded decision at each checkpoint.
  • How do you roll back a database migration after writes on the target? Only with reverse replication or a replay strategy rehearsed beforehand; otherwise the rollback loses transactions. It is better to plan the point of no return than to improvise.
  • How do you reduce the blast radius of a cut-over? Phase by segment, shift traffic gradually with a health signal, keep a short TTL, hold hypercare staffing, and keep the old system intact for the rollback window.

Certification practice

Completion checklist

  • I can name the five phases of a cut-over and the point of no return
  • I ran the evaluator and the monitor and understood each output
  • I wrote at least five go/no-go criteria with evidence references
  • I can explain how weighted DNS gives a rollback path and where it falls short
  • I ran the cleanup steps and confirmed nothing is still billing

Key takeaways

  • Decide go/no-go criteria and rollback triggers before the rehearsal, and write them as tests with evidence.
  • The point of no return is a decision, not an accident; make it as late as possible.
  • Rollback must be rehearsed, timed and have a data-divergence plan.
  • Shift traffic gradually with a low TTL and health signals; expect DNS caching tails.
  • Hold the rollback window and hypercare, and only then finalise and decommission.

Further reading

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page