Skip to Content
Cloud Engineering Academy
14. Advanced Banking Architecture

Disaster-recovery architecture for important business services

Map important business services and their impact tolerances to backup-and-restore, pilot-light, warm-standby or active-active designs, plan for cyber recovery and dependencies, and check the mapping with a runnable C# tolerance checker.

advanced⏱ 2 h 30 min🟢 FreeSAP-C03SAA-C03SCS-C03

Assumes: .NET 10 LTS (file-based apps) · AWS SDK for .NET v4 · Amazon.CDK.Lib 2.277.0 · Terraform AWS provider v6 · Region us-east-1 (DR example us-west-2). Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Explain the chain from important business service to impact tolerance to RTO and RPO to a DR strategy.
  • Choose between backup and restore, pilot light, warm standby and multi-site active-active for each service, and justify the cost.
  • Find the hidden dependencies that break recovery: identity, DNS, keys, quotas, partners, on-premises links and pipelines.
  • Distinguish DR for an Availability Zone or Region failure from cyber recovery, where replication copies the damage.
  • Design DR tests that produce evidence of measured recovery times, not assumed ones.
  • Run a C# checker that finds services whose chosen strategy cannot meet their tolerance, and mark regulatory points as needing review.

Prerequisites

Why this matters in banking

When an entire cloud Region, a data centre or a critical dependency fails, a bank is judged on one question: how quickly did customers get the important services back, and how much data did they lose? Regulators in many jurisdictions expect firms to identify their important business services, set a maximum tolerable disruption for each, and show by testing that they can stay within it ⚠ VERIFY which regimes apply to your firm (for example UK operational-resilience rules and EU digital operational resilience rules).

Teams often confuse "we have Multi-AZ" with "we have disaster recovery". Multi-AZ survives the loss of a data centre in a Region. It does nothing for a regional event, a bad deployment that corrupts data, an expired certificate chain, a ransomware attack or an unavailable identity provider. A DR architecture answers each of those with a named strategy and evidence.

Educational content, not regulatory advice

Impact tolerances, severe-but-plausible scenarios, testing frequency and reporting duties are set by your regulators and your firm's risk appetite ⚠ VERIFY. The numbers in this lesson are illustrative planning values for the fictional Harbourline Bank, not recommendations, and not claims that a design meets any regulation.

The idea in plain language

Start from the customer, not the server:

  1. Important business service (IBS): something the bank provides whose failure could harm customers or the market: "make a retail payment", "view balance and statements", "access cash through cards".
  2. Impact tolerance: the maximum disruption you accept for that service, for example "no longer than 2 hours".
  3. RTO (recovery time objective): how long recovery is allowed to take, set inside the tolerance with margin.
  4. RPO (recovery point objective): how much recent data you can afford to lose.
  5. Strategy: the design that can actually meet the RTO and RPO.
  6. Evidence: test results that show it did.

Analogy: a hospital's emergency plan. "Emergency surgery must be possible within 30 minutes of a power cut" is the tolerance. A generator that starts in 10 seconds is the strategy. The monthly generator test, with the timing written in the log, is the evidence. A generator nobody has tested is a hope.

The four AWS strategies

StrategyWhat runs in the recovery RegionTypical recoveryTypical data lossRelative cost
Backup and restoreNothing; backups are copied thereHours to a day or moreSince last backupLowest
Pilot lightCore data replicated; compute switched off or minimalTens of minutes to hoursSeconds to minutesLow
Warm standbyFull stack at reduced capacity, scaled up on failoverMinutesSecondsMedium
Multi-site active-activeFull stack serving traffic in both RegionsNear zero for the Region lossNear zero, with conflict handlingHighest, plus complexity

The ranges are planning assumptions. Your measured times from tests replace them. Never put a typical value in front of a regulator as if it were your result.

Why the weakest dependency sets the time

A service is only as recoverable as its slowest or least-protected dependency. If payment initiation has a 2-hour tolerance but depends on a reporting database that restores in 8 hours, the dependency chain sets the real answer.

What problem the AWS services solve

NeedService or featureHonest limits
Copy backups across Regions and accountsAWS Backup cross-Region and cross-account copyRestore speed depends on data size; test it
Backups safe from deletion and ransomwareAWS Backup Vault Lock, logically air-gapped vaultVault Lock compliance mode is irreversible after the cooling-off period; plan carefully
Relational data across RegionsAurora Global DatabaseAsynchronous replication, so RPO is not zero; typical lag is low but you must measure ⚠ VERIFY
NoSQL across RegionsDynamoDB global tablesActive-active writes need conflict thinking (last writer wins by default; check consistency options ⚠ VERIFY)
Object replicationS3 Cross-Region Replication, optionally with Replication Time ControlReplicates new objects after enabling; existing objects need batch replication; check delete-marker behaviour ⚠ VERIFY
Traffic failoverRoute 53 health checks, Amazon Application Recovery Controller (ARC) routing controlsARC Readiness Check is closed to new customers; use routing controls and your own tests ⚠ VERIFY status
Server-based workloadsAWS Elastic Disaster RecoveryReplicates servers; fits lift-and-shift estates more than cloud-native ones
Rebuild from codeCloudFormation, CDK, TerraformNeeds the pipeline and artefacts available in the DR Region
Multi-Region keys and secretsKMS multi-Region keys, Secrets Manager replicationReplication must be set up before the disaster

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
Azure Site RecoveryAWS Elastic Disaster RecoveryBoth replicate servers to a recovery location
Azure Backup, Recovery Services vaultAWS Backup with vaults and Vault LockVault Lock gives WORM protection
Azure paired regionsChosen Region pair (no automatic pairing)You pick the DR Region, including data-residency checks
Azure Traffic Manager / Front Door failoverRoute 53 failover, ARC routing controls, CloudFront origin failoverARC is the high-availability control-plane pattern
SQL geo-replication, failover groupsAurora Global Database, RDS cross-Region replicasAurora Global promotes a secondary Region
Cosmos DB multi-region writesDynamoDB global tablesSimilar active-active model with conflict resolution
Azure Chaos StudioAWS Fault Injection ServiceExperiments with stop conditions and IAM scoping

Architecture

Security-boundary view

The recovery account's backups cannot be deleted by workload administrators, which protects against ransomware and insider deletion. Failover itself is a controlled, authenticated, logged action.

The 15-section design

The fictional case: Harbourline's important business services and a strategy for each.

Sections 1 to 3: requirements

SectionContent
1 BusinessKeep retail payments, balance and statement access, and card use within agreed impact tolerances during severe but plausible disruptions, and be able to evidence this
2 FunctionalF1 Fail over payment initiation to a second Region. F2 Restore from immutable backups after a cyber event. F3 Fail back after recovery. F4 Communicate status to customers. F5 Run regular tests with timed results
3 Non-functionalSee the service mapping below. Test cadence: failover test twice a year, restore test quarterly ⚠ VERIFY with risk. Evidence retained 7 years ⚠ VERIFY

Illustrative service mapping:

Important business serviceImpact toleranceRTO targetRPO targetStrategyWhy
Retail payment initiation2 hours30 minutes1 minuteWarm standbyTolerance short; asynchronous replication is acceptable if reconciliation covers the gap
Balance and statement view1 hour30 minutes5 minutesWarm standbyRead-heavy; replica data serves reads quickly
Card authorisation15 minutes5 minutes1 minuteActive-active (⚠ business decision)Shortest tolerance; may rely on scheme stand-in processing ⚠ VERIFY
Regulatory reporting3 days24 hours24 hoursBackup and restoreBatch, tolerant; restore tested
Customer notifications8 hours4 hours1 hourPilot lightQueue backlog can be replayed

Section 5: service selection

ChoiceWhyRejected alternativeTrade-off
Aurora Global Database for the ledgerCross-Region replication with fast promotionCross-Region snapshots onlyHigher cost; non-zero RPO to be covered by reconciliation
DynamoDB global tables for idempotency and stateMulti-Region replicas with low lagSingle-Region table with backupsConflict behaviour must be understood
Warm standby for paymentsMeets 30-minute RTO at moderate costActive-activeFailover is a manual or semi-automatic decision
AWS Backup with a locked recovery-account vaultProtection against deletion and ransomwareBackups in the workload accountAnother account and key to manage
Route 53 ARC routing controls for failoverReliable data-plane control with approvalsManual DNS editsSetup and rehearsal

Sections 6 and 7: network and security

  • Network. Mirror the VPC layout in the DR Region with non-overlapping CIDRs if you plan cross-Region connectivity. On-premises links must exist in the DR Region too: a second Direct Connect location or VPN to the DR Region, or the core is unreachable after failover (core banking integration).
  • Identity. Prefer workforce identity that survives a Region event. Keep a break-glass role, tested, stored under dual control. Remember that IAM control-plane operations should not be required during recovery: pre-create roles, policies and keys.
  • Keys. KMS multi-Region keys or separately replicated key material for encrypted replicas. Test decryption in the DR Region.
  • Secrets and certificates. Replicate secrets; ensure certificates for DR hostnames are issued and not about to expire.
  • Data residency. The DR Region must also satisfy data-residency and transfer rules ⚠ VERIFY with your data-protection officer. A UK bank might pair two Regions that meet its own policy; this lesson uses us-east-1 and us-west-2 for the lab only.
  • Cyber recovery. Replication faithfully copies corruption and malicious deletes. Keep immutable, versioned, delayed-copy backups in a separate account, and rehearse restoring into an isolated clean environment.

Sections 8 to 10: API, data and flow

  • Failover API (internal, privileged). POST /dr/failover requires two approvals and a reason; it flips the routing control, scales the standby, promotes the database and records a timeline. Never expose it to customers.
  • Database. A dr_events table records every step with timestamps. The timeline is the evidence: detection, decision, each action and the time customers regained service.
  • Failover flow. (1) Detect via synthetic checks and alarms. (2) Incident commander decides against pre-agreed criteria. (3) Freeze deployments. (4) Promote database. (5) Scale standby. (6) Flip routing. (7) Verify with synthetic payments. (8) Communicate. (9) Later: reconcile any replication gap, then fail back deliberately.

Section 11: failure scenarios (of the DR design itself)

FailureDetectionAutomatic responseImpactManual step
Replication lag larger than RPOLag alarmAlertData-loss risk at failoverDecide with business; reconcile
DR Region quotas too lowPre-failover capacity testNoneScale-up failsRaise quotas in advance
Secrets or keys not replicatedDR testNoneStandby cannot startFix replication, retest
DNS or routing change not effectiveSynthetic probesNoneCustomers still hit failed RegionLower TTLs; verify
Partner allow-lists exclude DR addressesDR testNonePartner calls failPre-register DR egress IPs
Split brain (both Regions write)MonitoringFencingData conflictsStop one side; reconcile
Ransomware corrupts data and replicasAnomaly or integrity checksBackups immutablePrimary and replica unusableRestore clean point in recovery account
Runbook out of dateGame-day failuresNoneSlow recoveryUpdate after each test
Failback causes second outageTestNoneExtra downtimeRehearse failback; plan window
Control-plane dependency during recoveryTest with simulated outageStatic stability designDelayPre-provision; avoid creating resources during the event

Sections 12 to 15

  • Scalability. In warm standby the standby is smaller, so the recovery plan includes scaling up; verify quotas and capacity in the DR Region, and consider reserved capacity for critical services ⚠ VERIFY options.
  • Cost. Estimate each strategy's recurring cost: replicated storage, cross-Region data transfer, standby compute, backup storage and copies, and test run cost. Relative cost grows with strategy tier; active-active also pays for operating two live Regions. Compare with the business cost of an hour of downtime for each service ⚠ VERIFY prices.
  • Implementation. Phase 1 service inventory, tolerances and dependency maps. Phase 2 backups with vault lock and cross-account copy. Phase 3 data replication. Phase 4 standby stack from IaC. Phase 5 routing controls and approvals. Phase 6 timed tests and runbooks. Phase 7 cyber-recovery rehearsal.
  • Compliance touchpoints.
AreaWhere it touchesReviewing teamEvidence
Operational resilience: important business services and impact tolerances ⚠ VERIFYMapping table, tolerancesOperational resilience, riskMapping, approved tolerances, scenario results
Testing obligationsTest cadence and scopeRisk, internal auditTimed test reports
Third-party and cloud concentration riskCloud provider and partner dependenciesThird-party riskExit and stress-exit plans
Data residency and transferDR Region choiceData protection officer, legalRegion decision record
Backup retention and immutabilityVault lock retentionComplianceRetention schedule

Step-by-step implementation

Inventory important business services and dependencies

For each service list every component, data store, partner and network link it needs. Draw the dependency graph and mark the slowest-to-recover item.

Agree tolerances with the business, then derive RTO and RPO

Tolerance comes from the business and risk function. Set RTO inside the tolerance with margin for detection and decision time.

Pick the cheapest strategy that meets the numbers

Use the checker in this lesson to compare each service's choice against its needs, then record the choice and the rejected alternatives in an ADR.

Build the protection layers

Backups with cross-Region and cross-account copy and a locked vault. Replication for data stores. KMS and secret replication. IaC for the standby.

Script failover and failback

Write a runbook with owners, commands and checkpoints. Automate what you can, but keep the decision with a named person.

Test, time, record, improve

Run a game day with timers. Record detection time, decision time, action time and the time to a verified payment. Fix every gap and update the runbook.

AWS Console instructions

  • Console → AWS Backup → Backup vaults → Create backup vault. Pick a KMS key, then Vault lock to set minimum retention (and, in a real recovery account, a cooling-off period for compliance mode).
  • Console → AWS Backup → Backup plans → Create plan. Add a rule with a schedule, a retention, and Copy to destination with a Region and vault.
  • Console → RDS → Databases → your Aurora cluster → Actions → Add AWS Region to create a global database secondary (billing starts).
  • Console → DynamoDB → Tables → Global tables → Create replica for a global table replica.
  • Console → Route 53 Application Recovery Controller to review routing controls where available ⚠ VERIFY current console layout.

AWS CLI

# Create a vault in the DR Region and a plan copy rule is then added in the plan (see IaC).
aws backup create-backup-vault --backup-vault-name hl-m14-dr-vault \
  --backup-vault-tags Project=cea-lab,Lab=m14-l16,Environment=learn,Owner=learner \
  --profile learn-sandbox --region us-west-2

# Create a logically air-gapped vault (retention bounds are required).
aws backup create-logically-air-gapped-backup-vault --backup-vault-name hl-m14-airgap \
  --min-retention-days 7 --max-retention-days 35 \
  --profile learn-sandbox --region us-west-2

# See recent copy jobs: did the cross-Region copy actually happen?
aws backup list-copy-jobs --by-state COMPLETED --max-results 5 \
  --profile learn-sandbox --region us-east-1

# Check Aurora global cluster membership (read-only).
aws rds describe-global-clusters --profile learn-sandbox --region us-east-1

# Add a DynamoDB replica in the DR Region (billing starts for the replica).
aws dynamodb update-table --table-name hl-m14-idempotency \
  --replica-updates '[{"Create":{"RegionName":"us-west-2"}}]' \
  --profile learn-sandbox --region us-east-1

Cross-Region replication and global databases bill for as long as they exist; treat them as design-only unless you will delete them within the hour ⚠ VERIFY prices. Check each command with help in your CLI version.

Infrastructure as Code

DrBackupStack.cs
using Amazon.CDK;
using Amazon.CDK.AWS.Backup;
using Amazon.CDK.AWS.Events;
using Amazon.CDK.AWS.KMS;
using Constructs;

namespace Harbourline.Architecture.DisasterRecovery;

public class DrBackupStack : Stack
{
    public DrBackupStack(Construct scope, string id, IStackProps? props = null) : base(scope, id, props)
    {
        var key = new Key(this, "BackupKey", new KeyProps { EnableKeyRotation = true });

        var vault = new BackupVault(this, "PrimaryVault", new BackupVaultProps
        {
            BackupVaultName = "hl-m14-primary-vault",
            EncryptionKey = key,
            // Min retention without a cooling-off period gives Governance-style lock; production
            // compliance mode (with a cooling-off period) is a risk and compliance decision.
            LockConfiguration = new LockConfiguration { MinRetention = Duration.Days(7) },
            RemovalPolicy = RemovalPolicy.DESTROY,
        });

        // The destination vault lives in the DR Region (created there, see the CLI step).
        var drVault = BackupVault.FromBackupVaultArn(this, "DrVault",
            $"arn:aws:backup:us-west-2:{Aws.ACCOUNT_ID}:backup-vault:hl-m14-dr-vault");

        var plan = new BackupPlan(this, "DailyPlan", new BackupPlanProps
        {
            BackupPlanName = "hl-m14-daily",
            BackupVault = vault,
        });

        plan.AddRule(new BackupPlanRule(new BackupPlanRuleProps
        {
            RuleName = "daily-with-dr-copy",
            ScheduleExpression = Schedule.Cron(new CronOptions { Hour = "3", Minute = "0" }),
            DeleteAfter = Duration.Days(35),
            CopyActions =
            [
                new BackupPlanCopyActionProps
                {
                    DestinationBackupVault = drVault,
                    DeleteAfter = Duration.Days(35),
                },
            ],
        }));

        // Anything tagged Backup=daily is protected.
        plan.AddSelection("TaggedResources", new BackupSelectionOptions
        {
            Resources = [BackupResource.FromTag("Backup", "daily")],
        });

        Amazon.CDK.Tags.Of(this).Add("Project", "cea-lab");
        Amazon.CDK.Tags.Of(this).Add("Lab", "m14-l16");
        Amazon.CDK.Tags.Of(this).Add("Environment", "learn");
        Amazon.CDK.Tags.Of(this).Add("Owner", "learner");
    }
}

A vault with a lock configuration may be hard to delete during cleanup; use a very short minimum retention in the sandbox and expect deletion to wait for retention.

Complete code

The checker below compares each service's impact tolerance and data-loss tolerance with what its chosen strategy can deliver, and proposes the cheapest strategy that would meet both. The strategy numbers are planning assumptions: replace them with measured values from your tests.

DrTolerance.cs
// Run with: dotnet run DrTolerance.cs
// Compares each important business service's impact tolerance and data-loss tolerance with what its
// chosen DR strategy can deliver. The strategy numbers are PLANNING ASSUMPTIONS: replace them with
// the times you measure in game days.
var strategies = new Dictionary<string, Strategy>
{
    ["backup-restore"] = new("backup-restore", RtoMinutes: 24 * 60, RpoMinutes: 24 * 60, RelativeCost: 1),
    ["pilot-light"]    = new("pilot-light",    RtoMinutes: 4 * 60,  RpoMinutes: 15,      RelativeCost: 2),
    ["warm-standby"]   = new("warm-standby",   RtoMinutes: 30,      RpoMinutes: 1,       RelativeCost: 4),
    ["active-active"]  = new("active-active",  RtoMinutes: 2,       RpoMinutes: 1,       RelativeCost: 8),
};

var services = new[]
{
    new Service("Payment initiation",   ToleranceMinutes: 120,     DataLossMinutes: 1,   Chosen: "pilot-light"),
    new Service("Account balances",     ToleranceMinutes: 60,      DataLossMinutes: 5,   Chosen: "warm-standby"),
    new Service("Regulatory reporting", ToleranceMinutes: 3 * 24 * 60, DataLossMinutes: 24 * 60, Chosen: "backup-restore"),
    new Service("Notifications",        ToleranceMinutes: 8 * 60,  DataLossMinutes: 60,  Chosen: "backup-restore"),
};

var gaps = 0;
foreach (var s in services)
{
    var chosen = strategies[s.Chosen];
    var meets = chosen.RtoMinutes <= s.ToleranceMinutes && chosen.RpoMinutes <= s.DataLossMinutes;
    var cheapest = strategies.Values.Where(x => x.RtoMinutes <= s.ToleranceMinutes && x.RpoMinutes <= s.DataLossMinutes)
                             .OrderBy(x => x.RelativeCost).FirstOrDefault();
    if (!meets) gaps++;
    Console.WriteLine($"{s.Name,-22} chosen={s.Chosen,-15} {(meets ? "MEETS" : "GAP  ")}  cheapest that meets: {cheapest?.Name ?? "none (redesign)"}");
}

// Downtime budget arithmetic for an availability target (30-day month).
foreach (var target in new[] { 99.9, 99.95, 99.99 })
    Console.WriteLine($"{target}% availability = {(100 - target) / 100 * 30 * 24 * 60:0.0} minutes of downtime per 30 days");
return gaps == 0 ? 0 : 1;

public sealed record Strategy(string Name, int RtoMinutes, int RpoMinutes, int RelativeCost);

public sealed record Service(string Name, int ToleranceMinutes, int DataLossMinutes, string Chosen);

Expected output (the program exits with code 1 because it finds gaps, which is useful in a CI step):

Payment initiation     chosen=pilot-light     GAP    cheapest that meets: warm-standby
Account balances       chosen=warm-standby    MEETS  cheapest that meets: warm-standby
Regulatory reporting   chosen=backup-restore  MEETS  cheapest that meets: backup-restore
Notifications          chosen=backup-restore  GAP    cheapest that meets: pilot-light
99.9% availability = 43.2 minutes of downtime per 30 days
99.95% availability = 21.6 minutes of downtime per 30 days
99.99% availability = 4.3 minutes of downtime per 30 days

Code explained line by line

  1. Strategy holds the planning RTO and RPO (in minutes) and a relative cost index, not a price. The index lets the program rank options without pretending to know your bill.
  2. Service captures the three facts the business gave you: impact tolerance, data-loss tolerance and the strategy currently chosen.
  3. The dictionary initialiser uses named arguments (RtoMinutes: 24 * 60) so each number's meaning is visible at the call site.
  4. meets is true only when both the RTO is within the tolerance and the RPO is within the data-loss tolerance. Passing one is not enough.
  5. cheapest filters strategies that meet both conditions and sorts by RelativeCost. FirstOrDefault returns null if none qualifies, which the output reports as "redesign": a requirement that no listed strategy can satisfy needs a different approach.
  6. The payments row shows a gap: pilot light's planning RTO (4 hours) exceeds the 2-hour tolerance. Notifications show a gap because 24 hours of backup-and-restore exceeds their 8-hour tolerance.
  7. The availability loop computes the downtime budget: (100 - target) / 100 * 30 * 24 * 60 minutes in a 30-day month. These are the numbers to put next to an availability claim.
  8. return gaps == 0 ? 0 : 1 makes the checker usable as a pipeline gate: a design that fails its own tolerance cannot merge.

Security considerations

⚠ Learning shortcut — not for production

The lab vault uses a short minimum retention, a single account and a tag-based selection. Production recovery uses a separate recovery account with its own administrators, a compliance-mode vault lock after a cooling-off period agreed with risk, multi-Region encryption keys, tested restore permissions that do not rely on the workload account, and a break-glass role stored under dual control.

  • Protect the failover lever. Failover can itself cause an outage or data loss. Require two approvals, log everything, and allow only a small named group.
  • Assume the attacker is in the primary Region. Replicated data may be corrupted. Immutable backups in another account are the line of last defence.
  • Test restoring into isolation. A clean-room account lets you scan restored data before reconnecting it.
  • Do not widen security to make DR easier. Pre-provision narrow roles in the DR Region rather than granting broad permissions "for the emergency".
  • Keep DR runbooks safe but reachable. They must be accessible when your primary Region and identity provider are down, yet not public.

Banking use case

During a game day, Harbourline simulates the loss of the primary Region at 10:00. Synthetic probes alarm at 10:03. The incident commander, with the head of payments as second approver, declares failover at 10:08 according to pre-agreed criteria. The runbook promotes the Aurora secondary, scales the standby from 4 to 40 tasks, flips the routing control and runs 20 synthetic payments. Customers see payments succeed again at 10:41, 33 minutes after the decision and 41 minutes after the start. Against a 30-minute target RTO the test shows a gap of 11 minutes, mostly caused by a DR-Region vCPU quota that limited scale-up. The team raises the quota, adds a pre-warmed capacity step and retests in the next quarter. The timeline and the gap analysis are the evidence for the operational-resilience file ⚠ VERIFY what your firm must submit.

Common mistakes

  • Equating Multi-AZ with DR. It protects against a data-centre failure only.
  • Setting RTO equal to tolerance. Detection and decision time consume part of it; leave margin.
  • Forgetting quotas in the DR Region. Quotas are per Region and are not replicated.
  • Untested failback. Recovering in the DR Region is half the job; returning without data loss is the other half.
  • Backups in the same account as the workloads. One compromised admin deletes both.
  • Ignoring partners and on-premises links. The core or a screening provider may not accept calls from the DR Region.
  • Assuming typical RTO figures apply to you. Only measured results count.

Troubleshooting

SymptomLikely causeFix
Cross-Region copy job failsDestination vault missing, KMS key not usable in the destination, or role permissionsCreate the vault first; check key policy and the plan role
Restore from backup is far slower than plannedData volume larger than the test, or throughput limitsRe-time with production-scale data; consider continuous backup or replicas
Aurora secondary promotion succeeds but app cannot connectSecrets, endpoints or security groups not mirroredReplicate secrets, use a Region-agnostic connection abstraction, mirror networking
DNS failover does not move trafficHigh TTL or health check misconfigurationLower TTL ahead of time; test health checks; use routing controls
Standby cannot scale upService quota or capacity limit in the DR RegionRaise quotas in advance; reserve capacity where necessary
Decryption fails in the DR RegionSingle-Region keyUse multi-Region keys and test decryption
Vault cannot be deletedVault Lock retention activeWait out retention in the lab; use short retention only

Cost and cleanup

The C# checker is free. Everything else here is design-only by default: Aurora Global Database secondaries, DynamoDB replicas, cross-Region backup copies and warm-standby compute bill continuously ⚠ VERIFY. If you deploy the backup stack, create small vaults, use a tiny test resource, and delete them the same day. Vault Lock retention can delay deletion.

# Delete recovery points before the vault (list first to get ARNs).
aws backup list-recovery-points-by-backup-vault --backup-vault-name hl-m14-dr-vault \
  --profile learn-sandbox --region us-west-2

aws backup delete-backup-vault --backup-vault-name hl-m14-dr-vault \
  --profile learn-sandbox --region us-west-2

# Remove replicas and global clusters if you created any, then tear down IaC.
aws dynamodb update-table --table-name hl-m14-idempotency \
  --replica-updates '[{"Delete":{"RegionName":"us-west-2"}}]' \
  --profile learn-sandbox --region us-east-1
cdk destroy --profile learn-sandbox
terraform destroy

# Confirm nothing tagged for the course remains, in both Regions.
aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=cea-lab \
  --profile learn-sandbox --region us-east-1
aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=cea-lab \
  --profile learn-sandbox --region us-west-2

Exercises

  • 🟢 Guided: run the checker, then change Payment initiation to warm-standby and Notifications to pilot-light. Predict the exit code and the output.
  • 🟡 Challenge: extend the checker to read measured RTO and RPO from a JSON test-results file and flag any service whose measured time exceeds its target, even if the planning value passed.
  • 🔴 Live problem: a ransomware scenario encrypts the primary database and the replicated copy within minutes. Write the recovery plan: what you restore from, which account performs it, how you decide the clean restore point, how you test the restored data before reconnecting, and what you tell the board about the RPO.

Assignment

Add architecture/disaster-recovery/ to your portfolio: a service-to-tolerance mapping table for five fictional services, the 15-section document for one of them, a dependency map highlighting the slowest-to-recover component, an ADR choosing warm standby over active-active for payments, a runbook with timing checkpoints, and the checker with unit tests.

Solutions

Knowledge check

Interview questions

  • How do you decide between warm standby and active-active? Compare the tolerance, RTO and RPO with each strategy's measured performance and cost; choose the cheapest that meets them and accept the complexity only when the tolerance demands it.
  • What are the hidden dependencies in a Region failover? Identity, DNS, keys and secrets, certificates, quotas, partner allow-lists, on-premises connectivity, deployment pipelines and runbook access.
  • How is cyber recovery different from DR? The damage may be replicated, so you need immutable, delayed, separate-account backups and clean-room restoration.
  • How do you prove your DR works? Timed tests with recorded timelines, gap analysis, fixes and retests, retained as evidence.
  • Who decides to fail over? A named incident commander against pre-agreed criteria, with a second approver, because failover has its own risk.

Certification practice

Completion checklist

  • I can chain important business service, impact tolerance, RTO, RPO and strategy
  • I ran the checker and understood why two services show gaps
  • My dependency map names the slowest-to-recover component
  • My plan separates AZ or Region DR from cyber recovery
  • I ran cleanup in both Regions and confirmed nothing tagged Project=cea-lab remains

Key takeaways

  • Start from the customer service and its impact tolerance, then derive RTO, RPO and strategy.
  • Choose the cheapest strategy that meets the numbers, and replace planning figures with measured test results.
  • Dependencies (identity, keys, quotas, partners, networks) usually decide real recovery time.
  • Cyber recovery needs immutable backups in a separate account and a clean-room restore.
  • Tolerances, testing duties and Region choices need risk, compliance and data-protection review.

Further reading

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page