Module 12: High Availability & Disaster Recovery
Design banking systems that keep running when a server, an Availability Zone or a whole Region fails, and prove it with tested recovery. You learn auto scaling, resilience patterns, the four disaster-recovery strategies with RTO and RPO targets, backup and restore testing, and chaos engineering.
Assumes: .NET 10 LTS · AWS SDK for .NET v4 · Region us-east-1 (DR Region us-west-2 for design exercises). Verified against official sources on 2026-11-18.
Design banking systems that keep running when a server, an Availability Zone or a whole Region fails, and prove it with tested recovery. You learn auto scaling, resilience patterns, the four disaster-recovery strategies with RTO and RPO targets, backup and restore testing, and chaos engineering.
| Estimated time | About 16 hours (lessons 10 h · labs 4 h · project 2 h) |
| Priority for you | High — operational resilience is a regulatory expectation in banking |
| Level | Intermediate → advanced |
| Certifications | SAA-C03 · SOA-C03 · DOP-C02 · SAP-C03 |
| Cost profile | 🟢 design + local resilience code · 🟡 short FIS and restore tests · 🔴 multi-Region (design-only) |
Why this matters in banking
Payments, card authorisations and online banking are expected to be available around the clock, and UK and EU regulators expect banks to set impact tolerances for important business services and prove they can stay within them. A disaster-recovery plan that has never been tested is a hope, not a plan. In this module you learn to turn "the system must be resilient" into measurable targets — RTO (recovery time objective: how long you may be down) and RPO (recovery point objective: how much data you may lose) — then pick an architecture that meets them at a cost the business accepts, and rehearse the failover.
Analogy: a bank branch keeps spare keys in a second safe (backups), has a second branch on standby across town (warm standby), and runs fire drills (game days). Availability Zones are separate buildings on the same street; Regions are different cities.
From .NET/Azure to AWS
| You know (Microsoft/Azure) | AWS equivalent | What's different |
|---|---|---|
| Availability Zones / availability sets | Availability Zones (Multi-AZ deployments) | Most AWS managed services offer a Multi-AZ switch; spreading across AZs is your job for EC2/ECS |
| VM Scale Sets / App Service scale-out | EC2 Auto Scaling, ECS service auto scaling, Lambda concurrency | Target-tracking policies on CPU, requests or queue depth |
| Azure Load Balancer / Application Gateway | NLB / ALB | Health checks decide which targets receive traffic |
| Azure Traffic Manager / Front Door | Route 53 health checks and failover routing, CloudFront | DNS-based failover; TTLs matter |
| SQL Server Always On / Azure SQL geo-replication | RDS Multi-AZ, Aurora Global Database | Aurora Global gives cross-Region replicas with managed failover |
| Cosmos DB multi-region writes | DynamoDB global tables | Multi-Region, multi-active tables with conflict resolution |
| Azure Backup / Recovery Services vault | AWS Backup with vaults, Vault Lock and cross-account copies | Restore testing is a built-in feature |
| Azure Site Recovery | AWS Elastic Disaster Recovery, pilot light / warm standby designs | Many AWS DR designs are built from services rather than one product |
| Azure Chaos Studio | AWS Fault Injection Service (FIS) | Experiments with stop conditions tied to CloudWatch alarms |
| Polly in .NET | Polly / Microsoft.Extensions.Http.Resilience running on AWS | The patterns are the same; AWS SDK v4 has its own retry modes |
Learning objectives
By the end of this module you can:
- Configure auto scaling for EC2, ECS services, Lambda concurrency, DynamoDB and Aurora, and explain what each scales on.
- Explain Multi-AZ vs multi-Region and choose between active-active and active-passive designs.
- Apply timeouts, retries with jitter, bulkheads, circuit breakers and load shedding in a .NET API.
- Compare backup and restore, pilot light, warm standby and multi-site active-active against RTO, RPO and cost.
- Build backup plans with AWS Backup, cross-Region and cross-account copies, Vault Lock, and automated restore testing.
- Design failover with Route 53, Aurora Global Database and DynamoDB global tables.
- Use CloudFront and ElastiCache to protect back ends and improve performance.
- Run a controlled chaos experiment with FIS and a game day, and assess a workload with Resilience Hub and a Well-Architected review.
Prerequisites
- Module 2: Networking — AZs, subnets, load balancers, Route 53
- Module 3: .NET on AWS — ECS services, health checks and scaling basics
- Module 6: Databases — RDS, Aurora, DynamoDB, backups and PITR
- Module 11: Monitoring — alarms that drive scaling and failover
How the pieces fit
Moving right reduces downtime and data loss but increases cost and complexity. A bank usually mixes them: active-active for card authorisation, warm standby for online banking, backup and restore for internal reporting.
Lessons
- Availability vocabulary — fault tolerance vs high availability vs DR, impact tolerances, RTO and RPO in plain language.
- Auto Scaling everywhere — EC2 Auto Scaling groups, ECS service scaling, Lambda reserved and provisioned concurrency, DynamoDB and Aurora scaling.
- Elastic Load Balancing for resilience — cross-zone balancing, health checks, deregistration delay and zonal shift basics.
- Multi-AZ by default — what Multi-AZ means for RDS, Aurora, ElastiCache, ECS and your own stateless .NET services.
- Resilience patterns in .NET — timeouts, retries with backoff and jitter, bulkheads, circuit breakers and load shedding with
Microsoft.Extensions.Http.Resilience. - The four DR strategies — backup and restore, pilot light, warm standby and multi-site active-active, with RTO/RPO and cost.
- Active-active vs active-passive — data consistency, write conflicts and why most banking ledgers stay single-writer.
- Failover with Route 53 — health checks, failover and weighted routing, TTLs and the risk of split brain.
- Lab: Resilient Account API — the hands-on checkpoint for lessons 1 to 8 (see the labs section below).
- Database failover — Aurora Global Database, DynamoDB global tables, RDS cross-Region read replicas and promotion runbooks.
- AWS Backup in depth — plans, vaults, Vault Lock, cross-Region and cross-account copies, and restore testing.
- Caching for performance and protection — CloudFront and ElastiCache, cache-aside in .NET, and what must never be cached.
- Performance and cost optimisation — right-sizing scaled fleets, Graviton, Savings Plans awareness and the cost of idle DR capacity.
- Chaos engineering with AWS Fault Injection Service — experiment templates, stop conditions, and a safe first experiment.
- AWS Resilience Hub and game days — resilience policies, assessments, and running a realistic failover exercise.
- Well-Architected reviews — the Reliability pillar and the Financial Services Industry Lens applied to a banking API.
Labs and module project
- Lab: Resilient Account API covers Lab 12.1 (🟢) and Lab 12.2 (🟡): a layered Polly-based resilience pipeline proven locally against a fault-injecting stub (86 tests that run without an AWS account), then an ECS service with target-tracking scaling behind an ALB across three AZs; generate load, drill an AZ failure, then destroy. The code is in
labs/m12-resilient-account-api/. - Lab 12.3 (🟡) — AWS Backup plan for a DynamoDB table with a restore test, inside AWS Backup in depth; delete the plan, vault contents and table afterwards.
- Lab 12.4 (🟡) — A short FIS experiment that stops ECS tasks, with a CloudWatch alarm as the stop condition, inside Chaos engineering with AWS Fault Injection Service.
- Lab 12.5 (🟢) — Tabletop DR drill: walk through the failover runbook for a Region outage and time each step on paper, inside AWS Resilience Hub and game days.
Module project — highly available banking API with a tested DR plan. Design the Account API for a stated RTO and RPO (for example, RTO 30 minutes, RPO 5 minutes), choose a DR strategy and justify it, write the failover and failback runbook, and run a failover drill (live in one Region across AZs, design-only across Regions). The design half is Part 6 of the lab. This feeds Capstone 5 — Highly Available Banking Application.
When you finish, keep the Module 12 cheat sheet handy for revision.
Cost profile
| Part | Label | Free alternative |
|---|---|---|
| Resilience patterns, DR strategy design, runbooks | 🟢 free | Local code and diagrams |
| ECS/EC2 auto scaling with ALB | 🟡 credits | Deploy, load test and destroy within an hour |
| AWS Backup restore test | 🟡 credits | Use a tiny DynamoDB table; delete recovery points afterwards |
| FIS experiment | 🟡 credits | Keep it to minutes; or simulate faults locally with a stub |
| Resilience Hub assessment | 🟡 credits | Run one assessment then delete the application ⚠ VERIFY current pricing |
| Multi-Region warm standby, Aurora Global Database, DynamoDB global tables | 🔴 paid | Design-only exercise plus a tabletop failover drill |
| ElastiCache cluster | 🔴 paid | Run Redis/Valkey in Docker locally; design the ElastiCache version |
Certification alignment
| Exam | Domains (weight) | What this module covers |
|---|---|---|
| SAA-C03 | Resilient (26%) · High-Performing (24%) | Multi-AZ, DR strategies, scaling, caching |
| SOA-C03 | Reliability (22%) | Auto scaling, backups, restore testing, failover |
| DOP-C02 | Resilience (15%) | Resilient deployments, multi-Region designs, recovery automation |
| SAP-C03 | Resilience & Migration (weight ⚠ VERIFY) | Complex DR trade-offs across Regions and accounts |
The domain labels are short forms; see each exam page for the official wording.
Service availability notes
Service changes — verified 2026-11-17
Amazon Application Recovery Controller (ARC) – Readiness Check is in maintenance (announced 2026-03-31) and closed to new customers. Other ARC capabilities such as routing control and zonal shift are not listed as affected ⚠ VERIFY before building a lab on them.
Key AWS services
Amazon EC2 Auto Scaling · Application Auto Scaling · Amazon ECS · AWS Lambda · Elastic Load Balancing · Amazon Route 53 · Amazon RDS Multi-AZ · Amazon Aurora (Global Database, Serverless v2) · Amazon DynamoDB (global tables) · AWS Backup · AWS Elastic Disaster Recovery · Amazon CloudFront · Amazon ElastiCache · AWS Fault Injection Service · AWS Resilience Hub · AWS Well-Architected Tool · Amazon CloudWatch
Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.
Module 11 cheat sheet
One-page revision summary for Monitoring, Logging & Troubleshooting: services, Azure mapping, CLI commands, limits, decision rules, exam traps and cost traps.
Availability vocabulary: HA, fault tolerance, DR, RTO and RPO
Speak precisely about failure: separate high availability from fault tolerance and disaster recovery, turn a target such as 99.95% into minutes of allowed downtime, and set RTO, RPO and impact-tolerance numbers a bank can test.