Skip to Content
Cloud Engineering Academy
5. Security for Banking & Finance

UK GDPR and data residency

Design AWS workloads that respect UK GDPR: lawful basis, minimisation, the right to erasure versus immutable logs (pseudonymisation and crypto-shredding in .NET), international transfers, Region choice and residency guardrails. Concepts only — legal and DPO review required.

advanced⏱ 2 h 30 min🟢 FreeSCS-C03SAA-C03CLF-C02

Assumes: .NET 10 LTS · AWS SDK for .NET v4 (AWSSDK.KeyManagementService, AWSSDK.DynamoDBv2 4.x) · AWS CLI v2 · CDK v2 (Amazon.CDK.Lib 2.277.0) · Terraform AWS provider v6 · UK GDPR as amended by the Data (Use and Access) Act 2025. Verified against official sources on 2026-11-18.

Read this first

This lesson explains data-protection concepts so engineers can design sensibly and ask the right questions. It is not legal advice. Decisions about lawful basis, retention periods, erasure exceptions, international transfers and DPIAs belong to Harbourline's Data Protection Officer (DPO) and legal team. No AWS service, Region or architecture makes a workload "GDPR compliant" by itself.

Learning objectives

By the end of this lesson you can:

  • Explain the UK data-protection framework (UK GDPR, Data Protection Act 2018, and the changes made by the Data (Use and Access) Act 2025) and the controller/processor split between Harbourline and AWS.
  • Apply the principles that most affect engineering: lawful basis, purpose limitation, minimisation, storage limitation, integrity and confidentiality, and accountability.
  • Resolve the tension between the right to erasure and immutable audit logs with pseudonymisation, retention-by-design and crypto-shredding, implemented in .NET with KMS.
  • Distinguish data residency, data sovereignty and international transfers, and choose Regions for a UK bank with an EU subsidiary.
  • Enforce Region choice with SCPs and Control Tower controls, and know the gaps (global services, support access, replication).
  • Recognise when a change needs a DPIA, a record-of-processing update or a transfer assessment.

Prerequisites

Why this matters in banking

Harbourline holds exactly the data UK GDPR cares most about: names, addresses, dates of birth, account numbers, transaction histories that reveal health, religion or politics by inference (a monthly payment to a church, a clinic or a trade union), and identity documents from KYC checks. Three conversations happen every month:

  • A former customer asks the bank to delete everything. Engineering points out that transaction records sit in S3 buckets under Object Lock and CloudTrail logs are immutable by design. Legal points out that anti-money-laundering rules require keeping customer due-diligence records and transaction records for five years after the relationship ends. Who wins, and what do you actually delete?
  • A product team wants to use a new US-based analytics SaaS. Is that a restricted transfer? What paperwork is needed?
  • An architect proposes us-east-1 "because the new features launch there first". Is that allowed for customer data?

The answers are design decisions. If they're made late, you end up with personal data in places you can't delete it, can't explain, and can't defend to the regulator, the Information Commissioner's Office (ICO).

The idea in plain language

UK GDPR is the UK's version of the EU General Data Protection Regulation, kept after Brexit and read with the Data Protection Act 2018. The Data (Use and Access) Act 2025 (DUAA) received Royal Assent on 19 June 2025 and amended UK GDPR in stages; according to published commentary, the main data-protection changes commenced in early 2026, including a new lawful basis called recognised legitimate interests and a duty for controllers to have a data-protection complaints procedure (required by 19 June 2026). ⚠ VERIFY commencement details against ICO guidance and the commencement regulations before relying on them.

Harbourline's EU subsidiary is subject to the EU GDPR as well. Day to day, the engineering implications are almost identical.

Roles:

RoleWhoMeaning for engineering
ControllerHarbourlineDecides why and how personal data is processed; accountable for compliance
ProcessorAWS (for customer content), SaaS vendorsProcesses on Harbourline's instructions under a contract (AWS's GDPR data processing addendum is incorporated into its service terms ⚠ VERIFY current wording)
Data subjectCustomers, staff, prospectsHave rights: access, rectification, erasure, restriction, portability, objection, and rights about automated decisions
Supervisory authorityICO (UK); EU authorities for the subsidiaryInvestigates complaints and breaches; can fine

The seven principles (Article 5), translated into engineering terms:

PrincipleWhat it means for an engineer
Lawfulness, fairness, transparencyEvery processing purpose has a lawful basis recorded by the DPO; don't invent new uses of data in code
Purpose limitationData collected for payments isn't reused for marketing without a compatible basis
Data minimisationCollect and copy only the fields you need; tokens and pseudonyms in non-production and analytics
AccuracyCustomers can correct data; corrections propagate to copies
Storage limitationEvery store has a retention period and automated deletion
Integrity and confidentialityEncryption, least privilege, logging, tested backups — most of Module 5
AccountabilityYou can prove the above: records of processing, DPIAs, logs, evidence

Lawful bases (Article 6) for a bank: contract (running the account), legal obligation (AML checks, tax reporting), legitimate interests (fraud detection, with a balancing test), recognised legitimate interests (a DUAA addition for listed purposes such as crime prevention and safeguarding, without the balancing test), consent (marketing), vital interests and public task (rarely). The DPO assigns them; engineers make sure data flows match what was assigned.

Analogy: the bank's paper archive. Every file has a label saying why it's kept, who may open it, and a shred date. If a customer asks for their file to be destroyed, the archivist checks whether a law requires keeping it; if so, it's moved to a restricted shelf until the date passes, then shredded. Crypto-shredding is the digital version: instead of finding every copy of every page, you destroy the only key that can read them.

Residency, sovereignty and transfers

TermMeaningIs it a UK GDPR requirement?
Data residencyWhere data is physically stored and processedNo general requirement to keep data in the UK. Contracts, sector rules or the bank's own policy may require it
Data sovereigntyWhich country's laws can compel access to dataA risk consideration in transfer assessments and supervisory expectations
Restricted transferPersonal data sent to, or made accessible from, a country outside the UKAllowed only with an adequacy regulation ("data bridge"), appropriate safeguards (the UK IDTA or the UK Addendum to the EU SCCs, BCRs) or a narrow exception, with a transfer risk assessment

Facts to anchor on (checked 2026-11-18):

  • The UK recognises the EEA countries as adequate, and has a UK–US data bridge (the UK Extension to the EU–US Data Privacy Framework) for certified US organisations since October 2023.
  • The European Commission renewed its adequacy decisions for the UK on 19 December 2025; they now run until 27 December 2031, after assessing the DUAA changes. Data can therefore flow from Harbourline's EU subsidiary to the UK parent without extra transfer tools.
  • Using an AWS Region in the UK (London, eu-west-2) doesn't by itself avoid all transfers: support access, global services and sub-processors can involve access from elsewhere. The DPO reviews AWS's sub-processor list and contractual terms.

What problem the AWS services solve

AWS doesn't decide your lawful basis or retention, but it gives you controls to implement and prove those decisions.

AWS capabilityData-protection useLimits and sharp edges
Region choice (eu-west-2 London, eu-west-1 Ireland, etc.)Content stays in the Regions you choose; AWS commits not to move customer content out of your chosen Region without your instruction, except as necessary to provide the service or comply with law ⚠ VERIFY current wording in the AWS Data Privacy FAQSome services are global (IAM, Organizations, Route 53, CloudFront) and process configuration data outside the Region
SCPs with aws:RequestedRegionDeny API calls in unapproved RegionsMust exempt global services; doesn't control where a SaaS you call stores data
AWS Control Tower Region deny controlManaged version of the above for landing zonesApplies to enrolled OUs; check the exempted services list
AWS European Sovereign CloudA separate AWS partition operated in the EU (first Region in Brandenburg, Germany, eusc-de-east-1, generally available January 2026)Separate partition: separate accounts, IAM and endpoints; not all services available; relevant to EU sovereignty requirements, not UK ones
KMS customer managed keysEncryption with keys you control; key deletion as crypto-shreddingPer-customer KMS keys are costly (USD 1/key/month) — use per-customer data keys wrapped by one CMK instead
Macie, CloudWatch Logs data protectionFind and mask personal data where it shouldn't bePattern-based; tune for UK identifiers
S3 Lifecycle, DynamoDB TTL, CloudWatch Logs retentionStorage limitation, automatedObject Lock retention overrides lifecycle deletion until retention ends
AWS ArtifactAWS's certifications and agreements (for example, its data processing terms and ISO 27701 reports) for your DPO's due diligenceEvidence of AWS's controls, not yours
AWS Config, CloudTrailAccountability evidence: who accessed what, configuration historyThere is no GDPR conformance pack; GDPR evidence is a manual, DPO-led activity

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
Azure "UK South / UK West" regionseu-west-2 (London); AWS has one UK RegionDR for UK-only data typically uses multiple AZs in London, or a second Region with DPO approval
Azure Policy "Allowed locations"SCP with aws:RequestedRegion, Control Tower Region denySCPs act on API calls rather than on resource location properties
Microsoft EU Data BoundaryAWS European Sovereign Cloud (separate partition)AWS's approach is a separate partition rather than a boundary inside the commercial cloud
Microsoft Purview data map / DSAR toolingMacie + your own data inventory + DSAR workflowNo managed DSAR product; you build or buy
Microsoft Products and Services DPAAWS GDPR data processing addendum (in AWS Service Terms)Both are processor terms; your DPO reviews either
Key Vault key deletion for crypto-shreddingKMS ScheduleKeyDeletion (7–30 day wait), or deleting wrapped data keysKMS deletion is delayed and irreversible; per-customer data keys avoid per-key costs

Architecture

The pattern:

  1. Directly identifying data (name, address, email, phone, date of birth) lives in one profile store, with each customer's fields encrypted under a per-customer data key.
  2. Transactions and logs use a pseudonymous reference (an HMAC of the customer ID) instead of names and emails.
  3. Erasure deletes the customer's data key (crypto-shredding the profile and any backups or copies encrypted with it) once legal retention has expired, and records that it did so.

Security-boundary view

StorePersonal data?Retention driverErasure method
Profile (encrypted fields)Directly identifyingContract + legal obligation (e.g. AML: five years after relationship ends)Delete the per-customer data key, then the row
TransactionsPseudonymous (linkable only via the key ring and HMAC key)Legal obligationRetain for the legal period; after crypto-shredding, records are no longer attributable through Harbourline's systems ⚠ legal view needed on whether this is anonymisation
Audit logs (Object Lock)Pseudonymous IDs, IPsSecurity and legal obligationsRetain to the Object Lock date; never put names or emails in them
BackupsSame as sourceSame as source, plus backup cycleEncrypted with the same data keys, so shredding covers them

Step-by-step implementation

Map the data before you design

Create a data inventory for the service: each field, its classification, the purpose and lawful basis (from the DPO's record of processing activities), where it's stored and copied, who can access it, and its retention. This feeds the threat model, the DPIA and every later erasure request. Macie and log data-protection policies help you find surprises.

Minimise and pseudonymise by default

Only the profile service holds direct identifiers. Everywhere else, use customer_ref = HMAC(customer_id) computed with a KMS HMAC key. Non-production environments get synthetic data (this platform's generator) or masked copies, never production personal data.

Encrypt each customer's personal fields with their own data key

Call KMS GenerateDataKey once per customer, store the encrypted data key in a key-ring table, and encrypt fields locally with AES-GCM. One CMK protects all the data keys; you pay per CMK, not per customer.

Separate retention from erasure

Give every store an explicit retention rule. When a customer asks for erasure, the workflow checks legal holds (AML retention, active complaints, litigation, fraud investigations). If a hold applies, it restricts processing (flag the record, remove it from marketing and analytics) and schedules erasure for when the hold ends. If not, it crypto-shreds now.

Keep personal data out of immutable stores

Object Lock and CloudTrail log validation are for evidence. Design log formats with pseudonymous IDs, and configure log data-protection policies as a safety net. If a name gets into an Object Lock bucket in compliance mode, you can't remove it until retention ends — which is a data-protection incident in its own right.

Constrain Regions and document transfers

Apply a Region-deny SCP (or Control Tower control) allowing only approved Regions. List every service and vendor outside them (global AWS services, SaaS tools) in the record of processing, and have the DPO assess any restricted transfer.

Run a DPIA for high-risk processing

New processing that is likely to result in high risk — large-scale profiling for fraud, new biometric login, automated credit decisions — needs a Data Protection Impact Assessment before it starts. Engineers provide the data flows, controls and threat model; the DPO runs the assessment.

AWS Console instructions

  1. Which Regions are enabled? Console → account menu (top right) → Account → AWS Regions. Opt-in Regions are disabled by default. Note which are enabled.
  2. Where are my buckets? Console → S3 → General purpose buckets — the AWS Region column shows each bucket's Region.
  3. Control Tower (if you have a landing zone): Console → AWS Control Tower → Landing zone settings → Region deny, and Governed Regions. In a single sandbox account you won't have Control Tower; read only.
  4. AWS's data processing terms: Console → AWS Artifact → Agreements shows account-level agreements available to accept; the GDPR DPA is incorporated into the AWS Service Terms rather than accepted separately ⚠ VERIFY. Share the relevant reports with your DPO.
  5. Log retention: Console → CloudWatch → Log groups — the Retention column; "Never expire" on a log group containing personal data is a storage-limitation finding.

AWS CLI

# 1. List Regions and whether they are enabled for this account.
aws account list-regions --region-opt-status-contains ENABLED ENABLED_BY_DEFAULT \
  --query "Regions[].RegionName" --output text --profile learn-sandbox

# 2. Every S3 bucket and its Region (a quick residency inventory).
for b in $(aws s3api list-buckets --query "Buckets[].Name" --output text --profile learn-sandbox); do
  echo "$b $(aws s3api get-bucket-location --bucket "$b" --query LocationConstraint --output text --profile learn-sandbox)"
done
# A LocationConstraint of "None" means us-east-1.

# 3. Log groups with no retention (storage-limitation risk).
aws logs describe-log-groups \
  --query "logGroups[?retentionInDays==null].logGroupName" --output text --profile learn-sandbox

# 4. Set a retention period on a log group (example: 400 days).
aws logs put-retention-policy --log-group-name /hl/m05/customer-api --retention-in-days 400 --profile learn-sandbox

# 5. Simulate whether the Region-deny SCP would block an API call in eu-central-1 (in an org with the SCP attached,
#    just try a harmless call; expect an explicit deny from a service control policy).
aws ec2 describe-vpcs --region eu-central-1 --profile learn-sandbox

Labs on this platform run in us-east-1 with fictional data only. A production Harbourline workload with real UK customer data would use eu-west-2 or another DPO-approved Region.

Infrastructure as Code

A Region-deny SCP for the workloads OU. It allows only London and Ireland, exempting global services whose APIs are served from us-east-1. The exempt list here is illustrative; base yours on AWS's current example and test it.

RegionGuardrailStack.cs
using System.Collections.Generic;
using System.Text.Json;
using Amazon.CDK;
using Amazon.CDK.AWS.Organizations;
using Constructs;

namespace Harbourline.Platform.Infra;

public sealed class RegionGuardrailStack : Stack
{
    private static readonly string[] GlobalServiceActions =
    {
        "iam:*", "organizations:*", "sts:*", "account:*", "support:*", "health:*",
        "route53:*", "route53domains:*", "cloudfront:*", "waf:*", "shield:*",
        "budgets:*", "ce:*", "cur:*", "globalaccelerator:*", "trustedadvisor:*",
        "artifact:*",
    };

    public RegionGuardrailStack(Construct scope, string id, string workloadsOuId, IStackProps? props = null)
        : base(scope, id, props)
    {
        var policy = new
        {
            Version = "2012-10-17",
            Statement = new object[]
            {
                new
                {
                    Sid = "DenyOutsideApprovedRegions",
                    Effect = "Deny",
                    NotAction = GlobalServiceActions,
                    Resource = "*",
                    Condition = new Dictionary<string, object>
                    {
                        ["StringNotEquals"] = new Dictionary<string, object>
                        {
                            ["aws:RequestedRegion"] = new[] { "eu-west-2", "eu-west-1" },
                        },
                        ["ArnNotLike"] = new Dictionary<string, object>
                        {
                            ["aws:PrincipalArn"] = new[] { "arn:aws:iam::*:role/hl-platform-breakglass" },
                        },
                    },
                },
            },
        };

        _ = new CfnPolicy(this, "RegionGuardrail", new CfnPolicyProps
        {
            Name = "hl-approved-regions",
            Description = "Only eu-west-2 and eu-west-1 for workload accounts (DPO-approved list)",
            Type = "SERVICE_CONTROL_POLICY",
            Content = JsonSerializer.Serialize(policy),
            TargetIds = new[] { workloadsOuId },
        });
    }
}

Keep the exemption list as short as possible: every exempted action can be called in any Region. Don't exempt regional services such as KMS, S3 or DynamoDB, or the guardrail stops protecting the data it exists for.

SCPs need an AWS Organization; in a standalone sandbox, treat this as a design exercise. If you have a lab organisation, attach it to a test OU first, because a wrong exemption list can break console sign-in or billing pages.

Complete code

Crypto-shredding with per-customer data keys, and an erasure workflow that respects legal holds.

src/Harbourline.Customers.Privacy/CustomerKeyRing.cs
using System.Security.Cryptography;
using System.Text;
using Amazon.KeyManagementService;
using Amazon.KeyManagementService.Model;

namespace Harbourline.Customers.Privacy;

/// <summary>Stores each customer's wrapped (KMS-encrypted) data key. DynamoDB in production.</summary>
public interface IKeyRingStore
{
    Task<byte[]?> GetWrappedKeyAsync(string customerId, CancellationToken ct);
    Task PutWrappedKeyIfAbsentAsync(string customerId, byte[] wrappedKey, CancellationToken ct);
    Task DeleteWrappedKeyAsync(string customerId, CancellationToken ct);
}

/// <summary>Encrypted field: nonce + ciphertext + tag, Base64 for storage.</summary>
public readonly record struct EncryptedField(string Value);

/// <summary>
/// Per-customer field encryption. Deleting a customer's wrapped key makes every field
/// encrypted with it unreadable everywhere it was copied (tables, backups, exports): crypto-shredding.
/// </summary>
public sealed class CustomerKeyRing(IAmazonKeyManagementService kms, IKeyRingStore store, string cmkId)
{
    private const int NonceSize = 12; // AES-GCM standard nonce
    private const int TagSize = 16;

    public async Task<EncryptedField> EncryptAsync(string customerId, string plaintext, CancellationToken ct = default)
    {
        using var key = await GetOrCreateKeyAsync(customerId, ct).ConfigureAwait(false);
        var nonce = RandomNumberGenerator.GetBytes(NonceSize);
        var data = Encoding.UTF8.GetBytes(plaintext);
        var cipher = new byte[data.Length];
        var tag = new byte[TagSize];

        using (var aes = new AesGcm(key.Bytes, TagSize))
        {
            // The customer ID is authenticated (not encrypted) so a field can't be swapped between customers.
            aes.Encrypt(nonce, data, cipher, tag, Encoding.UTF8.GetBytes(customerId));
        }
        return new EncryptedField(Convert.ToBase64String([.. nonce, .. cipher, .. tag]));
    }

    public async Task<string?> DecryptAsync(string customerId, EncryptedField field, CancellationToken ct = default)
    {
        using var key = await GetExistingKeyAsync(customerId, ct).ConfigureAwait(false);
        if (key is null)
            return null; // key shredded: the data is gone for good

        var blob = Convert.FromBase64String(field.Value);
        var nonce = blob.AsSpan(0, NonceSize);
        var tag = blob.AsSpan(blob.Length - TagSize);
        var cipher = blob.AsSpan(NonceSize, blob.Length - NonceSize - TagSize);
        var plain = new byte[cipher.Length];

        using var aes = new AesGcm(key.Bytes, TagSize);
        aes.Decrypt(nonce, cipher, tag, plain, Encoding.UTF8.GetBytes(customerId));
        return Encoding.UTF8.GetString(plain);
    }

    /// <summary>Crypto-shred: remove the only copy of the wrapped data key.</summary>
    public Task ShredAsync(string customerId, CancellationToken ct = default) =>
        store.DeleteWrappedKeyAsync(customerId, ct);

    private async Task<DataKey> GetOrCreateKeyAsync(string customerId, CancellationToken ct)
    {
        var existing = await GetExistingKeyAsync(customerId, ct).ConfigureAwait(false);
        if (existing is not null)
            return existing;

        var generated = await kms.GenerateDataKeyAsync(new GenerateDataKeyRequest
        {
            KeyId = cmkId,
            KeySpec = DataKeySpec.AES_256,
            EncryptionContext = Context(customerId),
        }, ct).ConfigureAwait(false);

        await store.PutWrappedKeyIfAbsentAsync(customerId, generated.CiphertextBlob.ToArray(), ct).ConfigureAwait(false);
        // Re-read so that concurrent first writes for the same customer converge on one stored key.
        generated.Plaintext.Dispose();
        return await GetExistingKeyAsync(customerId, ct).ConfigureAwait(false)
            ?? throw new InvalidOperationException("Key ring write did not persist.");
    }

    private async Task<DataKey?> GetExistingKeyAsync(string customerId, CancellationToken ct)
    {
        var wrapped = await store.GetWrappedKeyAsync(customerId, ct).ConfigureAwait(false);
        if (wrapped is null)
            return null;

        var decrypted = await kms.DecryptAsync(new DecryptRequest
        {
            KeyId = cmkId,
            CiphertextBlob = new MemoryStream(wrapped),
            EncryptionContext = Context(customerId),
        }, ct).ConfigureAwait(false);
        return new DataKey(decrypted.Plaintext.ToArray());
    }

    private static Dictionary<string, string> Context(string customerId) =>
        new() { ["purpose"] = "customer-pii", ["customerId"] = customerId };

    /// <summary>Wipes the plaintext key from memory when disposed.</summary>
    private sealed class DataKey(byte[] bytes) : IDisposable
    {
        public byte[] Bytes { get; } = bytes;
        public void Dispose() => CryptographicOperations.ZeroMemory(Bytes);
    }
}
src/Harbourline.Customers.Privacy/ErasureWorkflow.cs
namespace Harbourline.Customers.Privacy;

public enum ErasureDecision { ShreddedNow, RestrictedUntilHoldEnds, Rejected }

public sealed record LegalHold(string Reason, DateOnly Until);

public sealed record ErasureResult(ErasureDecision Decision, DateOnly? EraseOn, IReadOnlyList<string> Reasons);

/// <summary>Holds that may override erasure. Real rules come from Legal and the DPO, not from engineering.</summary>
public interface ILegalHoldProvider
{
    Task<IReadOnlyList<LegalHold>> GetHoldsAsync(string customerId, CancellationToken ct);
}

public interface IProcessingRestrictions
{
    Task RestrictAsync(string customerId, DateOnly until, CancellationToken ct); // e.g. remove from marketing, analytics
    Task ScheduleErasureAsync(string customerId, DateOnly on, CancellationToken ct);
}

public interface IPrivacyAudit
{
    Task RecordAsync(string customerRef, string action, string outcome, CancellationToken ct); // pseudonymous ref only
}

public sealed class ErasureWorkflow(
    CustomerKeyRing keyRing,
    ILegalHoldProvider holds,
    IProcessingRestrictions restrictions,
    IPrivacyAudit audit,
    Func<string, string> pseudonymise,
    TimeProvider clock)
{
    public async Task<ErasureResult> HandleAsync(string customerId, bool identityVerified, CancellationToken ct = default)
    {
        var customerRef = pseudonymise(customerId);
        if (!identityVerified)
        {
            await audit.RecordAsync(customerRef, "erasure.request", "rejected:identity-not-verified", ct).ConfigureAwait(false);
            return new ErasureResult(ErasureDecision.Rejected, null, ["Requester identity not verified"]);
        }

        var today = DateOnly.FromDateTime(clock.GetUtcNow().UtcDateTime);
        var active = (await holds.GetHoldsAsync(customerId, ct).ConfigureAwait(false))
            .Where(h => h.Until >= today)
            .ToList();

        if (active.Count > 0)
        {
            var until = active.Max(h => h.Until);
            await restrictions.RestrictAsync(customerId, until, ct).ConfigureAwait(false);
            await restrictions.ScheduleErasureAsync(customerId, until.AddDays(1), ct).ConfigureAwait(false);
            await audit.RecordAsync(customerRef, "erasure.request", $"restricted-until:{until:yyyy-MM-dd}", ct).ConfigureAwait(false);
            return new ErasureResult(ErasureDecision.RestrictedUntilHoldEnds, until.AddDays(1), active.Select(h => h.Reason).ToList());
        }

        await keyRing.ShredAsync(customerId, ct).ConfigureAwait(false);
        await audit.RecordAsync(customerRef, "erasure.request", "shredded", ct).ConfigureAwait(false);
        return new ErasureResult(ErasureDecision.ShreddedNow, today, []);
    }
}
tests/Harbourline.Customers.Privacy.Tests/ErasureWorkflowTests.cs
using Harbourline.Customers.Privacy;
using Microsoft.Extensions.Time.Testing;
using Xunit;

public class ErasureWorkflowTests
{
    private sealed class Holds(params LegalHold[] h) : ILegalHoldProvider
    {
        public Task<IReadOnlyList<LegalHold>> GetHoldsAsync(string id, CancellationToken ct) => Task.FromResult<IReadOnlyList<LegalHold>>(h);
    }

    private sealed class Recorder : IProcessingRestrictions, IPrivacyAudit
    {
        public List<string> Events { get; } = [];
        public Task RestrictAsync(string id, DateOnly until, CancellationToken ct) { Events.Add($"restrict:{until:yyyy-MM-dd}"); return Task.CompletedTask; }
        public Task ScheduleErasureAsync(string id, DateOnly on, CancellationToken ct) { Events.Add($"schedule:{on:yyyy-MM-dd}"); return Task.CompletedTask; }
        public Task RecordAsync(string r, string a, string o, CancellationToken ct) { Events.Add($"audit:{r}:{o}"); return Task.CompletedTask; }
    }

    [Fact]
    public async Task Aml_hold_restricts_and_schedules_instead_of_shredding()
    {
        var clock = new FakeTimeProvider(new DateTimeOffset(2026, 11, 18, 0, 0, 0, TimeSpan.Zero));
        var rec = new Recorder();
        var holds = new Holds(new LegalHold("AML record keeping: 5 years after relationship ended", new DateOnly(2030, 3, 31)));
        var keyRing = new CustomerKeyRing(null!, new InMemoryKeyRing(), "alias/unused"); // not called on this path
        var wf = new ErasureWorkflow(keyRing, holds, rec, rec, id => "ref-" + id.GetHashCode().ToString("x"), clock);

        var result = await wf.HandleAsync("CUS-00012345", identityVerified: true, TestContext.Current.CancellationToken);

        Assert.Equal(ErasureDecision.RestrictedUntilHoldEnds, result.Decision);
        Assert.Equal(new DateOnly(2030, 4, 1), result.EraseOn);
        Assert.Contains("restrict:2030-03-31", rec.Events);
        Assert.DoesNotContain(rec.Events, e => e.Contains("CUS-00012345")); // audit uses the pseudonym only
    }

    private sealed class InMemoryKeyRing : IKeyRingStore
    {
        private readonly Dictionary<string, byte[]> _keys = new();
        public Task<byte[]?> GetWrappedKeyAsync(string id, CancellationToken ct) => Task.FromResult(_keys.GetValueOrDefault(id));
        public Task PutWrappedKeyIfAbsentAsync(string id, byte[] k, CancellationToken ct) { _keys.TryAdd(id, k); return Task.CompletedTask; }
        public Task DeleteWrappedKeyAsync(string id, CancellationToken ct) { _keys.Remove(id); return Task.CompletedTask; }
    }
}

The test project uses xunit.v3 4.0.3 and Microsoft.Extensions.TimeProvider.Testing for FakeTimeProvider (look up the current version on NuGet). In production the pseudonymiser is a KMS HMAC (as in PCI DSS on AWS), not GetHashCode.

Code explained line by line

CustomerKeyRing

  1. IKeyRingStore — the wrapped data keys live in their own table. Only the profile service's role can read it, and the table has point-in-time recovery disabled or short by design, because restoring a deleted key would undo a shred (discuss with your DPO and resilience team).
  2. AesGcm with a 12-byte random nonce and 16-byte tag — authenticated encryption in the .NET base library; tampering causes Decrypt to throw.
  3. Associated data = customerId — the field is cryptographically bound to its owner, so copying Alice's encrypted email into Bob's row fails to decrypt.
  4. Convert.ToBase64String([.. nonce, .. cipher, .. tag]) — a C# 12 collection expression concatenating the three parts into one storable value.
  5. DecryptAsync returns null when the key is gone — callers treat that as "erased", not as an error.
  6. GenerateDataKeyAsync with EncryptionContext {purpose, customerId} — the wrapped key can only be unwrapped with the same context, and CloudTrail records which customer's key was used without showing any personal data beyond the ID. (Use the pseudonymous ref in the context if even IDs in CloudTrail are a concern.)
  7. PutWrappedKeyIfAbsentAsync then re-read — if two requests create a key for a new customer at the same time, a conditional write keeps one and both callers use it.
  8. DataKey : IDisposable with CryptographicOperations.ZeroMemory — plaintext key material is wiped after use.
  9. ShredAsync deletes the wrapped key. Every copy of the customer's encrypted fields — in replicas, backups, exports — becomes unreadable, which is why crypto-shredding is useful when you can't find and edit every copy.

ErasureWorkflow

  1. identityVerified — erasure requests are a fraud vector (an attacker erasing a victim's data); verification is part of the workflow.
  2. pseudonymise(customerId) — audit records about erasure contain only a pseudonym, so the audit trail of the erasure doesn't keep the personal data alive.
  3. LegalHold list from ILegalHoldProvider — the rules (AML retention, complaints, litigation, fraud) are owned by Legal and the DPO; code only applies them.
  4. Active hold → RestrictAsync + ScheduleErasureAsync — the right to restriction applies while a legitimate retention reason exists; erasure is scheduled for the day after the latest hold ends.
  5. No hold → ShredAsync and an audit record. The customer's transactions remain (pseudonymised) for any remaining obligations.

Security considerations

⚠ Learning shortcut — not for production

The workflow above simplifies several things that need legal and DPO input: which holds apply and for how long, whether pseudonymised transactions remain personal data after shredding (often yes, if they can still be linked by other means), how to handle backups that must be restorable for resilience, and how to respond within the statutory one-month deadline (extendable in some cases). Treat the code as a pattern for discussion, not a policy.

  • Pseudonymised data is still personal data under UK GDPR while you (or anyone reasonably likely) can re-identify it. It reduces risk; it doesn't remove obligations.
  • Crypto-shredding is only as good as key hygiene. If the plaintext data key was ever logged, cached in a long-lived store, or the wrapped key is in a backup of the key-ring table, the data isn't shredded.
  • Breach notification: a personal-data breach likely to result in risk to individuals must be reported to the ICO within 72 hours of becoming aware of it. Your incident runbooks must include the DPO early.
  • Non-production data: copying production personal data to test environments needs a lawful basis and equivalent protection; this platform's synthetic data generator avoids the problem.
  • Global services and support: configuration data in IAM, Route 53 and CloudFront is global; avoid putting personal data in resource names, tags or descriptions.

Banking use case

Harbourline receives about 40 erasure requests a month. The privacy operations team logs each in a case tool and triggers the erasure workflow via an internal API. For closed accounts within five years of closure, the workflow restricts processing (removes the customer from marketing lists, analytics extracts and model-training datasets) and schedules erasure. For prospects who never became customers and have no other holds, it crypto-shreds immediately. Monthly, the DPO receives a report generated from the pseudonymous audit events: requests received, restricted, completed, and average time to completion. That report, the data inventory and the code's tests form part of the accountability evidence (see Audit logging and evidence collection).

For residency, Harbourline's SCP allows only eu-west-2 and eu-west-1. When the fraud team wanted a new ML service that wasn't yet in London, the request went to the DPO and architecture board: the options were waiting, using synthetic or anonymised features in another Region, or a documented transfer assessment. They chose to wait for the service in London and use an interim model on SageMaker in eu-west-2.

Common mistakes

  • "GDPR means data must stay in the UK" → not in general; transfers need a mechanism and assessment. Your bank's policy may still require UK residency.
  • "Choosing eu-west-2 makes us compliant" → Region choice is one control among many; principles, rights and accountability still apply.
  • Personal data in Object Lock compliance-mode buckets → can't be erased until retention ends. Pseudonymise before writing.
  • Per-customer KMS keys → USD 1 per key per month adds up to millions for millions of customers; use per-customer data keys wrapped by one CMK.
  • Erasure that deletes from the primary database only → copies in search indexes, caches, analytics, exports and backups remain. Map copies in the data inventory, or encrypt with per-customer keys.
  • Logs with "Never expire" → storage limitation breach waiting to happen; set retention on every log group.
  • Engineers choosing lawful bases → that's the DPO's call; engineers implement and flag mismatches.

Troubleshooting

SymptomLikely causeFix
Console pages for billing or IAM fail after attaching the Region SCPA global service action isn't in the exemption listCompare with AWS's current Region-deny example; test in a sandbox OU
InvalidCiphertextException when unwrapping a data keyEncryption context differs (e.g. customer ID normalised differently)Use exactly the same context keys/values; normalise IDs before use
AuthenticationTagMismatchException in AesGcm.DecryptField copied between customers, data corrupted, or wrong keyExpected protection; investigate the copy path
Erased customer's data reappears after a restoreKey-ring table restored from backupExclude the key ring from routine restores or replay shredding events after restores
Macie finds emails in a "pseudonymous" log bucketAn application logs request bodiesFix logging, add a log data protection policy, assess whether it's a breach with the DPO
aws account list-regions returns AccessDeniedMissing account:ListRegionsAdd the permission or check in the console

Cost and cleanup

This lesson is free as written: concepts, local code with fakes, and read-only CLI commands.

If you try CustomerKeyRing against real KMS: one CMK at USD 1 per month prorated (about USD 0.03 for a day); GenerateDataKey and Decrypt calls fall within the 20,000-request monthly free tier at lab volumes. Clean up:

aws kms schedule-key-deletion --key-id alias/hl-m05-customer-pii --pending-window-in-days 7 --profile learn-sandbox
aws resourcegroupstaggingapi get-resources --tag-filters Key=Lab,Values=m05-gdpr --profile learn-sandbox

Exercises

  • 🟢 Guided: Build a data inventory table for the customers and transactions datasets in CONVENTIONS.md: for each field, classification (direct identifier / indirect / not personal), proposed purpose, where it would be stored in the module project, and a proposed retention (marked "for DPO review").
  • 🟡 Challenge: Extend ErasureWorkflow so it also removes the customer from a search index and an analytics extract table, and records each step. Make it idempotent: running it twice for the same request must not fail or duplicate audit records.
  • 🔴 Live problem: During an incident review you discover that, for three months, the Payments API logged full request bodies (names, IBANs, references) to a CloudWatch log group that streams to an S3 bucket with Object Lock in compliance mode for seven years. What do you do in the next 24 hours, and what do you change?

Assignment

Write a privacy design note for the module project's Account API: data inventory, lawful basis column left for the DPO, Region and residency decision, pseudonymisation approach for logs and analytics, erasure and restriction design, and a list of open questions for the DPO. Add CustomerKeyRing and ErasureWorkflow with tests to your portfolio repository.

Solutions

Knowledge check

Interview questions

  • "How do you design for the right to erasure in a system with immutable logs?" — Keep identifiers out of immutable stores (pseudonyms via HMAC), encrypt profile data with per-customer data keys, crypto-shred on erasure, apply legal holds by restricting processing, and record the outcome pseudonymously.
  • "What's the difference between data residency and data sovereignty?" — Residency is where data is stored and processed; sovereignty is whose laws can compel access. A UK Region addresses residency; sovereignty also depends on the provider's jurisdiction, contracts and access controls.
  • "How do you enforce Region restrictions on AWS?" — An SCP or Control Tower Region-deny control on aws:RequestedRegion, exempting global services, plus Config and inventory checks, and a process for exceptions approved by the DPO.
  • "Who decides retention periods?" — Legal and the DPO, based on legal obligations and purposes; engineering implements them as lifecycle rules, TTLs, log retention and Object Lock periods, and proves they work.
  • "What changed with the Data (Use and Access) Act 2025?" — Among other things, a new recognised-legitimate-interests lawful basis for listed purposes, changes to automated decision-making rules, and a requirement for controllers to handle data-protection complaints; the EU reviewed it and renewed UK adequacy.

Certification practice

Completion checklist

  • I can explain controller vs processor, the principles that matter for engineering, and who chooses lawful bases
  • I can explain residency, sovereignty and restricted transfers, and the current UK–EU adequacy position
  • I can design erasure that coexists with immutable logs (pseudonymisation, crypto-shredding, legal holds)
  • I ran the erasure workflow test
  • I can write a Region-deny SCP and explain its gaps
  • I created no ongoing charges (or scheduled any test KMS key for deletion)

Key takeaways

  • UK GDPR (as amended by the DUAA 2025) is about principles, rights and accountability; Region choice is one control, not compliance.
  • There's no general UK data-residency requirement, but transfers need a mechanism; the EU renewed UK adequacy until December 2031.
  • Keep personal data out of immutable stores, pseudonymise everywhere else, and crypto-shred with per-customer data keys.
  • Legal holds (such as AML retention) turn erasure into restriction plus scheduled deletion; Legal and the DPO own those rules.
  • Enforce approved Regions with SCPs or Control Tower, mind global services, and document every exception.

Further reading

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page