Skip to Content
Cloud Engineering Academy
5. Security for Banking & Finance

Encryption at rest and in transit, and field-level encryption in .NET

Encrypt every Harbourline data store with a customer managed key, enforce TLS 1.2 or later on every connection, and add field-level encryption with the AWS Encryption SDK, choosing between random and deterministic encryption with your eyes open.

intermediate⏱ 2 h 30 min🟡 Low costSCS-C03SAA-C03DVA-C03CLF-C02

Assumes: .NET 10 LTS · AWS SDK for .NET v4 (KMS 4.1.1.1, S3 4.1.1, EC2 4.1.4, RDS 4.1.0.2, DynamoDB 4.1.0.2) · AWS Encryption SDK for .NET 5.0.0 · Amazon.CDK.Lib 2.277.0 · Terraform AWS provider 6 · Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Explain the three layers of data protection on AWS (in transit, at rest by the service, and in the application) and say which threat each layer stops and which it does not.
  • Turn on and verify encryption at rest with a customer managed KMS key for S3, EBS, RDS and DynamoDB, and explain what you cannot change after creation.
  • Enforce TLS 1.2 or later on S3, a load balancer, a database connection and a Kestrel endpoint, and prove the policy works with a handshake probe.
  • Implement field-level encryption in C# with the AWS Encryption SDK, bind each value to its customer with an encryption context, and reduce KMS calls.
  • Choose between randomised and deterministic encryption, and add a blind index when a column must be searchable, stating the leakage each choice accepts.
  • List what an auditor asks about encryption and show the evidence for each question.

Prerequisites

Why this matters in banking

Three questions every bank hears, usually from different people in the same quarter:

  1. The auditor: "Is all customer data encrypted at rest and in transit? Show me, for every data store, not a policy document." A "yes" without a list of resources, keys and settings is not evidence.
  2. The penetration tester: "Your load balancer still accepts TLS 1.0." The default policy that infrastructure-as-code gives a load balancer is older than the one the console shows. One forgotten line fails a test.
  3. The incident responder: "Someone dumped the customers table through a SQL injection. What did they get?" If only the database encrypted the data at rest, they got everything: the database decrypts for any query that reaches it. If the sensitive columns were encrypted by the application, they got ciphertext.

That third answer is why this lesson has three layers. Encryption at rest by the service protects a stolen disk, snapshot or backup. It does not protect against a compromised application or a SQL injection, because the service decrypts transparently for anyone who is authorised to read. Field-level encryption moves that decision into your code and your key policy.

Not legal or compliance advice

This lesson shows engineering controls. Whether a control satisfies PCI DSS, UK GDPR or a regulator's expectation is decided by your QSA, data protection officer and compliance team. Nothing here makes a workload compliant. Fictional data only.

The idea in plain language

Data lives in three states, and each needs a different control.

  • In transit: moving between a customer's phone and your API, between services, or between an application and its database. Threat: someone on the network reads or alters it. Control: TLS.
  • At rest: sitting on a disk, in a snapshot, in a backup. Threat: someone obtains the storage without going through your application, for example a leaked snapshot, a decommissioned disk or a misconfigured sharing setting. Control: the service encrypts the bytes with a key.
  • In use, in the application: decrypted in memory while code works on it. Threat: a bug or an attacker with application access. Control: minimise what the application can decrypt, using field-level encryption and key policies that only allow the services that need a field.

Analogy: TLS is the armoured van between branches. Encryption at rest is the safe-deposit room: if someone carries the whole room away, they cannot open the boxes. Field-level encryption is a sealed envelope inside the box: even the room's staff, who can open boxes for customers, cannot read the letter without the key that only the right desk holds.

The service can encrypt, but who holds the key matters. With a key you control (a customer managed key) you decide who can use it, you can see every use in CloudTrail, you can disable it in an emergency and you can prove it. That is why the bank's standard asks for customer managed keys for anything sensitive.

What problem the AWS services solve

AWS removes most of the work of at-rest encryption: you tick a setting or write a line of infrastructure code and the service handles encryption and decryption. The limits are what you need to know.

Encryption at rest by service

ServiceWhat is encryptedKey choicesSharp edges
S3Every new object has been encrypted with SSE-S3 since January 2023; you can set a bucket default of SSE-KMS or dual-layer SSE-KMS (DSSE-KMS)SSE-S3 (AWS owned), aws/s3 AWS managed key, customer managed keyBucket Keys cut KMS request volume; a default encryption setting applies to new objects only, so old objects keep their old encryption until copied. SSE-C (customer-provided keys) is being restricted on new buckets ⚠ VERIFY the current default
EBSVolume data, snapshots, and data moving between the instance and the volumeaws/ebs or a customer managed key; account-wide "encrypt by default" per RegionYou cannot encrypt an existing volume in place: snapshot, copy the snapshot with encryption, create a volume. A snapshot encrypted with the AWS managed key cannot be shared with another account
RDS and AuroraStorage, automated backups, snapshots, read replicasChosen at creation; aws/rds or customer managedYou cannot turn encryption on for an existing unencrypted instance. Snapshot, copy encrypted, restore. A replica must match its source
DynamoDBAlways on, tables and indexes, backupsAWS owned (default), aws/dynamodb, or customer managedThe default AWS owned key leaves you no key policy and no CloudTrail trail for key use. Pick a customer managed key for sensitive tables
Secrets Manager, SQS, SNS, CloudWatch Logs, EFSEach has its own settingMostly AWS managed or customer managedCheck each one: the inventory is the control, not the assumption

"AWS owned key" means the service uses a key it manages across customers. You cannot see it, audit it or control it. "AWS managed key" (aliases such as aws/s3) is in your account and visible, but you cannot edit its policy or rotate it on your schedule. A customer managed key is the only type where you write the policy.

Encryption in transit

TLS 1.2 is the floor for a bank. TLS 1.3 is faster to set up and removes weak options. AWS announced that its service endpoints require TLS 1.2 as a minimum ⚠ VERIFY the effective date and any exceptions, but the connection you control, the one to your own API, database and load balancer, is yours to configure. The places a policy lives:

HopWhere the policy livesDefault risk
Client to Application Load BalancerSslPolicy on the HTTPS listenerIaC defaults to ELBSecurityPolicy-2016-08, which allows TLS 1.0 (see the certificates lesson)
Client to CloudFrontViewer minimum protocol version on the distributionThe default *.cloudfront.net certificate limits what you can set
Client to API Gateway custom domainSecurity policy on the domain nameLegacy policy names still exist
Application to S3Bucket policy conditions aws:SecureTransport and s3:TlsVersion ⚠ VERIFYWithout a deny, plain HTTP is accepted on the S3 endpoint
Application to RDSEngine parameter (rds.force_ssl for PostgreSQL) plus the client's SSL ModeThe server may allow plain connections; the client may skip certificate checks
ALB to your containerTarget group protocol HTTPSHTTP inside the VPC is common; many banks require encryption inside too
On-premises to AWSIPsec VPN, or MACsec on supported Direct ConnectA private line is not an encrypted line

An important detail: when TLS ends at the load balancer, the load balancer sees the plain-text request. Whether the next hop is encrypted is a separate decision, and the answer for a payments API at a bank is usually yes.

Field-level encryption and the trade-off it forces

Field-level (application-layer) encryption means your code encrypts a value before it is sent to the database. The database, the backup, a DBA and an SQL injection attacker all see ciphertext. The price is real:

  • You cannot search, sort, index or aggregate on ciphertext.
  • Each encryption needs a data key, and a data key needs a KMS call unless you reuse it.
  • Key and context design become part of the data model, and a mistake destroys data.

The next sections show the tools for each trade-off.

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
Azure Storage Service Encryption with customer-managed keysS3 SSE-KMS with a customer managed keyS3 always encrypts; the choice is who owns the key
Azure Disk Encryption / encryption at hostEBS encryption, "encrypt by default" Region settingSet once per Region per account; cannot encrypt in place
SQL Server Transparent Data Encryption (TDE)RDS storage encryption (KMS); TDE option on RDS for SQL Server and OracleStorage encryption is chosen at creation; TDE is an additional engine feature
SQL Server Always Encrypted (randomised and deterministic)AWS Encryption SDK, or AWS Database Encryption SDK for DynamoDB (⚠ VERIFY current features)No transparent driver feature for RDS: encryption happens in your code
Key Vault keysKMS keysSame idea, but the key policy and encryption context are central
SslProtocols on Kestrel / SChannel registry settingsALB security policy, Kestrel SslProtocols, OS settingsOn AWS the managed edge usually terminates TLS, so the policy is a named setting
Encrypt=True;TrustServerCertificate=False in SqlClientSSL Mode=VerifyFull in Npgsql, ssl-mode in MySQLVerify the certificate chain and host name, or encryption alone does not stop impersonation
Azure Policy "require TLS 1.2 on storage"Bucket policy deny on s3:TlsVersion, Config rules, Security Hub CSPM controlsPrevent in the policy, detect with Config

Architecture

Security-boundary view. Three controls overlap on purpose:

What each layer defeats, and what it does not:

ThreatTLSService at-rest encryptionField-level encryption
Sniffing on the networkStops itNo effectNo effect
Stolen snapshot or backupNo effectStops it (without the key)Stops it
Misconfigured bucket or snapshot shareNo effectPartly: the reader also needs key accessStops it
SQL injection or stolen database credentialsNo effectNo effect (the database decrypts for you)Stops it for encrypted columns
DBA reading a tableNo effectNo effectStops it
Compromised application with key accessNo effectNo effectDoes not stop it

Step-by-step implementation

Inventory what you have

You cannot report on encryption you have not measured. The lab has a read-only inventory that asks S3, EC2, RDS and DynamoDB how each resource is encrypted and what kind of key protects it. It needs only list and describe permissions. Run it first against your sandbox to see the real picture.

Decide the key layout

One customer managed key per data domain and environment is a sensible start: statements, customer-profile, token-vault. Keep the token vault in its own key so a role that can use the statements key cannot decrypt card data. Key policy details are in the KMS lesson.

Encrypt each data store with the key and verify it

Set the service-side encryption in infrastructure as code (the CDK and Terraform tabs below), then verify with the CLI. Remember the one-way doors: RDS and EBS encryption is chosen at creation.

Enforce TLS on the edge, the bucket and the database

Choose an explicit load balancer policy, deny non-TLS and old TLS on S3, set rds.force_ssl, and make the client verify the server certificate. Prove it with a handshake probe, not a screenshot.

Add field-level encryption for restricted fields

Use the Encryption SDK with an encryption context made of identifiers. Decide per field whether it needs random encryption (default), deterministic encryption (equality lookups) or a blind index next to random encryption.

Alarm on drift

AWS Config managed rules and Security Hub CSPM controls flag unencrypted volumes, databases and buckets and weak load balancer policies. Wire them so that an unencrypted resource creates a finding, not a surprise at audit time. Names of managed rules change, so look them up in the Config console when you set this up. ⚠ VERIFY

Choosing random, deterministic or blind-indexed encryption

ApproachSame value, same ciphertext?Equality searchLeaksUse for
Random (Encryption SDK, new data key each time)NoNoOnly length (roughly)Default for restricted fields you never search: date of birth, address
Deterministic (synthetic IV)YesYes, direct on the columnWhich rows share a value, and how oftenLow-cardinality lookups where you accept frequency leakage; a unique constraint on one field
Random plus blind index (HMAC beside the ciphertext)No in the ciphertext; yes in the indexYes, on the index columnEquality in the index; truncating the index adds false positives that hide uniquenessThe usual banking answer for "find the customer by IBAN"

Ask the same questions each time: who can see the column, how many distinct values exist, and could an attacker guess the values? A column with three possible values encrypted deterministically is almost plaintext, because the most frequent ciphertext is obviously the most common value. The lab test Deterministic_encryption_leaks_the_frequency_of_values_and_random_encryption_does_not shows this with 300 payments.

AWS Console instructions

Check S3 default encryption. Console → S3 → bucket → Properties → Default encryption. You see the type (SSE-S3, SSE-KMS, DSSE-KMS), the key and whether Bucket Key is enabled.

Turn on EBS encryption by default. Console → EC2 → EC2 Dashboard → Account attributes → Data protection and security → EBS encryption → Manage → Enable → choose the default KMS key. It is per Region; repeat in every Region you use.

Create an encrypted RDS instance. Console → RDS → Create database → Additional configuration → Encryption → Enable encryption → choose the customer managed key. For PostgreSQL create a parameter group with rds.force_ssl set to 1 and attach it. The console field names move around; use the infrastructure-as-code below as the source of truth.

Choose a DynamoDB key. Console → DynamoDB → table → Additional settings → Encryption → Manage encryption → pick AWS owned, AWS managed or Stored in your account, owned and managed by you.

Set a load balancer policy. Console → EC2 → Load balancers → ALB → Listeners and rules → HTTPS:443 → Edit listener → Secure listener settings → Security policy.

AWS CLI

# 1. Who encrypts what: S3 bucket default encryption (an error means no default is configured)
aws s3api get-bucket-encryption --bucket "$BUCKET" --profile learn-sandbox

# 2. Are new EBS volumes encrypted by default in this Region, and with which key?
aws ec2 get-ebs-encryption-by-default --profile learn-sandbox
aws ec2 get-ebs-default-kms-key-id --profile learn-sandbox

# 3. Which volumes are NOT encrypted?
aws ec2 describe-volumes --filters Name=encrypted,Values=false \
  --query "Volumes[].VolumeId" --profile learn-sandbox

# 4. RDS instances and their storage encryption
aws rds describe-db-instances \
  --query "DBInstances[].[DBInstanceIdentifier,StorageEncrypted,KmsKeyId]" \
  --output table --profile learn-sandbox

# 5. DynamoDB table encryption (SSEDescription absent = default AWS owned key)
aws dynamodb describe-table --table-name hl-m05-token-vault \
  --query "Table.SSEDescription" --profile learn-sandbox

# 6. Turn on EBS encryption by default for this Region
aws ec2 enable-ebs-encryption-by-default --profile learn-sandbox

# 7. Prove a TLS policy from the outside: this should FAIL against a TLS 1.2-and-up listener
openssl s_client -connect api.example.com:443 -tls1_1 </dev/null

# 8. Encrypt an unencrypted EBS volume: snapshot, copy with encryption, create a volume
aws ec2 create-snapshot --volume-id vol-0123456789abcdef0 --profile learn-sandbox
aws ec2 copy-snapshot --source-region us-east-1 --source-snapshot-id snap-0123456789abcdef0 \
  --encrypted --kms-key-id alias/hl-m05-at-rest --profile learn-sandbox

Commands 7 and 8 use placeholder identifiers. Replace them with your own. Some OpenSSL builds have TLS 1.0 and 1.1 compiled out, so a failure to connect can mean the client refused, not the server; test with the handshake probe below for a reliable answer.

Infrastructure as Code

The lab project labs/m05-data-protection/infra/cdk compiles with Amazon.CDK.Lib 2.277.0 and has three stacks. The data-store stack:

DataStoresStack.cs (excerpt)
using Amazon.CDK;
using Amazon.CDK.AWS.DynamoDB;
using Amazon.CDK.AWS.EC2;
using Amazon.CDK.AWS.IAM;
using Amazon.CDK.AWS.KMS;
using Amazon.CDK.AWS.RDS;
using Amazon.CDK.AWS.S3;
using Constructs;

namespace Harbourline.DataProtection.Cdk;

public sealed class DataStoresStack : Stack
{
    public DataStoresStack(Construct scope, string id, IStackProps? props = null) : base(scope, id, props)
    {
        var key = new Key(this, "DataKey", new KeyProps
        {
            Alias = "alias/hl-m05-at-rest",
            EnableKeyRotation = true,
            PendingWindow = Duration.Days(7),       // lab only
            RemovalPolicy = RemovalPolicy.DESTROY,  // lab only; production uses RETAIN
        });

        var bucket = new Bucket(this, "StatementsBucket", new BucketProps
        {
            Encryption = BucketEncryption.KMS,
            EncryptionKey = key,
            BucketKeyEnabled = true,
            EnforceSSL = true,
            BlockPublicAccess = BlockPublicAccess.BLOCK_ALL,
            Versioned = true,
            RemovalPolicy = RemovalPolicy.DESTROY,
        });

        // EnforceSSL denies plain HTTP; this statement also denies TLS older than 1.2.
        bucket.AddToResourcePolicy(new PolicyStatement(new PolicyStatementProps
        {
            Sid = "DenyOldTls",
            Effect = Effect.DENY,
            Principals = [new AnyPrincipal()],
            Actions = ["s3:*"],
            Resources = [bucket.BucketArn, bucket.ArnForObjects("*")],
            Conditions = new Dictionary<string, object>
            {
                ["NumericLessThan"] = new Dictionary<string, object> { ["s3:TlsVersion"] = 1.2 },
            },
        }));

        // Attribute is ambiguous with System.Attribute, so the type is fully qualified.
        _ = new Table(this, "TokenTable", new TableProps
        {
            PartitionKey = new Amazon.CDK.AWS.DynamoDB.Attribute { Name = "token", Type = AttributeType.STRING },
            Encryption = TableEncryption.CUSTOMER_MANAGED,
            EncryptionKey = key,
            BillingMode = BillingMode.PAY_PER_REQUEST,
            PointInTimeRecoverySpecification = new PointInTimeRecoverySpecification { PointInTimeRecoveryEnabled = true },
            RemovalPolicy = RemovalPolicy.DESTROY,
        });

        _ = new Volume(this, "DataVolume", new VolumeProps
        {
            AvailabilityZone = Fn.Select(0, Fn.GetAzs()),
            Size = Size.Gibibytes(8),
            Encrypted = true,
            EncryptionKey = key,
            RemovalPolicy = RemovalPolicy.DESTROY,
        });

        var vpc = new Vpc(this, "Vpc", new VpcProps
        {
            MaxAzs = 2,
            NatGateways = 0,
            SubnetConfiguration =
            [
                new SubnetConfiguration { Name = "isolated", SubnetType = SubnetType.PRIVATE_ISOLATED, CidrMask = 24 },
            ],
        });

        var parameters = new ParameterGroup(this, "PgParameters", new ParameterGroupProps
        {
            Engine = DatabaseInstanceEngine.Postgres(new PostgresInstanceEngineProps { Version = PostgresEngineVersion.VER_17 }),
            Parameters = new Dictionary<string, string> { ["rds.force_ssl"] = "1" },
        });

        _ = new DatabaseInstance(this, "Ledger", new DatabaseInstanceProps
        {
            Engine = DatabaseInstanceEngine.Postgres(new PostgresInstanceEngineProps { Version = PostgresEngineVersion.VER_17 }),
            InstanceType = Amazon.CDK.AWS.EC2.InstanceType.Of(InstanceClass.T4G, InstanceSize.MICRO),
            Vpc = vpc,
            VpcSubnets = new SubnetSelection { SubnetType = SubnetType.PRIVATE_ISOLATED },
            Credentials = Credentials.FromGeneratedSecret("hl_app"),
            StorageEncrypted = true,
            StorageEncryptionKey = key,
            ParameterGroup = parameters,
            AllocatedStorage = 20,
            DeletionProtection = false,   // lab only
            RemovalPolicy = RemovalPolicy.DESTROY,
        });

        // Amazon.CDK.Tags, not Tags: inside a Stack the bare name is the stack's own Tags property (CS0176).
        Amazon.CDK.Tags.Of(this).Add("Project", "cea-lab");
        Amazon.CDK.Tags.Of(this).Add("Lab", "m05-data-protection");
        Amazon.CDK.Tags.Of(this).Add("Environment", "learn");
    }
}

The edge stack attaches an HTTPS listener with an explicit policy. The IaC default is the old one:

EdgeTlsStack.cs (excerpt)
alb.AddListener("Https", new BaseApplicationListenerProps
{
    Port = 443,
    Protocol = ApplicationProtocol.HTTPS,
    Certificates = [ListenerCertificate.FromArn(certificateArn.ValueAsString)],
    // CDK, CloudFormation and the CLI default to ELBSecurityPolicy-2016-08, which accepts TLS 1.0.
    SslPolicy = SslPolicy.TLS13_12_RES_PQ,
    DefaultAction = ListenerAction.FixedResponse(200, new FixedResponseOptions
    {
        ContentType = "text/plain",
        MessageBody = "ok",
    }),
});

Complete code

The lab project labs/m05-data-protection builds on .NET 10 with xUnit v3 and passes without an AWS account. Three pieces carry the lesson.

1. Prove the TLS policy with a handshake

src/Harbourline.DataProtection/Tls/TlsProbe.cs
using System.Net.Security;
using System.Net.Sockets;
using System.Security.Authentication;

namespace Harbourline.DataProtection.Tls;

public sealed record TlsProbeResult(bool Succeeded, SslProtocols? Negotiated, string? Error);

public static class TlsProbe
{
    public static async Task<TlsProbeResult> ProbeAsync(
        string host, int port, SslProtocols offered, bool trustAnyCertificate = false, CancellationToken cancellationToken = default)
    {
        try
        {
            using var tcp = new TcpClient();
            await tcp.ConnectAsync(host, port, cancellationToken).ConfigureAwait(false);

            using var ssl = new SslStream(tcp.GetStream(), leaveInnerStreamOpen: false);
            var options = new SslClientAuthenticationOptions
            {
                TargetHost = host,
                EnabledSslProtocols = offered,
            };
            if (trustAnyCertificate)
            {
                // Localhost self-signed demo only. Never ship this callback.
                options.RemoteCertificateValidationCallback = (_, _, _, _) => true;
            }

            await ssl.AuthenticateAsClientAsync(options, cancellationToken).ConfigureAwait(false);
            return new TlsProbeResult(true, ssl.SslProtocol, null);
        }
        catch (Exception ex) when (ex is AuthenticationException or IOException or SocketException)
        {
            return new TlsProbeResult(false, null, ex.GetType().Name);
        }
    }
}

The test starts a real Kestrel server that only allows TLS 1.3, then checks that a TLS 1.2 client fails and a TLS 1.3 client succeeds:

tests/.../TlsTests.cs (excerpt)
[Fact]
public async Task Tls13_only_server_refuses_a_tls12_client_and_accepts_tls13()
{
    using var cert = CreateLocalhostCertificate();
    await using var app = DemoHost.Create(cert, SslProtocols.Tls13);
    await app.StartAsync(TestContext.Current.CancellationToken);
    var port = DemoHost.BoundAddress(app).Port;

    var tls12 = await TlsProbe.ProbeAsync("localhost", port, SslProtocols.Tls12, trustAnyCertificate: true,
        cancellationToken: TestContext.Current.CancellationToken);
    var tls13 = await TlsProbe.ProbeAsync("localhost", port, SslProtocols.Tls13, trustAnyCertificate: true,
        cancellationToken: TestContext.Current.CancellationToken);

    Assert.False(tls12.Succeeded);
    Assert.True(tls13.Succeeded);
    Assert.Equal(SslProtocols.Tls13, tls13.Negotiated);
    await app.StopAsync(TestContext.Current.CancellationToken);
}

The Kestrel side is one option on the HTTPS adapter:

Demo/DemoHost.cs (excerpt)
builder.WebHost.ConfigureKestrel(kestrel =>
    kestrel.Listen(IPAddress.Loopback, 0, listen =>
        listen.UseHttps(new HttpsConnectionAdapterOptions
        {
            ServerCertificate = certificate,
            SslProtocols = protocols,   // the same idea as an ALB security policy, enforced in-process
        })));

app.Use(async (context, next) =>
{
    // HSTS tells browsers to refuse plain HTTP for a year. An ALB does not add it for you.
    context.Response.Headers.StrictTransportSecurity = "max-age=31536000; includeSubDomains";
    await next();
});

2. Field-level encryption with the AWS Encryption SDK

src/Harbourline.DataProtection/Fields/EsdkFieldEncryptor.cs
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using Amazon.KeyManagementService;
using AWS.Cryptography.EncryptionSDK;
using AWS.Cryptography.MaterialProviders;

namespace Harbourline.DataProtection.Fields;

public sealed class EsdkFieldEncryptor
{
    public const string Prefix = "esdk:v1:";

    private readonly ESDK _esdk = new(new AwsEncryptionSdkConfig());
    private readonly IKeyring _keyring;

    private EsdkFieldEncryptor(IKeyring keyring) => _keyring = keyring;

    public static EsdkFieldEncryptor WithKeyring(IKeyring keyring) =>
        new(keyring ?? throw new ArgumentNullException(nameof(keyring)));

    public string Encrypt(string plaintext, IReadOnlyDictionary<string, string> context)
    {
        ArgumentNullException.ThrowIfNull(plaintext);
        ArgumentNullException.ThrowIfNull(context);

        var output = _esdk.Encrypt(new EncryptInput
        {
            Plaintext = new MemoryStream(Encoding.UTF8.GetBytes(plaintext)),
            Keyring = _keyring,
            EncryptionContext = new Dictionary<string, string>(context),
        });
        return Prefix + Convert.ToBase64String(output.Ciphertext.ToArray());
    }

    public string Decrypt(string token, IReadOnlyDictionary<string, string> expectedContext)
    {
        ArgumentNullException.ThrowIfNull(token);
        ArgumentNullException.ThrowIfNull(expectedContext);
        if (!token.StartsWith(Prefix, StringComparison.Ordinal))
        {
            throw new FormatException($"Value is not an '{Prefix}' token.");
        }

        var output = _esdk.Decrypt(new DecryptInput
        {
            Ciphertext = new MemoryStream(Convert.FromBase64String(token[Prefix.Length..])),
            Keyring = _keyring,
            EncryptionContext = new Dictionary<string, string>(expectedContext),
        });

        // The context stored in the message must contain every pair we expect.
        foreach (var (key, value) in expectedContext)
        {
            if (!output.EncryptionContext.TryGetValue(key, out var actual) || !string.Equals(actual, value, StringComparison.Ordinal))
            {
                throw new CryptographicException($"Encryption context mismatch for '{key}'.");
            }
        }

        return Encoding.UTF8.GetString(output.Plaintext.ToArray());
    }

    // One data key, so one KMS call, however many fields are in the record.
    public string EncryptRecord(IReadOnlyDictionary<string, string> fields, IReadOnlyDictionary<string, string> context) =>
        Encrypt(JsonSerializer.Serialize(fields), context);

    public IReadOnlyDictionary<string, string> DecryptRecord(string token, IReadOnlyDictionary<string, string> expectedContext) =>
        JsonSerializer.Deserialize<Dictionary<string, string>>(Decrypt(token, expectedContext))
        ?? throw new FormatException("Record payload was empty.");

    public static IReadOnlyDictionary<string, string> ContextFor(string customerId, string field) =>
        new Dictionary<string, string>(StringComparer.Ordinal) { ["customer_id"] = customerId, ["field"] = field };
}

public static class EsdkKeyrings
{
    private static readonly MaterialProviders Providers = new(new MaterialProvidersConfig());

    // Offline keyring for tests: the wrapping key is in process memory. Production uses KMS or an HSM.
    public static IKeyring RawAes(byte[] wrappingKey, string keyNamespace = "harbourline", string keyName = "test-wrapping-key-1") =>
        Providers.CreateRawAesKeyring(new CreateRawAesKeyringInput
        {
            KeyNamespace = keyNamespace,
            KeyName = keyName,
            WrappingKey = new MemoryStream(wrappingKey),
            WrappingAlg = AesWrappingAlg.ALG_AES256_GCM_IV12_TAG16,
        });

    // Pass the key ARN, not an alias, so the key identity is unambiguous.
    public static IKeyring Kms(IAmazonKeyManagementService kms, string keyArn) =>
        Providers.CreateAwsKmsKeyring(new CreateAwsKmsKeyringInput { KmsClient = kms, KmsKeyId = keyArn });
}

Used from an API, the call site is two lines:

var encryptor = EsdkFieldEncryptor.WithKeyring(EsdkKeyrings.Kms(kmsClient, keyArn));
var token = encryptor.Encrypt(iban, EsdkFieldEncryptor.ContextFor(customerId, "iban"));

3. Deterministic encryption and a blind index

src/Harbourline.DataProtection/Fields/DeterministicAndSearchable.cs (excerpt)
public byte[] Encrypt(string plaintext, string context)
{
    var data = Encoding.UTF8.GetBytes(plaintext);
    var aad = Encoding.UTF8.GetBytes(context);

    // Synthetic nonce: an HMAC of context and plaintext. Equal inputs give equal nonces and ciphertexts.
    byte[] seed = [.. aad, 0, .. data];
    var nonce = HMACSHA256.HashData(_macKey, seed).AsSpan(0, NonceSize).ToArray();

    var cipher = new byte[data.Length];
    var tag = new byte[TagSize];
    using var aes = new AesGcm(_encKey, TagSize);
    aes.Encrypt(nonce, data, cipher, tag, aad);

    return [Version, .. nonce, .. tag, .. cipher];
}

public static string Compute(byte[] key, string field, string normalisedValue, int outputBytes = 16)
{
    // The unit separator keeps ("ab","c") and ("a","bc") apart; the field stops cross-column joins.
    var mac = HMACSHA256.HashData(key, Encoding.UTF8.GetBytes(field + '\u001f' + normalisedValue));
    return Convert.ToHexStringLower(mac.AsSpan(0, outputBytes));
}

⚠ Learning shortcut: not for production

DeterministicFieldCipher composes standard primitives (HKDF, HMAC, AES-GCM) so you can see the leakage. Do not invent your own cipher mode in production code. Use a reviewed implementation: AES-SIV from a vetted library, SQL Server Always Encrypted deterministic mode, or the AWS Database Encryption SDK for DynamoDB (⚠ VERIFY the features it offers today), and have your cryptography reviewer approve the design.

Code explained line by line

TlsProbe

  1. SslClientAuthenticationOptions.EnabledSslProtocols = offered restricts what the client offers. A TLS 1.3-only server and a TLS 1.2-only client share no protocol, so the handshake fails with an AuthenticationException.
  2. ssl.SslProtocol after AuthenticateAsClientAsync is what was actually negotiated. Report that, not what you hoped for.
  3. trustAnyCertificate exists only so a self-signed localhost certificate works in a test. Production clients validate the chain and host name.

EsdkFieldEncryptor

  1. new ESDK(new AwsEncryptionSdkConfig()) creates the client. The defaults require commitment on both encrypt and decrypt, which prevents a class of attacks where a ciphertext decrypts differently under different keys. ⚠ VERIFY the current default commitment policy and algorithm suite in the SDK documentation.
  2. Keyring tells the SDK how to obtain and protect the data key. The KMS keyring calls GenerateDataKey on encrypt and Decrypt on decrypt, passing your encryption context to KMS, so KMS refuses a context it did not see at encryption time.
  3. EncryptionContext is authenticated but not secret. It is stored in the message header and appears in CloudTrail. Identifiers only; never put a name or a date of birth in it.
  4. Decrypt receives the expected context and then re-checks it. The SDK message carries the context it was encrypted with, and the decrypting code must verify it matches what the caller expects, or a valid ciphertext for one customer can be presented as another's.
  5. EncryptRecord serialises several fields into one message. One data key, one KMS call. The price is that reading one field decrypts the bundle, and that the context names the bundle rather than each field.

DeterministicFieldCipher

  1. Two subkeys are derived from the master key with HKDF and different info labels, so the MAC key and the encryption key are never the same bytes.
  2. The nonce is the first 12 bytes of an HMAC over the context, a separator and the plaintext. AES-GCM breaks if one nonce encrypts two different messages; here the same nonce only ever appears with the same message, which reveals equality and nothing more.
  3. The context goes in as AES-GCM associated data, so a ciphertext cannot be moved to another column.

BlindIndex.Compute

  1. A keyed hash (HMAC) is used, not a plain SHA-256. Dates of birth and sort codes have too few possible values; an attacker with a plain hash tries them all in seconds.
  2. outputBytes is a deliberate privacy lever. One byte over a thousand values gives at most 256 buckets, so many values share an index. Your code then decrypts the candidates and compares. You trade lookup cost for less information in the index.

Security considerations

⚠ Learning shortcut: not for production

The offline tests use EsdkKeyrings.RawAes, a wrapping key held in process memory, and a fake KMS that counts calls. That is right for fast, free tests and wrong for production: real data keys must be wrapped by a KMS key whose policy and CloudTrail entries you control. The demo host also reads the viewer role from a header; a real API takes it from a validated token.

  • Context is the audit label. customer_id and field in the context mean a CloudTrail search for one customer's decrypts is possible, and a decrypt for the wrong customer fails. Design the context before the first record is written; changing it later means re-encrypting.
  • Key policy limits who can decrypt. Give the API role kms:GenerateDataKey and kms:Decrypt with a condition on kms:EncryptionContextKeys (see the KMS lesson) and keep administrators out of the key-user list. A DBA must not hold kms:Decrypt.
  • Rotation does not re-encrypt. Automatic rotation of a symmetric KMS key keeps old key material to decrypt old data. It does not rewrite your ciphertext. Re-encryption is a deliberate batch job. See Backup security and key rotation.
  • Deleting a key destroys the data. This is the strength of the design and the danger. Use the waiting period, alarm on ScheduleKeyDeletion, and keep tested backups.
  • Verify certificates on database connections. SSL Mode=Require encrypts but may accept any certificate. Use VerifyFull with the RDS CA bundle (⚠ VERIFY the current RDS CA name and bundle location) so a man in the middle cannot present a different certificate.
  • TLS ends at the edge. An ALB decrypts the request. Decide deliberately whether traffic from the ALB to the container is encrypted, and note that an ALB does not validate the target's certificate chain in the way a client does ⚠ VERIFY. If your standard requires encryption between hops, use HTTPS target groups and a service mesh or an internal CA.
  • Do not log decrypted values. Encrypt at rest, then print the value in a debug log, and the control is gone. See the next lesson, data classification, masking and tokenisation.
  • Key commitment and algorithm suites. Keep the SDK defaults unless a cryptography reviewer says otherwise. The default message carries a header and a signature, so each tiny field costs hundreds of bytes ⚠ VERIFY. For many small fields prefer one record bundle (above) or a suite without signing, as a reviewed decision.

Banking use case

Harbourline's customer profile table holds name, email, IBAN, date of birth and phone number. The design:

  • The RDS instance has storage encryption with the customer-profile key and rds.force_ssl=1. This stops a stolen snapshot and a plain-text connection.
  • Name and email are Confidential: they sit in ordinary columns, protected by the service and by masking in logs.
  • IBAN, date of birth and phone number are Restricted: encrypted by the application with the Encryption SDK and the context {customer_id, field}.
  • Support staff look customers up by IBAN. The table stores iban_ct (random ciphertext) and iban_bidx (a 16-byte blind index). The API computes the index from the typed IBAN, queries iban_bidx, decrypts the candidate rows and compares. The HMAC key for the index is in Secrets Manager (see Secrets Manager and Parameter Store); it is a separate secret from any encryption key.
  • Country of residence has only about 200 values and is queried in reports. It is Internal, not encrypted at field level, because its encryption would be pointless (low cardinality, easily guessed).

The decision for each field is documented in the data inventory with an owner. When the auditor asks "which fields are encrypted by your application?" the answer is a query, not a meeting.

Common mistakes

  • "We use an AWS owned key, so it is encrypted." → True, and you cannot audit or revoke it → use a customer managed key for Confidential and Restricted data, and say why for the rest.
  • Trying to encrypt an existing RDS instance in place. → The setting is creation-only → snapshot, copy with encryption, restore, then cut over.
  • Leaving the ALB on the IaC default policy. → TLS 1.0 stays on → set SslPolicy explicitly and add a detective control.
  • StringNotEquals on the encryption header in a bucket policy. → It denies uploads that rely on bucket default encryption → use StringNotEqualsIfExists.
  • Deterministic encryption on a low-cardinality column. → Frequency analysis recovers the values → use random encryption, or do not encrypt a field that is not secret.
  • Plain SHA-256 as a search index. → Brute-forceable for small value spaces → use a keyed HMAC with a key kept out of the database.
  • Personal data in the encryption context. → It appears in clear text in CloudTrail → identifiers only.
  • One KMS call per field at scale. → Cost and throttling (see the quotas in the KMS lesson) → bundle fields per record, enable S3 Bucket Keys, or evaluate the Encryption SDK's hierarchical keyring ⚠ VERIFY.
  • TrustServerCertificate=True copied into production. → Encryption without identity → verify the chain and the host name.

Troubleshooting

SymptomLikely causeFix
AccessDeniedException from KMS on S3 uploadThe uploading role lacks kms:GenerateDataKey on the bucket's keyAdd the key-user statement to the key policy, scoped with kms:ViaService
InvalidCiphertextException on decryptEncryption context differs from the one used to encryptRebuild the context from the same identifiers; check for casing and extra keys
Handshake fails to the load balancer from an old clientThe policy no longer offers the client's protocol or ciphersUpgrade the client; do not weaken the policy without a documented exception
terraform apply fails: cannot modify storage_encryptedIt forces replacement of the instanceSnapshot, restore encrypted, import or recreate; plan the cutover
Uploads rejected after adding the "wrong header" denyStringNotEquals instead of StringNotEqualsIfExistsSwitch the operator
Object cannot be shared cross-accountSnapshot or object encrypted with an AWS managed keyRe-encrypt with a customer managed key whose policy allows the other account
ThrottlingException from KMS during a batchMany per-field data keys at onceBundle fields, add jittered retries, request a quota increase if justified
Npgsql: The remote certificate is invalidThe RDS CA bundle is not in the containerMount the bundle and set Root Certificate; do not switch to Trust Server Certificate
Tests pass locally but the TLS 1.0 check "fails to connect"The local OpenSSL refuses TLS 1.0Use a server-side probe or a scanning tool; do not conclude the server rejected it

Cost and cleanup

Free: the unit tests, terraform validate, dotnet build of the CDK project, and reading the inventory.

Credits (deploy, look, destroy within the hour):

  • Customer managed KMS keys: USD 1 per key per month, prorated hourly; keys pending deletion are not charged. Symmetric requests: USD 0.03 per 10,000 after a monthly free tier ⚠ VERIFY on the pricing page.
  • RDS db.t4g.micro: about an hour of instance time and 20 GB of storage; ⚠ VERIFY the current hourly price before deploying.
  • Application Load Balancer: an hourly charge plus capacity units while it exists, and an ACM public certificate is free. The edge stack needs a certificate ARN you supply; use a public certificate on a domain you own, or skip the edge stack and use the in-process Kestrel test.
  • DynamoDB on demand and an 8 GiB EBS volume: cents for an hour.

Design-only alternative: do not deploy. Run the tests, read the CDK and Terraform and fill in the exercise table.

# CDK: destroy everything the lab created
cd labs/m05-data-protection/infra/cdk
cdk destroy --all --profile learn-sandbox

# Terraform: destroy and then confirm the keys are pending deletion
cd labs/m05-data-protection/infra/terraform
terraform destroy -var "db_subnet_group_name=..." -var "alb_arn=..." -var "certificate_arn=..." 
aws kms list-keys --profile learn-sandbox
aws kms describe-key --key-id alias/hl-m05-at-rest --profile learn-sandbox --query "KeyMetadata.KeyState"

# If you turned on account-wide EBS encryption by default for the lab, turn it back off if the account policy allows
aws ec2 disable-ebs-encryption-by-default --profile learn-sandbox
aws ec2 reset-ebs-default-kms-key-id --profile learn-sandbox

Destroying the stacks schedules the keys for deletion after the 7-day waiting period; data encrypted under them becomes unreadable afterwards. That is expected in a lab. Check the billing console the next day.

Exercises

  • 🟢 Guided: Run the lab tests. Then read TlsTests and change SslProtocols.Tls13 to SslProtocols.Tls12 | SslProtocols.Tls13 in the first test. Predict which assertion fails, then run it to confirm.
  • 🟡 Challenge: Add a new field phone to the customer profile design. Decide random, deterministic or blind-indexed. Write the encryption context, extend the lab test with a round trip and a wrong-customer check, and write two sentences on the leakage you accepted.
  • 🔴 Live problem: A pen test reports that production accepts TLS 1.1 on one public API, that two RDS instances have StorageEncrypted=false, and that a developer stored customer IBANs with deterministic encryption in a column with a unique index. Write a one-page remediation plan: order of work, which fixes are online and which need downtime, what you tell the auditor, and how you prevent a repeat.

Assignment

Add a "data protection posture" report to your portfolio repository. Use AwsEncryptionInventorySource (or the CLI commands) against your sandbox, apply EncryptionPostureEvaluator, and commit the output as a Markdown table with one row per data store: service, resource, encrypted, key type, finding. Add a short paragraph on which findings you would fix first and why.

Solutions

Knowledge check

Interview questions

  • "Walk me through how you would encrypt customer data on AWS." Layers: TLS 1.2 or later on every hop with explicit policies; service encryption with customer managed keys per data domain; field-level encryption with an encryption context for Restricted data; key policy separating administrators from users; CloudTrail on every key use; Config and Security Hub CSPM for drift. Then name what each layer does not cover.
  • "Random or deterministic encryption?" Random by default. Deterministic only when equality lookup is needed and the leakage (equality, frequency) is acceptable, ideally on high-cardinality fields. For searchable restricted fields, random ciphertext plus a keyed blind index.
  • "How do you prove to an auditor that everything is encrypted?" An inventory query across services, with encrypted flag and key type per resource, backed by Config rule history and key policies. Not a policy document.
  • "A KMS call per field is too slow. What now?" Bundle fields per record, use S3 Bucket Keys, evaluate data-key caching or the hierarchical keyring, add retries with jitter and watch the account quota.
  • "What does TLS termination at the load balancer mean for security?" The load balancer sees plain text, so the next hop needs its own protection, and policies, certificates and logs on the load balancer are part of the control.

Certification practice

Completion checklist

  • I can name the three layers of data protection and one threat each does not stop
  • I ran the lab tests and understand why the TLS 1.3-only server refuses a TLS 1.2 client
  • I can say which encryption settings cannot be changed on an existing RDS instance or EBS volume
  • I can explain random, deterministic and blind-indexed encryption and the leakage of each
  • I completed the guided exercise and the data protection posture report
  • I ran the cleanup steps and confirmed nothing is still billing

Key takeaways

  • Encryption in transit, at rest and in the application are three controls; each stops different threats.
  • Service-side encryption protects disks and backups, not queries. For Restricted fields, encrypt in the application with a context that names the customer and the field.
  • Set TLS policies explicitly in infrastructure as code and prove them with a handshake, not an assumption.
  • Random encryption is the default; deterministic and blind indexes buy searchability with leakage you must state.
  • Evidence beats assertion: an inventory of every store, key and setting is what an auditor accepts.

Further reading

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page