Encryption at rest and in transit, and field-level encryption in .NET
Encrypt every Harbourline data store with a customer managed key, enforce TLS 1.2 or later on every connection, and add field-level encryption with the AWS Encryption SDK, choosing between random and deterministic encryption with your eyes open.
Assumes: .NET 10 LTS · AWS SDK for .NET v4 (KMS 4.1.1.1, S3 4.1.1, EC2 4.1.4, RDS 4.1.0.2, DynamoDB 4.1.0.2) · AWS Encryption SDK for .NET 5.0.0 · Amazon.CDK.Lib 2.277.0 · Terraform AWS provider 6 · Region us-east-1. Verified against official sources on 2026-11-18.
Learning objectives
By the end of this lesson you can:
- Explain the three layers of data protection on AWS (in transit, at rest by the service, and in the application) and say which threat each layer stops and which it does not.
- Turn on and verify encryption at rest with a customer managed KMS key for S3, EBS, RDS and DynamoDB, and explain what you cannot change after creation.
- Enforce TLS 1.2 or later on S3, a load balancer, a database connection and a Kestrel endpoint, and prove the policy works with a handshake probe.
- Implement field-level encryption in C# with the AWS Encryption SDK, bind each value to its customer with an encryption context, and reduce KMS calls.
- Choose between randomised and deterministic encryption, and add a blind index when a column must be searchable, stating the leakage each choice accepts.
- List what an auditor asks about encryption and show the evidence for each question.
Prerequisites
- AWS KMS: keys, key policies and envelope encryption: this lesson uses keys, encryption context and envelope encryption without re-explaining them
- Certificates, TLS and mTLS: ACM, Private CA and where TLS terminates
- S3 protection: bucket policies and Block Public Access
- Tools: .NET 10 SDK (or Docker), AWS CLI v2 with the
learn-sandboxprofile for the optional deploy steps - Lab code:
labs/m05-data-protection/
Why this matters in banking
Three questions every bank hears, usually from different people in the same quarter:
- The auditor: "Is all customer data encrypted at rest and in transit? Show me, for every data store, not a policy document." A "yes" without a list of resources, keys and settings is not evidence.
- The penetration tester: "Your load balancer still accepts TLS 1.0." The default policy that infrastructure-as-code gives a load balancer is older than the one the console shows. One forgotten line fails a test.
- The incident responder: "Someone dumped the
customerstable through a SQL injection. What did they get?" If only the database encrypted the data at rest, they got everything: the database decrypts for any query that reaches it. If the sensitive columns were encrypted by the application, they got ciphertext.
That third answer is why this lesson has three layers. Encryption at rest by the service protects a stolen disk, snapshot or backup. It does not protect against a compromised application or a SQL injection, because the service decrypts transparently for anyone who is authorised to read. Field-level encryption moves that decision into your code and your key policy.
Not legal or compliance advice
This lesson shows engineering controls. Whether a control satisfies PCI DSS, UK GDPR or a regulator's expectation is decided by your QSA, data protection officer and compliance team. Nothing here makes a workload compliant. Fictional data only.
The idea in plain language
Data lives in three states, and each needs a different control.
- In transit: moving between a customer's phone and your API, between services, or between an application and its database. Threat: someone on the network reads or alters it. Control: TLS.
- At rest: sitting on a disk, in a snapshot, in a backup. Threat: someone obtains the storage without going through your application, for example a leaked snapshot, a decommissioned disk or a misconfigured sharing setting. Control: the service encrypts the bytes with a key.
- In use, in the application: decrypted in memory while code works on it. Threat: a bug or an attacker with application access. Control: minimise what the application can decrypt, using field-level encryption and key policies that only allow the services that need a field.
Analogy: TLS is the armoured van between branches. Encryption at rest is the safe-deposit room: if someone carries the whole room away, they cannot open the boxes. Field-level encryption is a sealed envelope inside the box: even the room's staff, who can open boxes for customers, cannot read the letter without the key that only the right desk holds.
The service can encrypt, but who holds the key matters. With a key you control (a customer managed key) you decide who can use it, you can see every use in CloudTrail, you can disable it in an emergency and you can prove it. That is why the bank's standard asks for customer managed keys for anything sensitive.
What problem the AWS services solve
AWS removes most of the work of at-rest encryption: you tick a setting or write a line of infrastructure code and the service handles encryption and decryption. The limits are what you need to know.
Encryption at rest by service
| Service | What is encrypted | Key choices | Sharp edges |
|---|---|---|---|
| S3 | Every new object has been encrypted with SSE-S3 since January 2023; you can set a bucket default of SSE-KMS or dual-layer SSE-KMS (DSSE-KMS) | SSE-S3 (AWS owned), aws/s3 AWS managed key, customer managed key | Bucket Keys cut KMS request volume; a default encryption setting applies to new objects only, so old objects keep their old encryption until copied. SSE-C (customer-provided keys) is being restricted on new buckets ⚠ VERIFY the current default |
| EBS | Volume data, snapshots, and data moving between the instance and the volume | aws/ebs or a customer managed key; account-wide "encrypt by default" per Region | You cannot encrypt an existing volume in place: snapshot, copy the snapshot with encryption, create a volume. A snapshot encrypted with the AWS managed key cannot be shared with another account |
| RDS and Aurora | Storage, automated backups, snapshots, read replicas | Chosen at creation; aws/rds or customer managed | You cannot turn encryption on for an existing unencrypted instance. Snapshot, copy encrypted, restore. A replica must match its source |
| DynamoDB | Always on, tables and indexes, backups | AWS owned (default), aws/dynamodb, or customer managed | The default AWS owned key leaves you no key policy and no CloudTrail trail for key use. Pick a customer managed key for sensitive tables |
| Secrets Manager, SQS, SNS, CloudWatch Logs, EFS | Each has its own setting | Mostly AWS managed or customer managed | Check each one: the inventory is the control, not the assumption |
"AWS owned key" means the service uses a key it manages across customers. You cannot see it, audit it or control it. "AWS managed key" (aliases such as aws/s3) is in your account and visible, but you cannot edit its policy or rotate it on your schedule. A customer managed key is the only type where you write the policy.
Encryption in transit
TLS 1.2 is the floor for a bank. TLS 1.3 is faster to set up and removes weak options. AWS announced that its service endpoints require TLS 1.2 as a minimum ⚠ VERIFY the effective date and any exceptions, but the connection you control, the one to your own API, database and load balancer, is yours to configure. The places a policy lives:
| Hop | Where the policy lives | Default risk |
|---|---|---|
| Client to Application Load Balancer | SslPolicy on the HTTPS listener | IaC defaults to ELBSecurityPolicy-2016-08, which allows TLS 1.0 (see the certificates lesson) |
| Client to CloudFront | Viewer minimum protocol version on the distribution | The default *.cloudfront.net certificate limits what you can set |
| Client to API Gateway custom domain | Security policy on the domain name | Legacy policy names still exist |
| Application to S3 | Bucket policy conditions aws:SecureTransport and s3:TlsVersion ⚠ VERIFY | Without a deny, plain HTTP is accepted on the S3 endpoint |
| Application to RDS | Engine parameter (rds.force_ssl for PostgreSQL) plus the client's SSL Mode | The server may allow plain connections; the client may skip certificate checks |
| ALB to your container | Target group protocol HTTPS | HTTP inside the VPC is common; many banks require encryption inside too |
| On-premises to AWS | IPsec VPN, or MACsec on supported Direct Connect | A private line is not an encrypted line |
An important detail: when TLS ends at the load balancer, the load balancer sees the plain-text request. Whether the next hop is encrypted is a separate decision, and the answer for a payments API at a bank is usually yes.
Field-level encryption and the trade-off it forces
Field-level (application-layer) encryption means your code encrypts a value before it is sent to the database. The database, the backup, a DBA and an SQL injection attacker all see ciphertext. The price is real:
- You cannot search, sort, index or aggregate on ciphertext.
- Each encryption needs a data key, and a data key needs a KMS call unless you reuse it.
- Key and context design become part of the data model, and a mistake destroys data.
The next sections show the tools for each trade-off.
Microsoft / Azure comparison
| You know (Microsoft/Azure) | AWS equivalent | What's different |
|---|---|---|
| Azure Storage Service Encryption with customer-managed keys | S3 SSE-KMS with a customer managed key | S3 always encrypts; the choice is who owns the key |
| Azure Disk Encryption / encryption at host | EBS encryption, "encrypt by default" Region setting | Set once per Region per account; cannot encrypt in place |
| SQL Server Transparent Data Encryption (TDE) | RDS storage encryption (KMS); TDE option on RDS for SQL Server and Oracle | Storage encryption is chosen at creation; TDE is an additional engine feature |
| SQL Server Always Encrypted (randomised and deterministic) | AWS Encryption SDK, or AWS Database Encryption SDK for DynamoDB (⚠ VERIFY current features) | No transparent driver feature for RDS: encryption happens in your code |
| Key Vault keys | KMS keys | Same idea, but the key policy and encryption context are central |
SslProtocols on Kestrel / SChannel registry settings | ALB security policy, Kestrel SslProtocols, OS settings | On AWS the managed edge usually terminates TLS, so the policy is a named setting |
Encrypt=True;TrustServerCertificate=False in SqlClient | SSL Mode=VerifyFull in Npgsql, ssl-mode in MySQL | Verify the certificate chain and host name, or encryption alone does not stop impersonation |
| Azure Policy "require TLS 1.2 on storage" | Bucket policy deny on s3:TlsVersion, Config rules, Security Hub CSPM controls | Prevent in the policy, detect with Config |
Architecture
Security-boundary view. Three controls overlap on purpose:
What each layer defeats, and what it does not:
| Threat | TLS | Service at-rest encryption | Field-level encryption |
|---|---|---|---|
| Sniffing on the network | Stops it | No effect | No effect |
| Stolen snapshot or backup | No effect | Stops it (without the key) | Stops it |
| Misconfigured bucket or snapshot share | No effect | Partly: the reader also needs key access | Stops it |
| SQL injection or stolen database credentials | No effect | No effect (the database decrypts for you) | Stops it for encrypted columns |
| DBA reading a table | No effect | No effect | Stops it |
| Compromised application with key access | No effect | No effect | Does not stop it |
Step-by-step implementation
Inventory what you have
You cannot report on encryption you have not measured. The lab has a read-only inventory that asks S3, EC2, RDS and DynamoDB how each resource is encrypted and what kind of key protects it. It needs only list and describe permissions. Run it first against your sandbox to see the real picture.
Decide the key layout
One customer managed key per data domain and environment is a sensible start: statements, customer-profile, token-vault. Keep the token vault in its own key so a role that can use the statements key cannot decrypt card data. Key policy details are in the KMS lesson.
Encrypt each data store with the key and verify it
Set the service-side encryption in infrastructure as code (the CDK and Terraform tabs below), then verify with the CLI. Remember the one-way doors: RDS and EBS encryption is chosen at creation.
Enforce TLS on the edge, the bucket and the database
Choose an explicit load balancer policy, deny non-TLS and old TLS on S3, set rds.force_ssl, and make the client verify the server certificate. Prove it with a handshake probe, not a screenshot.
Add field-level encryption for restricted fields
Use the Encryption SDK with an encryption context made of identifiers. Decide per field whether it needs random encryption (default), deterministic encryption (equality lookups) or a blind index next to random encryption.
Alarm on drift
AWS Config managed rules and Security Hub CSPM controls flag unencrypted volumes, databases and buckets and weak load balancer policies. Wire them so that an unencrypted resource creates a finding, not a surprise at audit time. Names of managed rules change, so look them up in the Config console when you set this up. ⚠ VERIFY
Choosing random, deterministic or blind-indexed encryption
| Approach | Same value, same ciphertext? | Equality search | Leaks | Use for |
|---|---|---|---|---|
| Random (Encryption SDK, new data key each time) | No | No | Only length (roughly) | Default for restricted fields you never search: date of birth, address |
| Deterministic (synthetic IV) | Yes | Yes, direct on the column | Which rows share a value, and how often | Low-cardinality lookups where you accept frequency leakage; a unique constraint on one field |
| Random plus blind index (HMAC beside the ciphertext) | No in the ciphertext; yes in the index | Yes, on the index column | Equality in the index; truncating the index adds false positives that hide uniqueness | The usual banking answer for "find the customer by IBAN" |
Ask the same questions each time: who can see the column, how many distinct values exist, and could an attacker guess the values? A column with three possible values encrypted deterministically is almost plaintext, because the most frequent ciphertext is obviously the most common value. The lab test Deterministic_encryption_leaks_the_frequency_of_values_and_random_encryption_does_not shows this with 300 payments.
AWS Console instructions
Check S3 default encryption. Console → S3 → bucket → Properties → Default encryption. You see the type (SSE-S3, SSE-KMS, DSSE-KMS), the key and whether Bucket Key is enabled.
Turn on EBS encryption by default. Console → EC2 → EC2 Dashboard → Account attributes → Data protection and security → EBS encryption → Manage → Enable → choose the default KMS key. It is per Region; repeat in every Region you use.
Create an encrypted RDS instance. Console → RDS → Create database → Additional configuration → Encryption → Enable encryption → choose the customer managed key. For PostgreSQL create a parameter group with rds.force_ssl set to 1 and attach it. The console field names move around; use the infrastructure-as-code below as the source of truth.
Choose a DynamoDB key. Console → DynamoDB → table → Additional settings → Encryption → Manage encryption → pick AWS owned, AWS managed or Stored in your account, owned and managed by you.
Set a load balancer policy. Console → EC2 → Load balancers → ALB → Listeners and rules → HTTPS:443 → Edit listener → Secure listener settings → Security policy.
AWS CLI
# 1. Who encrypts what: S3 bucket default encryption (an error means no default is configured)
aws s3api get-bucket-encryption --bucket "$BUCKET" --profile learn-sandbox
# 2. Are new EBS volumes encrypted by default in this Region, and with which key?
aws ec2 get-ebs-encryption-by-default --profile learn-sandbox
aws ec2 get-ebs-default-kms-key-id --profile learn-sandbox
# 3. Which volumes are NOT encrypted?
aws ec2 describe-volumes --filters Name=encrypted,Values=false \
--query "Volumes[].VolumeId" --profile learn-sandbox
# 4. RDS instances and their storage encryption
aws rds describe-db-instances \
--query "DBInstances[].[DBInstanceIdentifier,StorageEncrypted,KmsKeyId]" \
--output table --profile learn-sandbox
# 5. DynamoDB table encryption (SSEDescription absent = default AWS owned key)
aws dynamodb describe-table --table-name hl-m05-token-vault \
--query "Table.SSEDescription" --profile learn-sandbox
# 6. Turn on EBS encryption by default for this Region
aws ec2 enable-ebs-encryption-by-default --profile learn-sandbox
# 7. Prove a TLS policy from the outside: this should FAIL against a TLS 1.2-and-up listener
openssl s_client -connect api.example.com:443 -tls1_1 </dev/null
# 8. Encrypt an unencrypted EBS volume: snapshot, copy with encryption, create a volume
aws ec2 create-snapshot --volume-id vol-0123456789abcdef0 --profile learn-sandbox
aws ec2 copy-snapshot --source-region us-east-1 --source-snapshot-id snap-0123456789abcdef0 \
--encrypted --kms-key-id alias/hl-m05-at-rest --profile learn-sandboxCommands 7 and 8 use placeholder identifiers. Replace them with your own. Some OpenSSL builds have TLS 1.0 and 1.1 compiled out, so a failure to connect can mean the client refused, not the server; test with the handshake probe below for a reliable answer.
Infrastructure as Code
The lab project labs/m05-data-protection/infra/cdk compiles with Amazon.CDK.Lib 2.277.0 and has three stacks. The data-store stack:
using Amazon.CDK;
using Amazon.CDK.AWS.DynamoDB;
using Amazon.CDK.AWS.EC2;
using Amazon.CDK.AWS.IAM;
using Amazon.CDK.AWS.KMS;
using Amazon.CDK.AWS.RDS;
using Amazon.CDK.AWS.S3;
using Constructs;
namespace Harbourline.DataProtection.Cdk;
public sealed class DataStoresStack : Stack
{
public DataStoresStack(Construct scope, string id, IStackProps? props = null) : base(scope, id, props)
{
var key = new Key(this, "DataKey", new KeyProps
{
Alias = "alias/hl-m05-at-rest",
EnableKeyRotation = true,
PendingWindow = Duration.Days(7), // lab only
RemovalPolicy = RemovalPolicy.DESTROY, // lab only; production uses RETAIN
});
var bucket = new Bucket(this, "StatementsBucket", new BucketProps
{
Encryption = BucketEncryption.KMS,
EncryptionKey = key,
BucketKeyEnabled = true,
EnforceSSL = true,
BlockPublicAccess = BlockPublicAccess.BLOCK_ALL,
Versioned = true,
RemovalPolicy = RemovalPolicy.DESTROY,
});
// EnforceSSL denies plain HTTP; this statement also denies TLS older than 1.2.
bucket.AddToResourcePolicy(new PolicyStatement(new PolicyStatementProps
{
Sid = "DenyOldTls",
Effect = Effect.DENY,
Principals = [new AnyPrincipal()],
Actions = ["s3:*"],
Resources = [bucket.BucketArn, bucket.ArnForObjects("*")],
Conditions = new Dictionary<string, object>
{
["NumericLessThan"] = new Dictionary<string, object> { ["s3:TlsVersion"] = 1.2 },
},
}));
// Attribute is ambiguous with System.Attribute, so the type is fully qualified.
_ = new Table(this, "TokenTable", new TableProps
{
PartitionKey = new Amazon.CDK.AWS.DynamoDB.Attribute { Name = "token", Type = AttributeType.STRING },
Encryption = TableEncryption.CUSTOMER_MANAGED,
EncryptionKey = key,
BillingMode = BillingMode.PAY_PER_REQUEST,
PointInTimeRecoverySpecification = new PointInTimeRecoverySpecification { PointInTimeRecoveryEnabled = true },
RemovalPolicy = RemovalPolicy.DESTROY,
});
_ = new Volume(this, "DataVolume", new VolumeProps
{
AvailabilityZone = Fn.Select(0, Fn.GetAzs()),
Size = Size.Gibibytes(8),
Encrypted = true,
EncryptionKey = key,
RemovalPolicy = RemovalPolicy.DESTROY,
});
var vpc = new Vpc(this, "Vpc", new VpcProps
{
MaxAzs = 2,
NatGateways = 0,
SubnetConfiguration =
[
new SubnetConfiguration { Name = "isolated", SubnetType = SubnetType.PRIVATE_ISOLATED, CidrMask = 24 },
],
});
var parameters = new ParameterGroup(this, "PgParameters", new ParameterGroupProps
{
Engine = DatabaseInstanceEngine.Postgres(new PostgresInstanceEngineProps { Version = PostgresEngineVersion.VER_17 }),
Parameters = new Dictionary<string, string> { ["rds.force_ssl"] = "1" },
});
_ = new DatabaseInstance(this, "Ledger", new DatabaseInstanceProps
{
Engine = DatabaseInstanceEngine.Postgres(new PostgresInstanceEngineProps { Version = PostgresEngineVersion.VER_17 }),
InstanceType = Amazon.CDK.AWS.EC2.InstanceType.Of(InstanceClass.T4G, InstanceSize.MICRO),
Vpc = vpc,
VpcSubnets = new SubnetSelection { SubnetType = SubnetType.PRIVATE_ISOLATED },
Credentials = Credentials.FromGeneratedSecret("hl_app"),
StorageEncrypted = true,
StorageEncryptionKey = key,
ParameterGroup = parameters,
AllocatedStorage = 20,
DeletionProtection = false, // lab only
RemovalPolicy = RemovalPolicy.DESTROY,
});
// Amazon.CDK.Tags, not Tags: inside a Stack the bare name is the stack's own Tags property (CS0176).
Amazon.CDK.Tags.Of(this).Add("Project", "cea-lab");
Amazon.CDK.Tags.Of(this).Add("Lab", "m05-data-protection");
Amazon.CDK.Tags.Of(this).Add("Environment", "learn");
}
}The edge stack attaches an HTTPS listener with an explicit policy. The IaC default is the old one:
alb.AddListener("Https", new BaseApplicationListenerProps
{
Port = 443,
Protocol = ApplicationProtocol.HTTPS,
Certificates = [ListenerCertificate.FromArn(certificateArn.ValueAsString)],
// CDK, CloudFormation and the CLI default to ELBSecurityPolicy-2016-08, which accepts TLS 1.0.
SslPolicy = SslPolicy.TLS13_12_RES_PQ,
DefaultAction = ListenerAction.FixedResponse(200, new FixedResponseOptions
{
ContentType = "text/plain",
MessageBody = "ok",
}),
});Complete code
The lab project labs/m05-data-protection builds on .NET 10 with xUnit v3 and passes without an AWS account. Three pieces carry the lesson.
1. Prove the TLS policy with a handshake
using System.Net.Security;
using System.Net.Sockets;
using System.Security.Authentication;
namespace Harbourline.DataProtection.Tls;
public sealed record TlsProbeResult(bool Succeeded, SslProtocols? Negotiated, string? Error);
public static class TlsProbe
{
public static async Task<TlsProbeResult> ProbeAsync(
string host, int port, SslProtocols offered, bool trustAnyCertificate = false, CancellationToken cancellationToken = default)
{
try
{
using var tcp = new TcpClient();
await tcp.ConnectAsync(host, port, cancellationToken).ConfigureAwait(false);
using var ssl = new SslStream(tcp.GetStream(), leaveInnerStreamOpen: false);
var options = new SslClientAuthenticationOptions
{
TargetHost = host,
EnabledSslProtocols = offered,
};
if (trustAnyCertificate)
{
// Localhost self-signed demo only. Never ship this callback.
options.RemoteCertificateValidationCallback = (_, _, _, _) => true;
}
await ssl.AuthenticateAsClientAsync(options, cancellationToken).ConfigureAwait(false);
return new TlsProbeResult(true, ssl.SslProtocol, null);
}
catch (Exception ex) when (ex is AuthenticationException or IOException or SocketException)
{
return new TlsProbeResult(false, null, ex.GetType().Name);
}
}
}The test starts a real Kestrel server that only allows TLS 1.3, then checks that a TLS 1.2 client fails and a TLS 1.3 client succeeds:
[Fact]
public async Task Tls13_only_server_refuses_a_tls12_client_and_accepts_tls13()
{
using var cert = CreateLocalhostCertificate();
await using var app = DemoHost.Create(cert, SslProtocols.Tls13);
await app.StartAsync(TestContext.Current.CancellationToken);
var port = DemoHost.BoundAddress(app).Port;
var tls12 = await TlsProbe.ProbeAsync("localhost", port, SslProtocols.Tls12, trustAnyCertificate: true,
cancellationToken: TestContext.Current.CancellationToken);
var tls13 = await TlsProbe.ProbeAsync("localhost", port, SslProtocols.Tls13, trustAnyCertificate: true,
cancellationToken: TestContext.Current.CancellationToken);
Assert.False(tls12.Succeeded);
Assert.True(tls13.Succeeded);
Assert.Equal(SslProtocols.Tls13, tls13.Negotiated);
await app.StopAsync(TestContext.Current.CancellationToken);
}The Kestrel side is one option on the HTTPS adapter:
builder.WebHost.ConfigureKestrel(kestrel =>
kestrel.Listen(IPAddress.Loopback, 0, listen =>
listen.UseHttps(new HttpsConnectionAdapterOptions
{
ServerCertificate = certificate,
SslProtocols = protocols, // the same idea as an ALB security policy, enforced in-process
})));
app.Use(async (context, next) =>
{
// HSTS tells browsers to refuse plain HTTP for a year. An ALB does not add it for you.
context.Response.Headers.StrictTransportSecurity = "max-age=31536000; includeSubDomains";
await next();
});2. Field-level encryption with the AWS Encryption SDK
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using Amazon.KeyManagementService;
using AWS.Cryptography.EncryptionSDK;
using AWS.Cryptography.MaterialProviders;
namespace Harbourline.DataProtection.Fields;
public sealed class EsdkFieldEncryptor
{
public const string Prefix = "esdk:v1:";
private readonly ESDK _esdk = new(new AwsEncryptionSdkConfig());
private readonly IKeyring _keyring;
private EsdkFieldEncryptor(IKeyring keyring) => _keyring = keyring;
public static EsdkFieldEncryptor WithKeyring(IKeyring keyring) =>
new(keyring ?? throw new ArgumentNullException(nameof(keyring)));
public string Encrypt(string plaintext, IReadOnlyDictionary<string, string> context)
{
ArgumentNullException.ThrowIfNull(plaintext);
ArgumentNullException.ThrowIfNull(context);
var output = _esdk.Encrypt(new EncryptInput
{
Plaintext = new MemoryStream(Encoding.UTF8.GetBytes(plaintext)),
Keyring = _keyring,
EncryptionContext = new Dictionary<string, string>(context),
});
return Prefix + Convert.ToBase64String(output.Ciphertext.ToArray());
}
public string Decrypt(string token, IReadOnlyDictionary<string, string> expectedContext)
{
ArgumentNullException.ThrowIfNull(token);
ArgumentNullException.ThrowIfNull(expectedContext);
if (!token.StartsWith(Prefix, StringComparison.Ordinal))
{
throw new FormatException($"Value is not an '{Prefix}' token.");
}
var output = _esdk.Decrypt(new DecryptInput
{
Ciphertext = new MemoryStream(Convert.FromBase64String(token[Prefix.Length..])),
Keyring = _keyring,
EncryptionContext = new Dictionary<string, string>(expectedContext),
});
// The context stored in the message must contain every pair we expect.
foreach (var (key, value) in expectedContext)
{
if (!output.EncryptionContext.TryGetValue(key, out var actual) || !string.Equals(actual, value, StringComparison.Ordinal))
{
throw new CryptographicException($"Encryption context mismatch for '{key}'.");
}
}
return Encoding.UTF8.GetString(output.Plaintext.ToArray());
}
// One data key, so one KMS call, however many fields are in the record.
public string EncryptRecord(IReadOnlyDictionary<string, string> fields, IReadOnlyDictionary<string, string> context) =>
Encrypt(JsonSerializer.Serialize(fields), context);
public IReadOnlyDictionary<string, string> DecryptRecord(string token, IReadOnlyDictionary<string, string> expectedContext) =>
JsonSerializer.Deserialize<Dictionary<string, string>>(Decrypt(token, expectedContext))
?? throw new FormatException("Record payload was empty.");
public static IReadOnlyDictionary<string, string> ContextFor(string customerId, string field) =>
new Dictionary<string, string>(StringComparer.Ordinal) { ["customer_id"] = customerId, ["field"] = field };
}
public static class EsdkKeyrings
{
private static readonly MaterialProviders Providers = new(new MaterialProvidersConfig());
// Offline keyring for tests: the wrapping key is in process memory. Production uses KMS or an HSM.
public static IKeyring RawAes(byte[] wrappingKey, string keyNamespace = "harbourline", string keyName = "test-wrapping-key-1") =>
Providers.CreateRawAesKeyring(new CreateRawAesKeyringInput
{
KeyNamespace = keyNamespace,
KeyName = keyName,
WrappingKey = new MemoryStream(wrappingKey),
WrappingAlg = AesWrappingAlg.ALG_AES256_GCM_IV12_TAG16,
});
// Pass the key ARN, not an alias, so the key identity is unambiguous.
public static IKeyring Kms(IAmazonKeyManagementService kms, string keyArn) =>
Providers.CreateAwsKmsKeyring(new CreateAwsKmsKeyringInput { KmsClient = kms, KmsKeyId = keyArn });
}Used from an API, the call site is two lines:
var encryptor = EsdkFieldEncryptor.WithKeyring(EsdkKeyrings.Kms(kmsClient, keyArn));
var token = encryptor.Encrypt(iban, EsdkFieldEncryptor.ContextFor(customerId, "iban"));3. Deterministic encryption and a blind index
public byte[] Encrypt(string plaintext, string context)
{
var data = Encoding.UTF8.GetBytes(plaintext);
var aad = Encoding.UTF8.GetBytes(context);
// Synthetic nonce: an HMAC of context and plaintext. Equal inputs give equal nonces and ciphertexts.
byte[] seed = [.. aad, 0, .. data];
var nonce = HMACSHA256.HashData(_macKey, seed).AsSpan(0, NonceSize).ToArray();
var cipher = new byte[data.Length];
var tag = new byte[TagSize];
using var aes = new AesGcm(_encKey, TagSize);
aes.Encrypt(nonce, data, cipher, tag, aad);
return [Version, .. nonce, .. tag, .. cipher];
}
public static string Compute(byte[] key, string field, string normalisedValue, int outputBytes = 16)
{
// The unit separator keeps ("ab","c") and ("a","bc") apart; the field stops cross-column joins.
var mac = HMACSHA256.HashData(key, Encoding.UTF8.GetBytes(field + '\u001f' + normalisedValue));
return Convert.ToHexStringLower(mac.AsSpan(0, outputBytes));
}⚠ Learning shortcut: not for production
DeterministicFieldCipher composes standard primitives (HKDF, HMAC, AES-GCM) so you can see the leakage. Do not invent your own cipher mode in production code. Use a reviewed implementation: AES-SIV from a vetted library, SQL Server Always Encrypted deterministic mode, or the AWS Database Encryption SDK for DynamoDB (⚠ VERIFY the features it offers today), and have your cryptography reviewer approve the design.
Code explained line by line
TlsProbe
SslClientAuthenticationOptions.EnabledSslProtocols = offeredrestricts what the client offers. A TLS 1.3-only server and a TLS 1.2-only client share no protocol, so the handshake fails with anAuthenticationException.ssl.SslProtocolafterAuthenticateAsClientAsyncis what was actually negotiated. Report that, not what you hoped for.trustAnyCertificateexists only so a self-signed localhost certificate works in a test. Production clients validate the chain and host name.
EsdkFieldEncryptor
new ESDK(new AwsEncryptionSdkConfig())creates the client. The defaults require commitment on both encrypt and decrypt, which prevents a class of attacks where a ciphertext decrypts differently under different keys. ⚠ VERIFY the current default commitment policy and algorithm suite in the SDK documentation.Keyringtells the SDK how to obtain and protect the data key. The KMS keyring callsGenerateDataKeyon encrypt andDecrypton decrypt, passing your encryption context to KMS, so KMS refuses a context it did not see at encryption time.EncryptionContextis authenticated but not secret. It is stored in the message header and appears in CloudTrail. Identifiers only; never put a name or a date of birth in it.Decryptreceives the expected context and then re-checks it. The SDK message carries the context it was encrypted with, and the decrypting code must verify it matches what the caller expects, or a valid ciphertext for one customer can be presented as another's.EncryptRecordserialises several fields into one message. One data key, one KMS call. The price is that reading one field decrypts the bundle, and that the context names the bundle rather than each field.
DeterministicFieldCipher
- Two subkeys are derived from the master key with HKDF and different
infolabels, so the MAC key and the encryption key are never the same bytes. - The nonce is the first 12 bytes of an HMAC over the context, a separator and the plaintext. AES-GCM breaks if one nonce encrypts two different messages; here the same nonce only ever appears with the same message, which reveals equality and nothing more.
- The context goes in as AES-GCM associated data, so a ciphertext cannot be moved to another column.
BlindIndex.Compute
- A keyed hash (HMAC) is used, not a plain SHA-256. Dates of birth and sort codes have too few possible values; an attacker with a plain hash tries them all in seconds.
outputBytesis a deliberate privacy lever. One byte over a thousand values gives at most 256 buckets, so many values share an index. Your code then decrypts the candidates and compares. You trade lookup cost for less information in the index.
Security considerations
⚠ Learning shortcut: not for production
The offline tests use EsdkKeyrings.RawAes, a wrapping key held in process memory, and a fake KMS that counts calls. That is right for fast, free tests and wrong for production: real data keys must be wrapped by a KMS key whose policy and CloudTrail entries you control. The demo host also reads the viewer role from a header; a real API takes it from a validated token.
- Context is the audit label.
customer_idandfieldin the context mean a CloudTrail search for one customer's decrypts is possible, and a decrypt for the wrong customer fails. Design the context before the first record is written; changing it later means re-encrypting. - Key policy limits who can decrypt. Give the API role
kms:GenerateDataKeyandkms:Decryptwith a condition onkms:EncryptionContextKeys(see the KMS lesson) and keep administrators out of the key-user list. A DBA must not holdkms:Decrypt. - Rotation does not re-encrypt. Automatic rotation of a symmetric KMS key keeps old key material to decrypt old data. It does not rewrite your ciphertext. Re-encryption is a deliberate batch job. See Backup security and key rotation.
- Deleting a key destroys the data. This is the strength of the design and the danger. Use the waiting period, alarm on
ScheduleKeyDeletion, and keep tested backups. - Verify certificates on database connections.
SSL Mode=Requireencrypts but may accept any certificate. UseVerifyFullwith the RDS CA bundle (⚠ VERIFY the current RDS CA name and bundle location) so a man in the middle cannot present a different certificate. - TLS ends at the edge. An ALB decrypts the request. Decide deliberately whether traffic from the ALB to the container is encrypted, and note that an ALB does not validate the target's certificate chain in the way a client does ⚠ VERIFY. If your standard requires encryption between hops, use HTTPS target groups and a service mesh or an internal CA.
- Do not log decrypted values. Encrypt at rest, then print the value in a debug log, and the control is gone. See the next lesson, data classification, masking and tokenisation.
- Key commitment and algorithm suites. Keep the SDK defaults unless a cryptography reviewer says otherwise. The default message carries a header and a signature, so each tiny field costs hundreds of bytes ⚠ VERIFY. For many small fields prefer one record bundle (above) or a suite without signing, as a reviewed decision.
Banking use case
Harbourline's customer profile table holds name, email, IBAN, date of birth and phone number. The design:
- The RDS instance has storage encryption with the
customer-profilekey andrds.force_ssl=1. This stops a stolen snapshot and a plain-text connection. - Name and email are Confidential: they sit in ordinary columns, protected by the service and by masking in logs.
- IBAN, date of birth and phone number are Restricted: encrypted by the application with the Encryption SDK and the context
{customer_id, field}. - Support staff look customers up by IBAN. The table stores
iban_ct(random ciphertext) andiban_bidx(a 16-byte blind index). The API computes the index from the typed IBAN, queriesiban_bidx, decrypts the candidate rows and compares. The HMAC key for the index is in Secrets Manager (see Secrets Manager and Parameter Store); it is a separate secret from any encryption key. - Country of residence has only about 200 values and is queried in reports. It is Internal, not encrypted at field level, because its encryption would be pointless (low cardinality, easily guessed).
The decision for each field is documented in the data inventory with an owner. When the auditor asks "which fields are encrypted by your application?" the answer is a query, not a meeting.
Common mistakes
- "We use an AWS owned key, so it is encrypted." → True, and you cannot audit or revoke it → use a customer managed key for Confidential and Restricted data, and say why for the rest.
- Trying to encrypt an existing RDS instance in place. → The setting is creation-only → snapshot, copy with encryption, restore, then cut over.
- Leaving the ALB on the IaC default policy. → TLS 1.0 stays on → set
SslPolicyexplicitly and add a detective control. StringNotEqualson the encryption header in a bucket policy. → It denies uploads that rely on bucket default encryption → useStringNotEqualsIfExists.- Deterministic encryption on a low-cardinality column. → Frequency analysis recovers the values → use random encryption, or do not encrypt a field that is not secret.
- Plain SHA-256 as a search index. → Brute-forceable for small value spaces → use a keyed HMAC with a key kept out of the database.
- Personal data in the encryption context. → It appears in clear text in CloudTrail → identifiers only.
- One KMS call per field at scale. → Cost and throttling (see the quotas in the KMS lesson) → bundle fields per record, enable S3 Bucket Keys, or evaluate the Encryption SDK's hierarchical keyring ⚠ VERIFY.
TrustServerCertificate=Truecopied into production. → Encryption without identity → verify the chain and the host name.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
AccessDeniedException from KMS on S3 upload | The uploading role lacks kms:GenerateDataKey on the bucket's key | Add the key-user statement to the key policy, scoped with kms:ViaService |
InvalidCiphertextException on decrypt | Encryption context differs from the one used to encrypt | Rebuild the context from the same identifiers; check for casing and extra keys |
| Handshake fails to the load balancer from an old client | The policy no longer offers the client's protocol or ciphers | Upgrade the client; do not weaken the policy without a documented exception |
terraform apply fails: cannot modify storage_encrypted | It forces replacement of the instance | Snapshot, restore encrypted, import or recreate; plan the cutover |
| Uploads rejected after adding the "wrong header" deny | StringNotEquals instead of StringNotEqualsIfExists | Switch the operator |
| Object cannot be shared cross-account | Snapshot or object encrypted with an AWS managed key | Re-encrypt with a customer managed key whose policy allows the other account |
ThrottlingException from KMS during a batch | Many per-field data keys at once | Bundle fields, add jittered retries, request a quota increase if justified |
Npgsql: The remote certificate is invalid | The RDS CA bundle is not in the container | Mount the bundle and set Root Certificate; do not switch to Trust Server Certificate |
| Tests pass locally but the TLS 1.0 check "fails to connect" | The local OpenSSL refuses TLS 1.0 | Use a server-side probe or a scanning tool; do not conclude the server rejected it |
Cost and cleanup
Free: the unit tests, terraform validate, dotnet build of the CDK project, and reading the inventory.
Credits (deploy, look, destroy within the hour):
- Customer managed KMS keys: USD 1 per key per month, prorated hourly; keys pending deletion are not charged. Symmetric requests: USD 0.03 per 10,000 after a monthly free tier ⚠ VERIFY on the pricing page.
- RDS
db.t4g.micro: about an hour of instance time and 20 GB of storage; ⚠ VERIFY the current hourly price before deploying. - Application Load Balancer: an hourly charge plus capacity units while it exists, and an ACM public certificate is free. The edge stack needs a certificate ARN you supply; use a public certificate on a domain you own, or skip the edge stack and use the in-process Kestrel test.
- DynamoDB on demand and an 8 GiB EBS volume: cents for an hour.
Design-only alternative: do not deploy. Run the tests, read the CDK and Terraform and fill in the exercise table.
# CDK: destroy everything the lab created
cd labs/m05-data-protection/infra/cdk
cdk destroy --all --profile learn-sandbox
# Terraform: destroy and then confirm the keys are pending deletion
cd labs/m05-data-protection/infra/terraform
terraform destroy -var "db_subnet_group_name=..." -var "alb_arn=..." -var "certificate_arn=..."
aws kms list-keys --profile learn-sandbox
aws kms describe-key --key-id alias/hl-m05-at-rest --profile learn-sandbox --query "KeyMetadata.KeyState"
# If you turned on account-wide EBS encryption by default for the lab, turn it back off if the account policy allows
aws ec2 disable-ebs-encryption-by-default --profile learn-sandbox
aws ec2 reset-ebs-default-kms-key-id --profile learn-sandboxDestroying the stacks schedules the keys for deletion after the 7-day waiting period; data encrypted under them becomes unreadable afterwards. That is expected in a lab. Check the billing console the next day.
Exercises
- 🟢 Guided: Run the lab tests. Then read
TlsTestsand changeSslProtocols.Tls13toSslProtocols.Tls12 | SslProtocols.Tls13in the first test. Predict which assertion fails, then run it to confirm. - 🟡 Challenge: Add a new field
phoneto the customer profile design. Decide random, deterministic or blind-indexed. Write the encryption context, extend the lab test with a round trip and a wrong-customer check, and write two sentences on the leakage you accepted. - 🔴 Live problem: A pen test reports that production accepts TLS 1.1 on one public API, that two RDS instances have
StorageEncrypted=false, and that a developer stored customer IBANs with deterministic encryption in a column with a unique index. Write a one-page remediation plan: order of work, which fixes are online and which need downtime, what you tell the auditor, and how you prevent a repeat.
Assignment
Add a "data protection posture" report to your portfolio repository. Use AwsEncryptionInventorySource (or the CLI commands) against your sandbox, apply EncryptionPostureEvaluator, and commit the output as a Markdown table with one row per data store: service, resource, encrypted, key type, finding. Add a short paragraph on which findings you would fix first and why.
Solutions
With Tls12 | Tls13 on the server, the Assert.False(tls12.Succeeded) line fails: the TLS 1.2 client now shares a protocol with the server, so its handshake succeeds and negotiates TLS 1.2. The TLS 1.3 assertion still passes. This is exactly how a policy with a lower minimum lets older clients in. The lesson is that the minimum version is a property of the server, and the only way to be sure is to attempt a handshake as the old client.
Phone numbers are high-cardinality and not usually searched exactly, so use random encryption with the context {customer_id, field=phone}. If support must find a customer by phone, add a blind index computed on the normalised number (digits only with country code) using a field-specific HMAC key; use a truncated index if you want to hide uniqueness. The accepted leakage is that two rows with the same phone have the same index. A test mirrors the IBAN one: encrypt, decrypt, then decrypt with customer_id changed and assert it throws. Note that the context contains only the customer id and the field name, not the phone number.
Order: (1) The deterministic IBAN column is the worst finding for privacy because equality and frequency leak: plan a migration to random ciphertext plus a blind index, backfill online (write both, read both), and drop the unique index in favour of a unique constraint on the blind index. (2) The two unencrypted RDS instances need snapshot, encrypted copy and restore: that needs a cutover window, so schedule it with the owners and test the restore first. (3) TLS 1.1: change the listener policy in infrastructure as code, check client telemetry first for who still uses TLS 1.1, and give any partner a dated notice. Tell the auditor what was found, the compensating controls in place (network segmentation, access restrictions), the dated plan and the owner. Prevent repeats with an SCP or Config rule that denies or flags unencrypted RDS creation, a Security Hub CSPM control for ELB policies, and a pull-request check that fails an IaC change omitting SslPolicy.
Knowledge check
Answer: The database decrypts transparently for any authorised query, and the injected query runs with the application's authority. Only application-layer (field-level) encryption, where the application role alone can decrypt, helps. TLS only protects the network path, so it is irrelevant here.
Answer: Snapshot it, copy the snapshot with encryption enabled and a customer managed key, restore a new instance from the copy and cut over. You cannot enable storage encryption on a running instance.
Answer: Equal values give equal ciphertexts, so the ciphertext frequencies reveal which value is which. With three values, the most common one is easily guessed. Random encryption has no such leak.
Answer: Non-secret key-value pairs bound cryptographically to the ciphertext, used for authentication and audit. They are stored in the message header and logged in CloudTrail in clear text, so they hold identifiers such as customer_id, never names or dates of birth.
Answer: The infrastructure-as-code default is an older policy that allows TLS 1.0 and 1.1, even though the console suggests a modern one. Set the policy explicitly and add a detective control.
Answer: A plain hash of a low-entropy value can be brute-forced by anyone who steals the table. An HMAC needs the key, which is stored elsewhere. The key must be a different secret from any encryption key.
Interview questions
- "Walk me through how you would encrypt customer data on AWS." Layers: TLS 1.2 or later on every hop with explicit policies; service encryption with customer managed keys per data domain; field-level encryption with an encryption context for Restricted data; key policy separating administrators from users; CloudTrail on every key use; Config and Security Hub CSPM for drift. Then name what each layer does not cover.
- "Random or deterministic encryption?" Random by default. Deterministic only when equality lookup is needed and the leakage (equality, frequency) is acceptable, ideally on high-cardinality fields. For searchable restricted fields, random ciphertext plus a keyed blind index.
- "How do you prove to an auditor that everything is encrypted?" An inventory query across services, with encrypted flag and key type per resource, backed by Config rule history and key policies. Not a policy document.
- "A KMS call per field is too slow. What now?" Bundle fields per record, use S3 Bucket Keys, evaluate data-key caching or the hierarchical keyring, add retries with jitter and watch the account quota.
- "What does TLS termination at the load balancer mean for security?" The load balancer sees plain text, so the next hop needs its own protection, and policies, certificates and logs on the load balancer are part of the control.
Certification practice
A bank stores customer profiles in Amazon RDS for PostgreSQL. A security review finds that a database administrator can read national identifiers with ordinary SQL and that a recent SQL injection finding exposed the same column. The bank wants only the application's service role to read the identifiers, with an audit trail of each decryption. What is the BEST solution?
A. Enable RDS storage encryption with an AWS managed key.
B. Encrypt the column in the application with the AWS Encryption SDK using a customer managed KMS key whose policy allows only the service role, with an encryption context containing the customer id.
C. Require TLS to the database with rds.force_ssl.
D. Move the column to an S3 bucket with SSE-S3.
Answer: B. The application encrypts before storage, only the service role holds kms:Decrypt, and CloudTrail logs each use with the context. A protects only disks and snapshots, and the database still decrypts for every query. C protects the network path. D uses a key owned by the service with no per-principal control.
A company must encrypt an existing unencrypted Amazon EBS volume attached to a production instance, using a customer managed key, with minimal risk. Which steps are correct?
A. Enable encryption on the volume in the console. B. Create a snapshot, copy the snapshot with encryption and the customer managed key, create a volume from the copy and attach it. C. Change the instance type to one that supports encryption. D. Attach an encrypted volume and let AWS migrate the data automatically.
Answer: B. EBS encryption cannot be switched on in place. A is not possible for an existing volume. C is irrelevant: modern instance types all support encryption. D has no such automatic migration.
A security engineer wants to guarantee that no client can talk to an S3 bucket over plain HTTP or with a TLS version older than 1.2. Which control enforces this?
A. A bucket policy with Deny statements conditioned on aws:SecureTransport being false and on s3:TlsVersion being less than 1.2.
B. Enable S3 Block Public Access.
C. Turn on default encryption with SSE-KMS.
D. Enable S3 versioning.
Answer: A. Only a policy condition evaluates the transport properties of the request. B controls public exposure, C controls storage encryption and D keeps previous versions; none of them looks at the protocol. ⚠ VERIFY the exact condition key semantics for s3:TlsVersion in the S3 documentation.
A developer encrypts payloads under 4 KB by calling KMS Encrypt directly, and the service now needs to encrypt 5 MB documents. What should the developer change?
A. Split the document into 4 KB pieces and call Encrypt for each.
B. Use envelope encryption: request a data key with GenerateDataKey, encrypt locally, and store the encrypted data key with the data.
C. Raise the KMS payload quota.
D. Use a larger key size.
Answer: B. Direct Encrypt is limited to 4 KB, and splitting into many calls is slow, costly and throttle-prone. The Encryption SDK implements B for you. The 4 KB limit is not a quota that can be raised, and key size does not change it.
Completion checklist
- I can name the three layers of data protection and one threat each does not stop
- I ran the lab tests and understand why the TLS 1.3-only server refuses a TLS 1.2 client
- I can say which encryption settings cannot be changed on an existing RDS instance or EBS volume
- I can explain random, deterministic and blind-indexed encryption and the leakage of each
- I completed the guided exercise and the data protection posture report
- I ran the cleanup steps and confirmed nothing is still billing
Key takeaways
- Encryption in transit, at rest and in the application are three controls; each stops different threats.
- Service-side encryption protects disks and backups, not queries. For Restricted fields, encrypt in the application with a context that names the customer and the field.
- Set TLS policies explicitly in infrastructure as code and prove them with a handshake, not an assumption.
- Random encryption is the default; deterministic and blind indexes buy searchability with leakage you must state.
- Evidence beats assertion: an inventory of every store, key and setting is what an auditor accepts.
Further reading
Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.
Certificates, TLS and mTLS
Issue and auto-renew certificates with ACM, decide when AWS Private CA is worth paying for, pick strict TLS policies on ALB and API Gateway, and build mutual TLS between a partner and a .NET API.
S3 protection: Block Public Access, bucket policies, Object Lock and Macie
Lock an S3 bucket down so it cannot be made public, accepts only TLS and the bank's KMS key, keeps WORM copies of records with Object Lock, and is scanned for personal data by Amazon Macie.