When banks need CloudHSM
Decide when a bank genuinely needs single-tenant HSMs: AWS CloudHSM clusters, KMS custom key stores (CloudHSM and external), AWS Payment Cryptography, the PKCS#11 programming model from .NET, and the real operational cost.
Assumes: .NET 10 LTS · Pkcs11Interop 5.3.0 · SoftHSM2 (Debian package) · AWS CLI v2 · Region us-east-1. Verified against official sources on 2026-11-18.
Learning objectives
By the end of this lesson you can:
- Explain what a hardware security module (HSM) is, and what AWS KMS already gives you before you reach for a dedicated one.
- Describe AWS CloudHSM: single-tenant HSMs in clusters, FIPS and non-FIPS modes, HSM types, users, backups and the shared-responsibility split.
- Choose between AWS KMS, a KMS CloudHSM key store, a KMS external key store (XKS), AWS CloudHSM used directly, and AWS Payment Cryptography for a given banking requirement.
- Write .NET code against the PKCS#11 interface and run it for free against SoftHSM2, knowing that the same code targets the CloudHSM PKCS#11 library.
- Estimate the monthly cost and staffing of a production CloudHSM deployment and present it honestly to an architecture board.
Prerequisites
- AWS KMS: keys, key policies and envelope encryption — this lesson assumes you understand KMS keys and key policies
- Module 2: Networking — VPCs, subnets and security groups (CloudHSM lives in your VPC)
- Tools: Docker Desktop (for the free SoftHSM2 lab), .NET 10 SDK in Docker, AWS CLI v2 for the read-only commands
Why this matters in banking
A typical conversation in a bank's architecture review board:
Head of Cryptography: "Our policy says keys protecting customer data must be held in an HSM under the bank's exclusive control." Cloud team: "KMS uses HSMs." Head of Cryptography: "Shared ones, operated by AWS. I want our own."
Both sides are partly right. AWS KMS already keeps key material inside FIPS-validated HSMs that never export it in plain text, and for the great majority of workloads — S3, RDS, field-level encryption, signing — it is the right answer. But some requirements genuinely need more:
- A regulator, scheme or contract requires single-tenant HSMs where the bank alone holds the cryptographic officer credentials.
- An existing application uses PKCS#11, JCE or Microsoft CNG directly and needs keys and algorithms KMS does not offer (for example, key wrapping with specific mechanisms, or a private CA's signing key).
- Payment card processing needs PIN translation, CVV generation, or key exchange with card schemes — a payment-HSM workload.
- The bank's policy requires keys to stay outside AWS entirely.
Choosing CloudHSM when KMS would do costs thousands of dollars a month and a team to operate it. Choosing KMS when a scheme requires a payment HSM fails an audit. This lesson teaches you to tell the difference and to explain the cost.
The idea in plain language
A hardware security module is a tamper-resistant computer whose only job is to generate, store and use cryptographic keys. Keys are created inside it and, if marked non-extractable, can never be read out — applications send data in and get ciphertext or signatures back. If someone tries to open the box, it destroys its keys.
AWS CloudHSM rents you such HSMs in the AWS cloud. Each HSM is single-tenant: no other customer's keys are on it. You put HSMs into a cluster spread across Availability Zones in your VPC; the cluster keeps keys synchronised between them. AWS manages the hardware, firmware, availability and backups. You manage everything inside: the crypto officers, the crypto users, the keys, and the applications.
Analogy: KMS is a bank's safe-deposit service: the bank owns the vault, staff check your identity, and you never hold the vault keys — but you decide who may open your box and everything is logged. CloudHSM is renting your own vault room inside the bank's building: the building's security, power and fire protection are the bank's job, but the room's combination is yours alone. Lose the combination and nobody, including the bank, can open it.
| Term | Plain meaning |
|---|---|
| Cluster | A group of HSMs (one per AZ recommended) that replicate keys and users; your unit of high availability |
| Crypto officer (CO) / admin | Manages users on the HSM; cannot use keys |
| Crypto user (CU) | Creates and uses keys |
| Client SDK | The software on your EC2 or container that talks to the cluster: PKCS#11 library, JCE provider, OpenSSL Dynamic Engine, and on Windows a KSP/CNG provider |
| Custom key store | A KMS feature that keeps a KMS key's material in your CloudHSM cluster (CloudHSM key store) or in your own external key manager (external key store) |
| FIPS 140 | The US/Canadian standard for cryptographic modules; Level 3 adds physical tamper resistance and identity-based authentication |
What problem the AWS services solve
AWS CloudHSM (verified 2026-11-18)
| Fact | Detail |
|---|---|
| Tenancy | Single-tenant HSMs in your VPC (an elastic network interface per HSM in your subnets) |
| Validation | FIPS 140-2 Level 3 or FIPS 140-3 Level 3 validated hardware for clusters in FIPS mode |
| Cluster modes | FIPS (only FIPS-approved mechanisms) or non-FIPS (all supported mechanisms). The mode cannot be changed after creation |
| HSM types | hsm1.medium (older generation, FIPS mode only, reaching end of support — new clusters cannot use it; AWS documents a migration path) and hsm2m.medium (current, launched August 2024; FIPS 140-3 Level 3 in FIPS mode, or non-FIPS mode) |
| High availability | Your responsibility: at least two HSMs in different AZs for production; AWS replaces failed HSMs and restores from backup |
| Backups | Taken by AWS, encrypted, usable only to restore into a cluster of the same mode; can be copied to other Regions |
| Programming interfaces | PKCS#11, JCE (Java), OpenSSL Dynamic Engine, Microsoft CNG and KSP (Windows) |
| Access control | IAM controls the management plane (create cluster, add HSM). HSM users and passwords control the data plane; AWS cannot see your keys or reset your crypto officer password |
| Billing | Per HSM, per hour (or partial hour) while it exists. No upfront fee. In us-east-1 AWS's own published example uses USD 1.60 per HSM-hour for current HSMs (⚠ VERIFY your Region in the AWS Pricing Calculator) |
KMS custom key stores
| Option | What it is | Why choose it | Limits |
|---|---|---|---|
| AWS CloudHSM key store | KMS keys whose AES-256 material lives in your CloudHSM cluster; KMS logs in as a dedicated crypto user kmsuser | You keep KMS's integrations (S3, RDS, EBS…) and API, but key material sits in single-tenant HSMs you control; you can disconnect the store to stop all use | Symmetric encryption keys only; no automatic rotation, imported material, asymmetric, HMAC or multi-Region keys; cluster needs two active HSMs in different AZs to create keys; 1,800 requests/second per key store, not adjustable; up to 10 custom key stores per account and Region |
| External key store (XKS) | KMS keys backed by a key manager outside AWS, reached through an XKS proxy you run | "Hold your own key": data cannot be decrypted if you cut the connection | You own the availability and latency of every encrypt/decrypt; an outage of your key manager is an outage of every AWS service using those keys |
AWS Payment Cryptography
A managed, elastic service for payment HSM functions — PIN translation, card verification values, EMV cryptograms, DUKPT, key exchange using TR-31/TR-34 key blocks — with PCI PIN and PCI P2PE-relevant controls handled by AWS ⚠ VERIFY current attestations in AWS Artifact. If the requirement is "we need a payment HSM", this is usually the AWS answer, not general-purpose CloudHSM. The .NET SDK packages are AWSSDK.PaymentCryptography (control plane) and AWSSDK.PaymentCryptographyData (data plane), version 4.1.0.2 at the time of writing.
Decision summary
Microsoft / Azure comparison
| You know (Microsoft/Azure) | AWS equivalent | What's different |
|---|---|---|
| Azure Key Vault Premium (HSM-backed keys, multi-tenant) | AWS KMS | Both keep keys in FIPS-validated HSMs shared across customers |
| Azure Key Vault Managed HSM (single-tenant, managed) | KMS CloudHSM key store, or CloudHSM | Managed HSM gives a Key Vault-style API on single-tenant HSMs; on AWS you combine CloudHSM with KMS's custom key store to get a similar result |
| Azure Dedicated HSM | AWS CloudHSM used directly | You manage users and keys through vendor/client tooling; check Microsoft's current guidance on Dedicated HSM's availability before comparing ⚠ VERIFY |
| Azure Payment HSM | AWS Payment Cryptography | Azure Payment HSM gives you dedicated payment HSM appliances; Payment Cryptography is an API-first managed service |
| Key Vault "Hold your own key" / external HSM integrations | KMS external key store (XKS) | XKS requires you to run a proxy with specified latency and availability |
| CNG / KSP on Windows Server with an on-premises HSM | CloudHSM KSP/CNG provider on Windows EC2 | Existing Windows apps (AD CS, SQL Server TDE EKM) can move with less code change |
Architecture
Responsibility split — the part architecture boards most often misunderstand:
| Area | AWS | Harbourline |
|---|---|---|
| HSM hardware, firmware, tamper response, data-centre security | ✅ | |
| Replacing a failed HSM, taking backups | ✅ | |
| Number of HSMs and AZ placement (availability) | ✅ | |
| Crypto officer and crypto user credentials, quorum authentication | ✅ — AWS cannot recover them | |
| Keys, key attributes (extractable or not), key lifecycle | ✅ | |
| Client SDK patching on application hosts | ✅ | |
| Audit logs of HSM user activity (sent to CloudWatch Logs) and their review | Delivery ✅ | Review ✅ |
| Capacity planning (operations per second per HSM) | ✅ |
Step-by-step implementation
This is a design-only exercise in your sandbox: you will run every step that is free (read-only CLI calls, local PKCS#11 against SoftHSM2) and write down the paid steps without running them.
1. Write the requirement in testable terms
Turn "keys must be in an HSM" into precise questions: Which keys? Single-tenant required, or FIPS 140 Level 3 only? Who must hold administrative credentials? Must keys stay outside AWS? Which algorithms and interfaces? What request rate and latency? What recovery time if one AZ fails? Many requirements turn out to be satisfied by KMS once written down — KMS's HSMs are FIPS 140-3 validated too (check the current certificate in the KMS documentation).
2. Pick the option with the decision flowchart
Record the reason in an architecture decision record (ADR). If you end with "CloudHSM used directly", write down which application needs PKCS#11/JCE/CNG and why KMS's API cannot serve it.
3. Design the cluster (paid — design only)
- Region and two or three AZs; one
hsm2m.mediumper AZ. - FIPS mode unless a specific non-FIPS mechanism is required (the mode is permanent).
- Private subnets; a security group that allows the HSM ports only from the application security group.
- Initialise the cluster: sign the cluster CSR with your CA (an offline or private CA), activate it, set the admin password, enable quorum authentication (M of N) for admin operations.
- Create crypto users per application; never share credentials between applications.
- Send HSM audit logs to CloudWatch Logs and alert on user creation and key deletion.
4. Prototype the application against SoftHSM2 (free)
PKCS#11 is a standard C interface. SoftHSM2 implements it in software. Code written against PKCS#11 runs against SoftHSM2 on your laptop and against the CloudHSM PKCS#11 library in AWS with only the library path and login string changed. The complete code is below.
5. Cost it honestly
Two HSMs × 730 hours × USD 1.60 ≈ USD 2,336 per month before EC2 for clients, data transfer and staff. Three HSMs for N+1 in three AZs ≈ USD 3,504. Add at least a fraction of a crypto engineer's time for user management, client patching, capacity monitoring and key ceremonies. Compare with KMS: a customer managed key is USD 1 per month plus requests.
6. Plan exit and recovery
Document how keys are backed up (cluster backups, cross-Region backup copies), how you would restore the cluster from backup, and what happens to data if you ever lose all crypto officer credentials (it is unrecoverable). For a CloudHSM key store, document the effect of disconnecting the store: every AWS service that uses those KMS keys stops being able to decrypt.
AWS Console instructions
Read-only exploration (free):
- Console → CloudHSM → Clusters. With no clusters you see Create cluster. Open the wizard to see the options — VPC, subnets per AZ, HSM type and mode (FIPS / non-FIPS) — then cancel before creating anything.
- Console → Key Management Service → Custom key stores. Note the two tabs: AWS CloudHSM key stores and External key stores. Open Create custom key store to see that it asks for a cluster ID, the cluster's trust anchor certificate and the
kmsuserpassword, then cancel. - Console → AWS Payment Cryptography. Browse Keys and the key-creation options (key usages such as PIN encryption, card verification) without creating a key.
- Console → AWS Artifact → Reports. Search for the PCI DSS Attestation of Compliance and the FIPS-related documentation for the services you might use. Your compliance team will ask for them.
AWS CLI
export AWS_PROFILE=learn-sandbox AWS_REGION=us-east-1
# Read-only: any clusters, HSMs and backups in this Region? (Expect empty lists in a sandbox.)
aws cloudhsmv2 describe-clusters --query 'Clusters[].{Id:ClusterId,State:State,Mode:Mode,Type:HsmType,Hsms:length(Hsms)}'
aws cloudhsmv2 describe-backups --query 'Backups[].{Id:BackupId,State:BackupState,Cluster:ClusterId}'
# Read-only: custom key stores known to KMS.
aws kms describe-custom-key-stores
# DESIGN ONLY — do not run in the sandbox. Creating a cluster and HSMs starts hourly billing.
# aws cloudhsmv2 create-cluster --hsm-type hsm2m.medium --mode FIPS \
# --subnet-ids subnet-aaa subnet-bbb \
# --tag-list Key=Project,Value=cea-lab Key=Lab,Value=m05-hsm Key=Environment,Value=learn
# aws cloudhsmv2 create-hsm --cluster-id cluster-xxxx --availability-zone us-east-1a
# aws kms create-custom-key-store --custom-key-store-name hl-prod-cks \
# --cloud-hsm-cluster-id cluster-xxxx --trust-anchor-certificate file://customerCA.crt \
# --key-store-password '<kmsuser password from your secrets process>'
# aws kms create-key --origin AWS_CLOUDHSM --custom-key-store-id cks-xxxx --description "HSM-backed key"
# If you ever create one by mistake: delete HSMs first (billing stops per HSM), then the cluster.
# aws cloudhsmv2 delete-hsm --cluster-id cluster-xxxx --hsm-id hsm-xxxx
# aws cloudhsmv2 delete-cluster --cluster-id cluster-xxxx--mode (FIPS or NON_FIPS) arrived with the hsm2m.medium generation, so update the AWS CLI if create-cluster help does not list it.
Infrastructure as Code
Both examples are design-only: they would bill per HSM-hour if applied. They are useful for review and for terraform validate / cdk synth.
using Amazon.CDK;
using Amazon.CDK.AWS.EC2;
using Constructs;
namespace Harbourline.Security.Infra;
/// <summary>DESIGN ONLY. Synthesises a two-AZ CloudHSM cluster; deploying it bills per HSM-hour.</summary>
public sealed class CloudHsmDesignStack : Stack
{
public CloudHsmDesignStack(Construct scope, string id, IStackProps? props = null) : base(scope, id, props)
{
var vpc = new Vpc(this, "SecurityVpc", new VpcProps
{
MaxAzs = 2,
NatGateways = 0,
SubnetConfiguration =
[
new SubnetConfiguration { Name = "hsm", SubnetType = SubnetType.PRIVATE_ISOLATED, CidrMask = 26 },
],
});
// CDK has no higher-level CloudHSM construct. A raw CfnResource works with any CDK version;
// ⚠ VERIFY the AWS::CloudHSM::Cluster property names in the CloudFormation resource reference.
var cluster = new CfnResource(this, "Cluster", new CfnResourceProps
{
Type = "AWS::CloudHSM::Cluster",
Properties = new Dictionary<string, object>
{
["HsmType"] = "hsm2m.medium",
["Mode"] = "FIPS",
["SubnetIds"] = vpc.IsolatedSubnets.Select(s => s.SubnetId).ToArray(),
},
});
Amazon.CDK.Tags.Of(this).Add("Project", "cea-lab");
Amazon.CDK.Tags.Of(this).Add("Lab", "m05-hsm");
Amazon.CDK.Tags.Of(this).Add("Environment", "learn");
_ = new CfnOutput(this, "ClusterId", new CfnOutputProps { Value = cluster.Ref });
}
}Complete code
A .NET 10 console app that uses PKCS#11 through the open-source Pkcs11Interop library: generate a non-extractable AES-256 key inside the token, prove it cannot be read, and encrypt/decrypt with AES-GCM inside the token. It runs for free against SoftHSM2 in Docker; pointed at /opt/cloudhsm/lib/libcloudhsm_pkcs11.so on a host with the CloudHSM Client SDK 5, the same code uses CloudHSM.
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
<TreatWarningsAsErrors>true</TreatWarningsAsErrors>
<RootNamespace>Harbourline.Security.Pkcs11Demo</RootNamespace>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Pkcs11Interop" Version="5.3.0" />
</ItemGroup>
</Project>using System.Security.Cryptography;
using System.Text;
using Net.Pkcs11Interop.Common;
using Net.Pkcs11Interop.HighLevelAPI;
// PKCS#11 demo: the same code talks to SoftHSM2 (free, local) or the AWS CloudHSM PKCS#11 library.
// SoftHSM2: PKCS11_LIB=/usr/lib/softhsm/libsofthsm2.so PKCS11_PIN=1234
// CloudHSM: PKCS11_LIB=/opt/cloudhsm/lib/libcloudhsm_pkcs11.so PKCS11_PIN=<crypto-user>:<password>
var libraryPath = Environment.GetEnvironmentVariable("PKCS11_LIB") ?? "/usr/lib/softhsm/libsofthsm2.so";
var pin = Environment.GetEnvironmentVariable("PKCS11_PIN")
?? throw new InvalidOperationException("Set PKCS11_PIN (never hard-code HSM credentials).");
var factories = new Pkcs11InteropFactories();
using var library = factories.Pkcs11LibraryFactory.LoadPkcs11Library(factories, libraryPath, AppType.MultiThreaded);
var slot = library.GetSlotList(SlotsType.WithTokenPresent).FirstOrDefault()
?? throw new InvalidOperationException("No initialised token found.");
Console.WriteLine($"Token: {slot.GetTokenInfo().Label.Trim()} ({slot.GetTokenInfo().ManufacturerId.Trim()})");
using var session = slot.OpenSession(SessionType.ReadWrite);
session.Login(CKU.CKU_USER, pin);
try
{
// 1. Generate a non-extractable AES-256 key INSIDE the HSM. CKA_TOKEN=false: a session key, gone at logout.
var keyTemplate = new List<IObjectAttribute>
{
factories.ObjectAttributeFactory.Create(CKA.CKA_CLASS, CKO.CKO_SECRET_KEY),
factories.ObjectAttributeFactory.Create(CKA.CKA_KEY_TYPE, CKK.CKK_AES),
factories.ObjectAttributeFactory.Create(CKA.CKA_VALUE_LEN, 32UL),
factories.ObjectAttributeFactory.Create(CKA.CKA_LABEL, "hl-m05-demo-aes"),
factories.ObjectAttributeFactory.Create(CKA.CKA_TOKEN, false),
factories.ObjectAttributeFactory.Create(CKA.CKA_SENSITIVE, true),
factories.ObjectAttributeFactory.Create(CKA.CKA_EXTRACTABLE, false),
factories.ObjectAttributeFactory.Create(CKA.CKA_ENCRYPT, true),
factories.ObjectAttributeFactory.Create(CKA.CKA_DECRYPT, true),
};
using var keyGen = factories.MechanismFactory.Create(CKM.CKM_AES_KEY_GEN);
var key = session.GenerateKey(keyGen, keyTemplate);
Console.WriteLine("1. Generated AES-256 key inside the token (sensitive, non-extractable).");
// 2. Try to read the key value: the token must refuse.
var value = session.GetAttributeValue(key, [CKA.CKA_VALUE])[0];
Console.WriteLine($"2. Reading CKA_VALUE → {(value.CannotBeRead ? "refused by the HSM, as expected" : "READABLE (misconfigured!)")}");
// 3. Encrypt with AES-GCM inside the HSM, binding the customer ID as associated data.
var iv = RandomNumberGenerator.GetBytes(12);
var aad = Encoding.UTF8.GetBytes("customer_id=CUS-00012345");
var plaintext = Encoding.UTF8.GetBytes("GB82WEST12345698765432");
var gcmParams = factories.MechanismParamsFactory.CreateCkGcmParams(iv, 96, aad, 128);
using var gcm = factories.MechanismFactory.Create(CKM.CKM_AES_GCM, gcmParams);
var ciphertext = session.Encrypt(gcm, key, plaintext);
Console.WriteLine($"3. Encrypted in the HSM: {ciphertext.Length} bytes (ciphertext + 16-byte tag)");
// 4. Decrypt with the same IV and associated data.
var decryptParams = factories.MechanismParamsFactory.CreateCkGcmParams(iv, 96, aad, 128);
using var gcmDecrypt = factories.MechanismFactory.Create(CKM.CKM_AES_GCM, decryptParams);
var roundTrip = session.Decrypt(gcmDecrypt, key, ciphertext);
Console.WriteLine($"4. Decrypted: {Encoding.UTF8.GetString(roundTrip)}");
session.DestroyObject(key);
}
finally
{
session.Logout();
}Run it for free in Docker from the project folder (Git Bash):
MSYS_NO_PATHCONV=1 docker run --rm -v "$PWD:/src" -w /src mcr.microsoft.com/dotnet/sdk:10.0 sh -c "
apt-get update -qq && apt-get install -y -qq softhsm2 >/dev/null &&
ln -s /usr/lib/x86_64-linux-gnu/libdl.so.2 /usr/lib/x86_64-linux-gnu/libdl.so &&
softhsm2-util --init-token --free --label hl-m05-lab --pin 1234 --so-pin 5678 >/dev/null &&
PKCS11_PIN=1234 dotnet run; rm -rf bin obj"Output (verified on 2026-11-18):
Token: hl-m05-lab (SoftHSM project)
1. Generated AES-256 key inside the token (sensitive, non-extractable).
2. Reading CKA_VALUE → refused by the HSM, as expected
3. Encrypted in the HSM: 38 bytes (ciphertext + 16-byte tag)
4. Decrypted: GB82WEST12345698765432Code explained line by line
Pkcs11Interop5.3.0 is a mature open-source .NET wrapper over the PKCS#11 C API. AWS does not ship a .NET-specific CloudHSM SDK; .NET applications use PKCS#11 (any OS) or, on Windows, the CNG/KSP provider throughSystem.Security.Cryptography.CngKey⚠ VERIFY the KSP provider name in the current Client SDK 5 docs.PKCS11_LIB/PKCS11_PINfrom environment variables — HSM credentials are secrets. In production the CloudHSM crypto-user password comes from Secrets Manager at start-up, never from source control.LoadPkcs11Library(..., AppType.MultiThreaded)loads the vendor's shared library (libsofthsm2.soorlibcloudhsm_pkcs11.so). Your code depends only on the PKCS#11 standard, which is the portability win.GetSlotList(SlotsType.WithTokenPresent)— a slot is a reader; a token is the HSM partition. SoftHSM2 reassigns the slot ID after initialisation, so the code picks the first token rather than hard-coding slot 0.session.Login(CKU.CKU_USER, pin)— logs in as a crypto user. With CloudHSM the "PIN" isusername:password.- The key template:
CKA_SENSITIVE=trueandCKA_EXTRACTABLE=falseare the two attributes that make a key unreadable and unexportable.CKA_TOKEN=falsemakes a session key; set it totruefor a persistent key that the cluster replicates. GetAttributeValue(key, [CKA_VALUE])— the HSM returns "cannot be read" for sensitive keys. This is the proof that application code, administrators and malware on the host cannot obtain the key.CreateCkGcmParams(iv, 96, aad, 128)— AES-GCM with a 96-bit IV, associated data and a 128-bit tag, executed inside the HSM. Every byte of plaintext crosses the network to the HSM, which is why HSM-side bulk encryption is slow; real designs often use the HSM to wrap data keys (envelope encryption) instead.DestroyObjectandLogoutinfinally— release HSM resources; CloudHSM has per-HSM limits on sessions and objects.- The
libdl.sosymlink in the Docker command works around Pkcs11Interop looking forlibdl.so, which newer glibc images only ship aslibdl.so.2.
Security considerations
⚠ Learning shortcut — not for production
SoftHSM2 is a software token: keys live in files under /var/lib/softhsm/tokens and the "HSM" offers no physical protection. It is perfect for learning the PKCS#11 programming model and for unit tests, and useless as a security control. Production keys belong in a FIPS-validated HSM: KMS, a CloudHSM cluster, or AWS Payment Cryptography.
- Credential custody is the whole game. If the bank loses all crypto officer credentials, nobody — including AWS — can administer the HSMs. Use quorum authentication, split knowledge and sealed, audited storage of break-glass credentials.
- Network isolation. HSMs sit in private subnets; restrict the HSM security group to the client application security group only.
- Client host hardening. The client SDK and the crypto-user password on the application host are the attack surface. Use dedicated hosts or tasks, short-lived retrieval of credentials from Secrets Manager, and patch the SDK.
- Audit logs. CloudHSM sends HSM audit logs to CloudWatch Logs. Alert on user creation, password changes, key deletion and login failures; management-plane calls appear in CloudTrail.
- Custom key stores add a new failure mode: disconnecting the key store, losing HSMs below the minimum, or changing the
kmsuserpassword outside KMS stops every dependent AWS service from decrypting. Treat it as a tier-0 dependency. - Do not claim compliance from tenancy. A single-tenant HSM does not make an application PCI DSS or PCI PIN compliant; your QSA assesses the whole process.
Banking use case
Harbourline's architecture board reviewed four cryptographic requirements in 2026. The outcomes:
| Requirement | Decision | Reason |
|---|---|---|
| Encrypt customer PII in Aurora, S3 and field-level in the Accounts API | AWS KMS customer managed keys | FIPS-validated HSMs, full integration, USD 1 per key per month, CloudTrail audit |
| Group policy: "keys protecting the core ledger backups must be in HSMs whose admin credentials only Harbourline holds" | KMS CloudHSM key store with a three-HSM cluster in eu-west-2 | Keeps AWS Backup and S3 integrations while meeting the single-tenant rule; cost accepted for one key store |
| Debit card PIN translation and CVV verification for the card-issuing platform | AWS Payment Cryptography | Purpose-built payment HSM functions; avoids building PIN logic on general-purpose HSMs |
| Private CA signing key for internal mTLS | AWS Private CA (keys in AWS-managed HSMs) | No need to run a CA on CloudHSM; see Certificates, TLS and mTLS |
The board minutes also recorded the cost of the CloudHSM key store (about USD 3,500 per month for three HSMs plus 0.2 FTE) and the risk that a key-store disconnection makes ledger backups unreadable — mitigated by alarms on the KMS CustomKeyStore connection state and a tested runbook.
Common mistakes
- "We need an HSM" → CloudHSM by default. → Write the requirement precisely; KMS already uses validated HSMs and is usually enough.
- One HSM in production. → An AZ failure stops all cryptography until AWS replaces the HSM and restores from backup. Use at least two AZs; a CloudHSM key store needs two active HSMs to create keys.
- Choosing non-FIPS mode casually. → The mode is permanent, and some auditors require FIPS mode. Decide deliberately.
- Expecting KMS features on custom key stores. → No automatic rotation, asymmetric, HMAC or multi-Region keys in a CloudHSM key store.
- Using general-purpose CloudHSM for card PIN processing. → Use AWS Payment Cryptography or a payment HSM solution designed for PCI PIN requirements.
- Leaving lab HSMs running. → Billing is per HSM-hour until the HSM is deleted, even if idle.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
DllNotFoundException: libdl running Pkcs11Interop on Linux | Image only has libdl.so.2 | Symlink libdl.so.2 to libdl.so (see the Docker command) |
CKR_PIN_INCORRECT / login fails | Wrong PIN; for CloudHSM the PIN must be user:password | Check the format and the crypto user name |
No initialised token found with SoftHSM2 | Token not initialised, or a different SOFTHSM2_CONF | Run softhsm2-util --init-token --free ... |
KMS CustomKeyStoreInvalidStateException | Key store disconnected | aws kms connect-custom-key-store; check cluster health |
| KMS cannot create keys in the CloudHSM key store | Fewer than two active HSMs, or HSMs in one AZ | Add an HSM in another AZ |
KMSInternalException / CloudHsmClusterNotActiveException | Cluster degraded or kmsuser password changed outside KMS | Follow the KMS custom key store troubleshooting guide; reset the password through KMS |
| Throttling at about 1,800 req/s | Custom key store request quota (not adjustable) | Data-key caching, spread across key stores, or reconsider standard KMS |
Cost and cleanup
This lesson is design-only by default: cost USD 0. The SoftHSM2 lab is free; the CLI calls used are read-only.
If you decide to deploy a real cluster (not recommended for learning), the cost in us-east-1 at USD 1.60 per HSM-hour is:
| Scenario | Calculation | Cost |
|---|---|---|
| One HSM for one hour (smallest possible test) | 1 × 1 × 1.60 | ≈ USD 1.60 |
| Two HSMs for one hour (needed for a CloudHSM key store) | 2 × 1 × 1.60 | ≈ USD 3.20 |
| Two HSMs for a month | 2 × 730 × 1.60 | ≈ USD 2,336 |
| Three HSMs for a month | 3 × 730 × 1.60 | ≈ USD 3,504 |
Teardown if you created anything — delete every HSM first, then the cluster, then confirm:
aws cloudhsmv2 describe-clusters --query 'Clusters[].{Id:ClusterId,Hsms:Hsms[].HsmId}'
aws cloudhsmv2 delete-hsm --cluster-id <cluster-id> --hsm-id <hsm-id> # repeat for each HSM
aws cloudhsmv2 delete-cluster --cluster-id <cluster-id>
aws cloudhsmv2 describe-clusters --query 'Clusters[?State!=`DELETED`]' # expect []Locally, the Docker container is removed with --rm; nothing else remains.
Exercises
- 🟢 Guided: Run the SoftHSM2 demo. Then change
CKA_EXTRACTABLEtotrueandCKA_SENSITIVEtofalse, run again, and note what step 2 prints. Change them back and explain in two sentences why a bank's key-generation template must fix these attributes. - 🟡 Challenge: Extend the demo to generate an ECDSA P-256 key pair inside the token (
CKM_EC_KEY_PAIR_GENwith the P-256 curve OID), sign the SHA-256 digest of a payment instruction withCKM_ECDSA, export only the public key, and verify the signature in .NET withECDsa. Compare with the KMS approach in Labs 5.3 and 5.7. - 🔴 Live problem: Harbourline's Group Security policy says: "All production encryption keys shall reside in HSMs exclusively controlled by the bank." The cloud programme has 140 workloads using KMS. Write a one-page paper for the architecture board proposing how to interpret and meet the policy, with options, costs and risks.
Assignment
Write an architecture decision record (ADR) in your portfolio repository titled "Key management options for Harbourline's card-issuing platform". Cover requirements, the five options from this lesson, a decision, consequences (cost, operations, availability, exit), and the questions you would need your QSA and compliance team to answer. Include the decision flowchart.
Solutions
With CKA_SENSITIVE=false and CKA_EXTRACTABLE=true, step 2 prints that the value is readable: the key's raw bytes can be pulled out of the token, so the HSM no longer protects it against a compromised application host or an insider with the crypto-user password. A bank's key-generation templates (and HSM policies where available) must force SENSITIVE=true and EXTRACTABLE=false for operational keys, with extraction only through wrapping under another HSM key during audited key ceremonies.
Use CKM_EC_KEY_PAIR_GEN with a public template containing CKA_EC_PARAMS set to the DER encoding of the P-256 OID (06 08 2A 86 48 CE 3D 03 01 07) and CKA_VERIFY=true, and a private template with CKA_SIGN=true, CKA_SENSITIVE=true, CKA_EXTRACTABLE=false. Sign the 32-byte SHA-256 digest with CKM_ECDSA: PKCS#11 returns the raw r‖s (64 bytes), which is the IEEE P1363 format, so verify in .NET with ecdsa.VerifyHash(digest, signature, DSASignatureFormat.IeeeP1363FixedFieldConcatenation). Read CKA_EC_POINT from the public key to build the .NET key. KMS, by contrast, returns DER-encoded signatures and handles key custody, rotation policy and CloudTrail logging for you.
Interpretation: "exclusively controlled" usually targets who can administer and use keys, not physical tenancy. Option A: classify workloads and accept KMS customer managed keys for most, arguing that key policies, SCPs and CloudTrail give exclusive control and that KMS HSMs are FIPS-validated — needs Group Security sign-off. Option B: a CloudHSM key store for the small set of crown-jewel keys (ledger, payments), keeping integrations; about USD 2,300–3,500 per month per key store plus operations, with a disconnection risk. Option C: XKS for keys that must stay on premises — strongest control, highest availability risk. Option D: CloudHSM key stores for everything — highest cost and operational load for little gain. Recommend A + B, with criteria for which workloads qualify, a policy wording update, and a review with compliance and internal audit.
Knowledge check
Answer: AWS manages the HSM hardware, firmware, availability of individual HSMs (replacement) and backups. The customer manages the number and placement of HSMs, HSM users and credentials, keys and their attributes, the client SDK on application hosts, and application integration. AWS cannot recover lost crypto officer credentials.
Answer: Asymmetric keys, HMAC keys, imported key material, automatic key rotation and multi-Region keys. Keys are AES-256 symmetric encryption keys only.
Answer: AWS Payment Cryptography. It provides payment-HSM functions (PIN translation, card verification, EMV) as a managed service; general-purpose CloudHSM or KMS do not offer these payment-specific operations.
Answer: To keep key material outside AWS so the bank can cut off decryption at will ("hold your own key"). The risk is availability and latency: every KMS operation depends on the bank's external key manager and proxy, so their outage becomes an outage of every AWS service using those keys.
Answer: No. The mode is fixed at creation, and backups restore only into clusters of the same mode.
Interview questions
- "When would you recommend CloudHSM over KMS?" — When there is a specific requirement KMS cannot meet: single-tenant HSMs with bank-only admin credentials, direct PKCS#11/JCE/CNG use, algorithms or key types KMS lacks, or migrating an HSM-dependent application. Otherwise KMS, because it is cheaper, integrated and operated by AWS.
- "What is a KMS custom key store and why use one?" — A KMS key whose material lives in your CloudHSM cluster (or external key manager). You keep KMS's integrations and API while meeting single-tenant or hold-your-own-key requirements, at the cost of HSM spend, quotas and new failure modes.
- "How do .NET applications use CloudHSM?" — Through PKCS#11 with a wrapper such as Pkcs11Interop on any OS, or through the CNG/KSP provider on Windows. There is no CloudHSM-specific .NET SDK for data-plane operations; the AWS SDK covers only the management API.
- "What does an HSM not protect you from?" — Misuse by an authorised caller. If an attacker controls an application host with a valid crypto-user session, they can ask the HSM to decrypt. HSMs protect keys, not data flows; you still need least privilege, monitoring and segmentation.
Certification practice
A bank must encrypt S3 objects and EBS volumes with keys whose material is stored in single-tenant HSMs under the bank's exclusive administrative control. The bank wants to keep native S3 and EBS encryption integration. What should a security engineer recommend?
A. AWS managed keys with automatic rotation. B. Customer managed KMS keys in a KMS custom key store backed by an AWS CloudHSM cluster. C. AWS CloudHSM with client-side encryption in every application. D. SSE-S3 and EBS default encryption.
Answer: B. It keeps KMS integrations while placing key material in the bank's CloudHSM cluster. A and D use multi-tenant AWS-controlled keys. C meets the HSM requirement but loses native integration and adds heavy application changes.
A company creates a CloudHSM key store but CreateKey with origin AWS_CLOUDHSM fails. The cluster is initialised and has one active HSM in us-east-1a. What is the most likely fix?
A. Enable automatic rotation on the key store. B. Add a second active HSM in a different Availability Zone. C. Switch the cluster to non-FIPS mode. D. Increase the KMS custom key store request quota.
Answer: B. Creating KMS keys in a CloudHSM key store requires at least two active HSMs in different AZs. A is not supported. C is impossible after creation. D is unrelated and not adjustable.
A start-up neobank wants encryption at rest for RDS and S3 with keys it can audit and restrict by IAM, at the lowest cost. A consultant proposes a three-HSM CloudHSM cluster. What should the architect recommend instead?
A. KMS customer managed keys with key policies and CloudTrail logging. B. A single CloudHSM HSM to save cost. C. Store encryption keys in Secrets Manager and encrypt in the application. D. Disable encryption and rely on network isolation.
Answer: A. KMS customer managed keys meet the stated needs at about USD 1 per key per month. B removes availability and is still costly. C hand-builds key management without HSM protection. D fails the requirement.
Completion checklist
- I can explain what CloudHSM adds over KMS and what it costs
- I can choose between KMS, CloudHSM key store, XKS, CloudHSM direct and Payment Cryptography with reasons
- I ran the PKCS#11 demo against SoftHSM2 and saw the key refuse to be read
- I can describe the CloudHSM shared-responsibility split
- I created no paid resources, or I deleted every HSM and cluster
Key takeaways
- KMS already uses FIPS-validated HSMs; CloudHSM is for specific single-tenant, interface or algorithm requirements.
- CloudHSM is billed per HSM-hour (about USD 1.60 in AWS's us-east-1 example); production needs at least two AZs — thousands of dollars a month.
- A KMS CloudHSM key store keeps AWS integrations but supports only symmetric keys, no automatic rotation, and 1,800 requests/second per store.
- Payment card cryptography belongs in AWS Payment Cryptography, not general-purpose HSMs.
- PKCS#11 code is portable: prototype against SoftHSM2 for free, run against CloudHSM in production.
Further reading
Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.
AWS KMS: keys, key policies and envelope encryption
Create and govern customer managed KMS keys for a bank: key policies, grants, encryption context, envelope encryption from .NET, rotation, multi-Region keys, quotas, pricing and safe deletion.
Secrets Manager and Parameter Store
Store, rotate and read secrets the way a bank should: Secrets Manager rotation strategies and resource policies, Parameter Store tiers and SecureString, caching in .NET, the Lambda extension, and how to choose between them.