Skip to Content
Cloud Engineering Academy
17. Daily-Work Playbooks & Career

Playbook: KMS AccessDeniedException

Diagnose a KMS AccessDeniedException by checking the key policy, IAM, grants, encryption context, key state and cross-account rules in order, and tell service-principal and SSE-KMS failures apart from ordinary IAM gaps.

advanced⏱ 1 h 25 min🟢 FreeSCS-C03SAA-C03DVA-C03

Assumes: .NET 10 LTS · AWS SDK for .NET v4 (AWSSDK.KeyManagementService) · AWS CLI v2 · AWS KMS · Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this playbook you can:

  • Explain why a key policy is the root of trust for a KMS key, and how that differs from an ordinary IAM resource.
  • Work through the KMS authorisation chain in order: key state, key policy, IAM, grants, conditions (kms:ViaService, encryption context), guardrails.
  • Diagnose the cross-account case where both the key policy and the caller's IAM policy must allow.
  • Recognise service-principal failures (SNS to SQS, EventBridge, CloudTrail, S3) and write the key policy statement they need.
  • Tell AccessDeniedException from DisabledException, KMSInvalidStateException, InvalidCiphertextException, IncorrectKeyException and throttling.
  • Analyse a key policy programmatically and run a safe encrypt/decrypt self-test from .NET.

Prerequisites

Why this matters in banking

Harbourline's reconciliation service runs in the Operations account. Overnight it must read statement files from an S3 bucket in the Data Platform account. The bucket policy allows the role, the role's IAM policy allows s3:GetObject, and the first run still fails with:

User: arn:aws:sts::111122223333:assumed-role/hl-recon-role/... is not authorized to perform:
kms:Decrypt on resource: arn:aws:kms:us-east-1:444455556666:key/1234abcd-... 
because no resource-based policy allows the kms:Decrypt action

Two engineers start "fixing" it. One attaches kms:* on * to the role. The other asks for an SCP exemption. Neither helps, because the message already says the problem: the key's own policy in the other account does not name the role. If either "fix" had worked, it would have given the role far more power over encryption keys than the task ever needed, and encryption keys protect the most sensitive data in the bank. Reading the message, finding the key policy and asking the key owner for one precise statement takes ten minutes and leaves a clean audit trail.

The idea in plain language

A KMS key is a locked box where the box itself carries the list of who may use it. That list is the key policy. Unlike most AWS resources, a KMS key has no "default open to the account" behaviour: if the key policy does not allow access (directly, or by handing control to IAM), then nothing in IAM can open it.

There are three ways a request gets allowed:

  1. The key policy names the principal directly (works on its own, no IAM statement needed in the same account).
  2. The key policy contains the statement that delegates to IAM (principal arn:aws:iam::ACCOUNT:root), and an IAM policy on the caller allows the action. Both pieces must exist.
  3. A grant gives a principal specific operations on the key (used heavily by AWS services acting on your behalf, for example EBS and RDS).

Add conditions on top (the call must come through a given service, the encryption context must match), and guardrails (SCP, RCP, VPC endpoint policy) which can still say no.

Analogy: The vault's deposit boxes each have their own owner's list taped inside. Head office (IAM) can issue you a card that says "may open boxes", but a box opens only if its own list also says "head office's cards are accepted" or lists you by name. In a cross-branch case, you need the card from your branch and the other branch's box listing you.

The KMS errors and what each means

ErrorMeaningWhere to look
AccessDeniedExceptionAuthorisation failedKey policy, IAM, conditions, grants, SCP, endpoint policy
DisabledExceptionThe key is disableddescribe-key state. Enable the key (with approval)
KMSInvalidStateExceptionThe key is pending deletion, pending import, or unavailabledescribe-key. Cancel deletion if it was a mistake
NotFoundExceptionThe key or alias does not exist in this account and RegionCheck Region, alias, account, and that you used the full ARN for a cross-account key
InvalidCiphertextExceptionThe ciphertext is not valid for this key, or the encryption context does not matchCompare the encryption context used at encrypt time
IncorrectKeyExceptionThe key given does not match the one that encrypted the dataRemove the KeyId from Decrypt, or pass the right key
ThrottlingException / LimitExceededExceptionRequest rate or quotaCache data keys, use envelope encryption, back off. Quotas are shared per Region (⚠ VERIFY current values)
KeyUnavailableExceptionThe key is temporarily unavailable (for example an external or custom key store problem)Retry, check the key store

What problem the AWS service solves

KMS keeps key material inside hardware security modules, gives every use of a key an audit record in CloudTrail and lets the key owner control who can use the key independently of who owns the data. That separation is the point for a bank: the team that owns a data bucket does not have to be the team that controls its key.

Limits and sharp edges relevant here (verify against the KMS documentation for your Region):

  • AWS managed keys (aws/s3, aws/secretsmanager, aws/ebs...) have a key policy you cannot edit and cannot be used across accounts. Cross-account access requires a customer managed key.
  • A key policy has a size limit of 32 KB (⚠ VERIFY).
  • Grants are eventually consistent. A new grant can take a short time to become effective. Using a grant token with the call avoids waiting.
  • Deleting a key has a waiting period of 7 to 30 days, and data encrypted under a deleted key is unrecoverable. Disable first, delete later.
  • A customer managed key has a monthly charge and a per-request charge (⚠ VERIFY current price).
  • The key policy must name existing principals. Saving a policy that names a role that does not exist yet fails with an invalid principal error, which matters for IaC ordering.

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
Key Vault access policy or Azure RBAC for keysKMS key policy plus IAM and grantsIn AWS the key policy is mandatory and primary. IAM cannot override a key policy that does not allow it
Key Vault in another subscription or tenantCustomer managed key shared cross-accountNeeds the key policy to allow the external principal and the external IAM policy to allow it
"Key Vault Crypto User" rolekms:Encrypt, kms:Decrypt, kms:GenerateDataKey, kms:DescribeKeySeparate actions per operation, so least privilege is granular
Managed HSM and Key Vault access reviewsCloudHSM custom key stores and CloudTrailEvery KMS operation appears in CloudTrail as an event
Service-managed encryption with customer-managed keys (SSE with CMK)SSE-KMS on S3, EBS, RDS, SQS, SNSThe AWS service calls KMS as or on behalf of your principal, so it needs permission too
Azure RBAC deny assignmentsKey policy Deny statements, SCPsExplicit deny wins everywhere

Architecture

The KMS authorisation chain

The two common real-world shapes

Step-by-step implementation

Minutes 0 to 5: read the error and name the key

Copy the whole message. Extract: the operation (kms:Decrypt, kms:GenerateDataKey, kms:Encrypt, kms:DescribeKey), the key ARN (account and Region matter), and the caller (the role behind the session). Note which service triggered it: if you called S3, SQS, Secrets Manager or Lambda, the failing KMS call may be made by that service using your permissions, so the principal in the message is still yours.

Minutes 5 to 12: check the key exists and is usable

Run describe-key on the full ARN. Confirm: KeyState is Enabled, KeyManager is CUSTOMER (not an AWS managed key if cross-account), the Region matches, and the alias points where you think. If the state is Disabled or PendingDeletion, stop and escalate to the key owner. Re-enabling a key or cancelling deletion is a controlled change.

Minutes 12 to 25: read the key policy

Get the default key policy and answer these questions in order:

  1. Is there an Allow statement that names the caller's role ARN (or the account root of the caller's account, for cross-account)?
  2. For same-account use, is there the statement that enables IAM ("Principal": {"AWS": "arn:aws:iam::<this account>:root"})? If yes, IAM policies in this account can grant access, and the caller's role needs its own IAM allow on the key ARN. If no, only principals named in the key policy (or grants) get in.
  3. For cross-account use, does the key policy allow the external account root or the specific external role? Does the external role's IAM policy allow the action on the key's full ARN?
  4. Are there Condition blocks? Typical ones: kms:ViaService (only through a named service in a named Region), kms:CallerAccount, kms:EncryptionContext:..., aws:PrincipalOrgID.
  5. Is there a Deny statement that matches the caller?

Minutes 25 to 35: check IAM, grants and guardrails

  • Caller's IAM policy: the Resource must be the key's ARN, not * if the policy owner restricts it, and for cross-account use the full key ARN (with account ID), because the key is in another account.
  • Grants: list grants for the caller. Services like EBS, RDS and Secrets Manager create grants automatically. A missing or retired grant explains "worked before, fails now".
  • Guardrails: SCPs and RCPs that deny kms:* outside an allowed set; a VPC endpoint policy for the KMS interface endpoint that does not allow the caller.
  • Conditions: if the key policy restricts kms:ViaService to s3.us-east-1.amazonaws.com, a direct Decrypt call from your code (not through S3) fails by design.

Minutes 35 to 50: fix with one precise statement and test

The fix is almost always one statement, added by the key owner through their IaC and review. Then test with a real GenerateDataKey and Decrypt with the same encryption context the service will use. Use the self-test in Complete code.

AWS Console instructions

  1. Console → KMS → Customer managed keys → (the key). The General configuration panel shows ARN, state and Region. The Key policy tab shows the policy (switch to the policy view for JSON). Key users and Other AWS accounts list principals.
  2. Console → KMS → Customer managed keys → (key) → Key policy → Edit (key owner only) to add statements. For cross-account access, the policy view lets you add the external account or role.
  3. Console → KMS → (key) → Grants (in the Key policy area or Cryptographic configuration depending on console version) to see grants and their grantees.
  4. Console → IAM → Roles → (caller role) → Permissions to confirm an allow on the key's ARN.
  5. Console → CloudTrail → Event history with Event source kms.amazonaws.com to see the denied Decrypt or GenerateDataKey and its errorMessage and requestParameters.encryptionContext.

AWS CLI

# 1. Key state, manager (AWS or CUSTOMER), Region and rotation. Use the FULL ARN for cross-account keys.
aws kms describe-key --key-id arn:aws:kms:us-east-1:444455556666:key/1234abcd-12ab-34cd-56ef-1234567890ab \
  --query 'KeyMetadata.{State:KeyState,Manager:KeyManager,Enabled:Enabled,Origin:Origin,MultiRegion:MultiRegion,Arn:Arn}' \
  --profile learn-data-platform

# 2. The key policy (the only policy name is "default").
aws kms get-key-policy --key-id <key-arn> --policy-name default --query Policy --output text \
  --profile learn-data-platform

# 3. Grants for a specific grantee principal.
aws kms list-grants --key-id <key-arn> \
  --grantee-principal arn:aws:iam::111122223333:role/hl-recon-role --profile learn-data-platform

# 4. Is rotation enabled? (relevant for audits, not for access)
aws kms get-key-rotation-status --key-id <key-arn> --profile learn-data-platform

# 5. Safe self-test with the encryption context the service will use. Writes a 32-byte data key, no data is exposed.
aws kms generate-data-key --key-id <key-arn> --key-spec AES_256 \
  --encryption-context purpose=recon --query CiphertextBlob --output text --profile learn-sandbox \
  | base64 --decode > /tmp/dek.bin
aws kms decrypt --ciphertext-blob fileb:///tmp/dek.bin --encryption-context purpose=recon \
  --query KeyId --output text --profile learn-sandbox

# 6. Find the denied call and its encryption context in CloudTrail.
aws cloudtrail lookup-events \
  --lookup-attributes AttributeKey=EventSource,AttributeValue=kms.amazonaws.com \
  --start-time 2026-11-13T22:00:00Z --end-time 2026-11-13T23:00:00Z \
  --query 'Events[].CloudTrailEvent' --output text --profile learn-sandbox

# 7. Which key encrypted this S3 object?
aws s3api head-object --bucket hl-statements-prod --key 2026/11/stmt-0001.pdf \
  --query '{Encryption:ServerSideEncryption,KeyId:SSEKMSKeyId}' --profile learn-sandbox

The learn-data-platform profile stands for a profile in the key-owning account; in the course sandbox you can run both sides in one account for practice.

Logs Insights over a CloudTrail log group (if your trail sends events to CloudWatch Logs):

fields @timestamp, userIdentity.arn, eventName, errorCode, requestParameters.keyId
| filter eventSource = "kms.amazonaws.com" and ispresent(errorCode)
| stats count(*) as failures by userIdentity.arn, eventName, errorCode
| sort failures desc

Infrastructure as Code

The cross-account fix: the key owner adds one statement to the key policy, and the caller adds one IAM statement scoped to the key ARN.

using Amazon.CDK;
using Amazon.CDK.AWS.IAM;
using Amazon.CDK.AWS.KMS;
using Constructs;

// Account B (Data Platform): the key and the statement for the external role.
public class DataKeyStack : Stack
{
    public Key Key { get; }

    public DataKeyStack(Construct scope, string id, IStackProps props) : base(scope, id, props)
    {
        Key = new Key(this, "StatementsKey", new KeyProps
        {
            Alias = "alias/hl-statements",
            EnableKeyRotation = true
        });

        // The role must already exist when this policy is saved, or KMS rejects the principal.
        Key.AddToResourcePolicy(new PolicyStatement(new PolicyStatementProps
        {
            Sid = "AllowReconRoleToDecryptViaS3",
            Effect = Effect.ALLOW,
            Principals = new[] { new ArnPrincipal("arn:aws:iam::111122223333:role/hl-recon-role") },
            Actions = new[] { "kms:Decrypt", "kms:DescribeKey" },
            Resources = new[] { "*" },    // in a key policy, "*" means "this key"
            Conditions = new Dictionary<string, object>
            {
                ["StringEquals"] = new Dictionary<string, string> { ["kms:ViaService"] = "s3.us-east-1.amazonaws.com" }
            }
        }));

        Amazon.CDK.Tags.Of(this).Add("Project", "cea-lab");
        Amazon.CDK.Tags.Of(this).Add("Lab", "m17-playbook-kms");
    }
}

// Account A (Operations): the caller's IAM statement, scoped to the one key ARN.
public class ReconRoleKmsStatement
{
    public static PolicyStatement Create(string keyArn) => new(new PolicyStatementProps
    {
        Sid = "UseStatementsKey",
        Effect = Effect.ALLOW,
        Actions = new[] { "kms:Decrypt", "kms:DescribeKey" },
        Resources = new[] { keyArn }       // full ARN including account 444455556666
    });
}

Complete code

A key-policy analyser (pure function, easy to test)

KeyPolicyAnalyzer.cs
using System.Text.Json;

namespace Harbourline.Kms;

public sealed record KeyPolicyFindings(
    bool DelegatesToCallerAccountIam,
    bool NamesCallerDirectly,
    IReadOnlyList<string> DenyStatements,
    IReadOnlyList<string> Conditions,
    IReadOnlyList<string> Advice);

public static class KeyPolicyAnalyzer
{
    public static KeyPolicyFindings Analyze(string keyPolicyJson, string callerRoleArn, string callerAccountId)
    {
        using var doc = JsonDocument.Parse(keyPolicyJson);
        var statements = doc.RootElement.GetProperty("Statement");
        if (statements.ValueKind == JsonValueKind.Object) statements = WrapSingle(statements);

        bool delegates = false, named = false;
        var denies = new List<string>();
        var conditions = new List<string>();
        var rootArn = $"arn:aws:iam::{callerAccountId}:root";

        foreach (var st in statements.EnumerateArray())
        {
            var effect = st.GetProperty("Effect").GetString();
            var sid = st.TryGetProperty("Sid", out var s) ? s.GetString() : "(no Sid)";
            var principals = AwsPrincipals(st);

            var matchesCaller = principals.Contains(callerRoleArn) || principals.Contains("*");
            var matchesRoot = principals.Contains(rootArn);

            if (effect == "Allow")
            {
                if (matchesRoot && ActionsInclude(st, "kms:*")) delegates = true;
                if (principals.Contains(callerRoleArn)) named = true;
            }
            else if (effect == "Deny" && (matchesCaller || matchesRoot))
            {
                denies.Add(sid ?? "(no Sid)");
            }

            if (st.TryGetProperty("Condition", out var cond))
                foreach (var op in cond.EnumerateObject())
                    foreach (var key in op.Value.EnumerateObject())
                        conditions.Add($"{sid}: {op.Name} {key.Name}");
        }

        var advice = new List<string>();
        if (denies.Count > 0) advice.Add("A Deny statement matches the caller: it overrides every Allow. Review it first.");
        if (!named && !delegates)
            advice.Add("The key policy neither names the caller nor delegates to the caller's account: ask the key owner to add a statement. IAM alone cannot fix this.");
        if (!named && delegates)
            advice.Add("The key policy delegates to IAM: confirm the caller's IAM policy allows the KMS actions on this key's full ARN.");
        if (conditions.Count > 0)
            advice.Add("Conditions are present (for example kms:ViaService or encryption context): a direct call may not meet them.");
        if (advice.Count == 0) advice.Add("The key policy looks permissive for this caller. Check key state, grants and SCPs next.");

        return new KeyPolicyFindings(delegates, named, denies, conditions, advice);
    }

    private static JsonElement WrapSingle(JsonElement single)
    {
        using var d = JsonDocument.Parse($"[{single.GetRawText()}]");
        return d.RootElement.Clone();
    }

    private static HashSet<string> AwsPrincipals(JsonElement st)
    {
        var set = new HashSet<string>();
        if (!st.TryGetProperty("Principal", out var p)) return set;
        if (p.ValueKind == JsonValueKind.String) { set.Add(p.GetString()!); return set; }   // "*"
        if (p.TryGetProperty("AWS", out var aws))
        {
            if (aws.ValueKind == JsonValueKind.String) set.Add(aws.GetString()!);
            else foreach (var x in aws.EnumerateArray()) set.Add(x.GetString()!);
        }
        return set;
    }

    private static bool ActionsInclude(JsonElement st, string action)
    {
        if (!st.TryGetProperty("Action", out var a)) return false;
        return a.ValueKind == JsonValueKind.String
            ? a.GetString() == action
            : a.EnumerateArray().Any(x => x.GetString() == action);
    }
}

A live self-test against the real key

KmsSelfTest.cs
using Amazon.KeyManagementService;
using Amazon.KeyManagementService.Model;

namespace Harbourline.Kms;

public sealed class KmsSelfTest(IAmazonKeyManagementService kms)
{
    public async Task<string> RunAsync(string keyId, string purpose, CancellationToken ct)
    {
        try
        {
            var meta = (await kms.DescribeKeyAsync(new DescribeKeyRequest { KeyId = keyId }, ct)).KeyMetadata;
            if (meta.Enabled != true || meta.KeyState != KeyState.Enabled)
                return $"Key state is {meta.KeyState}: not usable. Escalate to the key owner.";
        }
        catch (AmazonKeyManagementServiceException ex) when (ex.ErrorCode == "AccessDeniedException")
        {
            // Not fatal: you may be allowed to use the key but not describe it.
            Console.WriteLine("DescribeKey denied (kms:DescribeKey missing). Continuing to the data-key test.");
        }

        var context = new Dictionary<string, string> { ["purpose"] = purpose };
        try
        {
            var dk = await kms.GenerateDataKeyAsync(new GenerateDataKeyRequest
            {
                KeyId = keyId, KeySpec = DataKeySpec.AES_256, EncryptionContext = context
            }, ct);

            await kms.DecryptAsync(new DecryptRequest
            {
                CiphertextBlob = dk.CiphertextBlob, EncryptionContext = context
            }, ct);
            return "OK: GenerateDataKey and Decrypt both succeeded with this encryption context.";
        }
        catch (DisabledException) { return "The key is DISABLED."; }
        catch (KMSInvalidStateException) { return "The key is pending deletion or otherwise in an invalid state."; }
        catch (NotFoundException) { return "Key not found: check the Region, alias and use the full ARN for cross-account keys."; }
        catch (InvalidCiphertextException) { return "Invalid ciphertext or encryption context mismatch."; }
        catch (AmazonKeyManagementServiceException ex) when (ex.ErrorCode == "AccessDeniedException")
        {
            return "AccessDeniedException. RequestId " + ex.RequestId + ". Read the message for the layer: " + ex.Message;
        }
    }
}

Tests for the analyser

KeyPolicyAnalyzerTests.cs
using Harbourline.Kms;
using Xunit;

public class KeyPolicyAnalyzerTests
{
    private const string Role = "arn:aws:iam::111122223333:role/hl-recon-role";

    [Fact]
    public void Policy_that_only_trusts_its_own_account_does_not_help_a_foreign_role()
    {
        const string policy = """
        { "Version": "2012-10-17", "Statement": [ {
            "Sid": "EnableIam", "Effect": "Allow",
            "Principal": { "AWS": "arn:aws:iam::444455556666:root" },
            "Action": "kms:*", "Resource": "*" } ] }
        """;

        var f = KeyPolicyAnalyzer.Analyze(policy, Role, "111122223333");

        Assert.False(f.NamesCallerDirectly);
        Assert.False(f.DelegatesToCallerAccountIam);
        Assert.Contains(f.Advice, a => a.Contains("ask the key owner"));
    }

    [Fact]
    public void Named_role_with_via_service_condition_is_reported()
    {
        const string policy = """
        { "Version": "2012-10-17", "Statement": [ {
            "Sid": "ReconDecrypt", "Effect": "Allow",
            "Principal": { "AWS": "arn:aws:iam::111122223333:role/hl-recon-role" },
            "Action": ["kms:Decrypt", "kms:DescribeKey"], "Resource": "*",
            "Condition": { "StringEquals": { "kms:ViaService": "s3.us-east-1.amazonaws.com" } } } ] }
        """;

        var f = KeyPolicyAnalyzer.Analyze(policy, Role, "111122223333");

        Assert.True(f.NamesCallerDirectly);
        Assert.Contains(f.Conditions, c => c.Contains("kms:ViaService"));
        Assert.Contains(f.Advice, a => a.Contains("Conditions are present"));
    }

    [Fact]
    public void Deny_statement_matching_the_caller_is_flagged_first()
    {
        const string policy = """
        { "Version": "2012-10-17", "Statement": [ {
            "Sid": "BlockEveryone", "Effect": "Deny", "Principal": "*",
            "Action": "kms:Decrypt", "Resource": "*" } ] }
        """;

        var f = KeyPolicyAnalyzer.Analyze(policy, Role, "111122223333");

        Assert.Contains("BlockEveryone", f.DenyStatements);
        Assert.StartsWith("A Deny statement", f.Advice[0]);
    }
}

Code explained line by line

  1. KeyPolicyAnalyzer.Analyze takes the policy JSON, the caller's role ARN and account ID. It never calls AWS, which makes it fast and safe to test and to run on a policy you were sent by the key owner.
  2. WrapSingle handles the legal case where Statement is a single object instead of an array.
  3. AwsPrincipals normalises the three ways a principal can appear: "*", a single ARN string, or an array of ARNs.
  4. delegates is true when an Allow names the caller's account root with kms:*: the "enable IAM policies" statement. Then IAM in the caller's account decides, so the caller's IAM policy matters.
  5. named is true when the key policy lists the caller's role directly, which works without any IAM statement for same-account callers (cross-account callers still need an IAM allow in their own account).
  6. A Deny that matches the caller (or *) is reported first, because it overrides every Allow.
  7. KmsSelfTest does GenerateDataKey then Decrypt with the same encryption context: the smallest realistic test of both permissions and the context rule, and it exposes no customer data.
  8. The catch order moves from specific KMS exceptions to the AccessDeniedException error code, so the message reflects which failure it was.
  9. Tests use fictional accounts (111122223333, 444455556666) and cover the three most common real situations.

Security considerations

⚠ Learning shortcut — not for production

The examples use the kms:Decrypt/kms:DescribeKey actions with Resource: "*" in the key policy (which means "this key" inside a key policy) and run self-tests with broad sandbox roles. In production, separate the key administrators (who manage the key, but cannot use it to decrypt data) from the key users (who can use it, but cannot change its policy). Test with the real service role in a non-production account, not an admin profile, because an admin profile hides exactly the gap you are testing for.

  • Never fix a KMS denial with kms:* on *. KMS permissions are the control over the bank's data confidentiality. Grant only the needed actions on the specific key ARN.
  • Do not re-enable a disabled key or cancel a deletion without the key owner's approval. A key may have been disabled deliberately (for example a suspected compromise) and the change belongs in the incident record.
  • Keep the encryption context meaningful and non-secret. It appears in CloudTrail in plain text and is bound to the ciphertext. Use it for authorisation conditions (for example kms:EncryptionContext:purpose), not for sensitive values.
  • Use kms:ViaService and aws:PrincipalOrgID conditions where they fit, so a stolen role session cannot call KMS directly for data that should only be decrypted through S3.
  • Cross-account grants widen the audit scope. Record who approved the external access and review it periodically.
  • Disable first, delete later. Because deletion is irreversible, use the waiting period and CloudTrail alarms on ScheduleKeyDeletion.
  • This playbook is engineering guidance; whether your key management satisfies PCI DSS or your regulator's expectations is for your security and compliance functions to decide.

Banking use case

Harbourline encrypts statements in S3 with a customer managed key in the Data Platform account. The reconciliation service (Operations account) reads them. The key policy has four statements: enable IAM in the key owner's account; key administrators (manage but not decrypt); the statements service role (encrypt and decrypt through S3); and the reconciliation role (decrypt through S3, kms:ViaService condition, only from the approved Region). When a new team asked to read the same bucket, the process was: pull request on the key's IaC adding one statement for one role, review by the key owners and security, deploy, and a CloudTrail query one week later confirming the role only ever made Decrypt calls through S3. A second key protects card-related fields with a kms:EncryptionContext:purpose condition so a role cannot use the statements key to decrypt card data even if it can call Decrypt.

Common mistakes

  • Fixing only the caller's IAM policy when the key policy does not allow the caller or the caller's account.
  • Using a key alias or key ID for a cross-account key. Use the full ARN (including the owner's account ID) in the caller's IAM policy and in API calls.
  • Trying to share an AWS managed key (aws/s3, aws/secretsmanager) cross-account. It is not possible; use a customer managed key.
  • Granting kms:Decrypt but forgetting kms:GenerateDataKey for writers. S3 and SQS producers need it to encrypt.
  • Forgetting the service principal. SNS to an encrypted SQS queue, EventBridge, CloudWatch alarms to an encrypted SNS topic, and CloudTrail log delivery all need statements for the service principal in the key policy.
  • A key policy that names a role before it exists. KMS rejects the policy as having an invalid principal. Deploy the role first.
  • Ignoring the encryption context. The context used to decrypt must match exactly the one used to encrypt, or you get InvalidCiphertextException.
  • Deleting a key to "clean up". Data encrypted under it is lost for good.

Troubleshooting

SymptomLikely causeFix
no resource-based policy allows the kms:Decrypt actionThe key policy does not name the caller or the caller's accountKey owner adds a statement. Add a matching allow in the caller's IAM policy for cross-account
no identity-based policy allows the kms:Decrypt actionKey policy delegates to IAM, but the caller's IAM policy lacks the action on this keyAdd kms:Decrypt scoped to the key ARN to the role
Works for the console user, fails for the service roleThe console user has broader rights, or the key policy names only humansTest with the real role. Add the role to the key policy or IAM
SQS queue with SSE-KMS stops receiving from SNSKey policy lacks sns.amazonaws.com for kms:GenerateDataKey* and kms:Decrypt, with a source ARN conditionAdd the service principal statement, scoped to the topic ARN
Lambda fails to decrypt an environment variableExecution role lacks kms:Decrypt, or the key policy does not include itGrant the role the action, check the key policy
InvalidCiphertextException after a code changeEncryption context changed between encrypt and decryptRestore the same context. Test with a round trip
DisabledExceptionKey was disabled (maybe deliberately)Escalate to the key owner. Do not re-enable without approval
NotFoundExceptionWrong Region, alias, or accountUse the full ARN. Check the Region of the call and the key
Intermittent ThrottlingException under loadPer-Region KMS request quota shared by many callersUse envelope encryption, cache data keys (S3 Bucket Keys help), back off
Works right after creation, fails a few minutes later with a grantGrant retired or never createdlist-grants, re-create through the owning service. Use a grant token for the first call

Cost and cleanup

Cost label: credits if you deploy a key. A customer managed key costs about one US dollar per month, prorated hourly, plus a small charge per 10,000 requests (⚠ VERIFY current pricing). Create it, run the exercise and schedule deletion the same day (minimum 7 days; the key is not billed while pending deletion, ⚠ VERIFY). The free alternative is to run the analyser and its tests, which need no AWS account.

# Disable first (reversible), then schedule deletion (minimum waiting period 7 days)
aws kms disable-key --key-id <key-id> --profile learn-sandbox
aws kms schedule-key-deletion --key-id <key-id> --pending-window-in-days 7 --profile learn-sandbox

# Delete the alias if you created one
aws kms delete-alias --alias-name alias/hl-statements --profile learn-sandbox

# Confirm nothing tagged for this lesson remains
aws resourcegroupstaggingapi get-resources --tag-filters Key=Lab,Values=m17-playbook-kms --profile learn-sandbox

Exercises

  • 🟢 Guided: Create a KeyPolicyAnalyzer project, paste the three tests and run dotnet test. Then change the first test's policy so it delegates to the caller's account root and confirm the advice changes to "confirm the caller's IAM policy allows...".
  • 🟡 Challenge: Extend the analyser to detect a kms:ViaService condition and return the service name. Add advice when the caller is not going through that service (for example calling Decrypt directly in code).
  • 🔴 Live problem: An encrypted SQS queue stopped receiving notifications from an SNS topic after the queue was moved to a customer managed key last night. Nothing was changed on the topic. SNS shows delivery failures to the queue. Describe how you confirm the cause and write the statement you would ask the key owner to add.

Assignment

Write a KMS access decision record for your portfolio: a table of the chain (state, key policy, IAM, grant, conditions, guardrails) with the CLI command for each, and a worked cross-account example with fictional IDs (both sides' statements). Commit it as playbooks/kms-access-denied.md with the analyser project and its tests.

Solutions

Knowledge check

Interview questions

  • Why is the KMS key policy different from other AWS resource policies? It is mandatory and is the root of trust. If it does not allow access (directly or by delegating to IAM), nothing in IAM can grant it.
  • How do you share an encrypted S3 bucket with another account? Use a customer managed key, allow the external principal in the key policy and in the bucket policy, and allow the KMS and S3 actions in the external role's IAM policy.
  • What is the encryption context and why use it? Non-secret key-value pairs bound to the ciphertext and logged in CloudTrail. It must match on decrypt and can be used in conditions to restrict what a principal can decrypt.
  • A service says access denied to KMS but your role has the permission. Why? The service may call KMS as its own principal (SNS, EventBridge, CloudTrail) and need a key policy statement, or a condition (ViaService, source ARN) is not met.
  • How would you reduce KMS throttling? Envelope encryption with cached data keys, S3 Bucket Keys, back off with jitter and request quota increases where justified.

Certification practice

Completion checklist

  • I can explain why the key policy outranks IAM for KMS
  • I can list the error types and say which are authorisation errors
  • I know both sides of a cross-account KMS grant
  • I ran the analyser tests and added one case
  • I can write the key policy statement a service principal needs
  • I ran the cleanup steps and confirmed nothing tagged m17-playbook-kms remains

Key takeaways

  • The key policy is the root of trust: allow directly or delegate to IAM; IAM alone is not enough.
  • Cross-account use needs both the key policy and the caller's IAM policy to allow, with the full key ARN.
  • Check state first, then key policy, IAM, grants, conditions and guardrails.
  • Services such as SNS, EventBridge and CloudTrail act as service principals and need their own statements.
  • Never use kms:* on * as a fix, and disable before you delete.

Further reading

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page