Skip to Content
Cloud Engineering Academy
15. Multi-Account Governance & FinOps

Compliance evidence: Config conformance packs, advanced queries and what still needs a human

Turn AWS Config into an evidence source an auditor can use: deploy a conformance pack, query resource inventory and configuration history, export a tamper-evident evidence bundle from C#, and be clear about what automation can never prove.

advanced⏱ 1 h 40 min🟡 Low costSCS-C03SAP-C03SOA-C03

Assumes: .NET 10 LTS · AWS SDK for .NET v4 (AWSSDK.ConfigService 4.x) · AWS CLI v2 · CDK v2 (Amazon.CDK.Lib 2.277.0) · Terraform AWS provider v6 · Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Explain the difference between a control, a check and evidence, and why a green dashboard is not an audit result.
  • Deploy a Config conformance pack (single account and organisation-wide) and read per-rule compliance.
  • Write AWS Config advanced queries for inventory questions and run them across an aggregator.
  • Retrieve configuration history for a single resource to answer "what did it look like on the date of the incident?"
  • Produce a timestamped, hashed evidence bundle with a C# tool, and store it somewhere it cannot be altered.
  • List the evidence that stays manual (people, process, legal) and say so plainly to an auditor.

Prerequisites

Why this matters in banking

A regulated bank is audited on a cycle: the internal audit team, the external auditor, the card schemes' assessor and sometimes the regulator. Each asks some version of the same question: "How do you know your controls are working, and show me." The painful way to answer is a two-week scramble that screenshots consoles and builds spreadsheets in the last days before the visit. The sustainable way is to have a small number of automated checks that run all the time, a history that can be queried, and an evidence bundle you can regenerate on request.

This is where AWS Config earns its fee. It records what a resource looked like at each point in time, evaluates it against rules, and keeps the history. That is real, system-generated evidence that a control was in place on a given date. But it is only one kind of evidence. It cannot show that the access review was done by a manager who understood it, that the disaster recovery test happened, or that staff completed training. A good engineer knows which is which and says so.

Never claim compliance from a tool

A conformance pack that is 100 percent COMPLIANT means the configured rules passed. It does not mean the organisation is PCI DSS compliant, UK GDPR compliant or compliant with any other framework. Compliance is a judgement made by qualified people (a QSA for PCI DSS, your data protection officer and legal team for UK GDPR) against the whole framework, most of which is not about AWS configuration. Wording in this lesson, in your reports and in your emails to auditors should say "supports evidence for control X", not "proves compliance".

The idea in plain language

Three words get mixed up, so fix them first.

  • Control: a rule your organisation sets, for example "storage holding customer data must be encrypted at rest".
  • Check: an automated test that looks at the world and says pass or fail, for example a Config rule that flags unencrypted EBS volumes.
  • Evidence: a record, tied to a point in time, that an auditor can examine and trust, for example "on 2026-11-01 at 10:00 UTC, all 412 volumes in these 9 accounts were encrypted, here is the list, and here is the hash of that list."

Automation is excellent at the second and third for configuration facts. It is poor or useless for facts about people and process. The aim of this lesson is a clean split: automate what can honestly be automated, and keep a short, explicit register of the manual evidence.

Analogy: Config is the branch's CCTV and door-log system. It shows who walked in and which doors were open at 14:05 last Tuesday, with a timestamp that is hard to fake. It does not show that the branch manager trained the staff or that the vault procedures are sensible. An auditor wants both the footage and the procedure manual.

What problem the AWS service solves

NeedAWS featureWhat you getLimit to remember
"Is this control in place right now?"Config managed rules (and custom rules)Per-resource COMPLIANT / NON_COMPLIANTOnly for recorded resource types; detection, not prevention
"Package 40 checks and deploy them everywhere"Conformance pack (account) or organisation conformance packOne YAML template, one compliance view, optional remediationPacks map to a framework's themes but do not make you compliant
"List all resources that match X"Advanced queriesSQL-like SELECT over current configurationCurrent state only; no JOIN, FROM, DISTINCT; not for resources the recorder does not record
"Across all accounts"Configuration aggregator (in the security tooling account)Same queries and compliance summaries across accounts and RegionsRead-only; needs the recorder in each source account
"What did it look like on 5 March?"Configuration history (get-resource-config-history) and configuration itemsA timeline of configuration items per resourceRetention of the history is configurable; deleted resources need a lookup of discovered resources
"Who changed it?"CloudTrailThe API call, caller, source IP, timeConfig tells you what changed, CloudTrail who did it; use both
"Packaged audit workflow"AWS Audit ManagerWas a managed evidence toolIn maintenance: cannot be set up in new accounts since 2026-04-30 per the module overview. Do not design new work on it.

On Audit Manager: AWS points existing teams towards Config conformance packs, Control Tower controls, Security Hub CSPM standards and partner tools for full evidence management. This lesson takes that guidance literally: you build a lightweight evidence collector yourself and keep the human evidence in a register.

Honest limits of Config evidence

  • Recorder coverage. If a resource type is not recorded, there is no evidence for it. Decide the recorded types deliberately; cost grows with change volume.
  • Rule coverage. There are hundreds of managed rules but not one for every control. A "PCI DSS v4.0 operational best practices" conformance pack (⚠ VERIFY the exact pack name and its current rule list) covers a subset of the AWS-configurable requirements. The module overview notes that there is no GDPR or SOC 2 conformance pack. UK GDPR evidence is mostly records, decisions and contracts.
  • Point-in-time, not continuous truth. Evaluations can lag changes by minutes, and a resource can be non-compliant for a short window. Document your tolerance.
  • A pass is a pass of that rule only. A bucket can pass "public access blocked" and still hold data that should never have been stored.

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
Azure Policy compliance viewConfig rules and conformance pack complianceConfig evaluates configuration items that it records; Azure Policy evaluates against resource provider APIs
Azure Policy initiatives (built-in regulatory compliance)Conformance packs (sample templates for PCI DSS, NIST and others)You deploy the template yourself; no automatic "compliance score" tied to a framework
Microsoft Purview Compliance ManagerAWS Audit Manager (now in maintenance), partner toolsNo equivalent managed assessment workflow for new accounts; build with Config + a register
Azure Resource Graph (KQL)Config advanced queries (SQL subset)Different language, current state only; aggregator queries span accounts
Azure Activity Log + Change AnalysisCloudTrail + Config configuration historyTwo services: who did it (CloudTrail), what changed (Config)
Defender for Cloud regulatory compliance dashboardSecurity Hub CSPM standardsSecurity Hub CSPM standards are control checks too; see the Security Hub lesson

Architecture

Security-boundary view: the collector runs under a role that can read Config and write only to the evidence bucket. Source accounts cannot delete or edit what the collector stored. The evidence bucket sits in the log archive account with Object Lock in compliance mode (see the S3 protection lesson), and the auditors' role can read but not write.

Step-by-step implementation

Decide what you are evidencing

Pick two or three controls only, for example: encryption at rest for volumes and databases, public access blocked for storage, and CloudTrail enabled with log file validation. For each, write who owns it and which Config rule produces the check. Starting small keeps the bundle readable.

Make sure the recorder covers the resource types

Confirm that the recorder records AWS::EC2::Volume, AWS::S3::Bucket, AWS::RDS::DBInstance and AWS::CloudTrail::Trail (or all supported types). Advanced queries do not return anything for types the recorder does not record.

Deploy the conformance pack

Deploy a small pack (the lab one below), or a sample framework pack for a short session in one Region. For organisation-wide deployment, use an organisation conformance pack from the management or delegated administrator account.

Check compliance and explain exceptions

Read per-rule compliance. For each non-compliant resource decide: fix, or record an approved exception with an owner and an expiry date. An exception register is evidence too.

Add queries that answer inventory questions

Auditors ask "show me all databases", "show me all unencrypted volumes", "show me security groups open to the internet". Save these as named advanced queries.

Automate the bundle

Run the collector on a schedule (EventBridge → Lambda or a pipeline job). Store the output in an immutable bucket, with a manifest of hashes, and keep every run.

Keep the manual register

Create a table of controls that cannot be evidenced from Config, with owner, frequency and where the evidence lives. Review it quarterly.

AWS Console instructions

  • Conformance packs: Console → AWS Config → Conformance packs → Deploy conformance pack → choose a sample template or upload your own, give it a name, and set the delivery bucket. The bucket name for conformance pack artefacts must begin with awsconfigconforms. ⚠ VERIFY
  • Compliance view: Config → Conformance packs → select a pack → see rules and compliance counts. Select a rule to list resources and their evaluation results.
  • Advanced queries: Config → Advanced queries → New query. Pick a Region or an aggregator, write a SELECT ... expression and run it. Save useful queries so they can be reused.
  • Configuration timeline: Config → Resources → search for a resource → Resource timeline shows each configuration item and the changes between them, including the CloudTrail events involved.
  • Aggregator: Config → Aggregators → Add aggregator (in the security tooling account), choose Add my organization. ⚠ VERIFY exact wording.

AWS CLI

# 1) Which conformance packs are deployed in this account and Region?
aws configservice describe-conformance-packs --profile learn-sandbox --region us-east-1

# 2) Deploy the lab pack from a local template. The delivery bucket must exist already
#    and its name must start with awsconfigconforms (⚠ VERIFY).
aws configservice put-conformance-pack \
  --conformance-pack-name hl-baseline \
  --template-body file://hl-baseline.yaml \
  --delivery-s3-bucket awsconfigconforms-hl-111122223333 \
  --profile learn-sandbox --region us-east-1

# 3) Overall compliance of the pack, then per-rule detail.
aws configservice get-conformance-pack-compliance-summary \
  --conformance-pack-names hl-baseline --profile learn-sandbox --region us-east-1
aws configservice describe-conformance-pack-compliance \
  --conformance-pack-name hl-baseline --profile learn-sandbox --region us-east-1

# 4) Resources that failed one rule inside the pack.
aws configservice get-conformance-pack-compliance-details \
  --conformance-pack-name hl-baseline \
  --filters 'ConfigRuleNames=["hl-ebs-encrypted"],ComplianceType=NON_COMPLIANT' \
  --profile learn-sandbox --region us-east-1

# 5) Advanced query on the current state: unencrypted EBS volumes.
aws configservice select-resource-config \
  --expression "SELECT resourceId, awsRegion, configuration.encrypted WHERE resourceType = 'AWS::EC2::Volume' AND configuration.encrypted = false" \
  --profile learn-sandbox --region us-east-1

# 6) Same idea across the organisation, run from the security tooling account.
aws configservice select-aggregate-resource-config \
  --configuration-aggregator-name hl-org-aggregator \
  --expression "SELECT accountId, awsRegion, COUNT(*) WHERE resourceType = 'AWS::RDS::DBInstance' GROUP BY accountId, awsRegion" \
  --profile learn-security --region us-east-1

# 7) Configuration history of one resource across a date range.
aws configservice get-resource-config-history \
  --resource-type AWS::S3::Bucket --resource-id hl-m15-evidence-demo \
  --later-time 2026-11-01T00:00:00Z --earlier-time 2026-11-18T00:00:00Z \
  --profile learn-sandbox --region us-east-1

# 8) Which resources of a type has Config discovered (including deleted ones if requested)?
aws configservice list-discovered-resources --resource-type AWS::S3::Bucket \
  --include-deleted-resources --profile learn-sandbox --region us-east-1

Reading the commands. Advanced queries accept a subset of SQL: SELECT, WHERE, GROUP BY, ORDER BY, LIMIT, aggregate functions such as COUNT, and LIKE with suffix wildcards of at least three characters. There is no FROM, JOIN, AS, DISTINCT or HAVING, and tags cannot be unpacked inside a query (query by a tag with a simple tags.tag = 'Key=value' match, ⚠ VERIFY the exact tag syntax for your resource). Property paths such as configuration.encrypted depend on the resource type; the open-source aws-config-resource-schema repository lists them. S3 settings like versioning sit under supplementaryConfiguration. ⚠ VERIFY each property path against one real configuration item before you publish a query to auditors.

For get-resource-config-history, --later-time is the more recent bound and --earlier-time the older one, which surprises most people; check the output ordering the first time.

The lab pack template:

hl-baseline.yaml
Parameters:
  RequiredTag1Key:
    Type: String
    Default: CostCenter
  RequiredTag2Key:
    Type: String
    Default: Owner
Resources:
  EbsEncrypted:
    Type: AWS::Config::ConfigRule
    Properties:
      ConfigRuleName: hl-ebs-encrypted
      Description: EBS volumes that are in use must be encrypted.
      Source:
        Owner: AWS
        SourceIdentifier: ENCRYPTED_VOLUMES
  S3PublicAccessBlocked:
    Type: AWS::Config::ConfigRule
    Properties:
      ConfigRuleName: hl-s3-public-access-blocked
      Description: S3 buckets must block public access at the bucket level.
      Source:
        Owner: AWS
        SourceIdentifier: S3_BUCKET_LEVEL_PUBLIC_ACCESS_PROHIBITED
  CloudTrailEnabled:
    Type: AWS::Config::ConfigRule
    Properties:
      ConfigRuleName: hl-cloudtrail-enabled
      Description: At least one CloudTrail trail must be enabled.
      Source:
        Owner: AWS
        SourceIdentifier: CLOUD_TRAIL_ENABLED
  RequiredTags:
    Type: AWS::Config::ConfigRule
    Properties:
      ConfigRuleName: hl-required-tags
      Description: Key resources must carry the mandatory cost and owner tags.
      InputParameters:
        tag1Key:
          Ref: RequiredTag1Key
        tag2Key:
          Ref: RequiredTag2Key
      Scope:
        ComplianceResourceTypes:
          - AWS::EC2::Instance
          - AWS::RDS::DBInstance
          - AWS::S3::Bucket
      Source:
        Owner: AWS
        SourceIdentifier: REQUIRED_TAGS

Infrastructure as Code

The stack deploys the same pack as an account-level conformance pack, reading the template from the file above. It assumes the Config recorder and delivery channel already exist (they usually come from the landing zone). L1 constructs; property names follow CloudFormation. ⚠ VERIFY

EvidencePackStack.cs
using System.IO;
using Amazon.CDK;
using Amazon.CDK.AWS.Config;
using Amazon.CDK.AWS.S3;
using Constructs;

namespace Harbourline.Governance.Infra;

public sealed class EvidencePackStack : Stack
{
    public EvidencePackStack(Construct scope, string id, IStackProps? props = null) : base(scope, id, props)
    {
        Amazon.CDK.Tags.Of(this).Add("Project", "cea-lab");
        Amazon.CDK.Tags.Of(this).Add("Lab", "m15-l8");
        Amazon.CDK.Tags.Of(this).Add("Environment", "learn");

        // Conformance pack artefacts bucket: name must begin with "awsconfigconforms" ⚠ VERIFY.
        var bucket = new Bucket(this, "PackBucket", new BucketProps
        {
            BucketName = $"awsconfigconforms-hl-{Aws.ACCOUNT_ID}",
            BlockPublicAccess = BlockPublicAccess.BLOCK_ALL,
            Encryption = BucketEncryption.S3_MANAGED,
            EnforceSSL = true,
            Versioned = true,
            RemovalPolicy = RemovalPolicy.DESTROY,   // lab only
            AutoDeleteObjects = true,                // lab only
        });

        new CfnConformancePack(this, "BaselinePack", new CfnConformancePackProps
        {
            ConformancePackName = "hl-baseline",
            TemplateBody = File.ReadAllText("hl-baseline.yaml"),
            DeliveryS3Bucket = bucket.BucketName,
        });
    }
}

For organisation-wide deployment, CfnOrganizationConformancePack takes the same template plus ExcludedAccounts, and must be deployed from the management account or a Config delegated administrator. ⚠ VERIFY.

Complete code

The evidence collector. It runs a list of named advanced queries, reads conformance pack compliance, writes each result as a JSON file into a timestamped folder, and finishes with a manifest.json that holds a SHA-256 hash of every file. Upload the folder to a bucket with Object Lock and the manifest becomes a tamper-evidence record: if one byte of a file changes, its hash will no longer match.

EvidenceCollector.csproj
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
    <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
  </PropertyGroup>
  <ItemGroup>
    <!-- Look up exact 4.x versions at https://api.nuget.org/v3-flatcontainer/<id>/index.json -->
    <PackageReference Include="AWSSDK.ConfigService" Version="4.*" />
  </ItemGroup>
</Project>
Program.cs
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using Amazon.ConfigService;
using Amazon.ConfigService.Model;

// Named queries an auditor can read. Each one answers one plain-English question.
var queries = new Dictionary<string, string>
{
    ["unencrypted-ebs-volumes"] =
        "SELECT resourceId, accountId, awsRegion, configuration.encrypted WHERE resourceType = 'AWS::EC2::Volume' AND configuration.encrypted = false",
    ["rds-instances-by-account"] =
        "SELECT accountId, awsRegion, COUNT(*) WHERE resourceType = 'AWS::RDS::DBInstance' GROUP BY accountId, awsRegion",
    ["s3-bucket-inventory"] =
        "SELECT resourceId, accountId, awsRegion, resourceCreationTime WHERE resourceType = 'AWS::S3::Bucket'",
};

string? aggregator = Environment.GetEnvironmentVariable("CONFIG_AGGREGATOR");   // null = single account
string packName = Environment.GetEnvironmentVariable("CONFORMANCE_PACK") ?? "hl-baseline";

var runId = DateTime.UtcNow.ToString("yyyyMMdd'T'HHmmss'Z'");
var folder = Path.Combine("evidence", runId);
Directory.CreateDirectory(folder);

using var config = new AmazonConfigServiceClient();
var jsonOptions = new JsonSerializerOptions { WriteIndented = true };

// ---- 1. Advanced queries ------------------------------------------------------------------
foreach (var (name, expression) in queries)
{
    var rows = new List<JsonElement>();
    string? token = null;
    do
    {
        if (aggregator is null)
        {
            var r = await config.SelectResourceConfigAsync(new SelectResourceConfigRequest
            {
                Expression = expression,
                Limit = 100,
                NextToken = token,
            });
            rows.AddRange((r.Results ?? []).Select(s => JsonDocument.Parse(s).RootElement.Clone()));
            token = string.IsNullOrEmpty(r.NextToken) ? null : r.NextToken;
        }
        else
        {
            var r = await config.SelectAggregateResourceConfigAsync(new SelectAggregateResourceConfigRequest
            {
                ConfigurationAggregatorName = aggregator,
                Expression = expression,
                Limit = 100,
                NextToken = token,
            });
            rows.AddRange((r.Results ?? []).Select(s => JsonDocument.Parse(s).RootElement.Clone()));
            token = string.IsNullOrEmpty(r.NextToken) ? null : r.NextToken;
        }
    }
    while (token is not null);

    var doc = new { question = name, expression, capturedAtUtc = DateTime.UtcNow, count = rows.Count, rows };
    await File.WriteAllTextAsync(Path.Combine(folder, $"query-{name}.json"), JsonSerializer.Serialize(doc, jsonOptions));
    Console.WriteLine($"query {name}: {rows.Count} rows");
}

// ---- 2. Conformance pack compliance per rule -----------------------------------------------
var rules = new List<object>();
string? packToken = null;
do
{
    var r = await config.DescribeConformancePackComplianceAsync(new DescribeConformancePackComplianceRequest
    {
        ConformancePackName = packName,
        Limit = 100,
        NextToken = packToken,
    });
    foreach (var rule in r.ConformancePackRuleComplianceList ?? [])
    {
        rules.Add(new { rule = rule.ConfigRuleName, compliance = rule.ComplianceType?.Value });
    }
    packToken = string.IsNullOrEmpty(r.NextToken) ? null : r.NextToken;
}
while (packToken is not null);

await File.WriteAllTextAsync(
    Path.Combine(folder, $"pack-{packName}.json"),
    JsonSerializer.Serialize(new { pack = packName, capturedAtUtc = DateTime.UtcNow, rules }, jsonOptions));
Console.WriteLine($"pack {packName}: {rules.Count} rules");

// ---- 3. Manifest with SHA-256 per file -------------------------------------------------------
var manifest = Directory.GetFiles(folder, "*.json")
    .OrderBy(f => f, StringComparer.Ordinal)
    .Select(f => new
    {
        file = Path.GetFileName(f),
        sha256 = Convert.ToHexString(SHA256.HashData(File.ReadAllBytes(f))).ToLowerInvariant(),
        bytes = new FileInfo(f).Length,
    })
    .ToList();

await File.WriteAllTextAsync(
    Path.Combine(folder, "manifest.json"),
    JsonSerializer.Serialize(new { runId, scope = aggregator ?? "single-account", files = manifest }, jsonOptions));

Console.WriteLine($"Evidence bundle written to {folder}");
Console.WriteLine("Upload the folder to the evidence bucket (Object Lock). Do not edit files after hashing.");

Code explained line by line

  1. The queries dictionary names each question in plain English. The name becomes the file name, so an auditor browsing the bundle can see what each file answers.
  2. CONFIG_AGGREGATOR switches the tool between a single account (SelectResourceConfig) and the organisation view (SelectAggregateResourceConfig). Run the second form from the security tooling account.
  3. The do ... while loop follows NextToken until it is empty. Results are returned as JSON strings, one per row, so each is parsed and cloned into a JsonElement that outlives its parent document.
  4. r.Results ?? [] guards the SDK v4 behaviour where an unset collection is null.
  5. Each query file records the expression and the capture time. That is what turns raw rows into evidence: it says what was asked, when, and what came back.
  6. DescribeConformancePackComplianceAsync returns one entry per rule with a ComplianceType. In the SDK this is a constant class, so .Value gives the string such as COMPLIANT or NON_COMPLIANT.
  7. The manifest hashes every file with SHA-256. Anyone can recompute the hash later. If the stored file differs, the evidence has been altered or damaged.
  8. The tool is read-only against AWS. It needs only config:SelectResourceConfig (or config:SelectAggregateResourceConfig) and config:DescribeConformancePackCompliance, plus write access to wherever you put the folder.

Security considerations

⚠ Learning shortcut — not for production

The lab runs the collector from your laptop with your SSO session and keeps the output on disk. In production the collector runs as a Lambda or pipeline job with a dedicated read-only role in the security tooling account, writes straight to an Object Lock bucket in the log archive account (compliance mode, retention matching your policy), uses a KMS key with a key policy that excludes the collector from decrypt, and publishes failures to the security team's channel. The hash manifest is also signed or logged somewhere independent, so tampering with both files and manifest is detectable.

  • Evidence is sensitive. A list of unencrypted databases is a map for an attacker. Encrypt the bucket, limit read access to auditors and security, and log every read.
  • Separation of duties. The people who fix findings should not be able to edit the evidence that records them. Keep the evidence bucket in the log archive account.
  • Retention and privacy. Configuration items can contain identifiers, tags and policy documents. Set a retention period tied to your legal requirement, not "forever". Under UK GDPR, keep personal data out of evidence unless needed.
  • Do not weaken controls for convenience. Turning off a failing rule so the dashboard is green is itself something an auditor will find in CloudTrail.

Banking use case

Harbourline's external assessor asks for evidence that cardholder-data environment storage is encrypted and non-public, across the year. The security team has run the collector on the first of each month for twelve months. The assessor receives twelve bundles: each lists the unencrypted volumes (two in March, both remediated within 3 days, with the ticket numbers in the exception register), the bucket inventory, and the pack compliance, together with the hash manifests. The assessor spot-checks a hash and compares it with the Object Lock retention record. The AWS-configuration part of the conversation takes an afternoon. The same assessor then asks for the access review sign-offs, the incident response test and the training records. Those come from the manual register, because no AWS service can generate them. The bank does not describe the result as "automated compliance". It describes it as "automated evidence for configuration controls, plus a managed register for the rest".

Common mistakes

  • Equating a green pack with compliance. The rules are a subset of one framework's technical controls.
  • Forgetting the recorder. Rules and queries silently cover nothing for resource types that are not recorded.
  • Running advanced queries expecting history. They return current state. Use get-resource-config-history for the past.
  • Trusting an unverified property path. A query that returns zero rows might mean "no non-compliant resources" or "the path is wrong". Test against a known bad resource.
  • Collecting evidence the day before the audit. Evidence has most value when it was generated regularly, before anyone knew it would be inspected.
  • No exception process. A permanent red rule with no owner is worse than an approved, dated exception.
  • Storing evidence in the account it describes. A compromised account can delete its own evidence.

Troubleshooting

SymptomLikely causeFix
NoAvailableConfigurationRecorderConfig recorder is not set up in this RegionEnable the recorder (see the Config lesson) before deploying packs
Conformance pack stuck in CREATE_IN_PROGRESS or failsDelivery bucket missing, wrong name prefix, or bucket policy denies ConfigCreate the bucket first, check the awsconfigconforms prefix and the bucket policy ⚠ VERIFY
Advanced query returns zero rowsResource type not recorded, wrong property path, or case mismatchRun SELECT * for the type; check list-discovered-resources; test against a known resource
InvalidExpressionExceptionUsed an unsupported keyword such as FROM, AS, DISTINCT or JOIN, or a wildcard under three charactersRewrite using supported syntax only
Aggregate query returns only some accountsAggregator authorisation missing for some accounts, or recorders offCheck aggregator source status and recorder state in each account
AccessDeniedException on SelectAggregateResourceConfigRole lacks aggregator permissions, or the call was made from the wrong accountRun from the aggregator account with config:SelectAggregateResourceConfig
Compliance shows INSUFFICIENT_DATARule has not evaluated yet or resource not in scopeWait for the evaluation, or trigger a re-evaluation
History query returns nothingTime bounds reversed or recording started after the periodSwap --earlier-time and --later-time; check when the recorder started

Cost and cleanup

Cost: credits. Config charges per configuration item recorded and per rule evaluation (and conformance pack rule evaluations are billed per evaluation). Advanced queries have no extra charge. S3 storage for a few small JSON files costs cents. A small pack of four rules in a lightly used sandbox account for an hour costs cents to low single dollars, but cost scales with the number of recorded resources and the number of changes. ⚠ VERIFY current Config pricing before you run this in a busy account. Free alternative: do the exercises with the sample query output in the solutions section, or read-only against a sandbox that already has Config running.

Teardown, in this order:

# Delete the conformance pack (this also deletes its rules).
aws configservice delete-conformance-pack --conformance-pack-name hl-baseline \
  --profile learn-sandbox --region us-east-1

# Empty and delete the pack artefacts bucket.
aws s3 rb s3://awsconfigconforms-hl-111122223333 --force --profile learn-sandbox

# If you enabled Config only for this lesson, stop the recorder and remove the channel.
aws configservice stop-configuration-recorder --configuration-recorder-name default \
  --profile learn-sandbox --region us-east-1

# Delete local evidence output.
rm -rf ./evidence

# Confirm nothing tagged for the lab remains billing.
aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=cea-lab \
  --region us-east-1 --profile learn-sandbox --query 'ResourceTagMappingList[].ResourceARN'

Note: stopping the recorder does not delete past configuration items or the delivery bucket. If the landing zone's own recorder is in use, do not stop it. Only stop a recorder you created.

Exercises

  • 🟢 Guided: deploy hl-baseline in the sandbox, create an unencrypted EBS volume tagged for the lab, find it with the advanced query, fix it by creating an encrypted replacement, delete the old one, and re-run the query.
  • 🟡 Challenge: extend the collector to include get-resource-config-history for one named bucket, so the bundle shows how the bucket's public access settings changed over 14 days.
  • 🔴 Live problem: the assessor asks you to "prove we are PCI DSS compliant for AWS" by Friday. Write the reply that sets out what the bundle shows, what it does not, what you will provide from the manual register, and who must sign off.

Assignment

Create governance/evidence/ in your portfolio repository with the pack template, the collector, a register.md listing at least ten manual evidence items (owner, frequency, location), and a one-page how-to-read-the-bundle.md for an auditor who does not know AWS.

Solutions

Knowledge check

Interview questions

  • How do you provide audit evidence for AWS controls without a two-week scramble? Continuous Config rules and a conformance pack, saved advanced queries, a scheduled collector writing hashed bundles to an immutable bucket, and a manual register for the rest.
  • What are the limits of automated compliance checks? They cover only recorded resource types and implemented rules, they are point-in-time and they cannot evidence people or process. A pass means "this check passed".
  • A rule has been non-compliant for six months. What do you do? Find the owner, check whether it is a false positive, fix it or record an approved exception with an expiry date and compensating control, and make sure the exception is visible to audit.
  • Audit Manager is in maintenance. What would you use? Config conformance packs and aggregators, Control Tower and Security Hub CSPM controls, and a purpose-built collector plus a register, or a partner product for full workflow. ⚠ VERIFY current AWS guidance.

Certification practice

Completion checklist

  • I can state the difference between control, check and evidence
  • I deployed a small conformance pack and read its compliance
  • I wrote three advanced queries and tested them against known resources
  • I ran the collector and checked a hash in the manifest
  • I wrote a manual evidence register of at least ten items
  • I ran the cleanup steps and confirmed nothing is still billing

Key takeaways

  • Evidence is a timestamped, trustworthy record. A dashboard colour is not.
  • Config gives configuration evidence: recorded history, rule results and queryable inventory.
  • Conformance packs package checks, they do not certify compliance. There is no GDPR or SOC 2 pack.
  • Keep evidence immutable, separate from the accounts it describes, and hashed.
  • Be explicit about manual evidence. Auditors trust engineers who say what automation cannot show.

Further reading

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page