Skip to Content
Cloud Engineering Academy
5. Security for Banking & Finance

Audit logging and evidence collection

Understand what auditors actually ask a bank for, and build evidence on AWS that answers them: AWS Artifact reports, CloudTrail and AWS Config as configuration and activity evidence, application audit events, and a repeatable .NET evidence collector that packages, hashes and locks what it gathers.

intermediate⏱ 2 h 30 min🟢 FreeSCS-C03SAA-C03CLF-C02

Assumes: .NET 10 LTS · AWS SDK for .NET v4 (AWSSDK.IdentityManagement, AWSSDK.ConfigService, AWSSDK.CloudTrail, AWSSDK.S3 4.x) · AWS CLI v2 · Amazon Athena engine v3 · Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Explain the three kinds of audit evidence — design, implementation and operating effectiveness — and how auditors use populations and samples.
  • List the questions auditors typically ask a bank about its cloud estate (access, change, operations, logging, data protection, third parties) and which AWS source answers each.
  • Use AWS Artifact for AWS's own reports and agreements, and explain why they're only half the story.
  • Use CloudTrail, AWS Config, IAM credential reports, Security Hub CSPM and application audit_events as evidence, and query CloudTrail logs with Athena.
  • Build a .NET evidence collector that gathers snapshots, writes a manifest with SHA-256 hashes, and stores the package in an Object Lock bucket.
  • Explain the current service landscape: AWS Audit Manager is in maintenance (no new onboarding from 2026-04-30), CloudTrail Lake closed to new customers on 2026-05-31, and there is no GDPR conformance pack.

Prerequisites

Why this matters in banking

Every year Harbourline's cloud estate is examined by several kinds of auditor: internal audit (third line of defence), external auditors reviewing IT general controls for the financial statements, the QSA for PCI DSS, ISO 27001 certification auditors, and occasionally the PRA or FCA through skilled-person reviews or thematic work. Each asks variations of the same questions: who has access, how do changes get approved, how would you know if something went wrong, and can you prove it — for the whole year, not just today.

Teams that treat audit as a once-a-year scramble spend weeks taking screenshots, chasing approvers in Slack history and arguing about whether a log is complete. Teams that design for evidence spend days. And evidence that can't be trusted (a spreadsheet anyone can edit, a screenshot with no date) gets rejected, which turns into a finding even when the control actually worked.

The idea in plain language

An audit checks that a control (a safeguard, such as "production access requires approval") is designed properly and works in practice. Evidence is anything that lets the auditor reach that conclusion without taking your word for it.

Evidence typeQuestion it answersExample on AWS
DesignIs the control well designed?Policy document; SCP JSON; architecture diagram; runbook
Implementation (point in time)Does it exist as designed today?Config snapshot showing all buckets have Block Public Access; IAM credential report
Operating effectiveness (over a period)Did it work all year?Config compliance timeline for 12 months; CloudTrail events; a sample of 25 change tickets each linked to a pipeline run and approval

Auditors test operating effectiveness with populations and samples: "give me the full list of production deployments between 1 January and 31 December" (the population), then "for these 25 that I picked, show approval before deployment" (the sample). They will also test completeness: how do they know your population isn't missing the deployments that went wrong? That's why evidence that comes straight from a system of record (CloudTrail, Config, the CI/CD tool) beats evidence assembled by hand.

Good evidence is:

  • Complete — the whole population, with a way to show nothing is missing.
  • Accurate and unaltered — produced by a system, hashed, stored where it can't be edited.
  • Timely and dated — clearly shows the period it covers.
  • Attributable — who or what produced it, and how.
  • Relevant — answers the control being tested, not a neighbouring one.

Analogy: evidence collection is like keeping receipts for a tax return. Shoving receipts in a drawer and reconstructing the year in April is painful and you'll lose some. Scanning each receipt as it arrives, filing it by category with the date, and keeping the originals in a fireproof box makes April easy — and when the inspector asks for "all fuel receipts for March", you can produce them in minutes and prove none are missing.

What problem the AWS services solve

SourceWhat it provesNotes (checked 2026-11-18)
AWS Artifact — ReportsAWS's own controls: SOC 1/2/3, ISO 27001/27017/27018/27701, PCI DSS AOC and Responsibility Summary, C5 and moreAWS's controls only; auditors still need yours. Reports are confidential under Artifact terms
AWS Artifact — AgreementsWhich AWS agreements (for example BAA for HIPAA in the US) are accepted for an account or organisationFew apply to UK banks; your legal team reviews AWS's contract terms separately
AWS CloudTrailWho did what, when, from where, on AWS APIs (management events; data events if enabled)Organisation trail into the log-archive account with log file validation is the trusted record. Event history in the console covers 90 days of management events only
AWS ConfigResource configuration over time, relationships, and rule compliance historyConfiguration timeline and compliance history are strong operating-effectiveness evidence; retention configurable (default 7 years ⚠ VERIFY)
Config conformance packsGrouped rules mapped to frameworks (PCI DSS 4.0, NIST, CIS, etc.)AWS states sample packs don't ensure compliance; there is no GDPR pack
Security Hub CSPM standardsContinuous control checks with history (AWS FSBP, CIS, PCI DSS v4.0.1, NIST)Configuration checks only
IAM credential report, Access Analyzer, Identity Center assignmentsAccess evidence: who has which access, unused access, external accessCredential report is a point-in-time CSV
AWS Backup Audit Manager (a feature of AWS Backup)Evidence that backup policies are applied and backups happenDifferent from AWS Audit Manager; still available ⚠ VERIFY
Application audit_eventsBusiness actions: payment approvals, limit changes, customer data accessYour code; hash-chained and stored immutably (previous lesson)
CI/CD system (GitHub, CodePipeline)Change evidence: PR approvals, pipeline runs, deployment recordsThe link between ticket, approval, commit and deployment is the change-management evidence

What's gone or going:

  • AWS Audit Manager — in maintenance; from 2026-04-30 it can't be set up in new accounts. Existing users continue for now; AWS points to Config conformance packs, Control Tower controls and Security Hub CSPM standards. Don't design new evidence processes around it.
  • CloudTrail Lake — closed to new customers on 2026-05-31. Query CloudTrail logs in S3 with Athena, or send them to CloudWatch Logs or your SIEM.

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
Microsoft Service Trust PortalAWS ArtifactArtifact is inside the AWS console with IAM-controlled access
Azure Activity Log (90 days) + diagnostic settings to storageCloudTrail event history (90 days) + organisation trail to S3Same idea: the 90-day view is not your evidence store
Azure Resource Graph + change analysisAWS Config advanced queries + configuration timelineConfig must be enabled and recording; history begins when recording starts
Azure Policy compliance history, regulatory compliance dashboardConfig rules/conformance packs + Security Hub CSPM standardsTwo services; evidence exports are your job
Microsoft Purview Compliance Manager(AWS Audit Manager — now in maintenance)Build your own collector + GRC tool integration
Log Analytics / KQL over logsAthena SQL over CloudTrail in S3, or CloudWatch Logs InsightsAthena charges per data scanned; partition your tables

Architecture

Security-boundary view

ConcernControl
Evidence can't be altered by the people being auditedEvidence and logs live in separate accounts; workload teams have no write or delete access; Object Lock
Collector can only readRead-only role in each account (ReadOnlyAccess-style, plus config:Select*, iam:GenerateCredentialReport) and write-only to the evidence bucket
Auditor access is time-boundRead-only access to the evidence bucket via a temporary role or a pre-signed export, granted per engagement
Evidence itself may contain personal dataMinimise (IDs, not names); restrict and expire access

Step-by-step implementation

Build a control-to-evidence map

List each control in scope, the question an auditor will ask, the source of truth, the evidence type, how often it's collected, and the owner. This is the most valuable artefact in the lesson; the table in the next section is a starting point.

Make sure the sources are on and trustworthy

An organisation CloudTrail trail with log file validation, delivered to the log-archive account with Object Lock; AWS Config recording in every account and Region in use, with an aggregator; Security Hub CSPM standards enabled where required; application audit events hash-chained. Evidence about evidence matters: auditors will ask how you know logging was on all year (Config rules such as cloudtrail-enabled, and CloudTrail digest files, answer that).

Automate point-in-time snapshots

Schedule a collector (EventBridge Scheduler → Lambda or an ECS task) to capture snapshots monthly or quarterly: IAM credential report, Config compliance by rule, trail status, Identity Center assignments, Security Hub CSPM control status. Write each snapshot with a manifest listing file names, SHA-256 hashes, collection time and collector identity.

Answer period questions from logs, not snapshots

For "all changes to security groups in Q3" or "every use of the break-glass role this year", query CloudTrail logs in S3 with Athena (partitioned by account, Region and date). Save the query text and the result together, so the auditor can see exactly how the population was produced.

For each production deployment the pipeline records: ticket ID, pull-request approvers, commit SHA, pipeline run ID, deployer identity and time. CloudTrail shows the deployment role's calls. The link from ticket to CloudTrail is what proves "every change was approved".

Get AWS's reports and package for the auditor

Download the relevant Artifact reports (SOC 1/2, ISO, PCI AOC) for the period, file them in the GRC tool with their dates, and add your own evidence links. Give auditors read-only, time-bound access, and keep a log of what was shared.

What auditors actually ask for

Control areaTypical requestAWS / system sourceEvidence type
User access — joiners, movers, leaversPopulation of access grants and removals; for a sample, approval before grant and removal within X days of leavingIdentity Center assignment events in CloudTrail; HR leaver list; IdP logsOperating effectiveness
Privileged accessWho has admin/break-glass; every use with justificationIdentity Center permission sets; CloudTrail AssumeRole/Federate events for privileged roles; alert ticketsImplementation + operating
Periodic access reviewsEvidence that managers reviewed access quarterly, with actions takenExported assignments + signed-off review in the GRC toolOperating
Root userMFA on root, no access keys, no useIAM credential report; Config rules; CloudTrail root eventsImplementation + operating
Change managementPopulation of production changes; sample showing approval, testing, segregation of dutiesGitHub/CodePipeline records + CloudTrail deployment eventsOperating
Logging and monitoringLogging on all year; alerts reviewed; retentionConfig compliance history for cloudtrail-enabled; CloudTrail digest validation; alert ticketsOperating
Encryption and key managementData stores encrypted; key rotation; key access restrictedConfig rules (s3-default-encryption-kms, etc.); KMS key policies; rotation statusImplementation
Vulnerability managementFindings and remediation within SLAInspector findings history; Security Hub CSPMOperating
Backup and recoveryBackups run; restores testedAWS Backup jobs and Backup Audit Manager reports; restore test recordsOperating
Incident managementIncidents logged, classified, resolved, lessons learnedIncident tool; GuardDuty findings; runbooksOperating
Third partiesDue diligence on cloud providerAWS Artifact SOC 2 / ISO reports; contract; exit plan (next lesson)Design

AWS Console instructions

  1. AWS Artifact: Console → AWS Artifact → Reports. Search SOC 2. Select the latest SOC 2 Type 2 report, accept the terms, and download. Note the period covered — auditors care that it overlaps your audit period. Repeat for ISO 27001 and, if relevant, PCI DSS.
  2. CloudTrail validation status: Console → CloudTrail → Trails → (trail) — confirm Log file validation: Enabled.
  3. Config compliance timeline: Console → AWS Config → Resources → (a bucket) → Resource timeline — shows configuration changes and rule compliance changes over time. This is the view auditors find most convincing.
  4. Config advanced query: Console → AWS Config → Advanced queries → New query — run the query from the CLI section below.
  5. IAM credential report: Console → IAM → Credential report → Download credentials report.
  6. Security Hub CSPM: Console → Security Hub CSPM → Security standards → (standard) → Controls — note the pass/fail status and the history of each control.

AWS CLI

# --- AWS's own reports ---
aws artifact list-reports --query "reports[].{name:name,period:periodStart}" --output table --profile learn-sandbox  # ⚠ VERIFY output fields

# --- Point-in-time access evidence ---
aws iam generate-credential-report --profile learn-sandbox
aws iam get-credential-report --query Content --output text --profile learn-sandbox | base64 --decode > credential-report.csv

# --- Logging evidence: trails, status, validation ---
aws cloudtrail describe-trails --query "trailList[].{name:Name,multiRegion:IsMultiRegionTrail,org:IsOrganizationTrail,validation:LogFileValidationEnabled}" \
  --output table --profile learn-sandbox
aws cloudtrail get-trail-status --name <trail-name> --query "{logging:IsLogging,lastDelivery:LatestDeliveryTime}" --profile learn-sandbox

# --- Prove log files were not modified over a period ---
aws cloudtrail validate-logs --trail-arn <trail-arn> --start-time 2026-10-01T00:00:00Z --end-time 2026-10-31T23:59:59Z --profile learn-sandbox

# --- Configuration evidence: Config advanced query for unencrypted EBS volumes ---
aws configservice select-resource-config --expression \
  "SELECT resourceId, awsRegion, configuration.encrypted WHERE resourceType = 'AWS::EC2::Volume' AND configuration.encrypted = false" \
  --profile learn-sandbox

# --- Rule compliance summary ---
aws configservice describe-compliance-by-config-rule \
  --query "ComplianceByConfigRules[].{rule:ConfigRuleName,status:Compliance.ComplianceType}" --output table --profile learn-sandbox

# --- Hash every evidence file you produce, for the manifest ---
sha256sum credential-report.csv

Athena query over CloudTrail logs in S3 (use the table that the CloudTrail console's Create Athena table option creates, or a partition-projected table; replace the table name):

-- Population: every security group change in Q3 2026, with who made it.
SELECT eventtime,
       useridentity.arn          AS actor,
       eventname,
       recipientaccountid        AS account,
       awsregion,
       json_extract_scalar(requestparameters, '$.groupId') AS group_id,
       sourceipaddress
FROM   cloudtrail_logs_org
WHERE  eventsource = 'ec2.amazonaws.com'
  AND  eventname IN ('AuthorizeSecurityGroupIngress', 'AuthorizeSecurityGroupEgress',
                     'RevokeSecurityGroupIngress', 'RevokeSecurityGroupEgress',
                     'ModifySecurityGroupRules', 'CreateSecurityGroup', 'DeleteSecurityGroup')
  AND  eventtime BETWEEN '2026-07-01T00:00:00Z' AND '2026-09-30T23:59:59Z'
ORDER  BY eventtime;

Athena costs USD 5 per TB scanned in us-east-1 ⚠ VERIFY current price; a partitioned table for one account and a quarter scans megabytes, costing fractions of a cent.

Infrastructure as Code

The evidence bucket: versioned, KMS-encrypted, Object Lock in governance mode with a short default retention for the lab (production uses compliance mode and a retention set by your records policy), and a policy that lets the collector write but nobody delete.

EvidenceBucketStack.cs
using Amazon.CDK;
using Amazon.CDK.AWS.IAM;
using Amazon.CDK.AWS.S3;
using Constructs;

namespace Harbourline.Audit.Infra;

public sealed class EvidenceBucketStack : Stack
{
    public EvidenceBucketStack(Construct scope, string id, IStackProps? props = null) : base(scope, id, props)
    {
        var bucket = new Bucket(this, "Evidence", new BucketProps
        {
            BucketName = $"hl-m05-evidence-{Account}",
            Versioned = true,
            ObjectLockEnabled = true,
            // Learning shortcut: governance mode, 1 day. Production: compliance mode, retention from the records policy.
            ObjectLockDefaultRetention = ObjectLockRetention.Governance(Duration.Days(1)),
            Encryption = BucketEncryption.KMS_MANAGED,
            BlockPublicAccess = BlockPublicAccess.BLOCK_ALL,
            EnforceSSL = true,
            RemovalPolicy = RemovalPolicy.RETAIN, // objects under retention can't be deleted early anyway
        });

        var collectorRole = new Role(this, "CollectorRole", new RoleProps
        {
            RoleName = "hl-m05-evidence-collector",
            AssumedBy = new ServicePrincipal("lambda.amazonaws.com"),
            ManagedPolicies = new[] { ManagedPolicy.FromAwsManagedPolicyName("SecurityAudit") },
        });
        collectorRole.AddToPolicy(new PolicyStatement(new PolicyStatementProps
        {
            Actions = new[] { "iam:GenerateCredentialReport", "iam:GetCredentialReport" },
            Resources = new[] { "*" },
        }));
        bucket.GrantPut(collectorRole); // write only: no read, no delete

        bucket.AddToResourcePolicy(new PolicyStatement(new PolicyStatementProps
        {
            Sid = "DenyDeleteAndRetentionBypass",
            Effect = Effect.DENY,
            Principals = new IPrincipal[] { new AnyPrincipal() },
            Actions = new[] { "s3:DeleteObject", "s3:DeleteObjectVersion", "s3:BypassGovernanceRetention", "s3:PutBucketObjectLockConfiguration" },
            Resources = new[] { bucket.BucketArn, bucket.ArnForObjects("*") },
        }));

        foreach (var (k, v) in new[] { ("Project", "cea-lab"), ("Lab", "m05-evidence"), ("Environment", "learn"), ("Owner", "you") })
        {
            Amazon.CDK.Tags.Of(this).Add(k, v);
        }
    }
}

The deny on s3:DeleteObject for every principal means even you can't delete objects; to tear down, remove that statement first (an action that itself appears in CloudTrail — which is the point).

Complete code

An evidence collector: a console app (or Lambda handler) that captures three snapshots, writes them with a manifest of SHA-256 hashes, and uploads the package. It only needs read permissions plus s3:PutObject on the evidence bucket.

src/Harbourline.Audit.Evidence/EvidenceCollector.cs
using System.Security.Cryptography;
using System.Text;
using System.Text.Json;
using Amazon.CloudTrail;
using Amazon.CloudTrail.Model;
using Amazon.ConfigService;
using Amazon.ConfigService.Model;
using Amazon.IdentityManagement;
using Amazon.IdentityManagement.Model;

namespace Harbourline.Audit.Evidence;

public sealed record EvidenceItem(string FileName, string Control, string Source, byte[] Content);

public sealed record ManifestEntry(string FileName, string Control, string Source, string Sha256, long Bytes);

public sealed record Manifest(
    string PackageId, string AccountId, string Region, DateTimeOffset CollectedAtUtc,
    string CollectorIdentity, IReadOnlyList<ManifestEntry> Items);

/// <summary>Destination for evidence packages (S3 with Object Lock in production; a folder in tests).</summary>
public interface IEvidenceSink
{
    Task WriteAsync(string key, byte[] content, CancellationToken ct);
}

public sealed class EvidenceCollector(
    IAmazonIdentityManagementService iam,
    IAmazonConfigService config,
    IAmazonCloudTrail cloudTrail,
    IEvidenceSink sink,
    TimeProvider clock)
{
    private static readonly JsonSerializerOptions Json = new(JsonSerializerDefaults.Web) { WriteIndented = true };

    public async Task<Manifest> CollectAsync(string accountId, string region, string collectorIdentity, CancellationToken ct = default)
    {
        var items = new List<EvidenceItem>
        {
            await CredentialReportAsync(ct).ConfigureAwait(false),
            await ConfigComplianceAsync(ct).ConfigureAwait(false),
            await TrailStatusAsync(ct).ConfigureAwait(false),
        };

        var now = clock.GetUtcNow();
        var packageId = $"{accountId}/{region}/{now:yyyy-MM-dd'T'HHmmss'Z'}";
        var entries = new List<ManifestEntry>();

        foreach (var item in items)
        {
            await sink.WriteAsync($"{packageId}/{item.FileName}", item.Content, ct).ConfigureAwait(false);
            entries.Add(new ManifestEntry(item.FileName, item.Control, item.Source,
                Convert.ToHexStringLower(SHA256.HashData(item.Content)), item.Content.LongLength));
        }

        var manifest = new Manifest(packageId, accountId, region, now, collectorIdentity, entries);
        await sink.WriteAsync($"{packageId}/manifest.json", JsonSerializer.SerializeToUtf8Bytes(manifest, Json), ct).ConfigureAwait(false);
        return manifest;
    }

    // Control: IAM-01 "No root access keys; MFA on all human users"
    private async Task<EvidenceItem> CredentialReportAsync(CancellationToken ct)
    {
        // The report is generated asynchronously; poll until it's ready (usually seconds).
        for (var attempt = 0; attempt < 10; attempt++)
        {
            var state = await iam.GenerateCredentialReportAsync(new GenerateCredentialReportRequest(), ct).ConfigureAwait(false);
            if (state.State == ReportStateType.COMPLETE)
                break;
            await Task.Delay(TimeSpan.FromSeconds(2), clock, ct).ConfigureAwait(false);
        }

        var report = await iam.GetCredentialReportAsync(new GetCredentialReportRequest(), ct).ConfigureAwait(false);
        return new EvidenceItem("iam-credential-report.csv", "IAM-01", "iam:GetCredentialReport", report.Content.ToArray());
    }

    // Control: CFG-01 "Resources comply with the bank's Config rules"
    private async Task<EvidenceItem> ConfigComplianceAsync(CancellationToken ct)
    {
        var rows = new List<object>();
        string? next = null;
        do
        {
            var page = await config.DescribeComplianceByConfigRuleAsync(
                new DescribeComplianceByConfigRuleRequest { NextToken = next }, ct).ConfigureAwait(false);
            foreach (var r in page.ComplianceByConfigRules ?? [])
            {
                rows.Add(new
                {
                    rule = r.ConfigRuleName,
                    compliance = r.Compliance?.ComplianceType?.Value,
                    nonCompliantResources = r.Compliance?.ComplianceContributorCount?.CappedCount,
                });
            }
            next = page.NextToken;
        } while (!string.IsNullOrEmpty(next));

        return new EvidenceItem("config-compliance-by-rule.json", "CFG-01", "config:DescribeComplianceByConfigRule",
            JsonSerializer.SerializeToUtf8Bytes(rows, Json));
    }

    // Control: LOG-01 "Organisation trail is logging, multi-Region, with log file validation"
    private async Task<EvidenceItem> TrailStatusAsync(CancellationToken ct)
    {
        var trails = await cloudTrail.DescribeTrailsAsync(new DescribeTrailsRequest(), ct).ConfigureAwait(false);
        var rows = new List<object>();
        foreach (var t in trails.TrailList ?? [])
        {
            var status = await cloudTrail.GetTrailStatusAsync(new GetTrailStatusRequest { Name = t.TrailARN }, ct).ConfigureAwait(false);
            rows.Add(new
            {
                name = t.Name,
                isOrganizationTrail = t.IsOrganizationTrail,
                isMultiRegion = t.IsMultiRegionTrail,
                logFileValidation = t.LogFileValidationEnabled,
                isLogging = status.IsLogging,
                latestDelivery = status.LatestDeliveryTime,
                latestDeliveryError = status.LatestDeliveryError,
            });
        }
        return new EvidenceItem("cloudtrail-status.json", "LOG-01", "cloudtrail:DescribeTrails+GetTrailStatus",
            JsonSerializer.SerializeToUtf8Bytes(rows, Json));
    }
}
src/Harbourline.Audit.Evidence/S3EvidenceSink.cs
using Amazon.S3;
using Amazon.S3.Model;

namespace Harbourline.Audit.Evidence;

public sealed class S3EvidenceSink(IAmazonS3 s3, string bucket) : IEvidenceSink
{
    public async Task WriteAsync(string key, byte[] content, CancellationToken ct)
    {
        await s3.PutObjectAsync(new PutObjectRequest
        {
            BucketName = bucket,
            Key = key,
            InputStream = new MemoryStream(content),
            ChecksumAlgorithm = ChecksumAlgorithm.SHA256, // S3 verifies integrity on upload and stores the checksum
            ServerSideEncryptionMethod = ServerSideEncryptionMethod.AWSKMS,
        }, ct).ConfigureAwait(false);
    }
}
src/Harbourline.Audit.Evidence/Program.cs
using Amazon.CloudTrail;
using Amazon.ConfigService;
using Amazon.IdentityManagement;
using Amazon.S3;
using Amazon.SecurityToken;
using Amazon.SecurityToken.Model;
using Harbourline.Audit.Evidence;

// Usage: dotnet run -- <evidence-bucket>   (uses the default credential chain, e.g. AWS_PROFILE=learn-sandbox)
if (args.Length != 1)
{
    Console.Error.WriteLine("Usage: dotnet run -- <evidence-bucket>");
    return 2;
}

using var sts = new AmazonSecurityTokenServiceClient();
var me = await sts.GetCallerIdentityAsync(new GetCallerIdentityRequest());
using var iam = new AmazonIdentityManagementServiceClient();
using var config = new AmazonConfigServiceClient();
using var trail = new AmazonCloudTrailClient();
using var s3 = new AmazonS3Client();
var region = s3.Config.RegionEndpoint?.SystemName ?? "us-east-1"; // Region resolved from profile/environment

var collector = new EvidenceCollector(iam, config, trail, new S3EvidenceSink(s3, args[0]), TimeProvider.System);
var manifest = await collector.CollectAsync(me.Account, region, me.Arn);

Console.WriteLine($"Evidence package {manifest.PackageId}");
foreach (var item in manifest.Items)
{
    Console.WriteLine($"  {item.FileName,-34} {item.Control,-7} sha256={item.Sha256[..16]}… ({item.Bytes} bytes)");
}
return 0;

Packages: AWSSDK.IdentityManagement, AWSSDK.ConfigService, AWSSDK.CloudTrail, AWSSDK.S3 and AWSSDK.SecurityToken, all 4.x (look up current versions on NuGet). In tests, pass mocked clients and an in-memory IEvidenceSink, and assert that the manifest's hashes match the bytes written — the test runs without an AWS account.

Code explained line by line

  1. EvidenceItem carries a control ID and a source — every file says which control it supports and which API produced it, which answers the auditor's "how was this produced?".
  2. Manifest records package ID, account, Region, UTC time and collector identity (the role ARN from STS) — attributable and dated evidence.
  3. IEvidenceSink — abstraction so tests write to memory and production writes to S3.
  4. packageId uses the account, Region and a sortable UTC timestamp, so packages are easy to find for a period.
  5. SHA256.HashData(item.Content) — the hash of exactly the bytes stored; anyone can recompute it later to show the file is unchanged.
  6. GenerateCredentialReportAsync loop — IAM generates the report asynchronously; poll with a delay that respects the injected TimeProvider (testable without real waiting).
  7. DescribeComplianceByConfigRuleAsync with NextToken — paginate; evidence must be complete, and silently taking only the first page is a classic completeness failure.
  8. page.ComplianceByConfigRules ?? [] — in SDK for .NET v4, collection properties may be null when the service returns none, so guard them.
  9. ComplianceContributorCount?.CappedCount — Config caps the count it returns ⚠ VERIFY the cap; the evidence notes the number, and details come from the rule's resource view.
  10. DescribeTrailsAsync + GetTrailStatusAsync — design (multi-Region, org trail, validation) and operation (IsLogging, latest delivery and errors) in one file.
  11. ChecksumAlgorithm.SHA256 on PutObject — S3 validates the upload against the checksum and stores it, giving a second, independent integrity record.
  12. Program.cs uses STS to record who ran the collector, and returns non-zero on bad usage so a scheduler can alert.

Security considerations

⚠ Learning shortcut — not for production

In the lab the evidence bucket is in your sandbox account with governance-mode Object Lock for one day, and the collector runs with your own credentials. In production: the evidence bucket lives in a dedicated audit or log-archive account; the collector runs on a schedule under a dedicated read-only role assumed into each account; Object Lock uses compliance mode with retention from the bank's records-retention schedule; and nobody in the audited teams can write to, delete from or change the lifecycle of the evidence store.

  • Evidence can contain sensitive data. Credential reports list user names; CloudTrail includes IP addresses and role session names. Minimise, restrict and expire access, and treat auditor access like any other privileged access.
  • Don't screenshot when you can export. Screenshots are hard to verify, easy to fake and lack completeness. Use API exports with hashes, and record the query that produced each population.
  • Separation of duties applies to evidence. The team operating a control shouldn't be the only source of evidence about it.
  • Retention: keep evidence and logs for as long as your records policy requires. PCI DSS expects at least 12 months of audit log history with 3 months immediately available ⚠ VERIFY against requirement 10.5.1; UK banking record-keeping rules often imply longer — Legal sets the schedule.
  • Logs are evidence for attackers too. Logging that's disabled for a week is itself something attackers try; alert on StopLogging, DeleteTrail, StopConfigurationRecorder, and protect them with SCPs.

Banking use case

Harbourline's internal audit runs an annual review of "Cloud access management". In previous years the platform team spent three weeks producing screenshots. This year:

  • The population of privileged-role sessions for the year came from one Athena query over the organisation trail (AssumeRoleWithSAML and Identity Center federation events for the PlatformAdmin and BreakGlass permission sets). The query text, run time, result file and its SHA-256 hash were stored together in the evidence bucket.
  • Auditors picked 25 sessions. For each, the platform team linked the change ticket or incident, the approval in the access-request tool, and the session's CloudTrail activity.
  • The monthly evidence packages (credential reports, Config compliance, trail status) showed the controls were in place all year; the manifests and Object Lock showed the files hadn't been changed.
  • AWS's SOC 2 Type 2 report from Artifact covered AWS's side of identity infrastructure for the period.

Two sessions lacked a ticket; that became an audit finding with an action (enforce ticket IDs as a session tag at sign-in). That's a good outcome: the evidence process worked, and it surfaced a real gap.

Common mistakes

  • Relying on CloudTrail event history → it shows 90 days of management events in one Region per view; your evidence is the organisation trail in S3.
  • Evidence stored where the audited team can edit it → auditors won't rely on it. Separate accounts and Object Lock.
  • Missing the "how do we know it was on all year?" question → keep Config compliance history for the logging rules and validate CloudTrail digests.
  • Taking the first page of an API result → incomplete populations. Always paginate.
  • Planning around AWS Audit Manager or CloudTrail Lake for new accounts → both are closed to new customers; use Config, Security Hub CSPM, Athena and your own collector.
  • Assuming a conformance pack equals a framework → packs map some technical checks; most framework requirements need process evidence.
  • Confusing AWS's reports with yours → Artifact proves AWS's controls only.

Troubleshooting

SymptomLikely causeFix
validate-logs reports missing digest filesTrail was stopped, the bucket policy changed, or validation was enabled mid-periodExplain the gap with Config history; fix the cause; never "backfill" logs
Athena query scans terabytesTable not partitioned or using SELECT * without date filtersUse partition projection by date/account/Region; select only needed columns
NoSuchEntity / report not ready from GetCredentialReportReport still generatingPoll GenerateCredentialReport until COMPLETE
Config compliance history has gapsRecorder off, or resource types not recordedRecord all required resource types; alert on recorder stops
AccessDenied writing to the evidence bucket with Object LockMissing s3:PutObject or KMS permissions; or headers conflict with default retentionGrant put + kms:GenerateDataKey; don't set conflicting retention headers
Auditor says sample evidence doesn't match populationDifferent time zones or filters between queriesUse UTC everywhere; store the exact query with the results

Cost and cleanup

This lesson is free if you only read reports and run the CLI commands: Artifact, credential reports, CloudTrail describe calls and Config describe calls cost nothing. Config advanced queries are free ⚠ VERIFY; Athena costs about USD 5 per TB scanned (a small lab query scans kilobytes to megabytes).

Optional: deploying the evidence bucket costs pennies (S3 storage for a few KB, KMS requests with the AWS managed key). Clean up after the one-day governance retention expires:

BUCKET=hl-m05-evidence-<account-id>
# 1. Remove the deny statement so objects can be deleted (this change is itself logged).
aws s3api delete-bucket-policy --bucket "$BUCKET" --profile learn-sandbox
# 2. After the 1-day retention has passed, delete all object versions and the bucket.
aws s3api list-object-versions --bucket "$BUCKET" --query "Versions[].{Key:Key,VersionId:VersionId}" --output json --profile learn-sandbox > versions.json
# Delete each version (or use the console's Empty bucket button), then:
aws s3api delete-bucket --bucket "$BUCKET" --profile learn-sandbox
# 3. Local files
rm -f credential-report.csv versions.json
# 4. Confirm nothing tagged for the lab remains
aws resourcegroupstaggingapi get-resources --tag-filters Key=Lab,Values=m05-evidence --profile learn-sandbox

Exercises

  • 🟢 Guided: Download the IAM credential report and the latest SOC 2 report from Artifact. For the control "No IAM users have console passwords without MFA", write the evidence description (source, period, how produced, hash) as you'd file it in a GRC tool.
  • 🟡 Challenge: Add a fourth evidence item to the collector: the list of Identity Center permission-set assignments for all accounts (using AWSSDK.SSOAdmin), with pagination, and add a test that checks the manifest hash for each item.
  • 🔴 Live problem: An external auditor emails: "Please provide evidence that all production changes to the Payments API in the last 12 months were approved by someone other than the author." You have GitHub, a deployment pipeline, and an organisation CloudTrail trail. Describe the population, how you'd prove completeness, the sample evidence, and one gap you'd expect to find.

Assignment

Create a control-to-evidence map for the module project (at least ten controls across access, change, logging, encryption, backup and vulnerability management), with source, evidence type, frequency and owner. Add the evidence collector and its tests to your portfolio repository, and include one Athena query with a sample output for a population question.

Solutions

Knowledge check

Interview questions

  • "An auditor asks you to prove production access was controlled all year. What do you provide?" — The design (permission sets, approval workflow, SCPs), the population of privileged sessions from the organisation trail, a sample linked to approvals, periodic access review sign-offs, and evidence that logging was on throughout (Config history, digest validation).
  • "How do you make evidence trustworthy?" — System-generated, complete (paginated, reconciled), dated in UTC, attributable to a collector identity, hashed, stored immutably in a separate account, with the query or API call that produced it.
  • "What changed in AWS's audit tooling recently?" — Audit Manager went into maintenance with no new onboarding from 2026-04-30; CloudTrail Lake closed to new customers on 2026-05-31; Security Hub was relaunched with the original becoming Security Hub CSPM. Evidence now leans on Config, Security Hub CSPM, Athena over CloudTrail, and custom collectors.
  • "Is there a GDPR conformance pack?" — No. GDPR accountability evidence (records of processing, DPIAs, data-subject request handling) is process evidence, collected with the DPO; technical controls can be evidenced with Config and CloudTrail.

Certification practice

Completion checklist

  • I can explain design, implementation and operating-effectiveness evidence, and populations and samples
  • I downloaded an AWS Artifact report and noted its coverage period
  • I produced a credential report and hashed it
  • I can write an Athena query that produces a population from CloudTrail logs
  • I understand the evidence collector, its manifest and why pagination matters
  • I removed any evidence bucket and local files, and confirmed nothing tagged Lab=m05-evidence remains

Key takeaways

  • Auditors test controls over a period using complete populations and samples; design evidence collection for that, not for screenshots.
  • CloudTrail (organisation trail, validated), AWS Config history, Security Hub CSPM and application audit events are your systems of record; AWS Artifact covers only AWS's side.
  • Make evidence trustworthy: system-generated, complete, dated, attributable, hashed and immutable, in an account the audited team can't change.
  • AWS Audit Manager and CloudTrail Lake are closed to new customers; use Config, Security Hub CSPM, Athena and your own collector.
  • There's no GDPR conformance pack; privacy evidence is process evidence led by the DPO.

Further reading

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page