Skip to Content
Cloud Engineering Academy
9. DevOps & CI/CD

Automated testing strategy for pipelines: unit, integration, contract, end-to-end and load

Decide which tests run at which pipeline stage, write fast unit tests, in-process integration tests, Testcontainers tests against PostgreSQL and LocalStack, a consumer contract test, a post-deploy smoke test and a k6 load test, and keep the whole suite trustworthy and quick.

intermediate⏱ 1 h 30 min🟢 FreeDOP-C02DVA-C03

Assumes: .NET 10 LTS · xUnit v3 4.0.3 on Microsoft.Testing.Platform · Microsoft.AspNetCore.Mvc.Testing 10.0.15 · Testcontainers.LocalStack 4.17.0 · Testcontainers.PostgreSql 4.17.0 · k6 · AWS SDK for .NET v4 · Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Place unit, integration, contract, end-to-end and load tests at the pipeline stage where they give the fastest useful feedback.
  • Write xUnit v3 unit tests and in-process integration tests with WebApplicationFactory that need no network and no AWS account.
  • Use Testcontainers to start a real PostgreSQL database and a LocalStack S3 in a test, and know when emulation is not good enough.
  • Write a lightweight consumer contract test that turns an accidental breaking change into a red build.
  • Add a post-deployment smoke test and a k6 load test with pass/fail thresholds.
  • Publish JUnit test results from CI and manage flaky tests without hiding them.

Prerequisites

Why this matters in banking

A rounding rule changes, a JSON field is renamed from balance to amount, a package update changes how dates serialise. None of these crashes the service. The API still returns 200 OK. The mobile app, however, shows a blank balance to 40,000 customers.

Tests are the pipeline's evidence that behaviour did not change by accident. For a bank they also serve a second purpose: the test result is a record. When an auditor asks how you know a release was tested, a retained JUnit report tied to the commit and the run is a better answer than "the team said so".

There is a trap in the other direction. A suite that takes 90 minutes, fails randomly and is rerun until green teaches engineers to ignore red. A good strategy therefore balances coverage (what is proven) against speed and trust (how soon, and how reliably).

The idea in plain language

Think of tests as concentric rings of confidence, each slower and more expensive than the last:

  1. Unit tests check one piece of logic (an account-number validator). Milliseconds, thousands of them.
  2. Integration tests check that pieces work together: the HTTP layer, serialisation and a real database. Seconds.
  3. Contract tests check that the shape of what you send or receive matches what another team depends on. Seconds.
  4. End-to-end (E2E) and smoke tests check that a deployed system works in a real environment. Minutes.
  5. Load tests check that the system keeps its promises under traffic. Many minutes.

The well-known picture is a pyramid: many fast tests at the bottom, few slow ones at the top. The reason is cost, not fashion. A failing unit test points at a function; a failing E2E test points at "something, somewhere".

Analogy: testing a payment-card machine. You check each chip on the bench (unit), then the assembled machine with a test bank (integration), then check the receipt layout matches what the till software expects (contract), then run a card through one real terminal in the shop (smoke), and finally rush 500 customers through (load). You would not rush 500 customers through to find a loose wire.

What each test type is for

TypeProvesNeedsTypical timeRuns
UnitA piece of logic is correctNothingUnder 1 second eachEvery commit and pull request
In-process integrationHTTP, routing, serialisation, validation and DI work togetherNothing externalSecondsEvery commit and pull request
Container integration (Testcontainers)Real PostgreSQL behaviour, real S3 API shape (emulated)Docker10 to 60 secondsPull request or main
ContractResponse shape matches the published consumer contractNothing externalSecondsEvery commit and pull request
Smoke (post-deploy)The deployed service is the right version and basic paths workA deployed environmentUnder 1 minuteAfter every deployment
End-to-endA business journey works across servicesA full environmentMinutesAfter test deployment, nightly
Load / performanceLatency and error rate under trafficA prod-like environment10+ minutesBefore major releases, scheduled

What problem the tools solve

ToolWhat it solvesLimits and sharp edges
xUnit v3 on Microsoft.Testing.Platform (MTP)A modern test framework; tests are executables, no VSTest adapterCommand-line options differ from VSTest ⚠ VERIFY the dotnet test syntax for your SDK; the lab uses dotnet test --solution
WebApplicationFactory<Program>Runs the whole ASP.NET Core pipeline in memoryYour app needs a public Program (a partial class or InternalsVisibleTo)
Testcontainers for .NETStarts real dependencies (PostgreSQL, LocalStack) in disposable containersNeeds a Docker daemon; slower than in-memory; images must be pulled
LocalStackEmulates many AWS APIs locallyAn emulator is not AWS: IAM enforcement, eventual consistency, limits and newer features can differ. Recent image releases need an auth token ⚠ VERIFY current licensing
NSubstituteFakes an AWS SDK client (IAmazonS3) in a unit testA fake only proves your code calls the SDK the way you expect
k6 (open source)Scripted load tests with thresholds that fail the runYou must run it from somewhere with network access to the target
JUnit XML reportsA machine-readable result that CI, CodeBuild and auditors can displayNeeds a reporter extension in MTP; the lab passes --report-junit

Be honest about emulation. LocalStack is excellent for checking that your code calls S3 correctly, that the request is well-formed and that your error handling works. It cannot tell you whether the real IAM policy permits the call. Use it for the first and an AWS test account (or the sandbox) for the second.

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS or open-source equivalentWhat is different
MSTest/NUnit/xUnit with VSTestxUnit v3 on Microsoft.Testing.PlatformTests are executables; same C# you already write
WebApplicationFactoryIdenticalIt is an ASP.NET Core feature, not AWS-specific
Azurite (Azure Storage emulator)LocalStack, or Testcontainers modulesLocalStack covers many services; Azurite covers Storage
Azure Test PlansNot an AWS service; tests live in code and CIThere is no AWS-native test-management product ⚠ VERIFY
Azure Load Testingk6, NBomber, or Distributed Load Testing on AWS (a solution template) ⚠ VERIFY current nameRun it as a pipeline step or a scheduled job
Azure Pipelines test results tabActions run summary / artefacts, CodeBuild report groupsPublish JUnit XML so results are viewable
Pact / consumer-driven contractsSame tools; or a simple schema testA lightweight test is a good start; a broker is optional

Architecture

Which test runs where

Security-boundary view

Tests that run on pull requests never use real credentials. LocalStack accepts any key pair, so tests use the obvious fake test/test. Tests that touch a real AWS account use a separate, narrowly scoped test identity and run only after merge.

Step-by-step implementation

Start from the lab's test layout

labs/m09-delivery-pipeline has two test projects: Harbourline.Accounts.Api.Tests (unit, in-process integration and contract tests) and Harbourline.Delivery.Tests (policy, image-tag, health-gate and evidence tests, including the LocalStack test). Both use xUnit v3 4.0.3 on Microsoft.Testing.Platform.

Write unit tests for pure logic

Pure logic is the cheapest thing to test. Use [Theory] with [InlineData] for edge cases. Include the bad inputs: null, whitespace, wrong case, wrong length.

Add in-process integration tests

Use WebApplicationFactory<Program> to call real endpoints through the real middleware. Override configuration per test with WithWebHostBuilder and UseSetting.

Add a contract test for every consumer-facing response

List the fields and JSON value kinds that consumers depend on, and assert them. This catches renames and type changes that compile cleanly.

Add Testcontainers for what an in-memory fake cannot prove

Use a real PostgreSQL container for SQL behaviour (constraints, transactions, collations) and LocalStack for S3 API shape. Gate the tests behind an environment variable so developers without Docker are not blocked.

Publish the results

Run tests with --report-junit, upload the XML as an artefact, and keep it for the evidence window your policy sets.

Add post-deploy smoke and scheduled load tests

The smoke test checks /health/ready and /version against the deployed environment. The load test runs on a schedule against staging, not on every commit.

AWS Console instructions

Most of this lesson is code, but two console views help:

  • CodeBuild test reports (if you run tests in CodeBuild): Console → CodeBuild → Report groups. Each build that declares a reports: section in the buildspec produces a report with pass and fail counts and test case durations.
  • CloudWatch for load tests: Console → CloudWatch → Dashboards. During a load test watch HTTPCode_Target_5XX_Count and TargetResponseTime for the load balancer, and CPU and memory for the ECS service.

For GitHub Actions, open a workflow run and look at Summary → Artifacts for the uploaded test-results.

AWS CLI

The test code itself does not need the AWS CLI. These commands help when tests run in CodeBuild or when a load test runs against an ALB.

# List CodeBuild report groups (test reports published by builds).
aws codebuild list-report-groups --profile learn-sandbox --region us-east-1

# Show the latest reports in a group (replace the ARN with yours).
aws codebuild list-reports-for-report-group \
  --report-group-arn arn:aws:codebuild:us-east-1:111122223333:report-group/hl-m09-tests \
  --profile learn-sandbox --region us-east-1

# Fetch the 5xx count for a load balancer over the last 15 minutes (replace the dimension value).
aws cloudwatch get-metric-statistics \
  --namespace AWS/ApplicationELB --metric-name HTTPCode_Target_5XX_Count \
  --dimensions Name=LoadBalancer,Value=app/hl-m09-alb/1234567890abcdef \
  --start-time "$(date -u -d '-15 minutes' +%Y-%m-%dT%H:%M:%SZ)" \
  --end-time "$(date -u +%Y-%m-%dT%H:%M:%SZ)" \
  --period 60 --statistics Sum \
  --profile learn-sandbox --region us-east-1

Infrastructure as Code

The one piece of infrastructure tests need is a place to publish results when you use CodeBuild: a report group. It is optional if you run tests only in GitHub Actions.

TestReportsStack.cs
using Amazon.CDK;
using Amazon.CDK.AWS.CodeBuild;
using Constructs;

namespace Harbourline.Pipelines;

public class TestReportsStack : Stack
{
    public TestReportsStack(Construct scope, string id, IStackProps? props = null) : base(scope, id, props)
    {
        // A report group collects JUnit results from CodeBuild builds. RemovalPolicy.DESTROY is a learning shortcut.
        _ = new ReportGroup(this, "Tests", new ReportGroupProps
        {
            ReportGroupName = "hl-m09-tests",
            RemovalPolicy = RemovalPolicy.DESTROY,
        });

        Amazon.CDK.Tags.Of(this).Add("Project", "cea-lab");
        Amazon.CDK.Tags.Of(this).Add("Lab", "m09-testing");
    }
}

Complete code

The lab already contains tested code; read it in this order. Everything below comes from labs/m09-delivery-pipeline and passes without an AWS account (74 tests passed, 1 skipped when last run).

1. Unit tests: pure logic

tests/Harbourline.Accounts.Api.Tests/ApiTests.cs (excerpt)
public class IdentifierTests
{
    [Theory]
    [InlineData("ACC-0000000001", true)]
    [InlineData("ACC-000000001", false)]
    [InlineData("acc-0000000001", false)]
    [InlineData("ACC-0000000001 ", false)]
    [InlineData(null, false)]
    public void Account_ids_follow_the_canonical_format(string? id, bool valid) =>
        Assert.Equal(valid, Identifiers.IsAccountId(id));
}

2. In-process integration tests

tests/Harbourline.Accounts.Api.Tests/ApiTests.cs (excerpt)
public class ApiIntegrationTests(WebApplicationFactory<Program> factory) : IClassFixture<WebApplicationFactory<Program>>
{
    [Fact]
    public async Task Forced_unready_returns_503_for_the_rollback_drill()
    {
        var client = factory.WithWebHostBuilder(b => b.UseSetting("Health:ForceUnready", "true")).CreateClient();

        var ready = await client.GetAsync("/health/ready", TestContext.Current.CancellationToken);
        var live = await client.GetAsync("/health/live", TestContext.Current.CancellationToken);

        Assert.Equal(HttpStatusCode.ServiceUnavailable, ready.StatusCode);
        Assert.Equal(HttpStatusCode.OK, live.StatusCode); // liveness must not fail, or ECS would restart-loop
    }
}

3. Consumer contract test

tests/Harbourline.Accounts.Api.Tests/ApiTests.cs (excerpt)
public class AccountContractTests(WebApplicationFactory<Program> factory) : IClassFixture<WebApplicationFactory<Program>>
{
    private static readonly Dictionary<string, JsonValueKind> Contract = new()
    {
        ["accountId"] = JsonValueKind.String,
        ["customerId"] = JsonValueKind.String,
        ["iban"] = JsonValueKind.String,
        ["currency"] = JsonValueKind.String,
        ["product"] = JsonValueKind.String,
        ["status"] = JsonValueKind.String,
        ["balance"] = JsonValueKind.Number,
        ["openedAt"] = JsonValueKind.String,
    };

    [Fact]
    public async Task Account_response_honours_the_published_contract()
    {
        var json = await factory.CreateClient().GetStringAsync("/v1/accounts/ACC-0000000001", TestContext.Current.CancellationToken);
        using var doc = JsonDocument.Parse(json);

        foreach (var (field, kind) in Contract)
        {
            Assert.True(doc.RootElement.TryGetProperty(field, out var value), $"Contract field '{field}' is missing.");
            Assert.Equal(kind, value.ValueKind);
        }
    }
}

4. Unit test with a faked AWS client, and a LocalStack integration test

The unit test substitutes IAmazonS3; the integration test starts LocalStack with Testcontainers, and is skipped unless you opt in:

tests/Harbourline.Delivery.Tests/EvidenceStoreTests.cs (excerpt)
[Fact]
public async Task Round_trips_evidence_through_s3_and_refuses_overwrite()
{
    Assert.SkipUnless(Environment.GetEnvironmentVariable("RUN_LOCALSTACK_TESTS") == "1",
        "Set RUN_LOCALSTACK_TESTS=1 (and have Docker running) to run LocalStack integration tests.");

    var ct = TestContext.Current.CancellationToken;
    var image = Environment.GetEnvironmentVariable("LOCALSTACK_IMAGE") ?? "localstack/localstack:4.14";
    var builder = new LocalStackBuilder(image);
    var token = Environment.GetEnvironmentVariable("LOCALSTACK_AUTH_TOKEN");
    if (!string.IsNullOrEmpty(token))
    {
        builder = builder.WithEnvironment("LOCALSTACK_AUTH_TOKEN", token);
    }

    await using var localstack = builder.Build();
    await localstack.StartAsync(ct);

    using var s3 = new AmazonS3Client(
        new BasicAWSCredentials("test", "test"), // LocalStack accepts any credentials; never real ones
        new AmazonS3Config { ServiceURL = localstack.GetConnectionString(), ForcePathStyle = true, AuthenticationRegion = "us-east-1" });
    // ... create bucket, save, load, and assert a second save is refused
}

5. A PostgreSQL Testcontainers test

This pattern is for a service with a database. It is not part of the lab (the lab API uses in-memory sample data), so treat it as a template ⚠ VERIFY by running it once in your own project.

AccountRepositoryTests.cs
using Npgsql;
using Testcontainers.PostgreSql;

namespace Harbourline.Accounts.Tests;

// IAsyncLifetime starts one container for the whole class and disposes it afterwards.
public class AccountRepositoryTests : IAsyncLifetime
{
    private readonly PostgreSqlContainer _db = new PostgreSqlBuilder("postgres:17-alpine")
        .WithDatabase("accounts")
        .WithUsername("test")
        .WithPassword("test-only-password")   // a throw-away container; never a real secret
        .Build();

    public async ValueTask InitializeAsync() => await _db.StartAsync();

    public async ValueTask DisposeAsync() => await _db.DisposeAsync();

    [Fact]
    public async Task Balance_column_rejects_nulls_and_keeps_two_decimals()
    {
        await using var conn = new NpgsqlConnection(_db.GetConnectionString());
        await conn.OpenAsync(TestContext.Current.CancellationToken);

        await using (var create = new NpgsqlCommand(
            "CREATE TABLE account (id text PRIMARY KEY, balance numeric(18,2) NOT NULL)", conn))
        {
            await create.ExecuteNonQueryAsync(TestContext.Current.CancellationToken);
        }

        await using var insert = new NpgsqlCommand("INSERT INTO account VALUES ('ACC-0000000001', 1520.455)", conn);
        await insert.ExecuteNonQueryAsync(TestContext.Current.CancellationToken);

        await using var read = new NpgsqlCommand("SELECT balance FROM account WHERE id = 'ACC-0000000001'", conn);
        var balance = (decimal)(await read.ExecuteScalarAsync(TestContext.Current.CancellationToken))!;

        Assert.Equal(1520.46m, balance);   // PostgreSQL rounds half away from zero for numeric
    }
}

The builder constructor that takes an image name is the current Testcontainers style; older tutorials use a parameterless builder with WithImage ⚠ VERIFY for your package version.

6. A post-deployment smoke test

The VerifyDeployed.cs program from CI/CD concepts is the smoke test. In a workflow it looks like this (the SHA is the commit the pipeline built):

smoke step (excerpt from a workflow)
- name: Smoke test the deployed version
  env:
    SMOKE_URL: ${{ vars.SMOKE_URL }}
    EXPECTED_SHA: ${{ github.sha }}
  run: |
    dotnet run VerifyDeployed.cs -- "$SMOKE_URL" "$EXPECTED_SHA" 10 6
    curl -fsS "$SMOKE_URL/health/ready"

7. A k6 load test with thresholds

load/accounts.js
import http from 'k6/http';
import { check, sleep } from 'k6';

// Ramp to 50 virtual users, hold, ramp down. The run FAILS if a threshold is missed.
export const options = {
  stages: [
    { duration: '1m', target: 50 },
    { duration: '5m', target: 50 },
    { duration: '1m', target: 0 },
  ],
  thresholds: {
    http_req_failed: ['rate<0.01'],      // fewer than 1% errors
    http_req_duration: ['p(95)<300'],    // 95% of requests under 300 ms
  },
};

const BASE = __ENV.BASE_URL;              // for example https://test.accounts.example.internal

export default function () {
  const res = http.get(`${BASE}/v1/accounts/ACC-0000000001`);
  check(res, {
    'status is 200': (r) => r.status === 200,
    'has a balance': (r) => r.json('balance') !== undefined,
  });
  sleep(1);
}

Run it locally against the lab container, or in a pipeline job after deployment to a test environment:

docker run --rm -i -e BASE_URL=http://host.docker.internal:8080 grafana/k6 run - < load/accounts.js

The image name and the run - form are the documented pattern ⚠ VERIFY against the current k6 installation page.

8. Running the suite

# From labs/m09-delivery-pipeline. Unit, in-process integration and contract tests: no AWS, no Docker.
dotnet test --solution DeliveryPipeline.slnx

# Include the LocalStack integration test (needs Docker, and possibly a LocalStack auth token).
RUN_LOCALSTACK_TESTS=1 dotnet test --solution DeliveryPipeline.slnx

# Produce JUnit XML for the pipeline's evidence pack.
dotnet test --solution DeliveryPipeline.slnx -c Release --report-junit --results-directory ./artifacts/test-results

Code explained line by line

Contract test

  1. Contract = new() { ["balance"] = JsonValueKind.Number ... } — the contract is data: field name to JSON kind. A consumer team can review this table without reading controller code.
  2. JsonDocument.Parse(json) — the test parses the wire format, not the C# type. Renaming a C# property while keeping [JsonPropertyName] would pass; changing the wire name would not. That is the point.
  3. TryGetProperty(field, out var value) — fails with a message naming the missing field, so the red build explains itself.
  4. Assert.Equal(kind, value.ValueKind) — catches "balance": "1520.45" (a string) where a number is expected.

Faked AWS client versus LocalStack

  1. Assert.SkipUnless(...) — xUnit v3 can skip dynamically with a reason; the skip appears in the report as "skipped", not as a silent pass.
  2. LocalStackBuilder(image) — Testcontainers starts a throw-away container and waits until it is ready.
  3. BasicAWSCredentials("test", "test") — LocalStack ignores credentials; using obviously fake ones means the test can never touch real AWS by accident.
  4. ForcePathStyle = true — S3 URLs become host/bucket/key, which works with a plain localhost endpoint (virtual-hosted style would need a wildcard DNS name).
  5. await using var localstack — the container is removed when the test ends, even on failure.

PostgreSQL test

  1. IAsyncLifetime — xUnit v3's InitializeAsync and DisposeAsync return ValueTask.
  2. PostgreSqlBuilder("postgres:17-alpine") — pin the same major version you run in production, so the test catches version-specific behaviour.
  3. numeric(18,2) with 1520.455 — demonstrates a real database rounding rule that an in-memory fake would not reproduce.

k6 script

  1. stages — a ramp avoids measuring cold-start effects as steady state.
  2. thresholds — turn the load test into a gate: a missed threshold makes the run exit non-zero.
  3. __ENV.BASE_URL — the target comes from the environment, so the same script runs against test and staging, and a pull request cannot point it at production.

Security considerations

⚠ Learning shortcut — not for production

The lab uses fake credentials (test/test) and an in-container PostgreSQL password written in the test. That is fine for a throw-away container on a developer or CI machine. It is not acceptable to reuse these patterns against real databases. Never put a real credential in a test file, and never run load tests against production without an explicit, approved plan.

  • Test data is fictional. Use generated or synthetic data. Real customer records in a test database are a GDPR and UK GDPR risk and usually a policy breach. Card numbers in tests come only from the payment brands' published test ranges.
  • Load tests can look like attacks. Notify your security team and the platform owner, and only target environments you own. AWS has a policy for load testing your own resources ⚠ VERIFY the current "penetration testing and load testing" policy page before running tests at high volume.
  • LocalStack tokens are secrets. If you use LOCALSTACK_AUTH_TOKEN, store it as a GitHub environment secret and run the job only on main, never on fork pull requests (the lab's ci.yml does exactly this).
  • Do not log secrets or personal data from test failures. Failing assertions print values; keep sensitive values out of assertions.
  • Tests are code under review. A test that is weakened to make a release pass is a control bypass. CODEOWNERS can require a second reviewer for changes under tests/ that remove assertions.

Banking use case

Harbourline's statements service and the mobile app are owned by different teams. The statements team once renamed balance to amount in the Accounts API response and merged after unit tests passed. The contract test now lives in the Accounts API repository and is reviewed by both teams. The next time someone changes a field, the build fails with Contract field 'balance' is missing, and the conversation happens before release instead of after.

Their pipeline runs unit, in-process, contract and PostgreSQL integration tests on every pull request in under six minutes. After deployment to test, the smoke test and a ten-minute end-to-end journey run (open account, post a fictional transaction, read the statement). A staging load test runs every night, and its p95 latency is charted against the service-level objective.

Common mistakes

  • Testing implementation, not behaviour → tests break on every refactor → assert observable outputs (HTTP responses, database state).
  • Sharing mutable state between tests → order-dependent failures → create fresh data per test or per class fixture, and let containers be disposable.
  • Using Thread.Sleep to wait → slow and flaky → poll with a timeout (VerifyDeployed.cs does).
  • Treating LocalStack as AWS → green locally, AccessDenied in AWS → verify IAM behaviour in a real test account.
  • Rerunning until green → flaky tests accumulate → quarantine with an owner and a deadline, or fix.
  • Running all tests on every commit → a 45-minute feedback loop → keep the pull-request path fast, and move slow suites to after-merge or scheduled runs.
  • No retention of results → no evidence → upload JUnit XML as an artefact with a defined retention.
  • Load testing from a laptop over Wi-Fi → you measure your network → run from a runner or load generator near the target.

Troubleshooting

SymptomLikely causeFix
Cannot connect to the Docker daemon in a Testcontainers testDocker not running, or runner without DockerStart Docker Desktop; GitHub-hosted Ubuntu runners include Docker; CodeBuild needs privileged mode
LocalStack test errors mentioning LOCALSTACK_AUTH_TOKENNewer LocalStack image requires a tokenSet the token from a secret, or pin LOCALSTACK_IMAGE to an older tag ⚠ VERIFY
WebApplicationFactory<Program> does not compile: Program is inaccessibleTop-level Program is internalAdd public partial class Program { } or InternalsVisibleTo
Tests pass locally, fail in CI onlyTime zone, culture, file path case or Docker availabilitySet invariant culture in tests; avoid absolute paths; run in the same Docker image locally
Test discovery finds zero testsWrong dotnet test syntax for Microsoft.Testing.PlatformUse dotnet test --solution X.slnx or --project ⚠ VERIFY; ensure global.json opts in to MTP
JUnit file is not producedReporter not enabledPass --report-junit and a --results-directory
k6 run exits 99A threshold was crossedRead the threshold summary; it is a failed test, not a tool error
Smoke test flaps after deployDeployment still shifting trafficRetry with a timeout; verify the version before the health check

Cost and cleanup

Cost: free. Everything here runs locally or on a GitHub-hosted runner. Testcontainers images (LocalStack, PostgreSQL) are downloaded once. A k6 load test against a real environment can cost money: Fargate tasks, load-balancer capacity units and data transfer. Estimate before running against AWS, and run for minutes, not hours.

Cleanup:

# Testcontainers removes its containers automatically; list anything left behind.
docker ps -a --filter "label=org.testcontainers=true"

# Remove stopped Testcontainers containers and unused images when you are done.
docker container prune -f --filter "label=org.testcontainers=true"
docker image rm localstack/localstack:4.14 postgres:17-alpine 2>/dev/null

# Remove build output from the lab.
find labs/m09-delivery-pipeline \( -name bin -o -name obj \) -type d -prune -exec rm -rf {} +

Exercises

  • 🟢 Guided: From the lab folder, run dotnet test --solution DeliveryPipeline.slnx (or the Docker form from the lab README). Then rename the Balance property of the account record in Accounts.cs to Amount (fix any compile errors the rename causes), run the tests again, and read the failure message. Revert the change.
  • 🟡 Challenge: Add a contract test for GET /v1/customers/{id}/accounts: the response is a JSON array, each element has the same required fields as the single-account response. Also assert that an unknown customer returns an empty array, not a 404.
  • 🔴 Live problem: Your pull-request pipeline now takes 28 minutes and fails about one run in ten with no code change. Product wants releases daily. Propose a plan: how to find where the time goes, how to treat the flaky tests, which tests to move to after-merge, and how you would prove to the team that the faster pipeline is still safe.

Assignment

In your portfolio repository, add tests/ to a small ASP.NET Core API with: one unit test class, one in-process integration test, one contract test with an explicit Contract dictionary, and one Testcontainers test gated by an environment variable. Add a workflow that runs them with --report-junit and uploads the XML. Include a TESTING.md of ten lines saying which test type runs at which stage.

Solutions

Knowledge check

Interview questions

  • "What is the test pyramid and when would you break it?" Many fast unit tests, fewer integration tests, few end-to-end tests, because cost and diagnosis difficulty rise with scope. You might add more integration tests where the risk is in the integration itself (SQL, serialisation), as in banking data layers.
  • "How do you stop a breaking API change reaching consumers?" Consumer contract tests in the provider's pipeline, reviewed by consumers, plus versioned endpoints.
  • "Emulator or real AWS for tests?" Emulator for fast feedback about request shape and error handling; a real test account for IAM, quotas and service behaviour. State what each cannot tell you.
  • "A test is flaky. What do you do?" Reproduce, find the shared state or timing cause, quarantine with an owner and deadline if blocking, fix; never normalise reruns.
  • "What test evidence would you give an auditor?" The JUnit report tied to the commit and run, retained for the policy period, referenced from the release record.

Certification practice

Completion checklist

  • I can explain which test type runs at which pipeline stage and why
  • I ran the lab test suite and saw the contract test fail after renaming a field
  • I completed the guided exercise and reverted the change
  • I can say what LocalStack cannot prove
  • I removed build output and Testcontainers leftovers

Key takeaways

  • Put fast, deterministic tests on the pull-request path; move slow ones after merge or onto a schedule.
  • A contract test turns a silent breaking change into a red build before consumers feel it.
  • Testcontainers gives you real dependencies in disposable containers; LocalStack proves request shape, not IAM.
  • A smoke test that checks the running version closes the loop between build and deployment.
  • Retain JUnit results with the commit; flaky tests must be fixed or quarantined with an owner, not retried into silence.

Further reading

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page