Skip to Content
Cloud Engineering Academy
3. .NET Application Development on AWS

AWS SDK for .NET v4 in depth

Use the AWS SDK for .NET v4 the way production code should: explicit Regions, the credential provider chain, SSO profiles for local work, retries and deadlines, paginators, error handling and the v4 breaking changes that bite during upgrades.

intermediate⏱ 1 h 30 min🟢 FreeDVA-C03

Assumes: .NET 10 LTS · AWS SDK for .NET v4 (AWSSDK.Core 4.1.x, AWSSDK.S3 4.1.1, AWSSDK.SecurityToken 4.1.0.2) · AWS CLI v2 · Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Explain how the SDK for .NET v4 is packaged and what a service client does for every call (endpoint resolution, SigV4 signing, retries, response parsing).
  • Describe the credential provider chain and the Region resolution order, and predict which credentials a process will use on your laptop, in a container and on EC2.
  • Run a C# console app against your sandbox using the learn-sandbox IAM Identity Center (SSO) profile, without a single access key on disk.
  • Configure retries, connect timeouts and per-call deadlines with CancellationToken, and explain why "retry everything" is dangerous for money-moving calls.
  • Read paginated results with paginators (IAsyncEnumerable) and with an explicit loop, and say when to use each.
  • Handle AmazonServiceException and AmazonClientException correctly and log the request ID.
  • List the v4 breaking changes that matter most when upgrading a v3 code base: nullable value types, null collections, Standard retry mode, IMDSv2 only and Region-strict S3 clients.

Prerequisites

Why this matters in banking

Harbourline's reconciliation service started failing at 02:00 one Sunday. Nothing had changed in the code. The cause took six hours to find: a developer had once copied an access key into appsettings.Production.json "just to test", the key was rotated by the security team on Saturday, and the SDK had been quietly picking up that stale key instead of the ECS task role, because explicit configuration beats the default chain.

Three other incidents in the same quarter had the same root cause — engineers did not know how the SDK decides who it is, where it calls and what it does when a call fails:

  • An S3 client created with no Region on a build agent defaulted to whatever the agent's profile said, and wrote a statement file to a bucket in the wrong Region — a data-residency question for the EU subsidiary.
  • A payment-status poller used the default retry settings plus its own Polly retry loop, so one throttled call became up to twelve HTTP requests, which made the throttling worse.
  • A migration from SDK v3 to v4 compiled cleanly and then threw NullReferenceException in production, because empty lists in responses are now null.

Regulators will not ask you about SDK internals, but they will ask why a payment was submitted twice, why a customer file landed in another jurisdiction or why a credential was hard-coded. Every one of those answers starts with understanding the SDK.

The idea in plain language

The AWS SDK for .NET is a set of NuGet packages that turn C# method calls into signed HTTPS requests to AWS service endpoints. You never build URLs or signatures yourself. You create a client for a service (for example AmazonS3Client), give it a Region and credentials (or let it find them), and call async methods such as ListObjectsV2Async.

Each call goes through the same pipeline: resolve the endpoint for the Region, sign the request with Signature Version 4 (SigV4) using temporary credentials, send it, retry if the failure is transient, then parse the JSON or XML response into C# objects.

Analogy: a service client is like a branch courier with a sealed pouch. The courier knows the address of head office for each region (endpoint resolution), carries a signed authority letter that expires at the end of the shift (temporary credentials and SigV4), tries again if the door is busy (retries) and hands you back a typed receipt (the response object). Your job is to tell the courier which region and whose authority to use — or to trust the courier to find the right letter in the right drawer (the credential chain).

What problem the SDK solves

Without the SDK you would have to:

  • canonicalise every request and compute SigV4 signatures with HMAC-SHA256;
  • refresh temporary credentials before they expire;
  • know each service's endpoint pattern per Region and partition;
  • implement exponential back-off with jitter and decide which errors are retryable;
  • parse dozens of response formats.

The SDK does all of that. v4 (generally available since 2025) is a major version with breaking changes. The most important are covered below. It targets .NET Framework 4.7.2, .NET Standard 2.0 and .NET 8 or later, so it runs everywhere from legacy Windows services to .NET 10 containers.

Packaging. AWSSDK.Core holds the runtime: credentials, signing, retries, HTTP. Each service has its own package: AWSSDK.S3, AWSSDK.DynamoDBv2, AWSSDK.SecurityToken, and so on. You reference only the services you use. All service packages in one application should come from the same 4.x line, so they agree on the AWSSDK.Core version.

Honest limitations:

  • The SDK cannot fix a wrong permission. AccessDenied is a policy problem, not an SDK problem.
  • The default retry settings are tuned for "most apps". A user-facing banking API usually wants fewer retries and a tighter deadline than the defaults.
  • Some high-level helpers live in separate packages and evolve on their own schedule. Examples are the S3 Transfer Utility, the DynamoDB object persistence model and the Secrets Manager caching client.
  • Mocking paginators in unit tests is awkward. You will see a workaround in this lesson.

The v4 changes that matter most

Change in v4What breaksWhat to do
Value-type properties on requests and responses are nullable (int?, bool?, long?, DateTime?)if (response.IsTruncated) no longer compiles; arithmetic on obj.Size needs a null checkCompare with == true, use ?? 0
Collections in requests and responses default to nullresponse.Items.Count throws NullReferenceException when nothing came backUse response.Items ?? [] or is { Count: > 0 }. AWSConfigs.InitializeCollections = true restores v3 behaviour as a stop-gap
Default RetryMode is Standard, and the Legacy mode is removedRetry counts and back-off differ from v3State your retry settings explicitly per client
DefaultConfigurationMode defaults to StandardTimeouts and other defaults differ from v3Set ConnectTimeout and per-call deadlines yourself
Only IMDSv2 is used on EC2Nothing, if your instances require IMDSv2 (they should)Enforce IMDSv2 on every launch template
STS-based credential providers always call the regional STS endpointFirewall rules that only allowed sts.amazonaws.comAllow the regional endpoint, or use a VPC endpoint for STS
An S3 client configured for us-east-1 can no longer reach buckets in other RegionsCode that relied on the global endpoint redirectOne S3 client per Region, configured for the bucket's Region
ECSTaskCredentials was removedCode that constructed it explicitlyUse the default chain; GenericContainerCredentials covers ECS and EKS Pod Identity
log4net integration moved out; Microsoft.Extensions.Logging integration is an adaptor packageNo SDK logs in your ILogger outputAdd AWSSDK.Extensions.Logging.ILoggerAdaptor and call ConfigureAWSSDKLogging()
Timestamps parsed from responses are UTCCode that assumed local timeTreat SDK DateTime values as UTC (which you should do in banking anyway)
SSO credentials no longer open a login flow by default (SupportsGettingNewToken = false)A desktop tool that relied on the SDK launching a browserRun aws sso login first, or opt in explicitly for interactive tools

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
Azure SDK client libraries (Azure.Storage.Blobs, Azure.Messaging.ServiceBus)AWSSDK.S3, AWSSDK.SQS and the other service packagesOne package per service on both sides. AWS method names mirror the HTTP API operation names (PutObject, ListObjectsV2)
DefaultAzureCredentialThe SDK's default credential provider chainBoth try environment, developer login and managed identity in turn. AWS has no "Visual Studio credential"; you use a CLI SSO profile
Managed identityIAM role for the EC2 instance, ECS task or Lambda functionThe SDK fetches and refreshes the role's temporary credentials automatically
RetryOptions on ClientOptionsRetryMode and MaxErrorRetry on the client configAWS Standard mode adds a client-side retry quota (token bucket) so retries cannot snowball
Pageable<T> / AsyncPageable<T>client.Paginators.<Operation>(request) returning IAsyncEnumerablePaginators are single-use. Create a new one each time
RequestFailedException (Status, ErrorCode)AmazonServiceException (StatusCode, ErrorCode, RequestId) plus service-specific subclassesAlways log the RequestId: AWS Support asks for it
CancellationToken on every async methodThe sameUse it as your per-call deadline
Azure Resource Manager endpoints per cloudEndpoint resolution per Region and partition (aws, aws-cn, aws-us-gov)The Region decides the endpoint; you rarely set ServiceURL except for local emulators

Architecture

The diagram shows what happens inside one SDK call.

And this is how the default credential chain decides who you are. The first source that produces credentials wins.

Security-boundary view. Your code never sees long-term secrets in a well-built system. On a laptop, temporary credentials come from the SSO cache that aws sso login fills. In AWS, they come from the platform's metadata or container endpoint for the role attached to the compute. Explicit keys in configuration (the top of the chain) override the role. That is exactly why they are dangerous: one forgotten key silently replaces a well-scoped role.

Region resolution follows a similar order when you do not set RegionEndpoint on the client:

  1. Application configuration files (app.config/web.config on .NET Framework).
  2. AWS_REGION, then AWS_DEFAULT_REGION.
  3. The region in the profile (from AWSConfigs.AWSProfileName, AWS_PROFILE or default).
  4. EC2 instance metadata, on EC2.

If nothing is found, the client throws when it is created. Inside ECS and Lambda, AWS sets AWS_REGION for you.

Step-by-step implementation

Sign in with your SSO profile

aws sso login --profile learn-sandbox
aws sts get-caller-identity --profile learn-sandbox

The second command should print your sandbox account ID and an ARN containing AWSReservedSSO_. This is your proof that the profile works before any C# is involved.

Create the console project

dotnet new console -n Harbourline.SdkTour -f net10.0
cd Harbourline.SdkTour
dotnet add package AWSSDK.S3 --version 4.1.1
dotnet add package AWSSDK.SecurityToken --version 4.1.0.2
dotnet add package AWSSDK.SSO --version 4.1.0.2
dotnet add package AWSSDK.SSOOIDC --version 4.1.0.2

AWSSDK.SSO and AWSSDK.SSOOIDC are not called directly by your code. The core runtime loads them when it meets an SSO profile. Without them you get a confusing "assembly not found" style error the first time you run with learn-sandbox.

Point the process at the profile, not the code

Set AWS_PROFILE in the environment rather than hard-coding a profile name. In Properties/launchSettings.json that looks like this:

Properties/launchSettings.json
{
  "profiles": {
    "SdkTour": {
      "commandName": "Project",
      "environmentVariables": {
        "AWS_PROFILE": "learn-sandbox",
        "AWS_REGION": "us-east-1"
      }
    }
  }
}

From a terminal, use $env:AWS_PROFILE = "learn-sandbox" in PowerShell or export AWS_PROFILE=learn-sandbox in bash. The same binary then uses the ECS task role in AWS with no code change, because there will be no profile there.

Replace Program.cs and run

Paste the complete code from the next sections, then run:

dotnet run
dotnet run -- --bucket hl-m03-statements-yourname   # optional, if you created a bucket earlier

Expected output (your values differ):

Account : 111122223333
Caller  : arn:aws:sts::111122223333:assumed-role/AWSReservedSSO_SandboxDeveloper_0123456789abcdef/alex
Region  : us-east-1
  bucket hl-m00-budget-reports (created 2026-11-02)
1 bucket(s) in us-east-1.

Break it on purpose

Learn the failure modes while they are cheap:

  1. Run aws sso logout, then dotnet run. You get an AmazonClientException about the SSO token. The code prints the hint to log in again.
  2. Set $env:AWS_REGION = "eu-west-2" and run again. The bucket list now shows only buckets whose Region is eu-west-2, because the request filters on BucketRegion.
  3. Run with --bucket and a bucket name that is not yours. You get AccessDenied (HTTP 403) with a request ID. That is a service error, not a client error.

AWS Console instructions

The SDK has no console of its own, but two console views help you check what your code did:

  • Console → CloudTrail → Event history. Filter by Event name = ListBuckets. Open the event and look at userIdentity.arn (your SSO role session) and userAgent, which names the .NET SDK and its version ⚠ VERIFY (exact user-agent format). This is how auditors link an action to a person and a tool.
  • Console → IAM Identity Center → AWS accounts → (your sandbox) → Permission sets. Check which permission set backs learn-sandbox. If a call fails with AccessDenied, this is where the missing permission lives.

AWS CLI

# 1. Log in (opens a browser once; caches a token under ~/.aws/sso/cache).
aws sso login --profile learn-sandbox

# 2. See which credentials and Region the CLI (and, by the same rules, the SDK) will use, and where each came from.
aws configure list --profile learn-sandbox

# 3. Who am I? The same call the C# code makes first.
aws sts get-caller-identity --profile learn-sandbox

# 4. List buckets in one Region (the same filter the C# code uses).
aws s3api list-buckets --bucket-region us-east-1 --max-buckets 100 --profile learn-sandbox

# 5. Short-lived credentials as environment variables, for a tool that cannot read SSO profiles.
#    They expire with the SSO role session. Never paste them into config files.
aws configure export-credentials --profile learn-sandbox --format powershell

Infrastructure as Code

The console app needs very little permission. In a real team the permission set (or, in AWS, the task role) would be defined in code. Here is a least-privilege customer managed policy for this lesson, scoped to the statements bucket prefix used later in the module.

SdkTourPolicyStack.cs
using Amazon.CDK;
using Amazon.CDK.AWS.IAM;
using Constructs;

namespace Harbourline.Learn.Infra;

public sealed class SdkTourPolicyStack : Stack
{
    public SdkTourPolicyStack(Construct scope, string id, IStackProps? props = null) : base(scope, id, props)
    {
        const string bucket = "hl-m03-statements-uk";

        var policy = new ManagedPolicy(this, "SdkTourPolicy", new ManagedPolicyProps
        {
            ManagedPolicyName = "hl-m03-sdk-tour",
            Statements =
            [
                // ListBuckets works only on "*": it is an account-level action.
                new PolicyStatement(new PolicyStatementProps
                {
                    Actions = ["s3:ListAllMyBuckets"],
                    Resources = ["*"],
                }),
                // Listing keys is limited to the statements/ prefix of one bucket.
                new PolicyStatement(new PolicyStatementProps
                {
                    Actions = ["s3:ListBucket"],
                    Resources = [$"arn:aws:s3:::{bucket}"],
                    Conditions = new Dictionary<string, object>
                    {
                        ["StringLike"] = new Dictionary<string, object> { ["s3:prefix"] = new[] { "statements/*" } },
                    },
                }),
            ],
        });

        Amazon.CDK.Tags.Of(this).Add("Project", "cea-lab");
        Amazon.CDK.Tags.Of(this).Add("Lab", "m03-sdk");
        Amazon.CDK.Tags.Of(this).Add("Environment", "learn");

        _ = new CfnOutput(this, "PolicyArn", new CfnOutputProps { Value = policy.ManagedPolicyArn });
    }
}

sts:GetCallerIdentity needs no permission: it always works for any valid credentials, which is why it is the standard "who am I?" check.

Complete code

Harbourline.SdkTour.csproj
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
    <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
    <RootNamespace>Harbourline.SdkTour</RootNamespace>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="AWSSDK.S3" Version="4.1.1" />
    <PackageReference Include="AWSSDK.SecurityToken" Version="4.1.0.2" />
    <!-- Required at run time to resolve an IAM Identity Center (SSO) profile. -->
    <PackageReference Include="AWSSDK.SSO" Version="4.1.0.2" />
    <PackageReference Include="AWSSDK.SSOOIDC" Version="4.1.0.2" />
  </ItemGroup>
</Project>
Program.cs
using Amazon;
using Amazon.Runtime;
using Amazon.Runtime.CredentialManagement;
using Amazon.S3;
using Amazon.S3.Model;
using Amazon.SecurityToken;
using Amazon.SecurityToken.Model;

// Usage:
//   dotnet run                       -> default credential chain (set AWS_PROFILE=learn-sandbox first)
//   dotnet run -- --profile NAME     -> explicit named profile (local development only)
//   dotnet run -- --bucket NAME      -> also list up to 50 object keys under statements/
string? profileName = ArgValue(args, "--profile");
string? bucketName = ArgValue(args, "--bucket");

// 1. Region: explicit beats implicit. Fall back to us-east-1 for this lab.
var region = RegionEndpoint.GetBySystemName(Environment.GetEnvironmentVariable("AWS_REGION") ?? "us-east-1");

// 2. Credentials: null means "let the SDK walk the default chain".
AWSCredentials? credentials = null;
if (profileName is not null)
{
    var chain = new CredentialProfileStoreChain();
    if (!chain.TryGetAWSCredentials(profileName, out credentials))
    {
        Console.Error.WriteLine($"Profile '{profileName}' was not found in ~/.aws/config or ~/.aws/credentials.");
        return 2;
    }
}

// 3. One config object per client: Region, retry behaviour and a connect timeout.
var stsConfig = new AmazonSecurityTokenServiceConfig
{
    RegionEndpoint = region,
    RetryMode = RequestRetryMode.Standard, // the v4 default, stated for readers
    MaxErrorRetry = 3,                     // first attempt + up to 3 retries
    ConnectTimeout = TimeSpan.FromSeconds(3),
};
var s3Config = new AmazonS3Config
{
    RegionEndpoint = region,
    RetryMode = RequestRetryMode.Standard,
    MaxErrorRetry = 3,
    ConnectTimeout = TimeSpan.FromSeconds(3),
};

// 4. Clients are thread-safe and expensive to build: create once, reuse, dispose at the end.
using var sts = credentials is null
    ? new AmazonSecurityTokenServiceClient(stsConfig)
    : new AmazonSecurityTokenServiceClient(credentials, stsConfig);
using var s3 = credentials is null
    ? new AmazonS3Client(s3Config)
    : new AmazonS3Client(credentials, s3Config);

try
{
    // 5. Every async call takes a CancellationToken: this is your per-call deadline.
    using var callTimeout = new CancellationTokenSource(TimeSpan.FromSeconds(10));
    var me = await sts.GetCallerIdentityAsync(new GetCallerIdentityRequest(), callTimeout.Token);
    Console.WriteLine($"Account : {me.Account}");
    Console.WriteLine($"Caller  : {me.Arn}");
    Console.WriteLine($"Region  : {region.SystemName}");

    // 6. Paginator: the SDK follows ContinuationToken for you. Never reuse a paginator object.
    using var listTimeout = new CancellationTokenSource(TimeSpan.FromSeconds(30));
    var buckets = s3.Paginators.ListBuckets(new ListBucketsRequest { BucketRegion = region.SystemName, MaxBuckets = 100 });
    var count = 0;
    await foreach (var bucket in buckets.Buckets.WithCancellation(listTimeout.Token))
    {
        Console.WriteLine($"  bucket {bucket.BucketName} (created {bucket.CreationDate:yyyy-MM-dd})");
        count++;
    }
    Console.WriteLine($"{count} bucket(s) in {region.SystemName}.");

    if (bucketName is not null)
    {
        var request = new ListObjectsV2Request { BucketName = bucketName, Prefix = "statements/", MaxKeys = 50 };
        var page = await s3.ListObjectsV2Async(request, listTimeout.Token);
        // SDK v4: S3Objects is null (not empty) when nothing matches.
        foreach (var obj in page.S3Objects ?? [])
        {
            Console.WriteLine($"  {obj.Key,-60} {obj.Size,10:N0} bytes");
        }
        Console.WriteLine(page.IsTruncated == true ? "  ...more keys exist (use a paginator to read them all)" : "  (end of list)");
    }

    return 0;
}
catch (AmazonServiceException ex)
{
    // The service answered with an error: AccessDenied, NoSuchBucket, ThrottlingException...
    Console.Error.WriteLine($"AWS error {ex.ErrorCode} (HTTP {(int)ex.StatusCode}), request id {ex.RequestId}: {ex.Message}");
    return 1;
}
catch (AmazonClientException ex)
{
    // Nothing reached AWS: no credentials, expired SSO token, no Region, DNS/network failure.
    Console.Error.WriteLine($"SDK client error: {ex.Message}");
    Console.Error.WriteLine("If you use SSO, run: aws sso login --profile learn-sandbox");
    return 1;
}
catch (OperationCanceledException)
{
    Console.Error.WriteLine("The call did not finish before its deadline.");
    return 1;
}

static string? ArgValue(string[] args, string name)
{
    var i = Array.IndexOf(args, name);
    return i >= 0 && i + 1 < args.Length ? args[i + 1] : null;
}

This code was compiled against the package versions above with TreatWarningsAsErrors on.

Code explained line by line

  1. ArgValue(args, "--profile") reads optional switches. Production code would use System.CommandLine or the generic host. Plain parsing keeps the SDK in focus here.
  2. RegionEndpoint.GetBySystemName(...) turns "us-east-1" into the SDK's Region object. Setting the Region explicitly on every client is the habit that prevents "it wrote to the wrong Region" incidents. Reading it from AWS_REGION keeps it configurable without code changes.
  3. AWSCredentials? credentials = null. A null here means "do not pass credentials". The client then walks the default chain, which is what you want in AWS.
  4. CredentialProfileStoreChain.TryGetAWSCredentials looks up a named profile in ~/.aws/config and ~/.aws/credentials. For an SSO profile it returns credentials that read the SSO token cache and exchange it for role credentials. Use this only for local tools that must switch profiles at run time. Services should rely on the chain.
  5. AmazonSecurityTokenServiceConfig / AmazonS3Config are per-service subclasses of ClientConfig. Configuration is fixed when the client is created and cannot change afterwards.
  6. RetryMode = RequestRetryMode.Standard is already the v4 default. Writing it out documents intent for reviewers. Standard mode retries throttling errors, 5xx errors and some network errors with exponential back-off and jitter. It also keeps a retry quota: when many calls are failing, the client stops retrying so it does not pile more load on a struggling service.
  7. MaxErrorRetry = 3 means one attempt plus up to three retries. The documented Standard-mode default is 2 retries for most services and 10 for DynamoDB. Choose a number that fits your latency budget, not "as many as possible".
  8. ConnectTimeout bounds the TCP and TLS connect phase. A dead endpoint then fails in 3 seconds instead of waiting for the operating system's default.
  9. using var sts = .... Clients implement IDisposable and own an HttpClient connection pool. In a console app, create them once and dispose at exit. In ASP.NET Core you register them as singletons instead (next lesson).
  10. new CancellationTokenSource(TimeSpan.FromSeconds(10)) is the per-call deadline. The SDK's guide recommends the CancellationToken for timeouts on async calls. It covers the whole operation, including retries. When it fires you get OperationCanceledException (or its subclass TaskCanceledException).
  11. GetCallerIdentityAsync returns the account, user ID and ARN of whoever the credentials belong to. It needs no IAM permission.
  12. s3.Paginators.ListBuckets(...) returns a paginator. .Buckets is the "key result": an IAsyncEnumerable of S3Bucket. The paginator fetches the next page with ContinuationToken only when you iterate past the current one. .Responses gives you whole response objects instead. A paginator object can be enumerated only once.
  13. BucketRegion = region.SystemName asks S3 to return only buckets in that Region. MaxBuckets sets the page size. Both arrived when ListBuckets became paginated.
  14. WithCancellation(listTimeout.Token) passes the deadline into the async enumeration.
  15. page.S3Objects ?? [] is the v4 null-collection rule in one line. In v3 this was an empty list; in v4 it is null when nothing matched.
  16. page.IsTruncated == true. IsTruncated is bool? in v4. Comparing with true treats "unknown" as "no".
  17. catch (AmazonServiceException ex). The request reached AWS and AWS said no. ErrorCode is the machine-readable code (AccessDenied, NoSuchBucket). StatusCode is the HTTP status. RequestId is what AWS Support needs to trace the call. Service packages add subclasses such as AmazonS3Exception and NoSuchBucketException that you can catch more specifically.
  18. catch (AmazonClientException ex). The SDK could not even make the call: no credentials, an expired SSO token, no Region or a network failure. Treat these as configuration or infrastructure problems, not business errors.
  19. catch (OperationCanceledException). Your deadline fired. In an API this becomes a 503 or 504 with a ProblemDetails body. A long-running worker logs it and moves on.

Retries and money: a warning

Retries are safe for reads and for writes that are idempotent. The SDK retries PutObject because writing the same object twice gives the same result. It also retries a DynamoDB PutItem with a condition, because the condition makes a duplicate harmless.

Your own business operation may not be idempotent. If your code calls a partner's payment API after an SQS message arrives, and you wrap the whole handler in another retry loop, you can submit the payment twice. Rules for Harbourline:

  • Do not stack your own retry loop (for example Polly) on top of SDK retries for the same AWS call. Pick one layer: usually the SDK for AWS calls, and Polly for non-AWS HTTP calls.
  • Make every money-moving write idempotent with a client-supplied key (Idempotency-Key, DynamoDB conditional writes). See Module 4.
  • Use Adaptive retry mode only when you understand it. It adds client-side rate limiting after throttling, which suits batch jobs hammering one table more than user-facing APIs.

Paginator or explicit loop?

Use a paginator when…Use an explicit loop when…
You want every item and the code is not unit-tested against a mockYou want to unit-test with NSubstitute or Moq. Mocking ListObjectsV2Async is easy; mocking IS3PaginatorFactory and its async enumerables is not
The volume is bounded (dozens of pages at most)You need to stop early, return one page to an API caller with a nextToken, or checkpoint progress
Readability matters mostYou need to control the page size per request

The lab code in labs/m03-accounts-api/src/Harbourline.Accounts.Api/Data/DynamoDbAccountRepository.cs uses the explicit loop for exactly the testing reason.

SDK logging without leaking data

To see SDK activity in your normal ILogger output, add the AWSSDK.Extensions.Logging.ILoggerAdaptor package and call ConfigureAWSSDKLogging() on the ILoggerFactory at startup. Keep the AWSSDK category at Warning in production. Never turn on response logging (LogResponse / AWSConfigs.LoggingConfig.LogResponses) in an environment that handles customer data. Responses contain account items, statement keys and sometimes secrets.

Security considerations

⚠ Learning shortcut — not for production

The console app accepts --profile and reads profiles from your user folder. That is fine on a developer laptop. Services in AWS must never ship a profile name or a credentials file: they get credentials from the role attached to the EC2 instance, ECS task or Lambda function. Keep AWS:Profile only in appsettings.Development.json (next lesson), or use the AWS_PROFILE environment variable locally.

  • No long-term access keys. Harbourline's rule is that IAM users with access keys do not exist outside break-glass procedures. Locally you use SSO; in AWS you use roles; in CI you use OIDC federation (Module 9).
  • Explicit credentials override roles. An AWS_ACCESS_KEY_ID environment variable left in a container definition beats the task role. Scan task definitions and Beanstalk environment properties for it.
  • Enforce IMDSv2. The v4 SDK only uses IMDSv2. Requiring IMDSv2 on instances (hop limit 1 unless containers need more) also blocks the server-side request forgery pattern that stole credentials through IMDSv1.
  • Pin the Region for data residency. For the EU subsidiary, the client's Region is part of your data-residency control. Do not leave it to whatever profile a build agent has.
  • Log request IDs, not payloads. Request IDs are safe to log and essential for investigations. Payloads may contain personal data under UK GDPR.
  • Keep packages patched. The SDK is a security-relevant dependency. Let Dependabot or Renovate raise 4.x patch updates weekly.

Banking use case

Harbourline's Statements Service produces monthly PDF statements for 1.2 million fictional current accounts. A nightly .NET 10 worker on ECS Fargate:

  1. Lists the statement objects already generated under statements/<account-id>/ with an explicit pagination loop, so it can checkpoint after each page.
  2. Writes new PDFs with PutObjectAsync and server-side encryption with a KMS key. The SDK retries these safely because each object key is deterministic.
  3. Uses the task role, which allows only s3:PutObject and s3:ListBucket on the statements bucket and kms:GenerateDataKey on one key.
  4. Sets MaxErrorRetry = 3 and a 15-second per-call deadline. A slow S3 call fails fast, the worker records the account in a retry queue and carries on, and the batch finishes inside its window.
  5. Logs one structured line per page with the S3 request ID and the number of objects, never the customer's name.

The EU subsidiary's run uses a second client pinned to eu-west-1 and a different bucket. Whether that arrangement meets the subsidiary's data-residency obligations is a question for Harbourline's legal and compliance teams. The SDK settings only enforce the decision they make.

Common mistakes

  • Creating a new client per request → it happens because clients look cheap → each one builds its own HttpClient, so you exhaust sockets and repeat credential lookups → create once and reuse (singleton).
  • Leaving out the Region → works on your laptop because your profile has a Region → fails or writes elsewhere on a build agent → set RegionEndpoint or AWS:Region explicitly.
  • .Result or .Wait() on SDK calls → copied from old samples → thread-pool starvation under load in ASP.NET Core → await everywhere.
  • Treating null as "the API returned nothing" after upgrading to v4 → NullReferenceException → ?? [] and == true patterns, or turn on AWSConfigs.InitializeCollections temporarily while you fix the code.
  • Reusing a paginator → the second enumeration yields nothing or throws → create a new paginator each time.
  • Catching Exception and retrying everything → you retry AccessDenied and validation errors forever → catch AmazonServiceException, check ErrorCode, and retry only transient errors (the SDK already did that for you).
  • Missing AWSSDK.SSO/AWSSDK.SSOOIDC → works for colleagues with old access keys, fails for you with SSO → add both packages to any app you run locally with SSO.
  • Using an S3 client in us-east-1 for a bucket in eu-west-2 → v3 used to follow the redirect; v4 does not → one client per Region.

Troubleshooting

SymptomLikely causeFix
AmazonClientException: Failed to resolve AWS credentials listing several providersNo profile, environment variables or role reachableSet AWS_PROFILE locally; attach a role in AWS. Read each provider's message in the exception: it tells you what was tried
Error mentioning the SSO token being expired or missingThe SSO session ended (typically after 8–12 hours, depending on your organisation's settings)aws sso login --profile learn-sandbox
Could not load file or assembly 'AWSSDK.SSOOIDC' (or similar) with an SSO profileSSO packages not referencedAdd AWSSDK.SSO and AWSSDK.SSOOIDC
No RegionEndpoint or ServiceURL configuredNo Region anywhere in the resolution orderSet RegionEndpoint, AWS_REGION or the profile's region
AccessDenied (403) with a request IDIAM policy, bucket policy, SCP or KMS key policy denies the callCheck the caller ARN with get-caller-identity, then the policies. CloudTrail shows the denied event
PermanentRedirect or AuthorizationHeaderMalformed from S3Client Region differs from the bucket's RegionCreate the client for the bucket's Region
NullReferenceException after upgrading to v4A response collection is now nullNull-coalesce; see the v4 changes table
TaskCanceledException after exactly your deadlineYour CancellationToken fired: network path, throttling or a slow serviceCheck retries in SDK logs (AWSSDK category at Debug locally); check NAT/VPC endpoint routing in AWS
ThrottlingException / SlowDown surfacing to callersSustained request rate above the service limit; retries exhaustedReduce concurrency, add jittered back-off in the caller, request a quota increase or redesign the access pattern
SignatureDoesNotMatch or clock-skew errorsMachine clock is wrongFix NTP; the SDK corrects small skews automatically

Cost and cleanup

This lesson is free:

  • sts:GetCallerIdentity has no charge.
  • S3 ListBuckets and ListObjectsV2 are LIST requests, billed at a fraction of a cent per thousand requests in us-east-1. A few runs cost effectively nothing.
  • No resources are created, unless you deploy the optional IaC policy, and IAM policies are free.

Cleanup:

# If you deployed the optional policy with Terraform:
terraform destroy
# or with CDK:
cdk destroy SdkTourPolicyStack --profile learn-sandbox

# End your SSO session when you finish for the day.
aws sso logout

# Confirm nothing tagged for this lesson remains.
aws resourcegroupstaggingapi get-resources --tag-filters Key=Lab,Values=m03-sdk \
  --region us-east-1 --profile learn-sandbox --query "ResourceTagMappingList[].ResourceARN"

IAM policies are global and do not appear in the Regional tagging API. Check IAM → Policies → Customer managed for hl-m03-sdk-tour if you created it.

Exercises

  • 🟢 Guided: Run the console app with AWS_PROFILE=learn-sandbox, then with --profile learn-sandbox, then with neither (expect the credential error). For each run, write down which link in the credential chain was used.
  • 🟡 Challenge: Change the bucket listing to use .Responses instead of .Buckets, and print the page number, the number of buckets per page and the ContinuationToken for each page. Set MaxBuckets = 1 to force several pages.
  • 🔴 Live problem: A colleague reports: "After we moved the reconciliation job to v4, it started failing only in the EU subsidiary account, with PermanentRedirect on GetObject. The UK account is fine." The job creates one AmazonS3Client with no Region, and the EU bucket is in eu-west-1. Diagnose and propose a fix that also makes the Region an auditable setting.

Assignment

Add a docs/sdk-conventions.md page to your portfolio repository that a new Harbourline developer could follow. It should cover: how to authenticate locally (SSO profile, AWS_PROFILE), the client lifetime rule, the default retry and deadline values for user-facing and batch code, how to handle AmazonServiceException, the paginator-versus-loop rule, and the v4 null rules. Include the console app as a working sample.

Solutions

Knowledge check

Interview questions

  • "How does your .NET service authenticate to AWS?" — It has no keys. The SDK's default credential chain finds the IAM role attached to the compute: the container credentials endpoint on ECS, IMDSv2 on EC2, environment variables injected by Lambda. Locally, developers use an IAM Identity Center profile through AWS_PROFILE. Explicit keys are forbidden because they override roles.
  • "What changed in SDK v4 that you had to handle?" — Nullable value types and null collections (we added null-coalescing and fixed tests), Standard retry mode by default, Region-strict S3 clients, IMDSv2 only, and the move of logging integrations into adaptor packages.
  • "How do you configure retries for a payments API?" — Few SDK retries (two or three) with a tight per-call deadline via CancellationToken. No second retry layer around AWS calls. Idempotency keys or conditional writes for anything that moves money.
  • "A call fails intermittently with throttling. What do you do?" — Confirm with CloudWatch metrics and SDK logs. Check for retry amplification (several retry layers). Reduce concurrency or batch requests. Consider Adaptive retry mode for batch jobs. Request a quota increase only after the access pattern is sane.

Certification practice

Completion checklist

  • I ran the console app with my SSO profile and saw my account and role ARN
  • I can draw the credential chain from memory and say which link wins in ECS, on EC2 and on my laptop
  • I set retries, a connect timeout and a per-call deadline, and can justify the values
  • I used a paginator and an explicit loop, and know when each is appropriate
  • I know the five v4 changes most likely to break an upgrade
  • I logged out of SSO and removed any optional policy I created

Key takeaways

  • A service client handles endpoints, SigV4 signing, retries and parsing. Create it once, configure it explicitly (Region, retries, connect timeout) and reuse it.
  • The default credential chain lets the same binary use SSO locally and roles in AWS. Anything explicit (constructor or environment keys) overrides the role.
  • Use a CancellationToken as your deadline. Keep SDK retries modest and never stack a second retry layer on AWS calls that move money.
  • Paginators are convenient and single-use. Explicit loops are easier to test and to stop early.
  • v4 returns null collections and nullable values. Most upgrade bugs come from that.

Further reading

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page