Interview preparation: .NET cloud developer
Prepare for a .NET cloud developer interview: explain SDK v4 behaviour, hosting choices, messaging and idempotency, and secure coding on AWS, and write a small idempotent SQS consumer live while explaining every decision.
Assumes: .NET 10 LTS · AWS SDK for .NET v4 (AWSSDK.SQS 4.x) · AWS CLI v2 · Region us-east-1. Verified against official sources on 2026-11-18.
Learning objectives
By the end of this lesson you can:
- Describe what a .NET cloud developer interview tests: code, design judgement, AWS knowledge and how you reason about failure.
- Explain the AWS SDK for .NET v4 behaviours that surprise people: null collections, nullable value types, the credential chain, explicit Regions, retries and client lifetime.
- Choose and defend a hosting option (Fargate, Lambda, Elastic Beanstalk) for a given banking workload.
- Explain at-least-once delivery, visibility timeout, dead-letter queues and idempotency, and write a consumer that is safe under duplicates and failures.
- Review a short piece of C# for AWS security problems: secrets, injection, SSRF, logging of personal data and token validation.
- Run a 45-minute live-coding or pair-programming round calmly: clarify, design, code in small steps, test and explain trade-offs.
Prerequisites
- Interview preparation: cloud engineer for the answer shapes and story bank used here
- AWS SDK for .NET v4 in depth and Dependency injection for AWS clients
- Lambda with C#, Deploying to Fargate and Hosting options compared
- SQS in depth, Dead-letter queues and redrive and Idempotency in .NET
- Secure code review for .NET
- Tools: .NET 10 SDK (or Docker with the SDK image) for the code sample, which runs locally with no AWS account
Why this matters in banking
A payment service written by a developer who has not internalised "messages arrive at least once" will, sooner or later, post a payment twice. A service written by someone who reads documentation but never runs failure cases will pass its review and fail on the first timeout. Hiring managers at banks know this, so a .NET cloud developer interview tends to probe three habits:
- Do you design for failure? Retries, duplicates, partial failure, timeouts, and what the customer sees when something breaks.
- Do you write code a reviewer can trust? Small, testable units, no secrets in code, safe logging, explicit configuration.
- Do you understand what the platform does for you and what it does not? For example, SQS retries a message for you, but does not make your handler idempotent.
Your advantage as a banking .NET developer is that you already think in transactions, audit trails and change control. This lesson shows you how to say that in AWS terms and how to write the code to match.
The idea in plain language
The interview has a typical shape, and each part wants something different from you:
| Part | Typical content | What gets you through |
|---|---|---|
| Fundamentals | C#, async, DI, testing, REST, SQL, then AWS basics | Short correct answers, honest limits |
| AWS developer questions | SDK, Lambda, SQS/SNS, DynamoDB, IAM for apps | Explaining behaviour under failure, not listing features |
| Live coding or pair programming | A small service, a handler, a bug fix | Clarify, small steps, tests, thinking aloud |
| Design discussion | A payment, notification or document flow | Constraints first, trade-offs, security and failure modes |
| Behavioural | Stories about quality, incidents, collaboration | Story shape from the previous lesson |
Analogy: A code interview is a driving test, not a theory test. The examiner watches mirrors and indicators: do you check what can go wrong before you pull out? A candidate who writes working code but never mentions failure is like a driver who is fast and never looks.
The four phrases to say while coding
Say these aloud and you will cover most of what an interviewer is scoring:
- "Let me clarify the requirement and the failure behaviour." (Scope.)
- "I will start with the simplest version and a test." (Small steps.)
- "This can be delivered twice, so I make it idempotent by…" (Failure awareness.)
- "What I have not handled is… and I would add it next." (Honesty about limits.)
What problem the interview solves
For the employer, it separates developers who can write C# from developers who can run C# safely in a distributed, regulated environment. For you, it is a way to check the team's engineering standards: how they test, review and release. Honest limits:
- A 45-minute coding exercise cannot show everything. Interviewers weight clear thinking and communication heavily. A half-finished but well-structured solution often scores better than a rushed complete one.
- AWS service trivia is a poor signal but is still asked. Anchor to purpose and failure behaviour, and say when you would check the documentation.
- Expect language and framework questions too (async/await, DI lifetimes, EF Core). Do not neglect them while studying AWS.
Microsoft / Azure comparison
| You know (Microsoft/Azure/.NET) | AWS equivalent | What to say in an interview |
|---|---|---|
DefaultAzureCredential | The AWS SDK credential provider chain (environment, SSO profile, container or instance role) | Same idea: the code does not choose credentials, the environment does. Locally I use an SSO profile; in ECS the task role |
Azure.Messaging.ServiceBus with dead-lettering | AWSSDK.SQS with a dead-letter queue and maxReceiveCount | SQS gives at-least-once delivery; I must make handlers idempotent. FIFO queues add deduplication and ordering, with throughput trade-offs |
| Azure Functions isolated worker | Lambda dotnet10 (Annotations or executable assembly) | One handler per function; configuration on the function; cold starts managed with memory, ARM64, Native AOT or SnapStart |
| Azure App Service or AKS | ECS on Fargate, or Elastic Beanstalk for lift-and-shift | Container-first; the task role replaces managed identity |
| Cosmos DB SDK retries | SDK retry mode Standard (default in v4) | Explicit timeouts and a limited number of attempts; retries must respect idempotency |
| Azure App Configuration / Key Vault references | Systems Manager Parameter Store and Secrets Manager | Fetch at start-up or with caching; never bake secrets into images or logs |
ILogger and Application Insights | ILogger to CloudWatch Logs with structured JSON, and X-Ray or OpenTelemetry | Include a correlation ID; never log personal data or tokens |
| Polly | Microsoft.Extensions.Http.Resilience (built on Polly v8) plus SDK retries | Retry at one layer only to avoid multiplying attempts |
| EF Core with transactions | EF Core with RDS or Aurora; transactional outbox for events | A database write and a message publish are not atomic; the outbox pattern fixes that |
Architecture
A message-processing path, and where duplicates come from
Duplicates come from three places, and you should name all three: the producer retries (a new message with the same business key), the queue redelivers (visibility timeout expired or the consumer crashed before delete), and the consumer is scaled out so two instances see the same work. The idempotency key must therefore be the business key (the payment ID), not the SQS message ID.
The review lens for security
Step-by-step implementation
Refresh the C# and .NET core
Prepare short answers to: async/await and what happens to the thread; Task versus ValueTask; DI lifetimes (singleton, scoped, transient) and why an AWS client is a singleton; IHttpClientFactory; CancellationToken; records and nullable reference types; exceptions versus result types; unit testing with fakes. You will be asked at least one, and gaps here hurt more than AWS gaps.
Learn the SDK behaviours that cause bugs
Be able to say, from memory: create one client per Region per application and reuse it; set the Region explicitly; v4 returns null collections and nullable value types; the credential chain finds credentials (environment, profile, container role); the default retry mode is Standard with exponential backoff and a retry quota; use paginators for list calls; catch AmazonServiceException and inspect ErrorCode and StatusCode.
Know how to choose hosting
Give a one-sentence rule for each, with a banking example: Fargate for long-running APIs and consumers (steady load, container control), Lambda for event-driven, spiky or short tasks (statement rendering, webhook handlers), Elastic Beanstalk for lift-and-shift of existing apps when containers are not ready. Say what would change your mind: latency needs, runtime limits, cost at steady load, operational skills.
Master the messaging story
Practise explaining: at-least-once delivery; visibility timeout and why it must exceed your processing time; maxReceiveCount and the dead-letter queue; redrive; partial batch failures in Lambda event sources (report failed item IDs rather than failing the whole batch); standard versus FIFO queues; the outbox pattern. Link every point to what the customer experiences if you get it wrong.
Practise the live-coding routine
Do the exercise in the Complete code section without looking at the solution. Time yourself: 10 minutes to clarify and design, 25 to code and test, 10 to review failures. Then practise explaining it aloud.
Prepare a review round
Take the "security review" checklist below and apply it to your own code from a lab. Interviewers sometimes hand you a short snippet and ask "what is wrong with this?". The checklist is the answer.
Core question bank with model answers
SDK and runtime
- How do you create and use AWS clients in an ASP.NET Core app? Register one client per service as a singleton through DI (using the SDK's configuration extensions), set the Region explicitly, and let the environment supply credentials. Clients are thread-safe and expensive to create; creating one per request wastes connections.
- What changed in SDK v4 that you had to handle? Collections that are empty come back as
null, many value types are nullable, the default retry mode is Standard, and S3 clients no longer follow cross-Region redirects. I use null-coalescing,== truecomparisons and one client per Region. - How do credentials work locally and in production? Locally an SSO profile (
AWS_PROFILE), in ECS the task role, in Lambda the execution role, and never access keys in code or config. The code is the same; the environment changes. - How do you handle a throttling error? The SDK retries with backoff. I set explicit timeouts, limit retries, make writes idempotent and reduce concurrency if throttling persists, rather than adding another retry layer.
Hosting
- Lambda or Fargate for a statement-rendering job? Lambda if it is event-driven, bursty and finishes within the timeout; Fargate if it needs long runs, large memory, or steady throughput that makes Lambda more expensive. State the trade-off and what you would measure.
- How do you reduce Lambda cold starts for .NET? Right-size memory, use ARM64, enable ReadyToRun, consider Native AOT where the code allows it, and SnapStart or provisioned concurrency for latency-sensitive functions. Measure before and after.
- What does a health check do on ECS behind a load balancer? The load balancer sends requests to the check path; failures remove the task from rotation and can lead to the task being replaced. A wrong path or a check that calls the database can cause restart loops.
Messaging and data
- Explain at-least-once delivery and what you do about it. A message may be delivered more than once, so each handler must be idempotent: claim a business key with a conditional write, do the work, mark it done, then delete the message.
- What is the visibility timeout and how do you choose it? The time a received message is hidden from other consumers. Set it longer than the worst-case processing time (plus margin), and extend it for long work, or messages are processed twice.
- What goes to the dead-letter queue and what then? Messages that exceed
maxReceiveCount. I alarm on queue depth, inspect messages (they may contain personal data, so access is restricted), fix the cause and redrive. - How do you write to a database and publish an event atomically? You cannot do both in one transaction across systems, so use the transactional outbox: write the event to an outbox table in the same transaction, and a relay publishes it with retries and idempotency.
- DynamoDB conditional writes: when do you use them? For idempotency keys (
attribute_not_exists), optimistic locking (version attribute) and invariants such as not overdrawing a balance.
Security and secrets
- Where do secrets live and how does the app read them? In Secrets Manager (or Parameter Store for non-secret config), read at start-up or cached with refresh, with the task or function role granted only that secret. Never in environment files, images or logs.
- What are the usual AWS-specific security bugs in C# services? Wildcard IAM permissions, logging tokens or card data, user-supplied URLs (SSRF) reaching the metadata endpoint or internal services, string-built SQL, missing JWT audience or issuer validation, and public S3 buckets from copied samples.
Security review checklist for a snippet
Apply in this order. It is the same checklist you would use in a pull request.
| Look for | Question to ask | Typical fix |
|---|---|---|
| Hard-coded keys or connection strings | Where do these come from in production? | Secrets Manager plus role |
HttpClient calls built from request data | Can a caller point this at an internal address? | Allow-list hosts, block metadata and private ranges, no redirects to unknown hosts |
| String-concatenated SQL | Is every value a parameter? | Parameterised queries or EF Core |
catch that returns or logs the whole exception | Does it leak details or personal data? | Log a code and correlation ID; return ProblemDetails without internals |
Console.WriteLine of request bodies | Are PII, tokens or card numbers included? | Structured logging with redaction |
| JWT handling | Are issuer, audience, expiry and signature validated? | Use the framework's JWT bearer validation with explicit parameters |
| AWS calls with no timeout or retries | What happens when a dependency stalls? | Explicit timeouts, bounded retries, circuit breaker |
New AmazonS3Client() per call | What does this cost under load? | One client via DI |
IAM "Resource": "*" | Which resources does this really need? | Specific ARNs and conditions |
AWS Console instructions
Not applicable: this lesson is about code and reasoning, not console steps. If asked about tooling, be ready to describe where you would inspect a queue (Console → SQS → select queue → Monitoring and Send and receive messages) and a function (Console → Lambda → Monitor → View CloudWatch logs).
AWS CLI
Commands a developer should know for diagnosing the messaging path.
# How deep is the queue and how many messages are in flight? (backlog and stuck-consumer check)
aws sqs get-queue-attributes \
--queue-url https://sqs.us-east-1.amazonaws.com/111122223333/hb-payments \
--attribute-names ApproximateNumberOfMessages ApproximateNumberOfMessagesNotVisible \
--profile learn-sandbox
# What are the redrive settings? Look for maxReceiveCount and the dead-letter queue ARN.
aws sqs get-queue-attributes \
--queue-url https://sqs.us-east-1.amazonaws.com/111122223333/hb-payments \
--attribute-names RedrivePolicy VisibilityTimeout --profile learn-sandbox
# Peek at a dead-letter message (it becomes invisible for the visibility timeout).
aws sqs receive-message \
--queue-url https://sqs.us-east-1.amazonaws.com/111122223333/hb-payments-dlq \
--max-number-of-messages 1 --visibility-timeout 30 --profile learn-sandbox
# Tail the consumer's logs.
aws logs tail /hb/payments-consumer --since 15m --follow --profile learn-sandboxInfrastructure as Code
Not applicable: no infrastructure is deployed in this lesson. If an interviewer asks how you would provision the queue, say: a standard queue with a dead-letter queue, maxReceiveCount of about 5, a visibility timeout greater than your worst-case processing time, encryption with a KMS key, and an alarm on the dead-letter queue depth, all in CDK or Terraform. The queue pattern is built in Module 8.
Complete code
An idempotent payment consumer is a realistic live-coding task and shows every habit that matters. The program below runs locally: it simulates five deliveries (a duplicate, one failure followed by a redelivery, and two normal payments) against an in-memory idempotency store. A real SQS loop is included and compiles, but the demo does not need AWS. The store's contract mirrors a DynamoDB conditional write.
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="AWSSDK.SQS" Version="4.*" />
</ItemGroup>
</Project>using System.Text.Json;
using Amazon.SQS;
using Amazon.SQS.Model;
// Local demo: no AWS account needed. Five deliveries of three payments:
// p-1 delivered twice (duplicate), p-2 fails once then succeeds, p-3 is normal.
var clock = () => DateTimeOffset.UtcNow;
var store = new InMemoryIdempotencyStore(clock, TimeSpan.FromSeconds(30));
var posted = new List<string>();
int p2Attempts = 0;
var handler = new PaymentMessageHandler(store, async payment =>
{
if (payment.PaymentId == "p-2" && ++p2Attempts == 1)
throw new InvalidOperationException("core banking timeout");
posted.Add(payment.PaymentId);
await Task.CompletedTask;
});
string[] deliveries =
[
"""{"paymentId":"p-1","amount":25.00,"currency":"GBP"}""",
"""{"paymentId":"p-1","amount":25.00,"currency":"GBP"}""", // SQS at-least-once: same message again
"""{"paymentId":"p-2","amount":80.10,"currency":"GBP"}""", // first attempt fails
"""{"paymentId":"p-2","amount":80.10,"currency":"GBP"}""", // redelivered after visibility timeout
"""{"paymentId":"p-3","amount":9.99,"currency":"GBP"}""",
];
foreach (var body in deliveries)
{
var outcome = await handler.HandleAsync(body);
Console.WriteLine($"{outcome,-10} {body}");
}
Console.WriteLine($"Posted payments: {string.Join(", ", posted)}");
return posted.Distinct().Count() == posted.Count ? 0 : 1;
public sealed record PaymentRequest(string PaymentId, decimal Amount, string Currency);
public enum HandlerOutcome { Processed, Duplicate, Retry }
public enum ClaimResult { Claimed, AlreadyDone, InProgressElsewhere }
public interface IIdempotencyStore
{
// Claim the key for processing. In DynamoDB this is one conditional PutItem:
// attribute_not_exists(pk) OR (#status = :inprogress AND leaseExpires < :now)
Task<ClaimResult> TryClaimAsync(string key);
Task CompleteAsync(string key);
Task ReleaseAsync(string key); // give the claim back after a failure so a retry can proceed
}
public sealed class InMemoryIdempotencyStore(Func<DateTimeOffset> clock, TimeSpan lease) : IIdempotencyStore
{
private readonly object _gate = new();
private readonly Dictionary<string, (bool Done, DateTimeOffset LeaseExpires)> _items = new();
public Task<ClaimResult> TryClaimAsync(string key)
{
lock (_gate)
{
var now = clock();
if (_items.TryGetValue(key, out var item))
{
if (item.Done) return Task.FromResult(ClaimResult.AlreadyDone);
if (item.LeaseExpires > now) return Task.FromResult(ClaimResult.InProgressElsewhere);
}
_items[key] = (false, now + lease);
return Task.FromResult(ClaimResult.Claimed);
}
}
public Task CompleteAsync(string key)
{
lock (_gate) _items[key] = (true, DateTimeOffset.MaxValue);
return Task.CompletedTask;
}
public Task ReleaseAsync(string key)
{
lock (_gate) _items.Remove(key);
return Task.CompletedTask;
}
}
public sealed class PaymentMessageHandler(IIdempotencyStore store, Func<PaymentRequest, Task> postPayment)
{
private static readonly JsonSerializerOptions Json = new(JsonSerializerDefaults.Web);
public async Task<HandlerOutcome> HandleAsync(string body)
{
var payment = JsonSerializer.Deserialize<PaymentRequest>(body, Json)
?? throw new InvalidDataException("empty payment message");
// The business key (not the SQS message ID) is the idempotency key:
// a producer retry creates a NEW SQS message with the same payment.
switch (await store.TryClaimAsync(payment.PaymentId))
{
case ClaimResult.AlreadyDone: return HandlerOutcome.Duplicate;
case ClaimResult.InProgressElsewhere: return HandlerOutcome.Retry;
}
try
{
await postPayment(payment);
await store.CompleteAsync(payment.PaymentId);
return HandlerOutcome.Processed;
}
catch (Exception)
{
// Production code logs the exception here with the correlation ID (never the full payload).
await store.ReleaseAsync(payment.PaymentId);
return HandlerOutcome.Retry; // leave the message; it reappears after the visibility timeout
}
}
}
public static class SqsConsumer
{
public static async Task RunAsync(IAmazonSQS sqs, string queueUrl,
PaymentMessageHandler handler, CancellationToken ct)
{
while (!ct.IsCancellationRequested)
{
var resp = await sqs.ReceiveMessageAsync(new ReceiveMessageRequest
{
QueueUrl = queueUrl,
MaxNumberOfMessages = 10,
WaitTimeSeconds = 20, // long polling: fewer empty receives, lower cost
}, ct);
foreach (var msg in resp.Messages ?? new List<Message>()) // v4: null when empty
{
HandlerOutcome outcome;
try { outcome = await handler.HandleAsync(msg.Body); }
catch (Exception) { continue; /* poison message: do not delete, the DLQ redrive policy handles it */ }
if (outcome is HandlerOutcome.Processed or HandlerOutcome.Duplicate)
await sqs.DeleteMessageAsync(queueUrl, msg.ReceiptHandle, ct);
// Retry: do nothing; the message becomes visible again after the visibility timeout.
}
}
}
}Run it:
dotnet runExpected output:
Processed {"paymentId":"p-1","amount":25.00,"currency":"GBP"}
Duplicate {"paymentId":"p-1","amount":25.00,"currency":"GBP"}
Retry {"paymentId":"p-2","amount":80.10,"currency":"GBP"}
Processed {"paymentId":"p-2","amount":80.10,"currency":"GBP"}
Processed {"paymentId":"p-3","amount":9.99,"currency":"GBP"}
Posted payments: p-1, p-2, p-3Code explained line by line
HandlerOutcomehas three results because a consumer has three decisions: delete the message (processed or duplicate), or leave it (retry). Naming them makes the decision visible and testable.IIdempotencyStoreis the seam for the real store. The comment shows the DynamoDB condition: write the item only if it does not exist, or if it isin progresswith an expired lease. That condition is evaluated atomically by DynamoDB, which is what makes it safe with several consumers.- The lease (
TimeSpan.FromSeconds(30)) covers a crash: a consumer that claimed the key and died leaves an in-progress record whose lease eventually expires, so another consumer can take over. Without a lease, a crash after claiming would block the payment forever. - Business key.
payment.PaymentIdis the idempotency key. The SQS message ID changes when a producer retries and sends a new message, so it cannot catch producer-side duplicates. TryClaimAsyncresults.AlreadyDonemeans a previous attempt finished, so this is a duplicate and the message can be deleted safely.InProgressElsewheremeans another consumer is working on it, so we leave the message and let it reappear.try/catcharound the work. On failure we release the claim so the redelivered message can try again, and returnRetryso the message is not deleted. If we deleted on failure we would lose the payment.- Complete after the side effect.
CompleteAsyncruns only afterpostPaymentsucceeded. A crash between the payment andCompleteAsyncis the remaining gap: the downstream system must also be idempotent (for example, accept the same payment ID twice and ignore the second). Mention this in an interview; it shows you know idempotency is end to end. SqsConsumer. Long polling (WaitTimeSeconds = 20) reduces empty receives and cost.resp.Messages ?? new List<Message>()is the v4 null-collection rule. A poison message (unparseable) is not deleted, so aftermaxReceiveCountreceives, SQS moves it to the dead-letter queue.- Deleting only on success or duplicate. Deletion is the commit point for the queue.
- Exit code. The demo returns
1if any payment was posted twice, so the demonstration doubles as a test.
Security considerations
⚠ Learning shortcut — not for production
The in-memory store works within one process and is lost on restart, so it does not protect against duplicates across several tasks. Production needs a shared store with a conditional write (DynamoDB, with a TTL on completed items) and the downstream payment call must accept an idempotency key as well. The demo also swallows exceptions without logging; production logs a correlation ID and an error code, never the payload.
- Message bodies are sensitive. Queues and dead-letter queues hold payment data. Encrypt queues with KMS, restrict who can receive from the DLQ, and set message retention to the shortest period that suits your recovery needs.
- Validate input from queues too. A message is untrusted input from the consumer's point of view. Validate types, ranges and currency codes before acting.
- Least privilege for the consumer role.
sqs:ReceiveMessage,sqs:DeleteMessage,sqs:ChangeMessageVisibilityandsqs:GetQueueAttributeson one queue, plus the DynamoDB actions on one table, pluskms:Decrypton the queue key. Nothing withResource: "*". - Logging. Log the payment ID and correlation ID, not the account number or amount of identifiable customers where avoidable.
- Do not claim more than you have. Idempotency reduces duplicates; it does not by itself make a system compliant, audit-complete or reconciled. Say what else is needed (audit events, reconciliation, reversals).
Banking use case
Harbourline's payments team processes outbound payments from a queue. A developer interviewing for the team is given 45 minutes and this prompt: "Write the consumer for payment messages. They are delivered at least once. The core banking call sometimes times out. We must never pay twice."
A strong candidate begins with questions: "Is there an ID on the message that is stable across producer retries? Does core banking accept an idempotency key? What should happen to a message that fails five times?" Then they write the smallest handler with a test for the duplicate case, add the failure case (release the claim, leave the message), explain the lease for crashed consumers, and say what is not covered: the gap between posting and marking complete, and the need for a downstream key. They finish by naming the operational pieces: dead-letter queue with an alarm, visibility timeout longer than the core banking timeout, an audit event for each decision and a reconciliation report. The interviewer does not need the code to be perfect; the sequence of questions and failure cases is the signal.
Common mistakes
- Using the SQS message ID as the idempotency key. Why: it looks unique. Avoid: use the business key, because producer retries create new messages.
- Deleting the message before the work is done. Why: wanting to avoid reprocessing. Avoid: delete only after success or confirmed duplicate.
- Visibility timeout shorter than processing time. Why: default values. Avoid: set it above worst-case processing time and extend for long jobs.
- Retrying at several layers. Why: SDK, HTTP client, Lambda and SQS all retry. Avoid: choose one layer and set explicit limits.
- Catching
Exceptionand doing nothing. Why: wanting the loop to continue. Avoid: log with a correlation ID and classify retryable versus poison. - Creating a client per call. Why: snippets do. Avoid: register singletons in DI.
- Ignoring null collections in v4. Why: v3 habits. Avoid:
?? []and== true. - Forgetting the lease. Why: only the happy path was considered. Avoid: ask "what if the consumer crashes after claiming?"
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| Same payment processed twice | Idempotency key is the message ID, or a non-atomic check-then-write | Use the business key and a conditional write |
| Messages processed again and again | Visibility timeout shorter than processing, or consumer not deleting | Raise the timeout, delete after success, check errors |
| Messages go straight to the DLQ | maxReceiveCount too low, or a poison message | Inspect the DLQ; fix the parser or data; redrive |
NullReferenceException on Messages | v4 returns null for empty collections | Use resp.Messages ?? [] |
| Payments stuck "in progress" | Consumer crashed after claiming and there is no lease expiry | Add a lease with an expiry time |
| Queue depth grows while consumers look healthy | Concurrency too low, long handler, or throttling downstream | Scale consumers, shorten handlers, check downstream limits |
AccessDenied on ReceiveMessage | Role lacks the SQS action, or the KMS key policy blocks the role | Add the action on the queue ARN and kms:Decrypt on the key |
| Interview code runs but the interviewer is uneasy | Failure cases were never mentioned | Narrate duplicates, crashes, poison messages and timeouts before you finish |
Cost and cleanup
Cost: free. The sample runs locally and needs no AWS account. If you point the consumer at a real SQS queue in the sandbox, expect cents: SQS has a monthly free allowance of requests ⚠ VERIFY, and long polling keeps request counts down. Delete the queue after the exercise.
# Delete the sandbox queues if you created them for practice.
aws sqs delete-queue --queue-url https://sqs.us-east-1.amazonaws.com/111122223333/hb-payments-practice \
--profile learn-sandbox
aws sqs delete-queue --queue-url https://sqs.us-east-1.amazonaws.com/111122223333/hb-payments-practice-dlq \
--profile learn-sandbox
# Remove build output from the local sample.
rm -rf bin objExercises
- 🟢 Guided: Run the sample. Change the delivery list so
p-3is delivered twice and confirm only one payment is posted. Then change the lease to 0 seconds and describe what could go wrong. - 🟡 Challenge: Write a DynamoDB-backed
IIdempotencyStoreusingPutItemRequestwith aConditionExpression. Keep the interface unchanged and add a TTL attribute for completed items. Explain how you would test it without AWS (a fake client or DynamoDB Local). - 🔴 Live problem: The interviewer says: "Our Lambda consumer processes SQS batches of ten. One poison message makes the whole batch retry, so nine good payments are processed again and again until the DLQ." Diagnose and propose a fix, including the Lambda event source setting and code change, and what you would check about idempotency.
Assignment
Add a "developer interview kit" to your portfolio: the idempotent consumer with tests (duplicate, failure-then-success, crash with expired lease), a one-page explanation of the design with a sequence diagram, a list of the five SDK v4 behaviours to remember, and the security review checklist applied to your own code. Use fictional data only.
Solutions
If p-3 appears twice, the second delivery returns Duplicate and posted still contains one p-3, so the exit code stays 0. With a lease of 0 seconds, an in-progress claim is treated as expired immediately, so a second consumer that sees the same message while the first is still working can claim it too. Both would post the payment. The lease must be longer than the worst-case handler time (plus a margin), or the claim loses its purpose.
Claim with PutItemAsync and ConditionExpression = "attribute_not_exists(pk) OR (#s = :inprogress AND leaseExpires < :now)", storing status = in-progress and leaseExpires. Catch ConditionalCheckFailedException: then read the item and return AlreadyDone or InProgressElsewhere. CompleteAsync updates status = done and sets a TTL epoch (for example 7 days) so old keys expire. ReleaseAsync deletes the item if its status is still in-progress. Test it with a fake IAmazonDynamoDB that throws ConditionalCheckFailedException on the second claim, and with DynamoDB Local for a real condition check. The reason for the condition expression is atomicity: the check and the write happen in one operation, so two consumers cannot both succeed.
Diagnosis: without partial batch response, the Lambda event source treats a failed invocation as a failure for every message in the batch, so all ten return to the queue. Fix: enable report batch item failures on the event source mapping (FunctionResponseTypes set to ReportBatchItemFailures) and return the IDs of only the failed messages in the function response. SQS then deletes the successes and leaves the failures. Also keep handlers idempotent (the nine good payments may already have been processed more than once), confirm maxReceiveCount and the DLQ with an alarm, and look at the poison message in the DLQ to fix the parser. ⚠ VERIFY the exact setting names in the current Lambda documentation for your runtime and tooling.
Knowledge check
Answer: When a producer retries after a timeout, it sends a new message with a new message ID but the same payment. Only a business key (such as the payment ID) identifies duplicates created by the producer as well as redeliveries by the queue.
Answer: Claim the key (conditional write), do the work, mark complete, then delete the message. Deleting first can lose the payment if the work fails; marking complete first can skip work that never happened.
Answer: A consumer crashing after claiming a key. Without an expiry, the key would stay in progress forever and the payment would never be retried. With a lease, another consumer can take over after it expires.
Answer: Treating a response list as non-null. In v4 an empty collection is null, so foreach or .Count throws NullReferenceException. Use null-coalescing and compare nullable booleans with == true.
Answer: Enable partial batch responses (report batch item failures) so only failed messages return to the queue, keep handlers idempotent, and set a dead-letter queue. Increasing the timeout or batch size does not address the cause.
Interview questions
- Design a payment consumer that never pays twice. Business-key idempotency with a conditional write and lease, delete only after success, downstream idempotency key, DLQ and alarms, an audit event per decision and reconciliation as the backstop.
- How do you choose between Lambda and Fargate? By workload shape and constraints: event-driven and bursty or steady and long-running, runtime limits, cold starts, cost at steady load, team skills and operations.
- How do you keep secrets out of code? Roles for access, Secrets Manager for values, retrieval at start-up with caching, no secrets in images, environment files, logs or source control, and secret scanning in the pipeline.
- What would you look for in a code review of AWS-calling C#? Client lifetime, explicit Region and timeouts, null handling in v4, idempotency, error classification, safe logging, least-privilege IAM and tests for failure paths.
- How do you test code that uses AWS? Unit tests with fakes behind interfaces, local emulators (DynamoDB Local) or Testcontainers for integration, and a small gated set of real-AWS tests in a sandbox account, not on every commit.
Certification practice
A developer's SQS consumer sometimes processes the same order twice. Messages are deleted after processing. What is the most likely reason and the best fix?
A. SQS standard queues deliver at least once, so the handler must be idempotent. B. The queue is encrypted, so delete fails. C. The consumer must use a larger instance type. D. SQS never delivers duplicates; the code has a bug elsewhere.
Answer: A. Standard queues provide at-least-once delivery, and producer retries also create duplicates. Make the handler idempotent with a business key. B and C are unrelated, and D is incorrect about standard queues.
A Lambda function consuming an SQS queue repeatedly reprocesses messages that already succeeded when one message in a batch fails. Which change best addresses this?
A. Increase the function memory. B. Enable partial batch responses and return the IDs of failed messages. C. Reduce the queue's retention period. D. Switch to a FIFO queue without other changes.
Answer: B. Without partial batch responses, one failure returns the whole batch to the queue. Memory (A) and retention (C) are unrelated, and FIFO (D) changes ordering and throughput, not the batch failure behaviour.
A payment queue and its dead-letter queue contain customer data. Which combination best protects it?
A. Server-side encryption with a KMS key, a key policy and queue policy limiting who can receive, and a short retention period. B. Public access with an obscure queue name. C. Encryption disabled for performance. D. Storing the messages in a text file on the consumer instead.
Answer: A. Encryption plus restricted access and limited retention reduces exposure. B and C leave data exposed, and D moves sensitive data to an unmanaged location.
Completion checklist
- I can state the four phrases to say while coding and used them in a practice round
- I can explain at-least-once delivery, visibility timeout, DLQ and idempotency in under two minutes
- I wrote the idempotent consumer without looking, including the duplicate and failure cases
- I can list five SDK v4 behaviours that cause bugs
- I can apply the security review checklist to a snippet
- I ran the cleanup steps and confirmed no practice queue remains
Key takeaways
- Interviewers want design for failure, not only working code: duplicates, crashes, poison messages and timeouts.
- The idempotency key is the business key, claimed with a conditional write and a lease, and the downstream call needs one too.
- Delete a message only after success or a confirmed duplicate.
- Know the SDK v4 behaviours: null collections, nullable values, explicit Regions, one client per Region.
- Say what you have not handled; honesty and sequence of reasoning beat a rushed complete answer.
Further reading
- Migrating to version 4 of the AWS SDK for .NET
- Retries and timeouts in the AWS SDK for .NET
- Resilience in .NET with Microsoft.Extensions.Http.Resilience
- Previous lesson: Interview preparation: cloud engineer
- Next lesson: Interview preparation: solutions architect
- Related: SQS messages stuck or redelivered and Module 17 overview
Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.
Interview preparation: cloud engineer
Prepare for a cloud engineer interview by turning your .NET and Azure experience into AWS stories, rehearsing a core bank of IAM, networking, operations and troubleshooting answers, and practising how to think aloud when you do not know the answer.
Interview preparation: solutions architect
Prepare for a solutions architect interview: elicit requirements, size a system on the back of an envelope, justify service choices against rejected alternatives, reason about security, resilience and cost, and record decisions the way a bank's design authority expects.