Human-in-the-loop and use-case triage for GenAI
Classify GenAI use cases as good fits, controlled fits or do-not-proceed, choose the right human-review pattern, and build a review queue with maker-checker rules and a Step Functions approval flow in C#, CDK and Terraform.
Assumes: .NET 10 LTS · AWS SDK for .NET v4 (AWSSDK.StepFunctions 4.1.0.2) · AWS CLI v2 · CDK v2 (Amazon.CDK.Lib 2.277.0) · Terraform AWS provider v6 · AWS Step Functions · Amazon SQS · Region us-east-1. Verified against official sources on 2026-11-18.
Service status: no managed human-review service to build on
Amazon Augmented AI (A2I) and SageMaker AI Ground Truth are closed to new customers (since 2026-07-30; Ground Truth Plus reached end of support on 2026-06-30), and AWS names no direct replacement. Bedrock Evaluations human-evaluation jobs cover foundation-model evaluation only. For reviewing individual production outputs, you build the review queue yourself, which is what this lesson does with Step Functions task tokens, SQS and your own application. A2I remains useful exam vocabulary.
Learning objectives
By the end of this lesson you can:
- Classify any proposed GenAI use case into good fit, controlled fit or do not proceed without strong controls, and justify the call with explicit factors.
- Choose a human-review pattern: sampling after release, approve before release, exception-based review or maker-checker.
- Explain the human failure modes (automation bias, rubber-stamping, review fatigue) and measure whether review is real.
- Implement triage rules and a review queue with segregation of duties in C#.
- Build an approval flow with Step Functions
waitForTaskTokenand SQS, in CDK (C#) and Terraform, with a timeout that fails safe. - Write a go/no-go recommendation the risk committee can accept.
Prerequisites
- Module 16 overview, GenAI in plain language and Bedrock Guardrails.
- Agents on AWS: the autonomy ladder and the approval gate.
- Data privacy and model risk: risk tiers and the inventory.
- Prompt injection and data leakage: why a tricked model must have limited power.
- Federation and segregation of duties: the maker-checker idea this lesson reuses.
- Tools: .NET 10 SDK, Docker. The core code runs offline.
Why this matters in banking
Every GenAI proposal reaches the same meeting. A business sponsor says "it will save thousands of hours". Risk says "what if it is wrong?". The conversation is unproductive until you replace "AI" with specific facts: who sees the output, what can it change, what data does it touch, can it be undone, and who checks it? A consistent triage turns an argument into a form with answers.
The second half of the problem is human review itself. "A human reviews every output" sounds reassuring and is often untrue in practice: reviewers see 300 items an hour, the model is right 98% of the time, and approving becomes a reflex. Regulators and auditors increasingly ask for evidence that human oversight is meaningful, not nominal. UK GDPR treats solely automated significant decisions about people as restricted (⚠ VERIFY the current wording after the Data (Use and Access) Act 2025), and the FCA's Consumer Duty expects firms to avoid foreseeable harm to retail customers ⚠ VERIFY current FCA wording with compliance. This lesson gives you designs where review does real work.
The idea in plain language
Triage sorts use cases by what could go wrong, not by how clever the technology is. Human-in-the-loop (HITL) means a person is part of the decision path: they approve, edit or reject before anything reaches a customer or changes a record. Other patterns keep the human on the loop (supervising samples and alerts) rather than in every decision.
Analogy: a bank already has this system for cheques and payments. Small items clear automatically with spot checks; larger items need a second signature from someone other than the person who prepared them; certain items (very large, unusual, high-risk) need senior sign-off. GenAI review should feel the same: effort proportional to consequence.
Three zones
| Zone | Meaning | Typical examples at Harbourline | What you need |
|---|---|---|---|
| Good fit | Low consequence, internal, reversible, non-sensitive data, suggestion or read-only | Searching internal policy documents; summarising a long public regulation; drafting meeting notes; explaining unfamiliar code; classifying internal tickets | Inventory entry, logging, sampled quality review, usage guidance |
| Controlled fit | Allowed with controls because output reaches customers, touches confidential data or prepares actions | Drafting complaint replies a person approves; extracting fields from documents for review; preparing (not sending) a refund request; fraud-case summaries for investigators | Tiered review, validation, DPIA, guardrails, approval before release, maker-checker for money, seeded test cases, monitoring |
| Do not proceed without strong controls | A red line is crossed: solely automated decisions about people, autonomous money movement, irreversible autonomous actions, restricted data to customers with no review | Auto-declining loans; autonomous payment release; AI-written final KYC outcomes; an unsupervised customer chatbot that can change account details | Redesign, or a documented senior risk and legal decision with additional safeguards; frequently the answer is "not yet" |
Factors that move a use case between zones
| Factor | Lower risk | Higher risk |
|---|---|---|
| Audience | Staff, internal | Customers, regulators, the public |
| Effect | Suggestion or draft | Decision, payment, change to rights |
| Reversibility | Easy to undo | Hard or impossible to undo |
| Data | Public or internal | Confidential, restricted, special-category |
| Autonomy | Suggests or reads | Proposes or commits actions |
| Explainability needed | Not needed | A person is owed an explanation |
| Volume and speed | Low, slow | Very high, real time (review is hard) |
| Error tolerance | A wrong draft is fixed by a person | A wrong output causes harm or a reportable event |
What problem the AWS service solves
Nothing in AWS "does human review" for GenAI outputs out of the box now that A2I is closed. What AWS gives you are building blocks for a review workflow:
| Need | AWS building block | Notes |
|---|---|---|
| Pause a process until a person decides | Step Functions waitForTaskToken pattern | The state machine hands out a task token and waits; your app calls SendTaskSuccess or SendTaskFailure to resume |
| Hold work items for reviewers | Amazon SQS (or DynamoDB for a queryable case list) | SQS gives buffering and a dead-letter queue; DynamoDB gives search and status |
| Notify reviewers | Amazon SNS, Amazon SES, or your case-management tool | Keep message content free of personal data; send a link instead |
| Authenticate reviewers and enforce roles | Cognito or IAM Identity Center plus your own authorisation | Approver role, plus rule "reviewer is not the requester" |
| Record decisions immutably | DynamoDB with conditional writes, S3 Object Lock, CloudTrail for API calls | The decision record is evidence |
| Evaluate quality before launch | Bedrock Evaluations (including human evaluation jobs) and fmeval | For testing models and prompts, not for per-item production review |
| Sample and measure in production | CloudWatch metrics and dashboards, Athena over logs | Override rate, time to review, agreement between reviewers |
Limits to be honest about: Step Functions task tokens have a maximum wait that is bounded by the state machine's maximum execution duration (one year for Standard workflows) ⚠ VERIFY current limits; the task token is a bearer secret that must not leak into logs; a queue is not a user interface, so you still build the reviewer screen.
Microsoft / Azure comparison
| You know (Microsoft/Azure) | AWS equivalent | What's different |
|---|---|---|
| Logic Apps "Send approval email" and wait | Step Functions waitForTaskToken with your own reviewer app | You build the approval UI; the token resumes the workflow |
| Power Automate approvals | Same, plus Cognito or Identity Center for reviewer identity | No built-in approval inbox |
Durable Functions human interaction pattern (WaitForExternalEvent) | Task token callback | A token you pass to the approver rather than an event name |
| Azure Service Bus queues | Amazon SQS | Visibility timeout and dead-letter queues; standard queues may deliver more than once |
| Azure AI Content Safety human review | Your own queue plus Guardrails findings | Guardrails provides signals; routing is yours |
| Maker-checker in Dynamics or core banking | Same concept, implemented in your own flow | Enforce it in code and in IAM, not in a policy document |
Architecture
Security-boundary view. The model output is untrusted until a person releases it. The reviewer application is the trust boundary: it authenticates the reviewer, enforces that the reviewer is not the requester and holds the approver role, and records the decision before it calls Step Functions. The task token is a bearer secret: store it server-side with the case and never send it to the browser. The default on timeout is do not release.
Step-by-step implementation
Step one: collect the facts for each use case
Fill the same form for every proposal: audience, effect, reversibility, data, autonomy, human review planned. The C# below encodes this form as UseCaseFacts.
Step two: run triage and read the reasons
The result names the zone, the review pattern and the reasons. A do not proceed result lists the red lines to remove; a controlled result lists the controls to build; a good fit result still needs inventory and logging.
Step three: choose the review pattern
| Pattern | When | How it works |
|---|---|---|
| Sample after release | Good fit | Review a deterministic percentage (for example 5%) after the fact; track quality trends |
| Approve before release | Controlled: customer-facing or confidential | Every output waits for a person to approve, edit or reject |
| Exception-based | High volume where most items are routine | Auto-release only items that pass validation, guardrails and confidence checks; route the rest to a person; still sample the auto-released items |
| Maker-checker | Money or rights | A second person, with the approver role, who is not the requester, approves |
Step four: design reviews that do real work
Show the reviewer evidence (the source passages and the policy clause), not just the answer. Require a reason code on every decision. Seed the queue with items that have known errors and check that reviewers catch them. Rotate reviewers and cap items per hour so attention stays real.
Step five: build the queue and the approval flow
Use the C# review queue and the Step Functions pattern below. Make the timeout fail safe.
Step six: measure the human
Track approval rate, edit rate, time per item, reviewer agreement and seeded-error catch rate. An approval rate near 100% with a few seconds per item is a warning, not a success.
Step seven: write the recommendation
Summarise the zone, the controls, the residual risks and a go, conditional go or no-go. The assignment gives a template.
AWS Console instructions
Console labels change, so ⚠ VERIFY them:
- Console → Step Functions → State machines → Create state machine. In Workflow Studio drag an Amazon SQS SendMessage state, choose Wait for callback with task token, and set a timeout. Add Catch branches for
ReviewRejectedandStates.Timeout. - Console → Step Functions → your state machine → Executions. Open a running execution to see it paused at the review state and inspect the input (never copy task tokens into tickets or chat).
- Console → SQS → your queue → Send and receive messages → Poll for messages to see review requests during testing.
- Console → CloudWatch → Log groups for the state machine log group.
AWS CLI
# Start an execution that will pause at the human-review step
aws stepfunctions start-execution \
--state-machine-arn "$STATE_MACHINE_ARN" \
--input '{"caseId":"CASE-1001","requesterId":"bob","summary":"Goodwill credit GBP 20 for fee dispute"}' \
--region us-east-1 --profile learn-sandbox
# Read the pending review request from the queue (this message contains the task token: treat it as a secret)
aws sqs receive-message --queue-url "$REVIEW_QUEUE_URL" --max-number-of-messages 1 \
--region us-east-1 --profile learn-sandbox
# Approve (for a test only; in real life your reviewer app does this after authenticating the reviewer)
aws stepfunctions send-task-success \
--task-token "$TASK_TOKEN" \
--task-output '{"approved":true,"reviewer":"carol","reason":"WITHIN_POLICY"}' \
--region us-east-1 --profile learn-sandbox
# Or reject
aws stepfunctions send-task-failure \
--task-token "$TASK_TOKEN" --error ReviewRejected --cause "Rejected by reviewer. Reason code AMOUNT_TOO_HIGH." \
--region us-east-1 --profile learn-sandbox
# Check the execution result
aws stepfunctions describe-execution --execution-arn "$EXECUTION_ARN" \
--region us-east-1 --profile learn-sandbox --query "{status:status,output:output}"Infrastructure as Code
The CDK stack below compiled against Amazon.CDK.Lib 2.277.0 on 2026-11-18. It creates an encrypted review queue with a dead-letter queue, and a state machine that waits for a human decision, handles a rejection and a 24-hour timeout, and releases only on approval.
using Amazon.CDK;
using Amazon.CDK.AWS.Logs;
using Amazon.CDK.AWS.SQS;
using Amazon.CDK.AWS.StepFunctions;
using Amazon.CDK.AWS.StepFunctions.Tasks;
using Constructs;
public class HumanReviewStack : Stack
{
public HumanReviewStack(Construct scope, string id, IStackProps props) : base(scope, id, props)
{
var dlq = new Queue(this, "ReviewDlq", new QueueProps
{
Encryption = QueueEncryption.SQS_MANAGED,
EnforceSSL = true,
RetentionPeriod = Duration.Days(14)
});
var queue = new Queue(this, "ReviewQueue", new QueueProps
{
Encryption = QueueEncryption.SQS_MANAGED,
EnforceSSL = true,
VisibilityTimeout = Duration.Minutes(5),
DeadLetterQueue = new DeadLetterQueue { Queue = dlq, MaxReceiveCount = 5 }
});
// The message carries the task token. It is a bearer secret: only the reviewer app reads this queue.
var requestReview = new SqsSendMessage(this, "RequestHumanReview", new SqsSendMessageProps
{
Queue = queue,
IntegrationPattern = IntegrationPattern.WAIT_FOR_TASK_TOKEN,
MessageBody = TaskInput.FromObject(new Dictionary<string, object>
{
["taskToken"] = JsonPath.TaskToken,
["caseId"] = JsonPath.StringAt("$.caseId"),
["requesterId"] = JsonPath.StringAt("$.requesterId"),
["summary"] = JsonPath.StringAt("$.summary")
}),
// Written out in full because System.Threading also defines Timeout.
TaskTimeout = Amazon.CDK.AWS.StepFunctions.Timeout.Duration(Duration.Hours(24)),
ResultPath = "$.review"
});
var released = new Pass(this, "ReleaseToCustomerChannel");
var rejected = new Pass(this, "RecordRejection");
var escalated = new Fail(this, "EscalateNoReviewInTime", new FailProps
{
Error = "ReviewTimeout",
Cause = "No human decision within 24 hours. Nothing was released."
});
requestReview.AddCatch(rejected, new CatchProps { Errors = ["ReviewRejected"], ResultPath = "$.error" });
requestReview.AddCatch(escalated, new CatchProps { Errors = ["States.Timeout"], ResultPath = "$.error" });
var logGroup = new LogGroup(this, "ReviewFlowLogs", new LogGroupProps
{
Retention = RetentionDays.ONE_MONTH,
RemovalPolicy = RemovalPolicy.DESTROY // lab only
});
var machine = new StateMachine(this, "ReviewFlow", new StateMachineProps
{
DefinitionBody = DefinitionBody.FromChainable(requestReview.Next(released)),
Timeout = Duration.Days(2),
TracingEnabled = true,
Logs = new LogOptions { Destination = logGroup, Level = LogLevel.ERROR }
});
Amazon.CDK.Tags.Of(this).Add("Project", "cea-lab");
Amazon.CDK.Tags.Of(this).Add("Lab", "m16-hitl");
Amazon.CDK.Tags.Of(this).Add("Environment", "learn");
new CfnOutput(this, "StateMachineArn", new CfnOutputProps { Value = machine.StateMachineArn });
new CfnOutput(this, "ReviewQueueUrl", new CfnOutputProps { Value = queue.QueueUrl });
}
}⚠ Learning shortcut — not for production
The release step is an empty Pass state, the reviewer is whoever can read the queue, and nothing checks the reviewer's identity. Production replaces Pass with the real release action (for example a Lambda that posts to the case-management system, using an idempotency key), puts a reviewer application with SSO and the approver role in front of the callback, stores the task token server-side in an encrypted table rather than in a message that humans can read, applies a customer managed KMS key to the queues and logs, and alarms on the age of the oldest message and on the dead-letter queue.
Complete code
Everything below compiled and ran in Docker with the .NET 10 SDK and AWSSDK.StepFunctions 4.1.0.2 on 2026-11-18. Namespace Harbourline.Triage.
<ItemGroup>
<PackageReference Include="AWSSDK.StepFunctions" Version="4.1.0.2" />
</ItemGroup>using System.Security.Cryptography;
using System.Text;
namespace Harbourline.Triage;
public enum DataSensitivity { Public = 0, Internal = 1, Confidential = 2, Restricted = 3 }
/// <summary>How far the system acts on its own: 0 suggests, 1 reads, 2 proposes a write, 3 commits a write.</summary>
public enum Autonomy { Suggest = 0, ReadOnlyTools = 1, ProposeAction = 2, CommitAction = 3 }
public enum Zone { GoodFit, Controlled, DoNotProceed }
public enum ReviewPattern { SampleAfterRelease, ApproveBeforeRelease, MakerChecker, NotPermitted }
public sealed record UseCaseFacts(
string Name,
bool AffectsCustomer,
bool MovesMoneyOrChangesRights,
bool DecidesAboutAPerson,
DataSensitivity Data,
Autonomy Autonomy,
bool Reversible,
bool HumanReviewPlanned);
public sealed record TriageResult(
Zone Zone,
ReviewPattern Pattern,
IReadOnlyList<string> Reasons,
IReadOnlyList<string> RequiredControls);
public static class UseCaseTriage
{
public static TriageResult Assess(UseCaseFacts u)
{
var reasons = new List<string>();
var controls = new List<string> { "Entry in the model inventory with an owner", "Invocation and decision logging with retention" };
// Red lines: do not proceed until the design changes or a senior risk decision is recorded.
if (u.DecidesAboutAPerson && !u.HumanReviewPlanned)
reasons.Add("Decides about a person with no human review (automated-decision rules apply; legal and DPO review).");
if (u.MovesMoneyOrChangesRights && u.Autonomy >= Autonomy.ProposeAction && !u.HumanReviewPlanned)
reasons.Add("Can move money or change customer rights without a human in the loop.");
if (u.Data == DataSensitivity.Restricted && u.AffectsCustomer && !u.HumanReviewPlanned)
reasons.Add("Restricted data reaches customers with no human review.");
if (u.Autonomy == Autonomy.CommitAction && !u.Reversible)
reasons.Add("Commits irreversible actions autonomously.");
if (reasons.Count > 0)
return new TriageResult(Zone.DoNotProceed, ReviewPattern.NotPermitted, reasons,
[.. controls, "Redesign to remove the red line, or obtain a documented senior risk decision"]);
bool controlled = u.AffectsCustomer || u.MovesMoneyOrChangesRights || u.DecidesAboutAPerson
|| u.Data >= DataSensitivity.Confidential || u.Autonomy >= Autonomy.ProposeAction
|| !u.Reversible;
if (!controlled)
{
reasons.Add("Internal, reversible, non-sensitive, suggestion or read-only use.");
controls.Add("Sampled quality review after release");
return new TriageResult(Zone.GoodFit, ReviewPattern.SampleAfterRelease, reasons, controls);
}
if (u.AffectsCustomer) reasons.Add("Output can reach or affect a customer.");
if (u.MovesMoneyOrChangesRights) reasons.Add("Relates to money or customer rights.");
if (u.DecidesAboutAPerson) reasons.Add("Informs a decision about a person.");
if (u.Data >= DataSensitivity.Confidential) reasons.Add("Confidential or restricted data in scope.");
if (u.Autonomy >= Autonomy.ProposeAction) reasons.Add("Prepares or performs actions.");
if (!u.Reversible) reasons.Add("Effects are hard to reverse.");
controls.AddRange(["Independent validation before launch", "DPIA where personal data is involved",
"Guardrails and output scanning", "Reviewer training and seeded test cases"]);
var pattern = u.MovesMoneyOrChangesRights ? ReviewPattern.MakerChecker : ReviewPattern.ApproveBeforeRelease;
if (pattern == ReviewPattern.MakerChecker) controls.Add("Maker-checker: reviewer differs from requester and holds the approver role");
return new TriageResult(Zone.Controlled, pattern, reasons, controls);
}
}
public sealed record ReviewItem(string CaseId, string RequesterId, string Summary, DateTimeOffset SubmittedAt);
public sealed record ReviewDecision(string CaseId, string ReviewerId, bool Approved, string ReasonCode, DateTimeOffset At);
public sealed class ReviewQueue(IReadOnlySet<string> approverIds)
{
private readonly Dictionary<string, ReviewItem> _open = new();
private readonly List<ReviewDecision> _log = [];
public IReadOnlyList<ReviewDecision> Log => _log;
public void Submit(ReviewItem item) => _open.Add(item.CaseId, item);
/// <summary>Segregation of duties: the requester cannot approve, and only approvers can decide.</summary>
public (bool Ok, string Message) Decide(string caseId, string reviewerId, bool approved, string reasonCode, DateTimeOffset at)
{
if (!_open.TryGetValue(caseId, out var item)) return (false, "Unknown or already decided case.");
if (!approverIds.Contains(reviewerId)) return (false, "Reviewer does not hold the approver role.");
if (string.Equals(item.RequesterId, reviewerId, StringComparison.Ordinal))
return (false, "Requester cannot review their own item (maker-checker).");
if (string.IsNullOrWhiteSpace(reasonCode)) return (false, "A reason code is required.");
_open.Remove(caseId);
_log.Add(new ReviewDecision(caseId, reviewerId, approved, reasonCode, at));
return (true, approved ? "Approved." : "Rejected.");
}
}
public static class Sampling
{
/// <summary>Deterministic sampling: the same case always gets the same answer, so reviews are reproducible.</summary>
public static bool ShouldSample(string caseId, double rate)
{
var hash = SHA256.HashData(Encoding.UTF8.GetBytes(caseId));
var value = BitConverter.ToUInt32(hash, 0) / (double)uint.MaxValue;
return value < rate;
}
}The callback that resumes the paused Step Functions execution after the decision has been recorded:
using Amazon.StepFunctions;
using Amazon.StepFunctions.Model;
namespace Harbourline.Triage;
/// <summary>Resumes a paused Step Functions execution after a human decision.</summary>
public sealed class TaskTokenCallback(IAmazonStepFunctions sfn)
{
public async Task CompleteAsync(string taskToken, ReviewDecision decision, CancellationToken ct = default)
{
if (decision.Approved)
{
// Output must be a JSON string. The state machine can read it with a Choice state.
var output = $"{{\"approved\":true,\"reviewer\":\"{decision.ReviewerId}\",\"reason\":\"{decision.ReasonCode}\"}}";
await sfn.SendTaskSuccessAsync(new SendTaskSuccessRequest { TaskToken = taskToken, Output = output }, ct);
}
else
{
await sfn.SendTaskFailureAsync(new SendTaskFailureRequest
{
TaskToken = taskToken,
Error = "ReviewRejected",
Cause = $"Rejected by reviewer. Reason code {decision.ReasonCode}."
}, ct);
}
}
}A driver that triages four fictional use cases, exercises the maker-checker rules and checks sampling:
using Harbourline.Triage;
var cases = new[]
{
new UseCaseFacts("Policy search for staff", AffectsCustomer: false, MovesMoneyOrChangesRights: false,
DecidesAboutAPerson: false, DataSensitivity.Internal, Autonomy.ReadOnlyTools, Reversible: true, HumanReviewPlanned: true),
new UseCaseFacts("Complaint reply drafts", AffectsCustomer: true, MovesMoneyOrChangesRights: false,
DecidesAboutAPerson: false, DataSensitivity.Confidential, Autonomy.Suggest, Reversible: true, HumanReviewPlanned: true),
new UseCaseFacts("Goodwill refund agent", AffectsCustomer: true, MovesMoneyOrChangesRights: true,
DecidesAboutAPerson: false, DataSensitivity.Confidential, Autonomy.ProposeAction, Reversible: true, HumanReviewPlanned: true),
new UseCaseFacts("Auto-decline loan applications", AffectsCustomer: true, MovesMoneyOrChangesRights: true,
DecidesAboutAPerson: true, DataSensitivity.Restricted, Autonomy.CommitAction, Reversible: false, HumanReviewPlanned: false),
};
foreach (var c in cases)
{
var r = UseCaseTriage.Assess(c);
Console.WriteLine($"{c.Name,-34} -> {r.Zone,-12} pattern={r.Pattern}");
foreach (var reason in r.Reasons) Console.WriteLine($" why: {reason}");
}
var queue = new ReviewQueue(new HashSet<string> { "bob", "carol" });
var now = new DateTimeOffset(2026, 11, 18, 10, 0, 0, TimeSpan.Zero);
queue.Submit(new ReviewItem("CASE-1", "bob", "Goodwill GBP 20", now));
Console.WriteLine(queue.Decide("CASE-1", "bob", true, "OK", now).Message);
Console.WriteLine(queue.Decide("CASE-1", "mallory", true, "OK", now).Message);
Console.WriteLine(queue.Decide("CASE-1", "carol", true, "", now).Message);
Console.WriteLine(queue.Decide("CASE-1", "carol", true, "WITHIN_POLICY", now).Message);
Console.WriteLine(queue.Decide("CASE-1", "carol", true, "WITHIN_POLICY", now).Message);
int sampled = Enumerable.Range(1, 1000).Count(i => Sampling.ShouldSample($"CASE-{i}", 0.05));
Console.WriteLine($"sampled {sampled} of 1000 at 5% (deterministic: {Sampling.ShouldSample("CASE-7", 0.05) == Sampling.ShouldSample("CASE-7", 0.05)})");Verified output:
Policy search for staff -> GoodFit pattern=SampleAfterRelease
why: Internal, reversible, non-sensitive, suggestion or read-only use.
Complaint reply drafts -> Controlled pattern=ApproveBeforeRelease
why: Output can reach or affect a customer.
why: Confidential or restricted data in scope.
Goodwill refund agent -> Controlled pattern=MakerChecker
why: Output can reach or affect a customer.
why: Relates to money or customer rights.
why: Confidential or restricted data in scope.
why: Prepares or performs actions.
Auto-decline loan applications -> DoNotProceed pattern=NotPermitted
why: Decides about a person with no human review (automated-decision rules apply; legal and DPO review).
why: Can move money or change customer rights without a human in the loop.
why: Restricted data reaches customers with no human review.
why: Commits irreversible actions autonomously.
Requester cannot review their own item (maker-checker).
Reviewer does not hold the approver role.
A reason code is required.
Approved.
Unknown or already decided case.
sampled 48 of 1000 at 5% (deterministic: True)Code explained line by line
UseCaseFactsis the triage form as a record. Making it a type means the form cannot be filled in half-way: every proposal supplies every fact.Autonomymirrors the ladder in the agents lesson: suggest, read, propose, commit.Assesschecks red lines first. Any red line returnsDoNotProceedwith the exact reasons, so the sponsor sees what must change. The checks combine facts (a decision about a person plus no review) rather than naming technologies, which keeps the rule stable as models change.- If no red line applies,
controlledis true when any risk factor is present. Only a use case with none of them is aGoodFit. This asymmetry is deliberate: one risk factor is enough to need controls. - The review pattern follows the facts: money or rights lead to
MakerChecker; other controlled cases getApproveBeforeRelease; good fits getSampleAfterRelease. ReviewQueue.Decideenforces segregation of duties in code: unknown case, reviewer lacks the approver role, requester equals reviewer, or no reason code all fail before any state changes. Notice the order: role first, then self-approval. The verified output shows each rejection.- A decided case is removed from the open set and appended to an immutable-style log, so a second decision on the same case fails ("Unknown or already decided case"), which also blocks double approval.
Sampling.ShouldSamplehashes the case ID with SHA-256 and compares the result to the rate. The result is deterministic (the same case always gives the same answer) and about 5% over many cases (48 of 1000 in the run), so a reviewer or auditor can reproduce which items should have been sampled.TaskTokenCallback.CompleteAsynccallsSendTaskSuccessorSendTaskFailure. The failure uses the error nameReviewRejected, which the state machine'sCatchmatches. Call it after the decision is recorded, and keep the task token server-side.
Security considerations
- Enforce maker-checker in code and in IAM. A policy document is not a control. The reviewer application must check role and requester, and the identity provider must prevent one person holding both maker and checker roles for the same process (segregation of duties).
- The task token is a bearer secret. Whoever holds it can complete the execution. Keep it server-side, out of logs (
include_execution_data = falsein the Terraform example), out of URLs and tickets. - Fail safe. Timeout and rejection must mean "not released". Never design a flow where silence approves.
- Show evidence, not just conclusions. Reviewers need the source text and policy clause. Otherwise review is a rubber stamp.
- Automation bias. People over-trust fluent text. Counter it with seeded errors, reason codes, rotation and rate limits for reviewers.
- Prompt injection reaches reviewers too. A model output can contain text aimed at the reviewer ("this is pre-approved by risk"). Render output as plain text and display system-generated verdicts separately (lesson 9).
- Privacy of the review UI. Reviewers see personal data. Apply least privilege, audit reads, and restrict screenshots and exports.
- Records. Store who decided, what they saw, when and why. Retain per policy. A decision log is evidence in a complaint or regulatory enquiry.
- Never claim compliance. The design supports oversight; legal and compliance decide whether it satisfies a specific obligation.
⚠ Learning shortcut — not for production
The review queue is an in-memory class, roles are a hard-coded set, and the CLI examples approve with your own credentials. A real system stores cases and decisions in a durable, access-controlled store with conditional writes (so two reviewers cannot both approve), authenticates reviewers through your identity provider, derives roles from groups, stores the task token encrypted next to the case, applies a service-level target and escalation for old items, and keeps decision records immutable for the retention period.
Banking use case
Harbourline portfolio triage (fictional), 10 proposals.
| # | Proposal | Zone | Pattern | Note |
|---|---|---|---|---|
| 1 | Staff search over approved policy documents | Good fit | Sample 5% | Citations required; internal data only |
| 2 | Summarise public regulatory consultations | Good fit | Sample | Public data; reversible |
| 3 | Developer assistant on bank code | Good fit with conditions | Sample plus pipeline gates | Enterprise tier only (lesson 7) |
| 4 | Draft replies to complaints | Controlled | Approve before release | A person edits and sends; DPIA |
| 5 | Extract fields from onboarding documents | Controlled | Exception-based plus sampling | Low-confidence items to a person; restricted data |
| 6 | Investigator case summaries (AML) | Controlled | Approve before use | Summaries support, never replace, the investigator |
| 7 | Prepare goodwill refund requests | Controlled | Maker-checker | Agent proposes, a different person approves |
| 8 | Customer-facing chatbot answering fee questions | Controlled (high tier) | Approve scripted answers; strong guardrails | Grounded in approved text; clear AI disclosure; no account actions |
| 9 | Auto-decline credit applications | Do not proceed | Not permitted | Automated decision about a person; EU AI Act high-risk for the EU subsidiary ⚠ VERIFY |
| 10 | Agent that releases payments | Do not proceed | Not permitted | Autonomous money movement |
Going back to the first meeting: for proposal 7 the recommendation is "conditional go for a pilot": maker-checker with a different approver, a GBP 25 cap enforced in the tool and in Policy (lesson 6), seeded errors, daily review of rejected and approved counts, and a stop condition if the override rate drifts.
Common mistakes
- Stopping at "a human reviews it". Ask: how many items per hour, with what evidence, with what measured catch rate?
- Letting the requester approve their own item.
- Defaulting to release on timeout.
- Putting the task token in the browser, in logs or in a ticket.
- Using the same bar for every use case. Over-controlling internal search wastes review capacity that high-risk items need.
- Treating 99% accuracy as "safe". At high volume, 1% is many customers; consequence matters more than average accuracy.
- Not re-triaging after changes. New data, a new tool or wider audience can move a use case to a higher zone.
- Building on A2I after the closure.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
Execution stays at RequestHumanReview forever | No callback was sent, or the task token was lost | Check the reviewer app logs; the 24-hour timeout should end it; resend with the correct token |
TaskTimedOut or InvalidToken on callback | Execution already timed out or token malformed or already used | Check execution status; tokens are single-use per task |
AccessDeniedException on SendTaskSuccess | Caller lacks states:SendTaskSuccess or states:SendTaskFailure | Grant the reviewer app role those actions for the state machine |
| State machine fails to send the message | Role lacks sqs:SendMessage or the queue policy blocks it | Add permission for the state machine role to the queue ARN |
| Reviewers approve everything in seconds | Review is nominal | Add seeded errors, show evidence, cap throughput, measure catch rate |
| Duplicate review items | Standard SQS queues can deliver a message more than once | Make the reviewer app idempotent on caseId |
| Dead-letter queue filling | Reviewer app errors repeatedly | Alarm on DLQ depth, fix the consumer, redrive after the fix |
| Triage output surprises the sponsor | Facts were entered optimistically | Require evidence per fact and a second reviewer for the form |
Cost and cleanup
Costs. The triage code is free. A Standard Step Functions workflow is billed per state transition, and SQS per request; a handful of test executions costs well under a cent ⚠ VERIFY current pricing. Logs cost pennies at this volume. Deploy and destroy within the hour.
# CDK path
cdk destroy hl-m16-human-review --profile learn-sandbox
# Terraform path
terraform destroy
# Stop any execution still waiting (find running executions first)
aws stepfunctions list-executions --state-machine-arn "$STATE_MACHINE_ARN" \
--status-filter RUNNING --region us-east-1 --profile learn-sandbox
aws stepfunctions stop-execution --execution-arn "$EXECUTION_ARN" \
--region us-east-1 --profile learn-sandbox
# Confirm nothing tagged for the lab remains
aws resourcegroupstaggingapi get-resources \
--tag-filters Key=Project,Values=cea-lab \
--region us-east-1 --profile learn-sandbox \
--query "ResourceTagMappingList[].ResourceARN"Exercises
- 🟢 Guided: run the triage sample, then add two proposals of your own (one good fit, one that lands in do-not-proceed) and explain each result in one sentence using the printed reasons.
- 🟡 Challenge: add exception-based review: a function that auto-releases an output only when the guardrail passed, the output scanner found nothing, and the answer cites at least one source; otherwise it routes to the queue. Always sample 5% of the auto-released items too. Prove it with tests for each routing path.
- 🔴 Live problem: a business head wants an AI assistant to answer customer chat messages 24/7 "with a human only if the customer asks". Write the triage, the three biggest risks, the controls you would require before a pilot, and the metrics that would stop it.
Assignment
Write the module's GenAI use-case assessment (one page) for the internal "policy Q&A" assistant, as required by the module overview: data flows, data classification, guardrails, human review, logging, model-risk questions and a go or no-go recommendation. Use the triage output, the inventory entry from lesson 8, and the red-team results from lesson 9. Add a short paragraph on what would change your recommendation if the assistant became customer-facing.
Solutions
Example good fit: "Summarise public rate announcements for staff", public data, no customer effect, suggestion only, reversible, review planned: result GoodFit, pattern SampleAfterRelease. Example do-not-proceed: "Auto-approve overdraft increases", DecidesAboutAPerson = true, HumanReviewPlanned = false, CommitAction: result DoNotProceed with reasons naming the automated decision and the autonomous money or rights change. The explanation should cite the printed reasons, because the rule names which fact caused each outcome.
Write Route(ValidationSignals s, string caseId) returning AutoRelease, AutoReleaseAndSample or HumanReview. Return HumanReview if !s.GuardrailPassed || s.ScannerFindings > 0 || s.CitationCount == 0. Otherwise return AutoReleaseAndSample when Sampling.ShouldSample(caseId, 0.05) is true, else AutoRelease. Tests: guardrail failed routes to review; one scanner finding routes to review; zero citations routes to review; all signals good routes to auto-release; and over 1000 case IDs about 5% are sampled. The always-on sample matters because the validation signals themselves can drift or be evaded, and sampling is how you notice.
Triage: customer-facing, confidential data, possibly decisions about complaints or rights: controlled at best, do-not-proceed if it can change account details or make eligibility decisions. Risks: wrong or invented policy statements (misleading customers, Consumer Duty exposure), prompt injection and data leakage, vulnerable customers not escalated. Required before a pilot: answers grounded only in approved, versioned content with citations; no account actions; clear AI disclosure; escalation paths and a human option at any time; guardrails and output scanning; red-team and independent validation; DPIA; scripted fallback messages; monitoring of complaint, escalation and thumbs-down rates; and a kill switch. Stop metrics: any leaked personal data, rising complaint rate, answers not matching approved text above a threshold, or failure of the kill-switch test. "A human only if the customer asks" is not enough because customers cannot detect an error they do not know about.
Knowledge check
Answer: Good fit, controlled fit, and do not proceed without strong controls. Red lines include solely automated decisions about people, autonomous money movement, restricted data reaching customers with no human review, and irreversible autonomous actions.
Answer: The person who prepares an action (maker) must differ from the person who approves it (checker), and the checker must hold the approver role. Enforce it in code (reviewer is not requester) and in identity and access management, not just in a policy document.
Answer: It is a bearer secret: anyone who holds it can complete or fail the paused task. Keep it server-side, out of logs, URLs and browsers.
Answer: Nothing is released. The flow fails safe, alerts, and escalates.
Answer: Amazon Augmented AI (A2I). It has been closed to new customers since 2026-07-30 with no named AWS replacement, so you build a queue and callback flow yourself.
Interview questions
- How do you decide whether a GenAI use case is safe to launch? Collect explicit facts (audience, effect, reversibility, data, autonomy, review), triage them, check for red lines, apply the controls for the zone, validate independently, and pilot with monitoring and a kill switch.
- How do you make human review meaningful? Show evidence, require reasons, seed known errors, measure catch rate and time per item, rotate reviewers and cap throughput.
- What is the difference between human-in-the-loop and human-on-the-loop? In the loop: a person approves each item before release. On the loop: a person supervises samples, dashboards and alerts while the system runs.
- Why is "99% accurate" not a sufficient safety argument? Impact matters: at scale 1% is many people, and some errors are severe or irreversible. Controls depend on consequence, not average accuracy.
- How would you implement an approval step on AWS? Step Functions
waitForTaskTokensending a message to SQS, a reviewer app that authenticates the reviewer, enforces maker-checker and records the decision, then callsSendTaskSuccessorSendTaskFailure, with a fail-safe timeout.
Certification practice
A bank wants a model to approve or decline personal loan applications automatically with no human involvement. What is the most appropriate response?
A. Proceed, because the model is accurate on average. B. Treat it as a high-risk use that needs human involvement, explanation, bias and fairness testing, and legal review before any release. C. Remove logging to protect privacy. D. Use the largest available model.
Answer: B. Decisions that significantly affect people are tightly regulated and need oversight and fairness controls. A ignores consequence, C removes evidence, and D does not address the risk.
An application must pause a GenAI-generated customer letter until a different employee approves it, then continue or stop. Which AWS design fits best?
A. A Step Functions workflow that sends the draft to an SQS queue using the wait-for-task-token pattern, with a reviewer application that calls SendTaskSuccess or SendTaskFailure and a timeout that does not release. B. An EC2 cron job that releases all drafts after one hour. C. A Lambda that approves automatically if the model's confidence is above 0.5. D. An S3 lifecycle rule.
Answer: A. It pauses durably, supports a different approver and fails safe. B and C release without a person, and D is unrelated.
A workflow passes a Step Functions task token to a web page so reviewers can approve items. A security review objects. What is the best fix?
A. Keep the token in the browser but shorten the URL. B. Store the token server-side with the case; the browser sends only the case ID and decision to an authenticated API, which looks up the token and calls Step Functions. C. Email the token to the reviewer. D. Hard-code the token in the front end.
Answer: B. The token is a bearer secret, so it must never reach untrusted clients. The others expose it.
Completion checklist
- I can place a use case in a zone and give the facts that justify it
- I can choose a review pattern and say why
- I ran the triage and queue code and understand each rejection reason
- I can explain the task-token approval flow and why timeouts fail safe
- I ran the cleanup steps and confirmed nothing tagged
Project=cea-labis still billing
Key takeaways
- Triage on facts (audience, effect, reversibility, data, autonomy, review), not on the technology.
- Match review effort to consequence: sample, approve before release, exception-based, or maker-checker.
- Human review only works if it is measured: evidence shown, reasons recorded, seeded errors caught.
- A2I is closed to new customers; build the queue and callback yourself with Step Functions task tokens and SQS.
- Segregation of duties belongs in code and in identity, and the default on silence is not released.
Further reading
- Step Functions: wait for a callback with the task token — the integration pattern behind the approval flow
- Amazon Bedrock Evaluations — evaluating models and prompts, including human evaluation jobs
- Policy in Amazon Bedrock AgentCore — deterministic limits that complement human review
- OWASP Top 10 for LLM Applications — excessive agency and related risks
Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.
Prompt injection and data leakage: defence in depth and red-teaming
Recognise direct and indirect prompt injection, system-prompt and data leakage, and unsafe output handling; build layered C# defences (untrusted-content wrapping, canary tokens, output scanning, Bedrock Guardrails) and red-team your own prototype with a repeatable test harness.
Module 16 cheat sheet
One-page revision summary for Generative AI on AWS for Banking: services, Azure mapping, CLI commands, limits, decision rules, exam traps and cost traps.