AWS Step Functions: workflows, retries and callbacks
Orchestrate multi-step banking processes with AWS Step Functions: Standard versus Express workflows, the state types, JSONata, service integrations, Retry and Catch, human-approval callbacks with task tokens, Map and Distributed Map, redrive, testing with TestState, and a .NET 10 client, CDK and Terraform.
Assumes: .NET 10 LTS · AWS SDK for .NET v4 (AWSSDK.StepFunctions 4.1.0.2) · Amazon.CDK.Lib 2.277.0 · Terraform AWS provider v6 · AWS CLI v2 · Region us-east-1. Verified against official sources on 2026-11-18.
Learning objectives
By the end of this lesson you can:
- Describe a state machine, its states and an execution, read Amazon States Language (ASL), and choose Standard or Express for a given workload.
- Write a workflow with JSONata (
Arguments,Output,Assign,$states) and explain how it differs from the older JSONPath fields. - Use the three service integration patterns: Request Response, Run a Job (
.sync) and Wait for Callback (.waitForTaskToken). - Configure Retry and Catch with the correct defaults, jitter and error names, and explain what Step Functions does when both apply.
- Build a human-approval step with a task token, complete it from .NET, and keep the token safe.
- Fan out with Map and know when to move to Distributed Map; redrive a failed execution.
- Lint and test a state machine without deploying it (structural linter,
ValidateStateMachineDefinition, theTestStateAPI with mocks). - Deploy the workflow with CDK (C#) and Terraform from one shared ASL file.
Prerequisites
- Asynchronous communication and event-driven architecture: why a workflow is not a message
- Retry strategies and Idempotency in .NET
- Amazon EventBridge: buses, rules and Pipes: the Pipe in that lesson starts the workflow you build here
- Tools: .NET 10 SDK or Docker, AWS CLI v2 with the
learn-sandboxSSO profile, optionally the CDK CLI or Terraform 1.14
The saga lesson comes next and builds a compensating payment saga on these foundations. This lesson is the toolbox; that one is a worked design.
Why this matters in banking
An outbound payment above a threshold at Harbourline goes through screening, then either straight through, or to a human reviewer who may take hours, then a posting record, then notifications to every party. The first implementation was a long-running service with an in-memory state machine and a database table of "where is this payment". It worked until:
- A deployment restarted the service while 400 payments were waiting for a reviewer. Their state was in memory. Operations spent a day reconstructing it from logs.
- A reviewer approved a payment a minute after the timeout job had rejected it. Nobody could say which decision won, because the logic for waiting, timing out and deciding was spread over three classes.
- An auditor asked for the path one payment took. The answer was a grep through four log streams.
Step Functions is an orchestrator: the workflow definition is the documentation, the service holds the state between steps (for up to a year), every execution has a visual, queryable history, and retries, timeouts and waiting are declared instead of coded. The costs are a new language (ASL), a per-transition price, and the discipline of keeping business logic in your code and flow in the definition.
The idea in plain language
A state machine is a diagram turned into JSON. Each box is a state. An execution is one run of the diagram with an input, a current state and a growing history. Step Functions remembers where every execution is, calls the code or AWS service each Task state names, passes the output to the next state, waits when told to, retries when a step fails, and follows the arrows.
You write the diagram in Amazon States Language (ASL), a JSON dialect. You rarely write it from nothing: the Workflow Studio designer generates it, and you keep the JSON in source control.
Analogy: A mortgage case file with a checklist stapled inside. The checklist says: check ID; run the credit search; if the score is borderline, send the file to the underwriter's tray and wait (the file sits there, possibly for days, without anyone watching it); when the underwriter returns it with a decision, continue; then send the letters. The file cabinet is Step Functions. It never forgets where a file is, and the checklist can be read by anyone, including the auditor.
The state types
| State | What it does | Banking example |
|---|---|---|
| Task | Calls Lambda or an AWS service (over 200 via the AWS SDK integrations), or waits for a callback | Screen the payment; write the decision to DynamoDB |
| Choice | Branches on the data | Risk score under 20 approve, over 80 reject, otherwise review |
| Map | Runs the same steps for each item of an array | Notify every party to the payment |
| Parallel | Runs different branches at once and waits for all | Sanctions check and fraud score in parallel |
| Wait | Pauses for a time or until a timestamp | Hold a scheduled payment until 06:00 |
| Pass | Passes data on, optionally reshaping it | Set defaults, test wiring |
| Succeed / Fail | Ends the execution, successfully or with an error name and cause | Reject with PaymentReviewFailed |
What problem AWS Step Functions solves
Step Functions gives you durable orchestration as a service: state persisted between steps, declarative retry and timeout, waiting that costs nothing while it waits, parallelism, and a record of every step. You use it where the sequence and the failure handling are the hard part: payment journeys, onboarding, loan decisions, data pipelines, long-running approvals.
Standard versus Express
| Standard | Express | |
|---|---|---|
| Maximum duration | 1 year | 5 minutes |
| Execution semantics | Exactly-once workflow execution. A task state runs once unless you wrote a Retry | Asynchronous: at-least-once. Synchronous (StartSyncExecution): at-most-once |
| Execution history | Kept by Step Functions for 90 days after the execution closes (30 days on request) | Not kept. You must turn on CloudWatch Logs |
| Start rate | 300 per second sustained in us-east-1 (bucket 1,300) | 6,000 per second |
| State transition rate | 5,000 per second in us-east-1 (soft quota) | Unlimited |
.sync and .waitForTaskToken | Supported | Not supported |
| Distributed Map, Activities | Supported | Not supported |
| Redrive | Yes, within 14 days | No |
| Billed by | State transitions | Number of executions, duration and memory |
| Best for | Long-running, auditable, non-idempotent work: payments, approvals | High-volume, short, idempotent work: stream processing, API back ends |
The workflow type cannot be changed after creation. A common pattern is a Standard parent that starts Express children for the high-volume steps.
Limits and defaults (checked 2026-11-18)
| Setting | Value |
|---|---|
| Input, output and task payload | 256 KiB per state, task and execution (UTF-8 string). Larger data: keep it in S3 or DynamoDB and pass a pointer |
| Standard execution history | 25,000 events. A long loop or a big Map can hit it: start a new execution or use child workflows |
| State machine definition size | 1 MB |
| Open Standard executions | 1,000,000 per account per Region |
| Execution and task names | Up to 80 characters; no whitespace, wildcards, brackets or special characters (for example double quote, hash, percent, backslash, caret, pipe, tilde, dollar sign, ampersand, comma, semicolon, colon or slash); use ASCII |
| Distributed Map parallel children | Up to 10,000 per Map Run |
| Published versions per state machine | 1,000; aliases 100 |
| Redrive window | 14 days |
| HTTP Task time | 60 seconds |
| New accounts | Have reduced state transition quotas that rise with use |
Honest limitations
- It is not a data store and not a message bus. The 256 KiB payload limit is deliberate.
- Per-transition pricing punishes tight loops. Polling every second in a Standard workflow costs money; use a Wait, a
.syncintegration or a callback. - "Exactly-once" is about the workflow, not your code. A Lambda task that timed out may still have done its work. Tasks must be idempotent.
- The history is personal data. Inputs and outputs of every state sit in the execution history for 90 days, and (if you log them) in CloudWatch Logs.
- JSON is the language. Large definitions are hard to review. Keep states small and push logic into Lambda or the service integrations.
Microsoft / Azure comparison
| You know (Microsoft/Azure) | AWS equivalent | What's different |
|---|---|---|
| Durable Functions orchestrator | Standard workflow | Durable Functions is code (C#) that replays; Step Functions is a declarative definition with a visual history. No determinism rules for your code, but you leave C# for JSON/ASL |
context.CallActivityAsync | Task state with a Lambda or service integration | Many AWS services are called directly with no Lambda in between |
context.WaitForExternalEvent | .waitForTaskToken callback | The token is an opaque secret the external system returns with SendTaskSuccess |
Task.WhenAll fan-out | Parallel and Map states | Map handles a dynamic array; Distributed Map handles millions of S3 items |
Retry policies in TaskOptions | Retry field | Declared per state; JitterStrategy: FULL available |
| Durable timers | Wait state | Costs nothing while it waits; up to the one-year execution limit |
| Logic Apps (Standard/Consumption) | Step Functions Standard, plus Pipes and EventBridge | Logic Apps is a designer-first connector platform; Step Functions is closer to code review and IaC |
| Durable Functions Eternal orchestration | A new execution per iteration (the 25,000-event history limit) | There is no "continue as new" call: a state starts the next execution |
| Orchestration instance ID | Execution name | A name is an idempotency key for Standard executions |
Amazon States Language and JSONata
Two query languages
A state machine declares "QueryLanguage": "JSONata" or the default "JSONPath". You can set it per state, so you can migrate gradually. The saga lesson uses JSONPath; this lesson uses JSONata, which AWS recommends for new workflows because it replaces five path fields with two and adds real expressions.
| Concept | JSONPath | JSONata |
|---|---|---|
| Select what a task receives | InputPath, Parameters | Arguments |
| Reshape the result | ResultSelector, ResultPath, OutputPath | Output |
| Expressions | .$ suffix on keys; limited intrinsic functions | "{% expression %}" in any string value |
| Variables | Assign with paths | Assign with expressions |
| Choice conditions | Variable + StringEquals, NumericGreaterThan and so on | Condition with a JSONata boolean |
| Map items | ItemsPath | Items |
| The state's input | $ | $states.input |
In a JSONata state the reserved variable $states holds input (the state's input), result (the task's raw result), errorOutput (inside a Catch) and context (execution metadata, including the task token).
Rules that bite:
AssignandOutputrun in parallel. A value you transform inAssignis not available toOutput: repeat the expression.- Expressions can fail at run time with
States.QueryEvaluationError(type error, a missing field, an undefined result, one second of evaluation time, memory). You canRetryorCatchit. - A string must start with
{%and end with%}with no spaces outside, or the definition is invalid. - Fields like
Typeand a Task'sResourcemust stay constant strings. - Built-in helpers replace intrinsic functions:
$uuid(),$hash(),$partition(),$range(),$random(),$parse()(JSONata's$evalis not available).
Here are the four lines that matter, from the workflow in this lesson:
"Assign": { "paymentId": "{% $states.input.paymentId %}" },
"Output": "{% $states.result.Payload %}",
"Condition": "{% $states.input.riskScore >= 80 %}",
"taskToken": "{% $states.context.Task.Token %}"Variables
Assign stores values in variables ($paymentId) that later states can read, so you stop threading data through every state's output. Use them for facts needed late in the workflow (the payment ID in the notification step, 20 states after the screening). Variables have size quotas ⚠ VERIFY the current figures, and like inputs they appear in the history.
Service integrations
A Task's Resource names what to call. The prefix arn:aws:states::: is a Step Functions namespace.
- Optimized integrations (
arn:aws:states:::lambda:invoke,dynamodb:putItem,sqs:sendMessage,sns:publish,events:putEvents,states:startExecution,ecs:runTask,glue:startJobRunand more) have tailored request and result shapes. - AWS SDK integrations (
arn:aws:states:::aws-sdk:<service>:<action>) call over two hundred services' API actions directly. - HTTP Task (
arn:aws:states:::http:invoke) calls any HTTPS API using an EventBridge connection for credentials (60-second limit per call).
Each integration supports a pattern:
| Pattern | Resource suffix | Step Functions waits for | Standard | Express |
|---|---|---|---|---|
| Request Response | none | The HTTP response of the call | Yes | Yes |
| Run a Job | .sync | The job to finish (it polls or listens to events for you) | Some services (ECS, Glue, Batch, Athena, nested workflows, Bedrock...) | No |
| Wait for Callback | .waitForTaskToken | SendTaskSuccess or SendTaskFailure with the token | Many services: SQS, SNS, Lambda, EventBridge, API Gateway, ECS and others (not DynamoDB) | No |
Two details from the integration table are worth remembering. DynamoDB supports Request Response only. Lambda, SQS and SNS support Request Response and callback, but not .sync. If the integration you want lacks the pattern, use .waitForTaskToken with a message or an event that carries the token.
.sync has a hidden cost: for the same account Step Functions listens to events and polls the service's describe API, consuming your quota. If a .sync task is aborted and cannot be cancelled (for example, the role lacks the permission), the job may run on and bill you.
Error handling: Retry and Catch
Every state except Pass and Wait can fail. By default an error fails the whole execution. Task, Parallel and Map can declare Retry and Catch.
Error names
| Name | Meaning |
|---|---|
States.ALL | Wildcard for known errors. Must be alone in ErrorEquals and last in the list. It does not catch States.DataLimitExceeded or States.Runtime |
States.TaskFailed | Any task failure except States.Timeout |
States.Timeout | Task exceeded TimeoutSeconds, a heartbeat was missed, or the whole execution exceeded its timeout |
States.HeartbeatTimeout | No heartbeat within HeartbeatSeconds |
States.Permissions | The execution role lacks permission |
States.DataLimitExceeded | A payload went over 256 KiB. Terminal unless you list it explicitly |
States.Runtime | An unexpected runtime failure (for example a path on null data). Never retriable or catchable |
States.QueryEvaluationError | A JSONata expression failed |
States.ExceedToleratedFailureThreshold | A Map ran past its failure threshold |
Lambda.ServiceException, Lambda.SdkClientException, Lambda.TooManyRequestsException, Lambda.AWSLambdaException | Transient Lambda service errors. Always retry these |
| Your own names | Anything not starting States.. A .NET exception type name becomes the error name |
Retry fields and defaults
| Field | Default | Notes |
|---|---|---|
ErrorEquals | required | Retriers are scanned in order |
IntervalSeconds | 1 | Delay before the first retry |
MaxAttempts | 3 | 0 means never retry this error |
BackoffRate | 2.0 | Multiplier per attempt |
MaxDelaySeconds | none | Caps the backoff; always set it |
JitterStrategy | NONE | FULL randomises each delay between 0 and the computed delay. Use FULL |
A retrier's counter belongs to the state, per retrier, across all visits. Retries count as state transitions (and are billed). When a state has both Retry and Catch, retriers run first, and the catcher runs only when retries are exhausted or the error did not match a retrier.
Catch
A catcher names ErrorEquals and a Next state, and (in JSONata) can Assign and shape the Output using $states.errorOutput, which holds Error and Cause. In JSONPath, ResultPath controls whether the error replaces or is added to the input.
The Lambda retry you should always write
Lambda can fail before your code runs (throttling, a transient service error). The documented defence is a retrier for Lambda.ServiceException, Lambda.AWSLambdaException, Lambda.SdkClientException and Lambda.TooManyRequestsException with a few attempts and jitter. The workflow in this lesson has it on the screening task.
Timeouts
Always set TimeoutSeconds on every Task (a hung worker would otherwise hold the execution for up to a year) and a top-level TimeoutSeconds on the state machine. For callbacks, add HeartbeatSeconds so a dead reviewer tool fails fast. The linter in this lesson flags a Task without a timeout.
The callback pattern: waiting for a human
A callback pauses the workflow at a Task until something outside calls SendTaskSuccess or SendTaskFailure with a task token. Nothing runs and nothing is billed while it waits (up to the one-year execution limit).
The token is a bearer secret: anyone who holds it can finish the task, from a principal in the same account (tokens do not work across accounts). Treat it like a session cookie: send it only to the review service, do not log it, do not put it in a URL that is e-mailed to a reviewer, and store it encrypted if you keep it. The review service's role needs states:SendTaskSuccess and states:SendTaskFailure, nothing else.
The workflow below uses a 24-hour TimeoutSeconds and routes States.Timeout to a rejection. Be deliberate about the race in the third incident above: whichever of SendTaskSuccess and the timeout arrives first wins, and the loser gets an error (TaskTimedOut) that the review service must handle gracefully and show to the reviewer.
Map and Distributed Map
Map runs the same sub-workflow for every item of an array. In the lesson's workflow, one notification per recipient.
| Inline Map | Distributed Map | |
|---|---|---|
| Each iteration | Part of the parent execution (shares its history) | A child workflow execution with its own history (Standard or Express) |
| Input | An array in the state's input (up to the 256 KiB limit) | The same, or a large S3 source: JSON, CSV, a list of objects, an inventory |
| Concurrency | Limited (the docs point to Distributed above 40 concurrent iterations) | Up to 10,000 parallel child executions |
| Result | An array | An array, or written to S3 with ResultWriter |
| Use when | Small arrays, a few hundred items | A file of a million transactions, a nightly reconciliation |
Both support MaxConcurrency and a tolerated failure threshold (ToleratedFailurePercentage or ToleratedFailureCount, default zero: any failure fails the Map). Set MaxConcurrency to what the downstream can take, not to what Step Functions can give. Distributed Map bills one state transition per iteration, so a 10-million-row file is a real cost.
Redrive
When a Standard execution fails, times out or is aborted, redrive restarts it from the failed state with the same input, definition and execution ARN, within 14 days. It reruns the unsuccessful state, keeps the history of the successful ones and resets retry counters. For a Parallel or inline Map only the failed branches or iterations rerun; for a Distributed Map only the unsuccessful child executions rerun. Executions older than 14 days, started before 15 November 2023 or with more than 24,999 history events cannot be redriven.
Redrive is the safe way to recover after you fix a bug or a permission, because a state that already completed will not run again. It is also why completed states must be correct: you cannot "replay from the start" with redrive.
Versions and aliases
Publish a version to freeze a definition, and point an alias (for example live) at one or two versions with weights for a gradual rollout. Callers start executions against the alias ARN. Executions run to completion on the version they started with, which matters for year-long workflows: you may have executions of several definitions in flight. Keep the definition changes backward compatible with the data that older executions carry.
Testing
Test in layers, cheapest first.
- Structural lint in CI (the
AslLintclass below): catches a misspeltNext, an unreachable state, a Task with no timeout, and JSONPath fields inside a JSONata machine. It needs no AWS account. ValidateStateMachineDefinition(aws stepfunctions validate-state-machine-definition): the service checks syntax and semantics. Use it in CI with a role that can only call that API. ⚠ VERIFY the exact CLI flags for your CLI version.TestStateAPI: runs one state (or a state from a full definition) in isolation and returns output, next state and status. Since November 2025 it can mock service integrations (--mock '{"result": "..."}'or{"errorOutput": {...}}), which also lets you test Map, Parallel,.syncand.waitForTaskTokenstates, simulate retry attempts (retrierRetryCount) and assert whichCatchorRetryapplied (catchIndex,retryIndex). With a mock you do not need an execution role. The console version has fewer features than the CLI and SDK.- A real execution in a sandbox account with the Lambdas stubbed, to prove IAM and wiring.
# Test the Route choice with a mock-free state: input riskScore 85 should go to RejectPayment.
aws stepfunctions test-state \
--definition '{"QueryLanguage":"JSONata","Type":"Choice","Choices":[{"Condition":"{% $states.input.riskScore >= 80 %}","Next":"RejectPayment"},{"Condition":"{% $states.input.riskScore < 20 %}","Next":"ApprovePayment"}],"Default":"ManualReview"}' \
--role-arn "$ROLE_ARN" --input '{"riskScore": 85}'
# Expected: "nextState": "RejectPayment", "status": "SUCCEEDED"
# Prove the screening retry: simulate a throttled Lambda on the second attempt with a mock error.
aws stepfunctions test-state --definition file://screen-state.json \
--input '{"paymentId":"PAY-1"}' \
--state-configuration '{"retrierRetryCount": 1}' \
--mock '{"errorOutput": {"error": "Lambda.TooManyRequestsException", "cause": "Rate exceeded"}}' \
--inspection-level DEBUG
# Expected: "status": "RETRIABLE" with errorDetails.retryIndex 0 and a backoff intervalObservability
- Console: the execution graph colours every state and shows input, output, retries and timing. For a Standard execution it is the audit trail for 90 days.
- CloudWatch Logs is the only history for Express. Choose a level (
ALL,ERROR,FATAL,OFF). SetIncludeExecutionDatato false unless you have decided that payment data may live in logs. - Metrics:
ExecutionsStarted,ExecutionsFailed,ExecutionsTimedOut,ExecutionThrottled,ExecutionTime. Alarm on failed and timed-out executions, and on throttling (new accounts have lower quotas). - EventBridge events: Step Functions emits execution status changes (succeeded, failed, timed out, aborted). A rule on
FAILEDandTIMED_OUTthat targets a ticket queue is the usual way to react to top-level failures, whichCatchcannot handle. - X-Ray: switch on tracing to see the whole path including the Lambda and service calls.
Architecture
Security-boundary view
The state machine's execution role is the most powerful identity here: it can do whatever the workflow does. Grant one statement per integration, scoped to one resource, and add aws:SourceArn and aws:SourceAccount conditions to its trust policy.
Step-by-step implementation
Create the state machine's dependencies
You need a screening Lambda (any function that returns {"riskScore": 50, ...}), the decisions table, the manual-review queue and the notification topic. The IaC below creates the last three and imports the Lambda by ARN, so you can use a stub.
Lint the definition
Run the linter over payment-review.asl.json (the test project does). It must report nothing. Break a Next on purpose and read the message.
Deploy with the IaC of your choice
CDK and Terraform both load the same ASL file and fill the ${Placeholders}. Keep the file in source control and review changes to it like code.
Run the three paths
Start three executions with riskScore 10 (straight through), 90 (rejected) and 50 (manual review). The third will wait. Open its task token from the review queue message.
Complete the callback from .NET
Call ApproveAsync with the token. Watch the execution continue, take the approved branch, write to DynamoDB and notify each recipient. Then run another review and let it time out (set a 60-second timeout in a sandbox copy) to see the rejection path.
Break it and redrive
Remove the dynamodb:PutItem permission, start an approved payment and watch it fail with States.Permissions at ApprovePayment. Restore the permission and redrive: only ApprovePayment and the states after it run.
AWS Console instructions
Console → Step Functions → State machines → Create state machine. Choose Standard, Design your workflow visually or Write your workflow in code. In Workflow Studio, set the Query language to JSONata at the top level, drag in Lambda Invoke, Choice, SQS SendMessage (tick Wait for callback), DynamoDB PutItem and Map. Under Config → Permissions let the console create an execution role to read what it generates, then replace it with your least-privilege role. Turn on Logging at the Error level and leave Include execution data off. Use Test state on a single state with the INFO or DEBUG inspection level.
To run: Start execution, paste the JSON input. Open an execution to see the Graph view, per-state Input/Output, Retries & redrives, and the Redrive button on failed Standard executions.
AWS CLI
export AWS_PROFILE=learn-sandbox AWS_REGION=us-east-1
# Create (use the IaC for anything beyond learning). The role must already exist.
aws stepfunctions create-state-machine --name hl-m08gap-payment-review --type STANDARD \
--role-arn "$ROLE_ARN" --definition file://payment-review.rendered.json \
--logging-configuration level=ERROR,includeExecutionData=false,destinations=[{cloudWatchLogsLogGroup={logGroupArn=$LOG_GROUP_ARN}}] \
--tracing-configuration enabled=true
# Ask the service to validate the definition without creating anything.
aws stepfunctions validate-state-machine-definition --definition file://payment-review.rendered.json
# Start an execution. The name is an idempotency key: the same name and input while it runs returns the existing execution.
aws stepfunctions start-execution --state-machine-arn "$SM_ARN" --name PAY-2026-0001 \
--input '{"paymentId":"PAY-2026-0001","amount":1250.00,"recipients":["CUST-0001"]}'
# Where is it? What happened?
aws stepfunctions describe-execution --execution-arn "$EXEC_ARN" --query '{status:status,start:startDate}'
aws stepfunctions get-execution-history --execution-arn "$EXEC_ARN" --reverse-order --max-items 10 \
--query 'events[].{id:id,type:type}'
# Complete a callback. The token comes from the SQS message body; never log it.
aws stepfunctions send-task-success --task-token "$TASK_TOKEN" --task-output '{"decision":"approved"}'
# Recover a failed Standard execution (within 14 days).
aws stepfunctions redrive-execution --execution-arn "$EXEC_ARN"
# Stop a runaway execution.
aws stepfunctions stop-execution --execution-arn "$EXEC_ARN" --error Operator --cause "Stopped during incident INC-123"Infrastructure as Code
The stack reads the ASL file, fills the ${Placeholders} through DefinitionSubstitutions, logs errors without execution data, enables tracing, sets an execution-level timeout and adds one least-privilege grant per integration.
using Amazon.CDK;
using Amazon.CDK.AWS.DynamoDB;
using Amazon.CDK.AWS.Lambda;
using Amazon.CDK.AWS.Logs;
using Amazon.CDK.AWS.SNS;
using Amazon.CDK.AWS.SQS;
using Amazon.CDK.AWS.StepFunctions;
using Constructs;
namespace Harbourline.Messaging.Infra;
public sealed class WorkflowStackProps : StackProps
{
public required string ScreeningFunctionArn { get; init; }
/// <summary>Path to payment-review.asl.json.</summary>
public required string DefinitionPath { get; init; }
}
/// <summary>
/// A Standard workflow from an ASL file kept in source control. The file uses ${Placeholders}; CDK fills them from
/// DefinitionSubstitutions at deploy time, so the same JSON works in every environment. Execution data is NOT logged:
/// payment inputs are personal data, and CloudWatch Logs is another copy to protect and expire.
/// </summary>
public sealed class WorkflowStack : Stack
{
public StateMachine StateMachine { get; }
public WorkflowStack(Construct scope, string id, WorkflowStackProps props) : base(scope, id, props)
{
var decisions = new Table(this, "Decisions", new TableProps
{
TableName = "hl-m08gap-decisions",
PartitionKey = new Amazon.CDK.AWS.DynamoDB.Attribute { Name = "pk", Type = AttributeType.STRING },
BillingMode = BillingMode.PAY_PER_REQUEST,
RemovalPolicy = RemovalPolicy.DESTROY, // lab only
});
var reviewQueue = new Queue(this, "ReviewQueue", new QueueProps
{
QueueName = "hl-m08gap-manual-review",
Encryption = QueueEncryption.SQS_MANAGED,
EnforceSSL = true,
});
var noticeTopic = new Topic(this, "NoticeTopic", new TopicProps { TopicName = "hl-m08gap-payment-notices", EnforceSSL = true });
var screening = Function.FromFunctionArn(this, "Screening", props.ScreeningFunctionArn);
var logs = new LogGroup(this, "Logs", new LogGroupProps
{
LogGroupName = "/hl-m08gap/payment-review",
Retention = RetentionDays.ONE_WEEK,
RemovalPolicy = RemovalPolicy.DESTROY,
});
StateMachine = new StateMachine(this, "PaymentReview", new StateMachineProps
{
StateMachineName = "hl-m08gap-payment-review",
StateMachineType = StateMachineType.STANDARD,
DefinitionBody = DefinitionBody.FromString(File.ReadAllText(props.DefinitionPath)),
DefinitionSubstitutions = new Dictionary<string, string>
{
["ScreeningFunctionArn"] = props.ScreeningFunctionArn,
["ReviewQueueUrl"] = reviewQueue.QueueUrl,
["DecisionsTableName"] = decisions.TableName,
["NoticeTopicArn"] = noticeTopic.TopicArn,
},
Logs = new LogOptions { Destination = logs, Level = LogLevel.ERROR, IncludeExecutionData = false },
TracingEnabled = true,
Timeout = Duration.Days(2), // a hard ceiling for the whole execution, on top of the per-task timeouts
});
// Least privilege, one grant per integration the ASL actually uses.
screening.GrantInvoke(StateMachine);
decisions.GrantWriteData(StateMachine);
reviewQueue.GrantSendMessages(StateMachine);
noticeTopic.GrantPublish(StateMachine);
_ = new CfnOutput(this, "StateMachineArn", new CfnOutputProps { Value = StateMachine.StateMachineArn });
}
}Complete code
The state machine (ASL, JSONata)
{
"Comment": "Harbourline outbound payment review (Standard workflow, JSONata)",
"QueryLanguage": "JSONata",
"StartAt": "ScreenPayment",
"States": {
"ScreenPayment": {
"Type": "Task",
"Resource": "arn:aws:states:::lambda:invoke",
"Arguments": {
"FunctionName": "${ScreeningFunctionArn}",
"Payload": "{% $states.input %}"
},
"Assign": {
"paymentId": "{% $states.input.paymentId %}",
"amount": "{% $states.input.amount %}",
"recipients": "{% $states.input.recipients %}"
},
"Output": "{% $states.result.Payload %}",
"TimeoutSeconds": 30,
"Retry": [
{
"ErrorEquals": ["Lambda.ServiceException", "Lambda.AWSLambdaException", "Lambda.SdkClientException", "Lambda.TooManyRequestsException"],
"IntervalSeconds": 2,
"MaxAttempts": 4,
"BackoffRate": 2.0,
"JitterStrategy": "FULL"
}
],
"Catch": [
{
"ErrorEquals": ["States.ALL"],
"Assign": { "failure": "{% $states.errorOutput %}" },
"Next": "RecordFailure"
}
],
"Next": "Route"
},
"Route": {
"Type": "Choice",
"Choices": [
{ "Condition": "{% $states.input.riskScore >= 80 %}", "Next": "RejectPayment" },
{ "Condition": "{% $states.input.riskScore < 20 %}", "Next": "ApprovePayment" }
],
"Default": "ManualReview"
},
"ManualReview": {
"Type": "Task",
"Resource": "arn:aws:states:::sqs:sendMessage.waitForTaskToken",
"Arguments": {
"QueueUrl": "${ReviewQueueUrl}",
"MessageBody": {
"paymentId": "{% $paymentId %}",
"taskToken": "{% $states.context.Task.Token %}"
}
},
"TimeoutSeconds": 86400,
"Catch": [
{ "ErrorEquals": ["States.Timeout"], "Assign": { "failure": "{% $states.errorOutput %}" }, "Next": "RejectPayment" }
],
"Next": "DecisionChoice"
},
"DecisionChoice": {
"Type": "Choice",
"Choices": [
{ "Condition": "{% $states.input.decision = 'approved' %}", "Next": "ApprovePayment" }
],
"Default": "RejectPayment"
},
"ApprovePayment": {
"Type": "Task",
"Resource": "arn:aws:states:::dynamodb:putItem",
"Arguments": {
"TableName": "${DecisionsTableName}",
"Item": {
"pk": { "S": "{% $paymentId %}" },
"decision": { "S": "approved" },
"amount": { "N": "{% $string($amount) %}" }
}
},
"TimeoutSeconds": 15,
"Output": { "decision": "approved" },
"Next": "NotifyCustomers"
},
"RejectPayment": {
"Type": "Task",
"Resource": "arn:aws:states:::dynamodb:putItem",
"Arguments": {
"TableName": "${DecisionsTableName}",
"Item": {
"pk": { "S": "{% $paymentId %}" },
"decision": { "S": "rejected" },
"amount": { "N": "{% $string($amount) %}" }
}
},
"TimeoutSeconds": 15,
"Output": { "decision": "rejected" },
"Next": "NotifyCustomers"
},
"NotifyCustomers": {
"Type": "Map",
"Items": "{% $recipients %}",
"MaxConcurrency": 5,
"ToleratedFailurePercentage": 10,
"ItemSelector": {
"customerId": "{% $states.context.Map.Item.Value %}",
"paymentId": "{% $paymentId %}",
"decision": "{% $states.input.decision %}"
},
"ItemProcessor": {
"ProcessorConfig": { "Mode": "INLINE" },
"StartAt": "PublishNotice",
"States": {
"PublishNotice": {
"Type": "Task",
"Resource": "arn:aws:states:::sns:publish",
"Arguments": {
"TopicArn": "${NoticeTopicArn}",
"Message": "{% $string($states.input) %}"
},
"TimeoutSeconds": 15,
"Retry": [
{ "ErrorEquals": ["States.TaskFailed"], "IntervalSeconds": 1, "MaxAttempts": 3, "BackoffRate": 2.0, "JitterStrategy": "FULL" }
],
"End": true
}
}
},
"End": true
},
"RecordFailure": {
"Type": "Fail",
"Error": "PaymentReviewFailed",
"Cause": "Screening failed after retries; see the execution history for the caught error."
}
}
}Read it as the diagram above. ScreenPayment calls a Lambda, stores three variables, and retries the four transient Lambda errors with full jitter, then routes any other error to RecordFailure. Route branches on the risk score. ManualReview sends the task token to SQS and waits up to 24 hours. Both decisions write one item to DynamoDB, then NotifyCustomers maps over the recipients with MaxConcurrency 5, tolerating 10 percent failures, publishing one SNS message per recipient with its own retry.
⚠ VERIFY before you rely on it
The definition was linted and unit-tested structurally, but it could not be validated against the live service from the machine that wrote this lesson. Run aws stepfunctions validate-state-machine-definition on it (with the ${Placeholders} replaced) before deploying. In particular check the ItemSelector field in a JSONata Map and the Assign inside Catch: the documentation for JSONata Map fields is spread across several pages.
The .NET client and the linter
using System.Text.Json;
using System.Text.RegularExpressions;
using Amazon.StepFunctions;
using Amazon.StepFunctions.Model;
namespace Harbourline.Messaging.Samples.StepFunctions;
public sealed record PaymentReviewInput(string PaymentId, decimal Amount, IReadOnlyList<string> Recipients);
/// <summary>
/// Starts payment-review executions and completes the manual-review callback.
/// Two Step Functions rules shape this class:
/// 1. A Standard execution NAME is an idempotency key. Starting the same name with the same input while it is
/// running returns the existing execution instead of a second one. Build the name from the business key.
/// 2. A callback task waits for SendTaskSuccess or SendTaskFailure carrying the task token. The token is a secret
/// capability: whoever holds it can finish the task, so only the review service may receive it.
/// </summary>
public sealed partial class PaymentReviewClient(IAmazonStepFunctions sfn, string stateMachineArn)
{
// Execution names: max 80 chars, no whitespace, wildcards, brackets or the special characters " # % \ ^ | ~ ` $ & , ; : /
[GeneratedRegex("[^A-Za-z0-9_-]")]
private static partial Regex Unsafe();
public static string ExecutionNameFor(string paymentId)
{
var cleaned = Unsafe().Replace(paymentId, "-");
return cleaned.Length <= 80 ? cleaned : cleaned[..80];
}
public async Task<string> StartAsync(PaymentReviewInput input, CancellationToken ct = default)
{
var response = await sfn.StartExecutionAsync(new StartExecutionRequest
{
StateMachineArn = stateMachineArn,
Name = ExecutionNameFor(input.PaymentId),
Input = JsonSerializer.Serialize(input, JsonSerializerOptions.Web),
}, ct);
return response.ExecutionArn;
}
public Task ApproveAsync(string taskToken, CancellationToken ct = default) =>
sfn.SendTaskSuccessAsync(new SendTaskSuccessRequest
{
TaskToken = taskToken,
Output = JsonSerializer.Serialize(new { decision = "approved" }),
}, ct);
public Task RejectAsync(string taskToken, CancellationToken ct = default) =>
sfn.SendTaskSuccessAsync(new SendTaskSuccessRequest
{
TaskToken = taskToken,
Output = JsonSerializer.Serialize(new { decision = "rejected" }),
}, ct);
/// <summary>For a reviewer tool that crashed: fails the task so the workflow's Catch can run instead of waiting for the timeout.</summary>
public Task FailAsync(string taskToken, string error, string cause, CancellationToken ct = default) =>
sfn.SendTaskFailureAsync(new SendTaskFailureRequest { TaskToken = taskToken, Error = error, Cause = cause }, ct);
}using System.Text.Json.Nodes;
namespace Harbourline.Messaging.Samples.StepFunctions;
/// <summary>
/// A tiny structural linter for Amazon States Language, meant to run in CI BEFORE the slower TestState API or a deploy.
/// It checks: StartAt exists, every Next / Default / Catch target exists, no state is unreachable, every Task has a
/// timeout, and a workflow is not mixing JSONPath-only fields into a JSONata machine. It does not replace the
/// service-side ValidateStateMachineDefinition API.
/// </summary>
public static class AslLint
{
private static readonly string[] JsonPathOnlyFields = ["InputPath", "Parameters", "ResultSelector", "ResultPath", "OutputPath"];
public static IReadOnlyList<string> Check(string aslJson)
{
var root = JsonNode.Parse(aslJson) as JsonObject ?? throw new ArgumentException("ASL must be a JSON object.", nameof(aslJson));
var problems = new List<string>();
var jsonata = root["QueryLanguage"]?.GetValue<string>() == "JSONata";
CheckMachine(root, "$", jsonata, problems);
return problems;
}
private static void CheckMachine(JsonObject machine, string path, bool jsonata, List<string> problems)
{
var startAt = machine["StartAt"]?.GetValue<string>();
var states = machine["States"] as JsonObject;
if (startAt is null || states is null) { problems.Add($"{path}: StartAt and States are required."); return; }
if (!states.ContainsKey(startAt)) problems.Add($"{path}: StartAt '{startAt}' is not a state.");
var reachable = new HashSet<string>();
var queue = new Queue<string>();
if (states.ContainsKey(startAt)) { queue.Enqueue(startAt); reachable.Add(startAt); }
while (queue.Count > 0)
{
var name = queue.Dequeue();
var state = (JsonObject)states[name]!;
foreach (var target in Targets(state))
{
if (!states.ContainsKey(target)) problems.Add($"{path}.{name}: target '{target}' is not a state.");
else if (reachable.Add(target)) queue.Enqueue(target);
}
}
foreach (var (name, node) in states)
{
var state = (JsonObject)node!;
var here = $"{path}.{name}";
if (!reachable.Contains(name)) problems.Add($"{here}: unreachable state.");
var type = state["Type"]?.GetValue<string>();
if (type == "Task" && state["TimeoutSeconds"] is null)
problems.Add($"{here}: Task without TimeoutSeconds (a hung worker would hold the execution for up to a year).");
if (type is "Task" or "Pass" or "Map" or "Parallel" or "Wait" && state["Next"] is null && state["End"] is null)
problems.Add($"{here}: needs Next or End.");
if (jsonata) foreach (var field in JsonPathOnlyFields)
if (state.ContainsKey(field)) problems.Add($"{here}: '{field}' is a JSONPath field and is invalid in a JSONata state.");
// Recurse into nested machines. Their names are scoped to the branch.
if (state["Branches"] is JsonArray branches)
for (var i = 0; i < branches.Count; i++) CheckMachine((JsonObject)branches[i]!, $"{here}.Branches[{i}]", jsonata, problems);
if ((state["ItemProcessor"] ?? state["Iterator"]) is JsonObject inner)
CheckMachine(inner, $"{here}.ItemProcessor", jsonata, problems);
}
}
private static IEnumerable<string> Targets(JsonObject state)
{
if (state["Next"] is JsonValue next) yield return next.GetValue<string>();
if (state["Default"] is JsonValue def) yield return def.GetValue<string>();
foreach (var rule in (state["Choices"] as JsonArray) ?? [])
if (rule?["Next"] is JsonValue n) yield return n.GetValue<string>();
foreach (var c in (state["Catch"] as JsonArray) ?? [])
if (c?["Next"] is JsonValue n) yield return n.GetValue<string>();
}
}The unit tests lint the real ASL file and prove the linter catches a broken Next, a Task without a timeout, and a JSONPath field in a JSONata machine.
[Fact]
public void The_lesson_state_machine_is_structurally_clean()
{
Assert.Empty(AslLint.Check(PaymentReview));
}
[Fact]
public void A_missing_target_is_reported()
{
var broken = PaymentReview.Replace("\"Next\": \"RecordFailure\"", "\"Next\": \"RecordFailur\"");
var problems = AslLint.Check(broken);
Assert.Contains(problems, p => p.Contains("'RecordFailur' is not a state"));
Assert.Contains(problems, p => p.Contains("RecordFailure") && p.Contains("unreachable"));
}dotnet add package AWSSDK.StepFunctions --version 4.1.0.2Code explained line by line
ExecutionNameForreplaces every character outside letters, digits,_and-with a dash and caps the name at 80 characters, the documented limits. Why use the payment ID as the name? A Standard execution name is an idempotency key: starting the same name with the same input while it runs returns the existing execution, so a retried API request cannot start a second review of the same payment. Reusing a name after an execution closed works only when the 90-day history has expired, so do not rely on that.StartAsyncserialises the input withJsonSerializerOptions.Web(camelCase) to match the ASL field namespaymentIdandrecipients.ApproveAsyncandRejectAsynccallSendTaskSuccesswith the decision as the task output, which the next state sees as$states.input.decision.FailAsyncsendsSendTaskFailure, so a crashed reviewer tool can fail the task now instead of waiting for the timeout.- In
AslLint.Check,QueryLanguagedecides whether JSONPath-only fields are errors.CheckMachineis recursive and scoped per nested machine, because state names inside a Parallel branch or MapItemProcessorare local to it. - The reachability walk is a plain breadth-first search from
StartAtoverNext,Default,Choices[].NextandCatch[].Next. A state never reached is reported, which catches the typo-in-Nextcase from both ends. - The Task rule (
TimeoutSecondsrequired) is the lesson's house style, not an ASL rule: it encodes "never let a hung worker hold an execution for a year".
Security considerations
⚠ Learning shortcut — not for production
The CDK and Terraform in this lesson use a placeholder screening function ARN and let the account's default key encrypt everything. Production uses customer managed KMS keys for the SQS queue and the Step Functions state machine (Step Functions supports customer managed keys for execution data ⚠ VERIFY current scope), an execution role per state machine, and a review service whose role can call only states:SendTaskSuccess and states:SendTaskFailure.
⚠ Learning shortcut — not for production
The lab puts the payment ID and amount into the execution input, so they are in the 90-day history. For real payments, put an opaque reference in the input, keep personal and account data in your own store, and fetch it in the Lambda that needs it. Set IncludeExecutionData false on logging, and tell the data-protection team that the console history is another copy.
- Execution role. One statement per integration, one resource each.
lambda:InvokeFunctionon the one function (and its versions),dynamodb:PutItemon the one table, and so on. Never*. Add a trust-policy condition onaws:SourceAccount(andaws:SourceArnwhere you can) to stop the confused-deputy problem. - Task tokens are bearer secrets. Send them only to the review service, do not log them, and do not e-mail them in links. A token works only for principals in the same account.
- Who can start executions.
states:StartExecutionon a payment workflow is equivalent to being able to submit a payment. Restrict it to the one Pipe, API or service, and use the execution name for idempotency. - Who can redrive or stop.
states:RedriveExecutionandstates:StopExecutionare operational break-glass actions. - Logging. Error level, no execution data; encrypt the log group, set retention.
- Immutable audit. The execution history expires after 90 days (30 on request). If you need a longer audit trail, write decisions to your own store (the DynamoDB write in the workflow is that record).
- Compliance. This lesson does not make a system compliant with GDPR, UK GDPR or PCI DSS. Retention of execution history and logs, minimisation of what enters them, and the audit trail all need review by your legal, data-protection and security teams.
Banking use case
Harbourline's outbound-payment review:
| Step | State | Why it is in the workflow |
|---|---|---|
| Sanctions and fraud screening | Task (Lambda) with retry | A transient error must not reject a customer's payment |
| Route on risk | Choice | The thresholds are visible to auditors and changed by version, not by redeploying code |
| Human review | Task with .waitForTaskToken, 24-hour timeout | Waits for days at no cost; the timeout is explicit policy |
| Decision record | Task (DynamoDB PutItem) | An idempotent record independent of the 90-day history |
| Notifications | Map over parties | One failing SMS gateway does not fail the payment (10 percent tolerance) |
The compensation logic for what happens when a later step (the scheme submission) fails after funds are reserved is the subject of the saga lesson.
Common mistakes
- No
TimeoutSecondson a Task. A hung worker holds the execution for up to a year. → Timeouts everywhere, plus a state machine timeout. - Forgetting the Lambda service-exception retry. A throttle rejects a good payment. → The four-error retrier with jitter.
States.ALLnot last, or not alone. The definition is rejected or masks other retriers. → Put it in its own, final retrier.- Expecting
States.ALLto catch everything. It missesStates.DataLimitExceededandStates.Runtime. → NameDataLimitExceededexplicitly, and keep payloads small. - Passing big payloads through states. The 256 KiB limit fails the execution. → Store in S3 or DynamoDB; pass an ID.
- Polling in a Standard workflow. A Wait and a Choice in a loop cost a transition per iteration. →
.sync, a callback, or an EventBridge event. - Logging execution data. Personal data in CloudWatch Logs. →
IncludeExecutionDatafalse. - Assuming "exactly-once" protects Lambda side effects. A task timeout can leave work done. → Idempotent tasks, with the payment ID as a key.
- Changing a long-running definition without versions. In-flight executions meet new logic. → Publish versions and use aliases.
- A big, hand-edited JSON file nobody lints. → The linter and
ValidateStateMachineDefinitionin CI.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
States.Permissions | Execution role lacks the action or resource, or a trust condition mismatch | Read the cause string; add the exact action and ARN; check aws:SourceAccount and aws:SourceArn |
States.QueryEvaluationError | JSONata type error or undefined value (a missing field) | Test the state with TestState; guard optional fields with $exists() |
| Execution stuck on the manual-review task | No SendTaskSuccess, or the token was lost | Check the queue; set HeartbeatSeconds and a timeout; resend from the message body |
TaskTimedOut returned to the reviewer service | The timeout fired first | Handle it and tell the user the case was auto-rejected |
ExecutionLimitExceeded or throttled starts | Account or Region quota; new accounts have lower quotas | Smooth the starts, request an increase |
States.DataLimitExceeded | A payload went over 256 KiB | Store the data elsewhere; pass a pointer |
| Execution fails after 25,000 events | History limit in a long loop or large inline Map | Start a new execution per batch or use Distributed Map |
ExecutionAlreadyExists on start | Same execution name with different input, or still running | Use the business key and the same input; check the existing execution |
.sync task keeps running after the execution stopped | Step Functions could not cancel the job | Grant the cancel permission; stop the job in its own service; check billing |
| Retries do not stop a failing step | The error name did not match any retrier | Look at the Error in the history; match it or use States.TaskFailed |
InvalidDefinition on create | A malformed {% %} string, a path field in a JSONata state, or a missing Next | Run the linter and validate-state-machine-definition |
| Redrive refused | Older than 14 days, over 24,999 events, or started before November 2023 | Start a new execution with the same business key |
Cost and cleanup
Cost: free at lab volumes. A Standard workflow costs about USD 0.000025 per state transition in us-east-1 (USD 25 per million), with a free tier of 4,000 transitions a month that does not expire after the first year. Each retry and each Distributed Map iteration is a transition. Working a rough example: one payment taking the manual-review path with five recipients is roughly 4 transitions plus 5 for the Map items, so about 10, which is 0.025 cents; a million of them is about USD 250. Express is billed by executions (about USD 1 per million) plus duration and memory ⚠ VERIFY the exact duration rates: the pricing page gives both per-GB-second and per-GB-hour forms. SQS, SNS, DynamoDB and Lambda are billed by their own services and are free at these volumes.
The lesson's three runs use fewer than 100 transitions. There is no hourly charge for an idle state machine, and no cost while a callback waits.
Free alternatives: run the structural tests (no account), and use TestState with mocks (no execution role and no resources). AWS also publishes Step Functions Local as a Docker image for offline runs ⚠ VERIFY the current image and its support for JSONata.
export AWS_PROFILE=learn-sandbox AWS_REGION=us-east-1
# Stop anything still running, then delete the state machine.
for exec in $(aws stepfunctions list-executions --state-machine-arn "$SM_ARN" --status-filter RUNNING \
--query 'executions[].executionArn' --output text); do
aws stepfunctions stop-execution --execution-arn "$exec" --error Cleanup --cause "Lab teardown"
done
aws stepfunctions delete-state-machine --state-machine-arn "$SM_ARN"
# Supporting resources.
aws dynamodb delete-table --table-name hl-m08gap-decisions
for q in hl-m08gap-manual-review; do
url=$(aws sqs get-queue-url --queue-name "$q" --query QueueUrl --output text 2>/dev/null) && aws sqs delete-queue --queue-url "$url"
done
aws sns delete-topic --topic-arn "$(aws sns list-topics --query "Topics[?contains(TopicArn,'hl-m08gap-payment-notices')].TopicArn" --output text)"
aws logs delete-log-group --log-group-name /hl-m08gap/payment-review
# If you deployed with IaC: cdk destroy HlM08Gap-Workflow | terraform destroy
aws resourcegroupstaggingapi get-resources --tag-filters Key=Lab,Values=m08-messaging-services \
--query 'ResourceTagMappingList[].ResourceARN'State machine deletion is asynchronous: it can take a short while to finish.
Exercises
- 🟢 Guided: Lint
payment-review.asl.json, deploy it, run the three paths (risk 10, 90 and 50), complete the manual review withApproveAsync, and write down the sequence of states in each execution history. - 🟡 Challenge: Add a Parallel state before
Routethat runs a sanctions check and a fraud score at the same time, and aRetryforStates.TimeoutwithMaxAttempts0 followed byStates.ALLwith three attempts (so timeouts are never retried but everything else is). Prove the retry order withTestStatemocks, and update the linter so a Parallel state's branches are checked. - 🔴 Live problem: Ticket from Operations: "Since last week's deploy, about 3% of payments in manual review show
TaskTimedOuterrors in the review tool and the customer is told 'rejected'. Reviewers swear they approved them within the hour. The workflow timeout is 24 hours." Work out what could be happening and how you would prove each theory from the execution histories.
Assignment
Add stepfunctions-design.md to your portfolio repository for one Harbourline process of your choice (card dispute, account opening, standing-order amendment). Include: the diagram, the ASL, the choice of Standard or Express and why, the timeouts and heartbeats with numbers, the retry table for every task (error names, attempts, jitter), the callback design with who holds the token, the data you keep out of the execution input, the IAM statements for the execution role and the callers, the tests (lint, TestState), and a cost estimate at one million executions a month.
Solutions
Risk 10: ScreenPayment, Route, ApprovePayment, NotifyCustomers (one PublishNotice per recipient). Risk 90: ScreenPayment, Route, RejectPayment, NotifyCustomers. Risk 50: ScreenPayment, Route, ManualReview (waiting), then after ApproveAsync the DecisionChoice takes approved to ApprovePayment, then NotifyCustomers.
Why: Route has two conditions and a Default, so everything in the middle band goes to review. The waiting ManualReview costs nothing while it waits. The decision reaches DecisionChoice as $states.input.decision because SendTaskSuccess's output becomes the task's result.
Add a Parallel state with two branches (each a Lambda Task with its own timeout), Output it with {% $merge($states.result) %} to flatten the branch results (the documented pattern), and give the first retrier ErrorEquals: ["States.Timeout"] with MaxAttempts: 0, then a last retrier ErrorEquals: ["States.ALL"], MaxAttempts: 3, jitter. Retriers are scanned in order, so a timeout matches the first and is never retried, while any other error falls through to the wildcard (which must be alone and last).
Prove it with TestState and a mock errorOutput of States.Timeout (expect retryIndex 0 and no retry, so the Catch runs) and of Lambda.ServiceException (expect retryIndex 1 and RETRIABLE). For the linter, recursion into Branches is already in CheckMachine: add a test with a Parallel state containing a Task with no timeout, and assert the message path includes Branches[0].
Theories, from most to least likely:
- The token was used after the timeout, or the wrong token. If the review service stores tokens in a cache or table, a stale token (from a redriven or retried execution, or a Task retry that generated a new token) will not match the waiting task. Look at the
TaskSubmitted,TaskTimedOutandTaskSucceededhistory events and compare the time the review service calledSendTaskSuccess. - The deploy changed the timeout or heartbeat. A
HeartbeatSecondsset to a short value with a service that does not heartbeat producesStates.Timeoutquickly. Compare the versions of the definition before and after. - The message sat in the review queue longer than reviewers took, because the review service consumed it late (queue backlog, visibility timeout). The 24-hour clock started when the task was scheduled, not when the reviewer saw the case.
- Clock or timezone assumptions in the tool that shows "time remaining".
Proof: for ten affected executions, get-execution-history and compare TaskScheduled time, queue receive time, the reviewer's decision timestamp and TaskTimedOut time. Also check CloudTrail for SendTaskSuccess calls with TaskTimedOut errors (the failed calls are in CloudTrail).
Fix: handle TaskTimedOut visibly in the tool, add a heartbeat only if the tool really sends one, alarm on the review queue's ApproximateAgeOfOldestMessage, and choose a timeout that includes queue delay. Consider a "reminder" branch at 20 hours instead of a hard reject at 24.
Knowledge check
Answer: Standard only. Express does not support the Run a Job (.sync) or Wait for Callback patterns, and runs at most five minutes, so a human approval that may take days needs Standard.
Answer: Retriers are scanned in order and matching ones run first. Only when retries are exhausted, or the error matches no retrier, does Step Functions follow a matching catcher to its Next state.
Answer: No. It must be alone and last, and it does not catch States.DataLimitExceeded (unless you list it explicitly) or States.Runtime.
Answer: In a Standard workflow, starting an execution with the name of a running execution and the same input returns the existing execution instead of creating another. Using the payment ID as the name makes retried start requests idempotent.
Answer: A Standard execution history is limited to 25,000 events. Start a new execution per batch, use child workflows, or use a Distributed Map.
Answer: Arguments replaces Parameters (and InputPath) in a JSONata state, and takes {% %} expressions on $states.input without the .$ key suffix. Output replaces ResultSelector, ResultPath and OutputPath.
Interview questions
- "When would you use Step Functions rather than a queue and a worker?" — When the sequence, the waiting and the failure handling are the problem: many steps, long waits, human approvals, compensation, an auditable path. For one kind of work with simple retry, a queue and a worker are simpler and cheaper.
- "Standard or Express?" — Standard for long-running, auditable, non-idempotent work such as payments, billed per transition and kept for 90 days. Express for short, high-volume, idempotent work, billed per execution and duration, with logs as the only history.
- "How do you build a human-approval step?" —
.waitForTaskTokenon a task that sends the token to the approver's service (SQS), a timeout (and optional heartbeat), andSendTaskSuccess/SendTaskFailurefrom that service with a narrowly-scoped role. Treat the token as a secret. - "How do you handle errors in a workflow?" —
Retrywith backoff and full jitter for transient errors, including the four Lambda service exceptions;Catchto route to a fallback or compensation state; explicit timeouts; a failed-execution alert from the EventBridge status events; and redrive after a fix. - "How do you test a state machine?" — A structural lint in CI,
ValidateStateMachineDefinition,TestStatewith mocks for individual states, and a sandbox execution with stubs.
Certification practice
A loan application workflow must pause until an underwriter approves, which can take up to three days, and then continue. The team wants no servers polling. Which solution should the developer use?
A. An AWS Step Functions Express workflow with a Wait state.
B. An AWS Step Functions Standard workflow with a task using the .waitForTaskToken integration pattern.
C. An AWS Lambda function that sleeps for three days.
D. An Amazon SQS queue with a 3-day visibility timeout.
Answer: B. Standard workflows can run for a year and support the callback pattern, which costs nothing while it waits. A fails: Express lasts five minutes. C exceeds Lambda's 15-minute limit. D has a maximum visibility timeout of 12 hours and is not an orchestrator.
A Step Functions task invokes a Lambda function. During traffic bursts the execution fails with Lambda.TooManyRequestsException. What should the developer do?
A. Increase the state machine timeout.
B. Add a Retry for the Lambda service exceptions with exponential backoff and jitter.
C. Change the workflow to Express.
D. Add a Catch that goes to a Succeed state.
Answer: B. Transient Lambda service errors are retried with backoff and jitter. A does not retry. C does not change throttling. D hides the failure instead of recovering.
A company processes insurance claims in six steps, some of which wait on external partners for hours. It needs an auditable record of each claim's path for 60 days and automatic retry of failed steps. Which service fits best?
A. Amazon SQS with a custom state table. B. AWS Step Functions Standard workflows. C. Amazon SNS with Lambda subscriptions. D. AWS Step Functions Express workflows.
Answer: B. Standard workflows retain the execution history for 90 days, support long waits and have declarative retry. A and C leave orchestration and audit to custom code. D lasts five minutes and keeps no history unless logging is configured.
A streaming application must run a short, idempotent, three-step transformation on 80,000 events per second. Cost per event must be low and executions last under a second. Which choice is best?
A. Step Functions Standard workflows. B. Step Functions Express workflows. C. A Standard workflow per event with a Wait state. D. A single long-running Standard workflow that loops.
Answer: B. Express workflows are built for high-volume, short, idempotent work and are billed by executions and duration. A and C would be limited by state-transition and start-rate quotas and cost far more. D hits the 25,000-event history limit.
Completion checklist
- I can explain state, execution and ASL, and name the state types
- I chose Standard or Express for a workload and said why
- I can write Retry and Catch with the right defaults,
States.ALLlast and jitter on - I completed a callback from .NET and kept the token out of logs
- I linted the definition and ran a state with
TestState - I redrove a failed execution after fixing a permission
- I ran the cleanup and the tag check returned
[]
Key takeaways
- A workflow definition is durable, visual documentation: Step Functions holds state, waits for free and records every step. Keep flow in ASL and business logic in code.
- Standard (exactly-once, up to a year, 90-day history, per transition) for payments and approvals; Express (5 minutes, at-least-once or at-most-once, logs only) for high-volume short work.
- JSONata replaces five path fields with
ArgumentsandOutput, addsAssignvariables and$states. Expressions can fail at run time. - Retry, then Catch. Defaults: 3 attempts, 2.0 backoff, no jitter, so set
JitterStrategy: FULLandMaxDelaySeconds.States.ALLmust be last and alone. Always retry the four Lambda service exceptions and set timeouts everywhere. - Callbacks pause for free; the task token is a secret. Redrive recovers Standard failures within 14 days. Payloads are limited to 256 KiB and histories to 25,000 events.
Further reading
- Choosing workflow type in Step Functions
- Step Functions service quotas
- Transforming data with JSONata in Step Functions
- Discover service integration patterns in Step Functions
- Handling errors in Step Functions workflows
- Using Map state in Distributed mode
- Restarting executions with redrive
- Testing state machines with the TestState API
- AWS Step Functions pricing
- Next: Distributed transactions and the saga pattern
Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.
Distributed transactions and the saga pattern
Replace the distributed transaction you can't have with a saga: compare orchestration and choreography, design compensating actions and pivot steps, and build a Harbourline payment saga in AWS Step Functions with retries, catches, DynamoDB integrations and idempotent .NET Lambda steps.
The outbox pattern with EF Core
Solve the dual-write problem: write business state and its events in one EF Core transaction, relay them to EventBridge with a leased, batch-aware BackgroundService, and handle relay crashes, partial PutEvents failures, poison rows and cleanup — with tests that run on SQLite.