Skip to Content
Cloud Engineering Academy
8. Messaging, Events & Microservices

AWS Step Functions: workflows, retries and callbacks

Orchestrate multi-step banking processes with AWS Step Functions: Standard versus Express workflows, the state types, JSONata, service integrations, Retry and Catch, human-approval callbacks with task tokens, Map and Distributed Map, redrive, testing with TestState, and a .NET 10 client, CDK and Terraform.

intermediate⏱ 3 h 🟢 FreeDVA-C03SAA-C03

Assumes: .NET 10 LTS · AWS SDK for .NET v4 (AWSSDK.StepFunctions 4.1.0.2) · Amazon.CDK.Lib 2.277.0 · Terraform AWS provider v6 · AWS CLI v2 · Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Describe a state machine, its states and an execution, read Amazon States Language (ASL), and choose Standard or Express for a given workload.
  • Write a workflow with JSONata (Arguments, Output, Assign, $states) and explain how it differs from the older JSONPath fields.
  • Use the three service integration patterns: Request Response, Run a Job (.sync) and Wait for Callback (.waitForTaskToken).
  • Configure Retry and Catch with the correct defaults, jitter and error names, and explain what Step Functions does when both apply.
  • Build a human-approval step with a task token, complete it from .NET, and keep the token safe.
  • Fan out with Map and know when to move to Distributed Map; redrive a failed execution.
  • Lint and test a state machine without deploying it (structural linter, ValidateStateMachineDefinition, the TestState API with mocks).
  • Deploy the workflow with CDK (C#) and Terraform from one shared ASL file.

Prerequisites

The saga lesson comes next and builds a compensating payment saga on these foundations. This lesson is the toolbox; that one is a worked design.

Why this matters in banking

An outbound payment above a threshold at Harbourline goes through screening, then either straight through, or to a human reviewer who may take hours, then a posting record, then notifications to every party. The first implementation was a long-running service with an in-memory state machine and a database table of "where is this payment". It worked until:

  1. A deployment restarted the service while 400 payments were waiting for a reviewer. Their state was in memory. Operations spent a day reconstructing it from logs.
  2. A reviewer approved a payment a minute after the timeout job had rejected it. Nobody could say which decision won, because the logic for waiting, timing out and deciding was spread over three classes.
  3. An auditor asked for the path one payment took. The answer was a grep through four log streams.

Step Functions is an orchestrator: the workflow definition is the documentation, the service holds the state between steps (for up to a year), every execution has a visual, queryable history, and retries, timeouts and waiting are declared instead of coded. The costs are a new language (ASL), a per-transition price, and the discipline of keeping business logic in your code and flow in the definition.

The idea in plain language

A state machine is a diagram turned into JSON. Each box is a state. An execution is one run of the diagram with an input, a current state and a growing history. Step Functions remembers where every execution is, calls the code or AWS service each Task state names, passes the output to the next state, waits when told to, retries when a step fails, and follows the arrows.

You write the diagram in Amazon States Language (ASL), a JSON dialect. You rarely write it from nothing: the Workflow Studio designer generates it, and you keep the JSON in source control.

Analogy: A mortgage case file with a checklist stapled inside. The checklist says: check ID; run the credit search; if the score is borderline, send the file to the underwriter's tray and wait (the file sits there, possibly for days, without anyone watching it); when the underwriter returns it with a decision, continue; then send the letters. The file cabinet is Step Functions. It never forgets where a file is, and the checklist can be read by anyone, including the auditor.

The state types

StateWhat it doesBanking example
TaskCalls Lambda or an AWS service (over 200 via the AWS SDK integrations), or waits for a callbackScreen the payment; write the decision to DynamoDB
ChoiceBranches on the dataRisk score under 20 approve, over 80 reject, otherwise review
MapRuns the same steps for each item of an arrayNotify every party to the payment
ParallelRuns different branches at once and waits for allSanctions check and fraud score in parallel
WaitPauses for a time or until a timestampHold a scheduled payment until 06:00
PassPasses data on, optionally reshaping itSet defaults, test wiring
Succeed / FailEnds the execution, successfully or with an error name and causeReject with PaymentReviewFailed

What problem AWS Step Functions solves

Step Functions gives you durable orchestration as a service: state persisted between steps, declarative retry and timeout, waiting that costs nothing while it waits, parallelism, and a record of every step. You use it where the sequence and the failure handling are the hard part: payment journeys, onboarding, loan decisions, data pipelines, long-running approvals.

Standard versus Express

StandardExpress
Maximum duration1 year5 minutes
Execution semanticsExactly-once workflow execution. A task state runs once unless you wrote a RetryAsynchronous: at-least-once. Synchronous (StartSyncExecution): at-most-once
Execution historyKept by Step Functions for 90 days after the execution closes (30 days on request)Not kept. You must turn on CloudWatch Logs
Start rate300 per second sustained in us-east-1 (bucket 1,300)6,000 per second
State transition rate5,000 per second in us-east-1 (soft quota)Unlimited
.sync and .waitForTaskTokenSupportedNot supported
Distributed Map, ActivitiesSupportedNot supported
RedriveYes, within 14 daysNo
Billed byState transitionsNumber of executions, duration and memory
Best forLong-running, auditable, non-idempotent work: payments, approvalsHigh-volume, short, idempotent work: stream processing, API back ends

The workflow type cannot be changed after creation. A common pattern is a Standard parent that starts Express children for the high-volume steps.

Limits and defaults (checked 2026-11-18)

SettingValue
Input, output and task payload256 KiB per state, task and execution (UTF-8 string). Larger data: keep it in S3 or DynamoDB and pass a pointer
Standard execution history25,000 events. A long loop or a big Map can hit it: start a new execution or use child workflows
State machine definition size1 MB
Open Standard executions1,000,000 per account per Region
Execution and task namesUp to 80 characters; no whitespace, wildcards, brackets or special characters (for example double quote, hash, percent, backslash, caret, pipe, tilde, dollar sign, ampersand, comma, semicolon, colon or slash); use ASCII
Distributed Map parallel childrenUp to 10,000 per Map Run
Published versions per state machine1,000; aliases 100
Redrive window14 days
HTTP Task time60 seconds
New accountsHave reduced state transition quotas that rise with use

Honest limitations

  • It is not a data store and not a message bus. The 256 KiB payload limit is deliberate.
  • Per-transition pricing punishes tight loops. Polling every second in a Standard workflow costs money; use a Wait, a .sync integration or a callback.
  • "Exactly-once" is about the workflow, not your code. A Lambda task that timed out may still have done its work. Tasks must be idempotent.
  • The history is personal data. Inputs and outputs of every state sit in the execution history for 90 days, and (if you log them) in CloudWatch Logs.
  • JSON is the language. Large definitions are hard to review. Keep states small and push logic into Lambda or the service integrations.

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
Durable Functions orchestratorStandard workflowDurable Functions is code (C#) that replays; Step Functions is a declarative definition with a visual history. No determinism rules for your code, but you leave C# for JSON/ASL
context.CallActivityAsyncTask state with a Lambda or service integrationMany AWS services are called directly with no Lambda in between
context.WaitForExternalEvent.waitForTaskToken callbackThe token is an opaque secret the external system returns with SendTaskSuccess
Task.WhenAll fan-outParallel and Map statesMap handles a dynamic array; Distributed Map handles millions of S3 items
Retry policies in TaskOptionsRetry fieldDeclared per state; JitterStrategy: FULL available
Durable timersWait stateCosts nothing while it waits; up to the one-year execution limit
Logic Apps (Standard/Consumption)Step Functions Standard, plus Pipes and EventBridgeLogic Apps is a designer-first connector platform; Step Functions is closer to code review and IaC
Durable Functions Eternal orchestrationA new execution per iteration (the 25,000-event history limit)There is no "continue as new" call: a state starts the next execution
Orchestration instance IDExecution nameA name is an idempotency key for Standard executions

Amazon States Language and JSONata

Two query languages

A state machine declares "QueryLanguage": "JSONata" or the default "JSONPath". You can set it per state, so you can migrate gradually. The saga lesson uses JSONPath; this lesson uses JSONata, which AWS recommends for new workflows because it replaces five path fields with two and adds real expressions.

ConceptJSONPathJSONata
Select what a task receivesInputPath, ParametersArguments
Reshape the resultResultSelector, ResultPath, OutputPathOutput
Expressions.$ suffix on keys; limited intrinsic functions"{% expression %}" in any string value
VariablesAssign with pathsAssign with expressions
Choice conditionsVariable + StringEquals, NumericGreaterThan and so onCondition with a JSONata boolean
Map itemsItemsPathItems
The state's input$$states.input

In a JSONata state the reserved variable $states holds input (the state's input), result (the task's raw result), errorOutput (inside a Catch) and context (execution metadata, including the task token).

Rules that bite:

  • Assign and Output run in parallel. A value you transform in Assign is not available to Output: repeat the expression.
  • Expressions can fail at run time with States.QueryEvaluationError (type error, a missing field, an undefined result, one second of evaluation time, memory). You can Retry or Catch it.
  • A string must start with {% and end with %} with no spaces outside, or the definition is invalid.
  • Fields like Type and a Task's Resource must stay constant strings.
  • Built-in helpers replace intrinsic functions: $uuid(), $hash(), $partition(), $range(), $random(), $parse() (JSONata's $eval is not available).

Here are the four lines that matter, from the workflow in this lesson:

"Assign": { "paymentId": "{% $states.input.paymentId %}" },
"Output": "{% $states.result.Payload %}",
"Condition": "{% $states.input.riskScore >= 80 %}",
"taskToken": "{% $states.context.Task.Token %}"

Variables

Assign stores values in variables ($paymentId) that later states can read, so you stop threading data through every state's output. Use them for facts needed late in the workflow (the payment ID in the notification step, 20 states after the screening). Variables have size quotas ⚠ VERIFY the current figures, and like inputs they appear in the history.

Service integrations

A Task's Resource names what to call. The prefix arn:aws:states::: is a Step Functions namespace.

  • Optimized integrations (arn:aws:states:::lambda:invoke, dynamodb:putItem, sqs:sendMessage, sns:publish, events:putEvents, states:startExecution, ecs:runTask, glue:startJobRun and more) have tailored request and result shapes.
  • AWS SDK integrations (arn:aws:states:::aws-sdk:<service>:<action>) call over two hundred services' API actions directly.
  • HTTP Task (arn:aws:states:::http:invoke) calls any HTTPS API using an EventBridge connection for credentials (60-second limit per call).

Each integration supports a pattern:

PatternResource suffixStep Functions waits forStandardExpress
Request ResponsenoneThe HTTP response of the callYesYes
Run a Job.syncThe job to finish (it polls or listens to events for you)Some services (ECS, Glue, Batch, Athena, nested workflows, Bedrock...)No
Wait for Callback.waitForTaskTokenSendTaskSuccess or SendTaskFailure with the tokenMany services: SQS, SNS, Lambda, EventBridge, API Gateway, ECS and others (not DynamoDB)No

Two details from the integration table are worth remembering. DynamoDB supports Request Response only. Lambda, SQS and SNS support Request Response and callback, but not .sync. If the integration you want lacks the pattern, use .waitForTaskToken with a message or an event that carries the token.

.sync has a hidden cost: for the same account Step Functions listens to events and polls the service's describe API, consuming your quota. If a .sync task is aborted and cannot be cancelled (for example, the role lacks the permission), the job may run on and bill you.

Error handling: Retry and Catch

Every state except Pass and Wait can fail. By default an error fails the whole execution. Task, Parallel and Map can declare Retry and Catch.

Error names

NameMeaning
States.ALLWildcard for known errors. Must be alone in ErrorEquals and last in the list. It does not catch States.DataLimitExceeded or States.Runtime
States.TaskFailedAny task failure except States.Timeout
States.TimeoutTask exceeded TimeoutSeconds, a heartbeat was missed, or the whole execution exceeded its timeout
States.HeartbeatTimeoutNo heartbeat within HeartbeatSeconds
States.PermissionsThe execution role lacks permission
States.DataLimitExceededA payload went over 256 KiB. Terminal unless you list it explicitly
States.RuntimeAn unexpected runtime failure (for example a path on null data). Never retriable or catchable
States.QueryEvaluationErrorA JSONata expression failed
States.ExceedToleratedFailureThresholdA Map ran past its failure threshold
Lambda.ServiceException, Lambda.SdkClientException, Lambda.TooManyRequestsException, Lambda.AWSLambdaExceptionTransient Lambda service errors. Always retry these
Your own namesAnything not starting States.. A .NET exception type name becomes the error name

Retry fields and defaults

FieldDefaultNotes
ErrorEqualsrequiredRetriers are scanned in order
IntervalSeconds1Delay before the first retry
MaxAttempts30 means never retry this error
BackoffRate2.0Multiplier per attempt
MaxDelaySecondsnoneCaps the backoff; always set it
JitterStrategyNONEFULL randomises each delay between 0 and the computed delay. Use FULL

A retrier's counter belongs to the state, per retrier, across all visits. Retries count as state transitions (and are billed). When a state has both Retry and Catch, retriers run first, and the catcher runs only when retries are exhausted or the error did not match a retrier.

Catch

A catcher names ErrorEquals and a Next state, and (in JSONata) can Assign and shape the Output using $states.errorOutput, which holds Error and Cause. In JSONPath, ResultPath controls whether the error replaces or is added to the input.

The Lambda retry you should always write

Lambda can fail before your code runs (throttling, a transient service error). The documented defence is a retrier for Lambda.ServiceException, Lambda.AWSLambdaException, Lambda.SdkClientException and Lambda.TooManyRequestsException with a few attempts and jitter. The workflow in this lesson has it on the screening task.

Timeouts

Always set TimeoutSeconds on every Task (a hung worker would otherwise hold the execution for up to a year) and a top-level TimeoutSeconds on the state machine. For callbacks, add HeartbeatSeconds so a dead reviewer tool fails fast. The linter in this lesson flags a Task without a timeout.

The callback pattern: waiting for a human

A callback pauses the workflow at a Task until something outside calls SendTaskSuccess or SendTaskFailure with a task token. Nothing runs and nothing is billed while it waits (up to the one-year execution limit).

The token is a bearer secret: anyone who holds it can finish the task, from a principal in the same account (tokens do not work across accounts). Treat it like a session cookie: send it only to the review service, do not log it, do not put it in a URL that is e-mailed to a reviewer, and store it encrypted if you keep it. The review service's role needs states:SendTaskSuccess and states:SendTaskFailure, nothing else.

The workflow below uses a 24-hour TimeoutSeconds and routes States.Timeout to a rejection. Be deliberate about the race in the third incident above: whichever of SendTaskSuccess and the timeout arrives first wins, and the loser gets an error (TaskTimedOut) that the review service must handle gracefully and show to the reviewer.

Map and Distributed Map

Map runs the same sub-workflow for every item of an array. In the lesson's workflow, one notification per recipient.

Inline MapDistributed Map
Each iterationPart of the parent execution (shares its history)A child workflow execution with its own history (Standard or Express)
InputAn array in the state's input (up to the 256 KiB limit)The same, or a large S3 source: JSON, CSV, a list of objects, an inventory
ConcurrencyLimited (the docs point to Distributed above 40 concurrent iterations)Up to 10,000 parallel child executions
ResultAn arrayAn array, or written to S3 with ResultWriter
Use whenSmall arrays, a few hundred itemsA file of a million transactions, a nightly reconciliation

Both support MaxConcurrency and a tolerated failure threshold (ToleratedFailurePercentage or ToleratedFailureCount, default zero: any failure fails the Map). Set MaxConcurrency to what the downstream can take, not to what Step Functions can give. Distributed Map bills one state transition per iteration, so a 10-million-row file is a real cost.

Redrive

When a Standard execution fails, times out or is aborted, redrive restarts it from the failed state with the same input, definition and execution ARN, within 14 days. It reruns the unsuccessful state, keeps the history of the successful ones and resets retry counters. For a Parallel or inline Map only the failed branches or iterations rerun; for a Distributed Map only the unsuccessful child executions rerun. Executions older than 14 days, started before 15 November 2023 or with more than 24,999 history events cannot be redriven.

Redrive is the safe way to recover after you fix a bug or a permission, because a state that already completed will not run again. It is also why completed states must be correct: you cannot "replay from the start" with redrive.

Versions and aliases

Publish a version to freeze a definition, and point an alias (for example live) at one or two versions with weights for a gradual rollout. Callers start executions against the alias ARN. Executions run to completion on the version they started with, which matters for year-long workflows: you may have executions of several definitions in flight. Keep the definition changes backward compatible with the data that older executions carry.

Testing

Test in layers, cheapest first.

  1. Structural lint in CI (the AslLint class below): catches a misspelt Next, an unreachable state, a Task with no timeout, and JSONPath fields inside a JSONata machine. It needs no AWS account.
  2. ValidateStateMachineDefinition (aws stepfunctions validate-state-machine-definition): the service checks syntax and semantics. Use it in CI with a role that can only call that API. ⚠ VERIFY the exact CLI flags for your CLI version.
  3. TestState API: runs one state (or a state from a full definition) in isolation and returns output, next state and status. Since November 2025 it can mock service integrations (--mock '{"result": "..."}' or {"errorOutput": {...}}), which also lets you test Map, Parallel, .sync and .waitForTaskToken states, simulate retry attempts (retrierRetryCount) and assert which Catch or Retry applied (catchIndex, retryIndex). With a mock you do not need an execution role. The console version has fewer features than the CLI and SDK.
  4. A real execution in a sandbox account with the Lambdas stubbed, to prove IAM and wiring.
# Test the Route choice with a mock-free state: input riskScore 85 should go to RejectPayment.
aws stepfunctions test-state \
  --definition '{"QueryLanguage":"JSONata","Type":"Choice","Choices":[{"Condition":"{% $states.input.riskScore >= 80 %}","Next":"RejectPayment"},{"Condition":"{% $states.input.riskScore < 20 %}","Next":"ApprovePayment"}],"Default":"ManualReview"}' \
  --role-arn "$ROLE_ARN" --input '{"riskScore": 85}'
# Expected: "nextState": "RejectPayment", "status": "SUCCEEDED"

# Prove the screening retry: simulate a throttled Lambda on the second attempt with a mock error.
aws stepfunctions test-state --definition file://screen-state.json \
  --input '{"paymentId":"PAY-1"}' \
  --state-configuration '{"retrierRetryCount": 1}' \
  --mock '{"errorOutput": {"error": "Lambda.TooManyRequestsException", "cause": "Rate exceeded"}}' \
  --inspection-level DEBUG
# Expected: "status": "RETRIABLE" with errorDetails.retryIndex 0 and a backoff interval

Observability

  • Console: the execution graph colours every state and shows input, output, retries and timing. For a Standard execution it is the audit trail for 90 days.
  • CloudWatch Logs is the only history for Express. Choose a level (ALL, ERROR, FATAL, OFF). Set IncludeExecutionData to false unless you have decided that payment data may live in logs.
  • Metrics: ExecutionsStarted, ExecutionsFailed, ExecutionsTimedOut, ExecutionThrottled, ExecutionTime. Alarm on failed and timed-out executions, and on throttling (new accounts have lower quotas).
  • EventBridge events: Step Functions emits execution status changes (succeeded, failed, timed out, aborted). A rule on FAILED and TIMED_OUT that targets a ticket queue is the usual way to react to top-level failures, which Catch cannot handle.
  • X-Ray: switch on tracing to see the whole path including the Lambda and service calls.

Architecture

Security-boundary view

The state machine's execution role is the most powerful identity here: it can do whatever the workflow does. Grant one statement per integration, scoped to one resource, and add aws:SourceArn and aws:SourceAccount conditions to its trust policy.

Step-by-step implementation

Create the state machine's dependencies

You need a screening Lambda (any function that returns {"riskScore": 50, ...}), the decisions table, the manual-review queue and the notification topic. The IaC below creates the last three and imports the Lambda by ARN, so you can use a stub.

Lint the definition

Run the linter over payment-review.asl.json (the test project does). It must report nothing. Break a Next on purpose and read the message.

Deploy with the IaC of your choice

CDK and Terraform both load the same ASL file and fill the ${Placeholders}. Keep the file in source control and review changes to it like code.

Run the three paths

Start three executions with riskScore 10 (straight through), 90 (rejected) and 50 (manual review). The third will wait. Open its task token from the review queue message.

Complete the callback from .NET

Call ApproveAsync with the token. Watch the execution continue, take the approved branch, write to DynamoDB and notify each recipient. Then run another review and let it time out (set a 60-second timeout in a sandbox copy) to see the rejection path.

Break it and redrive

Remove the dynamodb:PutItem permission, start an approved payment and watch it fail with States.Permissions at ApprovePayment. Restore the permission and redrive: only ApprovePayment and the states after it run.

AWS Console instructions

Console → Step Functions → State machines → Create state machine. Choose Standard, Design your workflow visually or Write your workflow in code. In Workflow Studio, set the Query language to JSONata at the top level, drag in Lambda Invoke, Choice, SQS SendMessage (tick Wait for callback), DynamoDB PutItem and Map. Under Config → Permissions let the console create an execution role to read what it generates, then replace it with your least-privilege role. Turn on Logging at the Error level and leave Include execution data off. Use Test state on a single state with the INFO or DEBUG inspection level.

To run: Start execution, paste the JSON input. Open an execution to see the Graph view, per-state Input/Output, Retries & redrives, and the Redrive button on failed Standard executions.

AWS CLI

export AWS_PROFILE=learn-sandbox AWS_REGION=us-east-1

# Create (use the IaC for anything beyond learning). The role must already exist.
aws stepfunctions create-state-machine --name hl-m08gap-payment-review --type STANDARD \
  --role-arn "$ROLE_ARN" --definition file://payment-review.rendered.json \
  --logging-configuration level=ERROR,includeExecutionData=false,destinations=[{cloudWatchLogsLogGroup={logGroupArn=$LOG_GROUP_ARN}}] \
  --tracing-configuration enabled=true

# Ask the service to validate the definition without creating anything.
aws stepfunctions validate-state-machine-definition --definition file://payment-review.rendered.json

# Start an execution. The name is an idempotency key: the same name and input while it runs returns the existing execution.
aws stepfunctions start-execution --state-machine-arn "$SM_ARN" --name PAY-2026-0001 \
  --input '{"paymentId":"PAY-2026-0001","amount":1250.00,"recipients":["CUST-0001"]}'

# Where is it? What happened?
aws stepfunctions describe-execution --execution-arn "$EXEC_ARN" --query '{status:status,start:startDate}'
aws stepfunctions get-execution-history --execution-arn "$EXEC_ARN" --reverse-order --max-items 10 \
  --query 'events[].{id:id,type:type}'

# Complete a callback. The token comes from the SQS message body; never log it.
aws stepfunctions send-task-success --task-token "$TASK_TOKEN" --task-output '{"decision":"approved"}'

# Recover a failed Standard execution (within 14 days).
aws stepfunctions redrive-execution --execution-arn "$EXEC_ARN"

# Stop a runaway execution.
aws stepfunctions stop-execution --execution-arn "$EXEC_ARN" --error Operator --cause "Stopped during incident INC-123"

Infrastructure as Code

The stack reads the ASL file, fills the ${Placeholders} through DefinitionSubstitutions, logs errors without execution data, enables tracing, sets an execution-level timeout and adds one least-privilege grant per integration.

WorkflowStack.cs
using Amazon.CDK;
using Amazon.CDK.AWS.DynamoDB;
using Amazon.CDK.AWS.Lambda;
using Amazon.CDK.AWS.Logs;
using Amazon.CDK.AWS.SNS;
using Amazon.CDK.AWS.SQS;
using Amazon.CDK.AWS.StepFunctions;
using Constructs;

namespace Harbourline.Messaging.Infra;

public sealed class WorkflowStackProps : StackProps
{
    public required string ScreeningFunctionArn { get; init; }
    /// <summary>Path to payment-review.asl.json.</summary>
    public required string DefinitionPath { get; init; }
}

/// <summary>
/// A Standard workflow from an ASL file kept in source control. The file uses ${Placeholders}; CDK fills them from
/// DefinitionSubstitutions at deploy time, so the same JSON works in every environment. Execution data is NOT logged:
/// payment inputs are personal data, and CloudWatch Logs is another copy to protect and expire.
/// </summary>
public sealed class WorkflowStack : Stack
{
    public StateMachine StateMachine { get; }

    public WorkflowStack(Construct scope, string id, WorkflowStackProps props) : base(scope, id, props)
    {
        var decisions = new Table(this, "Decisions", new TableProps
        {
            TableName = "hl-m08gap-decisions",
            PartitionKey = new Amazon.CDK.AWS.DynamoDB.Attribute { Name = "pk", Type = AttributeType.STRING },
            BillingMode = BillingMode.PAY_PER_REQUEST,
            RemovalPolicy = RemovalPolicy.DESTROY, // lab only
        });
        var reviewQueue = new Queue(this, "ReviewQueue", new QueueProps
        {
            QueueName = "hl-m08gap-manual-review",
            Encryption = QueueEncryption.SQS_MANAGED,
            EnforceSSL = true,
        });
        var noticeTopic = new Topic(this, "NoticeTopic", new TopicProps { TopicName = "hl-m08gap-payment-notices", EnforceSSL = true });
        var screening = Function.FromFunctionArn(this, "Screening", props.ScreeningFunctionArn);

        var logs = new LogGroup(this, "Logs", new LogGroupProps
        {
            LogGroupName = "/hl-m08gap/payment-review",
            Retention = RetentionDays.ONE_WEEK,
            RemovalPolicy = RemovalPolicy.DESTROY,
        });

        StateMachine = new StateMachine(this, "PaymentReview", new StateMachineProps
        {
            StateMachineName = "hl-m08gap-payment-review",
            StateMachineType = StateMachineType.STANDARD,
            DefinitionBody = DefinitionBody.FromString(File.ReadAllText(props.DefinitionPath)),
            DefinitionSubstitutions = new Dictionary<string, string>
            {
                ["ScreeningFunctionArn"] = props.ScreeningFunctionArn,
                ["ReviewQueueUrl"] = reviewQueue.QueueUrl,
                ["DecisionsTableName"] = decisions.TableName,
                ["NoticeTopicArn"] = noticeTopic.TopicArn,
            },
            Logs = new LogOptions { Destination = logs, Level = LogLevel.ERROR, IncludeExecutionData = false },
            TracingEnabled = true,
            Timeout = Duration.Days(2), // a hard ceiling for the whole execution, on top of the per-task timeouts
        });

        // Least privilege, one grant per integration the ASL actually uses.
        screening.GrantInvoke(StateMachine);
        decisions.GrantWriteData(StateMachine);
        reviewQueue.GrantSendMessages(StateMachine);
        noticeTopic.GrantPublish(StateMachine);

        _ = new CfnOutput(this, "StateMachineArn", new CfnOutputProps { Value = StateMachine.StateMachineArn });
    }
}

Complete code

The state machine (ASL, JSONata)

payment-review.asl.json
{
  "Comment": "Harbourline outbound payment review (Standard workflow, JSONata)",
  "QueryLanguage": "JSONata",
  "StartAt": "ScreenPayment",
  "States": {
    "ScreenPayment": {
      "Type": "Task",
      "Resource": "arn:aws:states:::lambda:invoke",
      "Arguments": {
        "FunctionName": "${ScreeningFunctionArn}",
        "Payload": "{% $states.input %}"
      },
      "Assign": {
        "paymentId": "{% $states.input.paymentId %}",
        "amount": "{% $states.input.amount %}",
        "recipients": "{% $states.input.recipients %}"
      },
      "Output": "{% $states.result.Payload %}",
      "TimeoutSeconds": 30,
      "Retry": [
        {
          "ErrorEquals": ["Lambda.ServiceException", "Lambda.AWSLambdaException", "Lambda.SdkClientException", "Lambda.TooManyRequestsException"],
          "IntervalSeconds": 2,
          "MaxAttempts": 4,
          "BackoffRate": 2.0,
          "JitterStrategy": "FULL"
        }
      ],
      "Catch": [
        {
          "ErrorEquals": ["States.ALL"],
          "Assign": { "failure": "{% $states.errorOutput %}" },
          "Next": "RecordFailure"
        }
      ],
      "Next": "Route"
    },

    "Route": {
      "Type": "Choice",
      "Choices": [
        { "Condition": "{% $states.input.riskScore >= 80 %}", "Next": "RejectPayment" },
        { "Condition": "{% $states.input.riskScore < 20 %}", "Next": "ApprovePayment" }
      ],
      "Default": "ManualReview"
    },

    "ManualReview": {
      "Type": "Task",
      "Resource": "arn:aws:states:::sqs:sendMessage.waitForTaskToken",
      "Arguments": {
        "QueueUrl": "${ReviewQueueUrl}",
        "MessageBody": {
          "paymentId": "{% $paymentId %}",
          "taskToken": "{% $states.context.Task.Token %}"
        }
      },
      "TimeoutSeconds": 86400,
      "Catch": [
        { "ErrorEquals": ["States.Timeout"], "Assign": { "failure": "{% $states.errorOutput %}" }, "Next": "RejectPayment" }
      ],
      "Next": "DecisionChoice"
    },

    "DecisionChoice": {
      "Type": "Choice",
      "Choices": [
        { "Condition": "{% $states.input.decision = 'approved' %}", "Next": "ApprovePayment" }
      ],
      "Default": "RejectPayment"
    },

    "ApprovePayment": {
      "Type": "Task",
      "Resource": "arn:aws:states:::dynamodb:putItem",
      "Arguments": {
        "TableName": "${DecisionsTableName}",
        "Item": {
          "pk": { "S": "{% $paymentId %}" },
          "decision": { "S": "approved" },
          "amount": { "N": "{% $string($amount) %}" }
        }
      },
      "TimeoutSeconds": 15,
      "Output": { "decision": "approved" },
      "Next": "NotifyCustomers"
    },

    "RejectPayment": {
      "Type": "Task",
      "Resource": "arn:aws:states:::dynamodb:putItem",
      "Arguments": {
        "TableName": "${DecisionsTableName}",
        "Item": {
          "pk": { "S": "{% $paymentId %}" },
          "decision": { "S": "rejected" },
          "amount": { "N": "{% $string($amount) %}" }
        }
      },
      "TimeoutSeconds": 15,
      "Output": { "decision": "rejected" },
      "Next": "NotifyCustomers"
    },

    "NotifyCustomers": {
      "Type": "Map",
      "Items": "{% $recipients %}",
      "MaxConcurrency": 5,
      "ToleratedFailurePercentage": 10,
      "ItemSelector": {
        "customerId": "{% $states.context.Map.Item.Value %}",
        "paymentId": "{% $paymentId %}",
        "decision": "{% $states.input.decision %}"
      },
      "ItemProcessor": {
        "ProcessorConfig": { "Mode": "INLINE" },
        "StartAt": "PublishNotice",
        "States": {
          "PublishNotice": {
            "Type": "Task",
            "Resource": "arn:aws:states:::sns:publish",
            "Arguments": {
              "TopicArn": "${NoticeTopicArn}",
              "Message": "{% $string($states.input) %}"
            },
            "TimeoutSeconds": 15,
            "Retry": [
              { "ErrorEquals": ["States.TaskFailed"], "IntervalSeconds": 1, "MaxAttempts": 3, "BackoffRate": 2.0, "JitterStrategy": "FULL" }
            ],
            "End": true
          }
        }
      },
      "End": true
    },

    "RecordFailure": {
      "Type": "Fail",
      "Error": "PaymentReviewFailed",
      "Cause": "Screening failed after retries; see the execution history for the caught error."
    }
  }
}

Read it as the diagram above. ScreenPayment calls a Lambda, stores three variables, and retries the four transient Lambda errors with full jitter, then routes any other error to RecordFailure. Route branches on the risk score. ManualReview sends the task token to SQS and waits up to 24 hours. Both decisions write one item to DynamoDB, then NotifyCustomers maps over the recipients with MaxConcurrency 5, tolerating 10 percent failures, publishing one SNS message per recipient with its own retry.

⚠ VERIFY before you rely on it

The definition was linted and unit-tested structurally, but it could not be validated against the live service from the machine that wrote this lesson. Run aws stepfunctions validate-state-machine-definition on it (with the ${Placeholders} replaced) before deploying. In particular check the ItemSelector field in a JSONata Map and the Assign inside Catch: the documentation for JSONata Map fields is spread across several pages.

The .NET client and the linter

PaymentReviewClient.cs
using System.Text.Json;
using System.Text.RegularExpressions;
using Amazon.StepFunctions;
using Amazon.StepFunctions.Model;

namespace Harbourline.Messaging.Samples.StepFunctions;

public sealed record PaymentReviewInput(string PaymentId, decimal Amount, IReadOnlyList<string> Recipients);

/// <summary>
/// Starts payment-review executions and completes the manual-review callback.
/// Two Step Functions rules shape this class:
///  1. A Standard execution NAME is an idempotency key. Starting the same name with the same input while it is
///     running returns the existing execution instead of a second one. Build the name from the business key.
///  2. A callback task waits for SendTaskSuccess or SendTaskFailure carrying the task token. The token is a secret
///     capability: whoever holds it can finish the task, so only the review service may receive it.
/// </summary>
public sealed partial class PaymentReviewClient(IAmazonStepFunctions sfn, string stateMachineArn)
{
    // Execution names: max 80 chars, no whitespace, wildcards, brackets or the special characters " # % \ ^ | ~ ` $ & , ; : /
    [GeneratedRegex("[^A-Za-z0-9_-]")]
    private static partial Regex Unsafe();

    public static string ExecutionNameFor(string paymentId)
    {
        var cleaned = Unsafe().Replace(paymentId, "-");
        return cleaned.Length <= 80 ? cleaned : cleaned[..80];
    }

    public async Task<string> StartAsync(PaymentReviewInput input, CancellationToken ct = default)
    {
        var response = await sfn.StartExecutionAsync(new StartExecutionRequest
        {
            StateMachineArn = stateMachineArn,
            Name = ExecutionNameFor(input.PaymentId),
            Input = JsonSerializer.Serialize(input, JsonSerializerOptions.Web),
        }, ct);
        return response.ExecutionArn;
    }

    public Task ApproveAsync(string taskToken, CancellationToken ct = default) =>
        sfn.SendTaskSuccessAsync(new SendTaskSuccessRequest
        {
            TaskToken = taskToken,
            Output = JsonSerializer.Serialize(new { decision = "approved" }),
        }, ct);

    public Task RejectAsync(string taskToken, CancellationToken ct = default) =>
        sfn.SendTaskSuccessAsync(new SendTaskSuccessRequest
        {
            TaskToken = taskToken,
            Output = JsonSerializer.Serialize(new { decision = "rejected" }),
        }, ct);

    /// <summary>For a reviewer tool that crashed: fails the task so the workflow's Catch can run instead of waiting for the timeout.</summary>
    public Task FailAsync(string taskToken, string error, string cause, CancellationToken ct = default) =>
        sfn.SendTaskFailureAsync(new SendTaskFailureRequest { TaskToken = taskToken, Error = error, Cause = cause }, ct);
}
AslLint.cs
using System.Text.Json.Nodes;

namespace Harbourline.Messaging.Samples.StepFunctions;

/// <summary>
/// A tiny structural linter for Amazon States Language, meant to run in CI BEFORE the slower TestState API or a deploy.
/// It checks: StartAt exists, every Next / Default / Catch target exists, no state is unreachable, every Task has a
/// timeout, and a workflow is not mixing JSONPath-only fields into a JSONata machine. It does not replace the
/// service-side ValidateStateMachineDefinition API.
/// </summary>
public static class AslLint
{
    private static readonly string[] JsonPathOnlyFields = ["InputPath", "Parameters", "ResultSelector", "ResultPath", "OutputPath"];

    public static IReadOnlyList<string> Check(string aslJson)
    {
        var root = JsonNode.Parse(aslJson) as JsonObject ?? throw new ArgumentException("ASL must be a JSON object.", nameof(aslJson));
        var problems = new List<string>();
        var jsonata = root["QueryLanguage"]?.GetValue<string>() == "JSONata";
        CheckMachine(root, "$", jsonata, problems);
        return problems;
    }

    private static void CheckMachine(JsonObject machine, string path, bool jsonata, List<string> problems)
    {
        var startAt = machine["StartAt"]?.GetValue<string>();
        var states = machine["States"] as JsonObject;
        if (startAt is null || states is null) { problems.Add($"{path}: StartAt and States are required."); return; }
        if (!states.ContainsKey(startAt)) problems.Add($"{path}: StartAt '{startAt}' is not a state.");

        var reachable = new HashSet<string>();
        var queue = new Queue<string>();
        if (states.ContainsKey(startAt)) { queue.Enqueue(startAt); reachable.Add(startAt); }

        while (queue.Count > 0)
        {
            var name = queue.Dequeue();
            var state = (JsonObject)states[name]!;
            foreach (var target in Targets(state))
            {
                if (!states.ContainsKey(target)) problems.Add($"{path}.{name}: target '{target}' is not a state.");
                else if (reachable.Add(target)) queue.Enqueue(target);
            }
        }

        foreach (var (name, node) in states)
        {
            var state = (JsonObject)node!;
            var here = $"{path}.{name}";
            if (!reachable.Contains(name)) problems.Add($"{here}: unreachable state.");

            var type = state["Type"]?.GetValue<string>();
            if (type == "Task" && state["TimeoutSeconds"] is null)
                problems.Add($"{here}: Task without TimeoutSeconds (a hung worker would hold the execution for up to a year).");
            if (type is "Task" or "Pass" or "Map" or "Parallel" or "Wait" && state["Next"] is null && state["End"] is null)
                problems.Add($"{here}: needs Next or End.");
            if (jsonata) foreach (var field in JsonPathOnlyFields)
                if (state.ContainsKey(field)) problems.Add($"{here}: '{field}' is a JSONPath field and is invalid in a JSONata state.");

            // Recurse into nested machines. Their names are scoped to the branch.
            if (state["Branches"] is JsonArray branches)
                for (var i = 0; i < branches.Count; i++) CheckMachine((JsonObject)branches[i]!, $"{here}.Branches[{i}]", jsonata, problems);
            if ((state["ItemProcessor"] ?? state["Iterator"]) is JsonObject inner)
                CheckMachine(inner, $"{here}.ItemProcessor", jsonata, problems);
        }
    }

    private static IEnumerable<string> Targets(JsonObject state)
    {
        if (state["Next"] is JsonValue next) yield return next.GetValue<string>();
        if (state["Default"] is JsonValue def) yield return def.GetValue<string>();
        foreach (var rule in (state["Choices"] as JsonArray) ?? [])
            if (rule?["Next"] is JsonValue n) yield return n.GetValue<string>();
        foreach (var c in (state["Catch"] as JsonArray) ?? [])
            if (c?["Next"] is JsonValue n) yield return n.GetValue<string>();
    }
}

The unit tests lint the real ASL file and prove the linter catches a broken Next, a Task without a timeout, and a JSONPath field in a JSONata machine.

StepFunctionsTests.cs (excerpt)
[Fact]
public void The_lesson_state_machine_is_structurally_clean()
{
    Assert.Empty(AslLint.Check(PaymentReview));
}

[Fact]
public void A_missing_target_is_reported()
{
    var broken = PaymentReview.Replace("\"Next\": \"RecordFailure\"", "\"Next\": \"RecordFailur\"");

    var problems = AslLint.Check(broken);

    Assert.Contains(problems, p => p.Contains("'RecordFailur' is not a state"));
    Assert.Contains(problems, p => p.Contains("RecordFailure") && p.Contains("unreachable"));
}
dotnet add package AWSSDK.StepFunctions --version 4.1.0.2

Code explained line by line

  1. ExecutionNameFor replaces every character outside letters, digits, _ and - with a dash and caps the name at 80 characters, the documented limits. Why use the payment ID as the name? A Standard execution name is an idempotency key: starting the same name with the same input while it runs returns the existing execution, so a retried API request cannot start a second review of the same payment. Reusing a name after an execution closed works only when the 90-day history has expired, so do not rely on that.
  2. StartAsync serialises the input with JsonSerializerOptions.Web (camelCase) to match the ASL field names paymentId and recipients.
  3. ApproveAsync and RejectAsync call SendTaskSuccess with the decision as the task output, which the next state sees as $states.input.decision. FailAsync sends SendTaskFailure, so a crashed reviewer tool can fail the task now instead of waiting for the timeout.
  4. In AslLint.Check, QueryLanguage decides whether JSONPath-only fields are errors. CheckMachine is recursive and scoped per nested machine, because state names inside a Parallel branch or Map ItemProcessor are local to it.
  5. The reachability walk is a plain breadth-first search from StartAt over Next, Default, Choices[].Next and Catch[].Next. A state never reached is reported, which catches the typo-in-Next case from both ends.
  6. The Task rule (TimeoutSeconds required) is the lesson's house style, not an ASL rule: it encodes "never let a hung worker hold an execution for a year".

Security considerations

⚠ Learning shortcut — not for production

The CDK and Terraform in this lesson use a placeholder screening function ARN and let the account's default key encrypt everything. Production uses customer managed KMS keys for the SQS queue and the Step Functions state machine (Step Functions supports customer managed keys for execution data ⚠ VERIFY current scope), an execution role per state machine, and a review service whose role can call only states:SendTaskSuccess and states:SendTaskFailure.

⚠ Learning shortcut — not for production

The lab puts the payment ID and amount into the execution input, so they are in the 90-day history. For real payments, put an opaque reference in the input, keep personal and account data in your own store, and fetch it in the Lambda that needs it. Set IncludeExecutionData false on logging, and tell the data-protection team that the console history is another copy.

  • Execution role. One statement per integration, one resource each. lambda:InvokeFunction on the one function (and its versions), dynamodb:PutItem on the one table, and so on. Never *. Add a trust-policy condition on aws:SourceAccount (and aws:SourceArn where you can) to stop the confused-deputy problem.
  • Task tokens are bearer secrets. Send them only to the review service, do not log them, and do not e-mail them in links. A token works only for principals in the same account.
  • Who can start executions. states:StartExecution on a payment workflow is equivalent to being able to submit a payment. Restrict it to the one Pipe, API or service, and use the execution name for idempotency.
  • Who can redrive or stop. states:RedriveExecution and states:StopExecution are operational break-glass actions.
  • Logging. Error level, no execution data; encrypt the log group, set retention.
  • Immutable audit. The execution history expires after 90 days (30 on request). If you need a longer audit trail, write decisions to your own store (the DynamoDB write in the workflow is that record).
  • Compliance. This lesson does not make a system compliant with GDPR, UK GDPR or PCI DSS. Retention of execution history and logs, minimisation of what enters them, and the audit trail all need review by your legal, data-protection and security teams.

Banking use case

Harbourline's outbound-payment review:

StepStateWhy it is in the workflow
Sanctions and fraud screeningTask (Lambda) with retryA transient error must not reject a customer's payment
Route on riskChoiceThe thresholds are visible to auditors and changed by version, not by redeploying code
Human reviewTask with .waitForTaskToken, 24-hour timeoutWaits for days at no cost; the timeout is explicit policy
Decision recordTask (DynamoDB PutItem)An idempotent record independent of the 90-day history
NotificationsMap over partiesOne failing SMS gateway does not fail the payment (10 percent tolerance)

The compensation logic for what happens when a later step (the scheme submission) fails after funds are reserved is the subject of the saga lesson.

Common mistakes

  • No TimeoutSeconds on a Task. A hung worker holds the execution for up to a year. → Timeouts everywhere, plus a state machine timeout.
  • Forgetting the Lambda service-exception retry. A throttle rejects a good payment. → The four-error retrier with jitter.
  • States.ALL not last, or not alone. The definition is rejected or masks other retriers. → Put it in its own, final retrier.
  • Expecting States.ALL to catch everything. It misses States.DataLimitExceeded and States.Runtime. → Name DataLimitExceeded explicitly, and keep payloads small.
  • Passing big payloads through states. The 256 KiB limit fails the execution. → Store in S3 or DynamoDB; pass an ID.
  • Polling in a Standard workflow. A Wait and a Choice in a loop cost a transition per iteration. → .sync, a callback, or an EventBridge event.
  • Logging execution data. Personal data in CloudWatch Logs. → IncludeExecutionData false.
  • Assuming "exactly-once" protects Lambda side effects. A task timeout can leave work done. → Idempotent tasks, with the payment ID as a key.
  • Changing a long-running definition without versions. In-flight executions meet new logic. → Publish versions and use aliases.
  • A big, hand-edited JSON file nobody lints. → The linter and ValidateStateMachineDefinition in CI.

Troubleshooting

SymptomLikely causeFix
States.PermissionsExecution role lacks the action or resource, or a trust condition mismatchRead the cause string; add the exact action and ARN; check aws:SourceAccount and aws:SourceArn
States.QueryEvaluationErrorJSONata type error or undefined value (a missing field)Test the state with TestState; guard optional fields with $exists()
Execution stuck on the manual-review taskNo SendTaskSuccess, or the token was lostCheck the queue; set HeartbeatSeconds and a timeout; resend from the message body
TaskTimedOut returned to the reviewer serviceThe timeout fired firstHandle it and tell the user the case was auto-rejected
ExecutionLimitExceeded or throttled startsAccount or Region quota; new accounts have lower quotasSmooth the starts, request an increase
States.DataLimitExceededA payload went over 256 KiBStore the data elsewhere; pass a pointer
Execution fails after 25,000 eventsHistory limit in a long loop or large inline MapStart a new execution per batch or use Distributed Map
ExecutionAlreadyExists on startSame execution name with different input, or still runningUse the business key and the same input; check the existing execution
.sync task keeps running after the execution stoppedStep Functions could not cancel the jobGrant the cancel permission; stop the job in its own service; check billing
Retries do not stop a failing stepThe error name did not match any retrierLook at the Error in the history; match it or use States.TaskFailed
InvalidDefinition on createA malformed {% %} string, a path field in a JSONata state, or a missing NextRun the linter and validate-state-machine-definition
Redrive refusedOlder than 14 days, over 24,999 events, or started before November 2023Start a new execution with the same business key

Cost and cleanup

Cost: free at lab volumes. A Standard workflow costs about USD 0.000025 per state transition in us-east-1 (USD 25 per million), with a free tier of 4,000 transitions a month that does not expire after the first year. Each retry and each Distributed Map iteration is a transition. Working a rough example: one payment taking the manual-review path with five recipients is roughly 4 transitions plus 5 for the Map items, so about 10, which is 0.025 cents; a million of them is about USD 250. Express is billed by executions (about USD 1 per million) plus duration and memory ⚠ VERIFY the exact duration rates: the pricing page gives both per-GB-second and per-GB-hour forms. SQS, SNS, DynamoDB and Lambda are billed by their own services and are free at these volumes.

The lesson's three runs use fewer than 100 transitions. There is no hourly charge for an idle state machine, and no cost while a callback waits.

Free alternatives: run the structural tests (no account), and use TestState with mocks (no execution role and no resources). AWS also publishes Step Functions Local as a Docker image for offline runs ⚠ VERIFY the current image and its support for JSONata.

export AWS_PROFILE=learn-sandbox AWS_REGION=us-east-1

# Stop anything still running, then delete the state machine.
for exec in $(aws stepfunctions list-executions --state-machine-arn "$SM_ARN" --status-filter RUNNING \
    --query 'executions[].executionArn' --output text); do
  aws stepfunctions stop-execution --execution-arn "$exec" --error Cleanup --cause "Lab teardown"
done
aws stepfunctions delete-state-machine --state-machine-arn "$SM_ARN"

# Supporting resources.
aws dynamodb delete-table --table-name hl-m08gap-decisions
for q in hl-m08gap-manual-review; do
  url=$(aws sqs get-queue-url --queue-name "$q" --query QueueUrl --output text 2>/dev/null) && aws sqs delete-queue --queue-url "$url"
done
aws sns delete-topic --topic-arn "$(aws sns list-topics --query "Topics[?contains(TopicArn,'hl-m08gap-payment-notices')].TopicArn" --output text)"
aws logs delete-log-group --log-group-name /hl-m08gap/payment-review

# If you deployed with IaC: cdk destroy HlM08Gap-Workflow   |   terraform destroy

aws resourcegroupstaggingapi get-resources --tag-filters Key=Lab,Values=m08-messaging-services \
  --query 'ResourceTagMappingList[].ResourceARN'

State machine deletion is asynchronous: it can take a short while to finish.

Exercises

  • 🟢 Guided: Lint payment-review.asl.json, deploy it, run the three paths (risk 10, 90 and 50), complete the manual review with ApproveAsync, and write down the sequence of states in each execution history.
  • 🟡 Challenge: Add a Parallel state before Route that runs a sanctions check and a fraud score at the same time, and a Retry for States.Timeout with MaxAttempts 0 followed by States.ALL with three attempts (so timeouts are never retried but everything else is). Prove the retry order with TestState mocks, and update the linter so a Parallel state's branches are checked.
  • 🔴 Live problem: Ticket from Operations: "Since last week's deploy, about 3% of payments in manual review show TaskTimedOut errors in the review tool and the customer is told 'rejected'. Reviewers swear they approved them within the hour. The workflow timeout is 24 hours." Work out what could be happening and how you would prove each theory from the execution histories.

Assignment

Add stepfunctions-design.md to your portfolio repository for one Harbourline process of your choice (card dispute, account opening, standing-order amendment). Include: the diagram, the ASL, the choice of Standard or Express and why, the timeouts and heartbeats with numbers, the retry table for every task (error names, attempts, jitter), the callback design with who holds the token, the data you keep out of the execution input, the IAM statements for the execution role and the callers, the tests (lint, TestState), and a cost estimate at one million executions a month.

Solutions

Knowledge check

Interview questions

  • "When would you use Step Functions rather than a queue and a worker?" — When the sequence, the waiting and the failure handling are the problem: many steps, long waits, human approvals, compensation, an auditable path. For one kind of work with simple retry, a queue and a worker are simpler and cheaper.
  • "Standard or Express?" — Standard for long-running, auditable, non-idempotent work such as payments, billed per transition and kept for 90 days. Express for short, high-volume, idempotent work, billed per execution and duration, with logs as the only history.
  • "How do you build a human-approval step?" — .waitForTaskToken on a task that sends the token to the approver's service (SQS), a timeout (and optional heartbeat), and SendTaskSuccess/SendTaskFailure from that service with a narrowly-scoped role. Treat the token as a secret.
  • "How do you handle errors in a workflow?" — Retry with backoff and full jitter for transient errors, including the four Lambda service exceptions; Catch to route to a fallback or compensation state; explicit timeouts; a failed-execution alert from the EventBridge status events; and redrive after a fix.
  • "How do you test a state machine?" — A structural lint in CI, ValidateStateMachineDefinition, TestState with mocks for individual states, and a sandbox execution with stubs.

Certification practice

Completion checklist

  • I can explain state, execution and ASL, and name the state types
  • I chose Standard or Express for a workload and said why
  • I can write Retry and Catch with the right defaults, States.ALL last and jitter on
  • I completed a callback from .NET and kept the token out of logs
  • I linted the definition and ran a state with TestState
  • I redrove a failed execution after fixing a permission
  • I ran the cleanup and the tag check returned []

Key takeaways

  • A workflow definition is durable, visual documentation: Step Functions holds state, waits for free and records every step. Keep flow in ASL and business logic in code.
  • Standard (exactly-once, up to a year, 90-day history, per transition) for payments and approvals; Express (5 minutes, at-least-once or at-most-once, logs only) for high-volume short work.
  • JSONata replaces five path fields with Arguments and Output, adds Assign variables and $states. Expressions can fail at run time.
  • Retry, then Catch. Defaults: 3 attempts, 2.0 backoff, no jitter, so set JitterStrategy: FULL and MaxDelaySeconds. States.ALL must be last and alone. Always retry the four Lambda service exceptions and set timeouts everywhere.
  • Callbacks pause for free; the task token is a secret. Redrive recovers Standard failures within 14 days. Payloads are limited to 256 KiB and histories to 25,000 events.

Further reading

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page