Skip to Content
Cloud Engineering Academy
3. .NET Application Development on AWS

Deploying .NET to ECS on Fargate

Deploy a containerised ASP.NET Core API to Amazon ECS on Fargate in private subnets behind an Application Load Balancer, with separate task and execution roles, safe rolling deployments and a C# rollout watcher.

intermediate⏱ 2 h 🟡 Low costDVA-C03SAA-C03

Assumes: .NET 10 LTS · AWS SDK for .NET v4 (AWSSDK.ECS 4.1.1.1) · AWS CLI v2 · Amazon.CDK.Lib 2.277.0 · Terraform AWS provider v6 · Region us-east-1. Verified against official sources on 2026-11-18.

Learning objectives

By the end of this lesson you can:

  • Explain the four ECS building blocks — cluster, task definition, task, service — and what Fargate removes from your to-do list.
  • Write a Fargate task definition for an ASP.NET Core API: CPU and memory sizing, ARM64, awsvpc networking, logging, secrets and a read-only root file system.
  • Tell the task execution role from the task role, and give each one least-privilege permissions.
  • Put tasks in private subnets behind an Application Load Balancer using an ip target group.
  • Configure rolling deployments with the deployment circuit breaker so a bad release rolls itself back.
  • Deploy with the AWS CLI, AWS CDK (C#) or Terraform, watch the rollout from C#, and tear everything down.

Prerequisites

  • Containerising .NET and Amazon ECR: you need an image in an ECR repository.
  • VPC fundamentals: public vs private subnets, route tables, NAT gateways.
  • IAM in depth: roles, trust policies and iam:PassRole.
  • Tools: AWS CLI v2 with the learn-sandbox SSO profile, Docker, .NET 10 SDK. Optional: AWS CDK CLI or Terraform 1.14.

Why this matters in banking

Harbourline Bank's Accounts API answers "what is my balance?" for the mobile app. It runs all day, every day, with predictable traffic peaks at 08:00 and on payday. Three things go wrong at banks that deploy containers carelessly:

  1. Over-permissioned containers. A team gives the container the same role ECS uses to pull images and write logs, then adds secretsmanager:* "to make it work". Now any remote-code-execution bug in a JSON library can read every secret in the account.
  2. Public tasks. Someone sets assignPublicIp to ENABLED because the image would not pull. The API is now reachable directly from the internet, bypassing the load balancer, the WAF and the TLS policy.
  3. Deployments that never finish. A release with a broken health endpoint keeps starting and killing tasks for hours. Customers see intermittent 503 errors; on-call engineers see nothing obvious because no alarm watches the deployment.

An auditor will ask: Which identity can read customer data? Is the workload reachable from the internet? How do you know a release succeeded, and how do you roll back? This lesson gives you concrete answers to all three.

The idea in plain language

Amazon ECS (Elastic Container Service) is AWS's own container orchestrator. You tell it "keep two copies of this container running, register them with this load balancer, and replace any that die". ECS does the bookkeeping.

AWS Fargate is a way of running ECS tasks without managing servers. You do not create EC2 instances, patch them or size a cluster. You say "this task needs 0.25 vCPU and 512 MB", and AWS finds the capacity, runs the task in its own isolated micro-VM, and bills you per second for what you asked for.

Four words to learn:

TermPlain meaning.NET analogy
ClusterA logical grouping of services and tasks. With Fargate it holds no servers, just a name and settings.A solution folder
Task definitionA versioned JSON recipe: which image, how much CPU and memory, which ports, environment variables, roles and log settings. Every change creates a new revision (family:1, family:2 …).A .csproj plus launchSettings.json, frozen and versioned
TaskOne running copy of a task definition: one or more containers sharing a network interface.One running dotnet process
ServiceA controller that keeps N tasks running, replaces failed ones, registers them with a load balancer and rolls out new revisions.IIS application pool with automatic recycling, plus a deployment tool

Analogy: Think of a bank branch staffing rota. The task definition is the job description (skills, desk, keycard level). A task is one person on shift. The service is the branch manager who makes sure there are always two cashiers at the counter, replaces anyone who goes home sick, and phases in new staff one at a time when the job description changes. Fargate is the staffing agency: you never hire, house or train anybody yourself; you just request "two cashiers, standard skills" and pay by the hour.

What problem ECS on Fargate solves

Without an orchestrator, running containers in production means writing your own scripts to start them, restart them, spread them across Availability Zones, update them without downtime and wire them to a load balancer. ECS does all of that. Fargate additionally removes the servers underneath, which matters to a bank because every EC2 instance is something you must patch, harden, scan and evidence to auditors.

What you still own: the image (and its vulnerabilities), the task definition, IAM roles, network placement, security groups, scaling rules, logs and alarms.

Limitations and sharp edges (checked 2026-11-18):

  • Fixed CPU/memory combinations. For example, 0.25 vCPU (256 CPU units) allows 512 MiB, 1 GB or 2 GB of memory. Larger tasks go up to 16 vCPU and 120 GB ⚠ VERIFY the current upper limits in the task-size table before sizing large jobs.
  • Per-second billing with a one-minute minimum, priced per vCPU-hour and GB-hour. In us-east-1, Linux/x86 costs about USD 0.0405 per vCPU-hour and USD 0.0044 per GB-hour; Linux/ARM (Graviton) about USD 0.0324 and USD 0.0036. ARM is roughly 20% cheaper.
  • 20 GB of ephemeral storage is included per task; more is charged per GB-hour.
  • No host access. You cannot SSH into a Fargate task. For debugging there is ECS Exec (Session Manager into the container), but a chiseled .NET image has no shell, so in practice you debug through logs and metrics.
  • No GPUs on Fargate. Use ECS on EC2 for GPU work.
  • Image pull time counts towards start-up. Small images (chiseled, trimmed) start faster and scale out faster.

Where Express Mode fits. In late 2025 AWS launched Amazon ECS Express Mode. You give it an image, an execution role and an infrastructure role, and it creates the Fargate service, an HTTPS ALB with a certificate, auto scaling, alarms and a URL like servicename.ecs.region.on.aws. There is no extra charge beyond those resources. It is the official replacement for App Runner, which is closed to new customers. This lesson builds everything by hand first, because you cannot operate (or defend to an auditor) what you do not understand. The final exercise uses Express Mode so you can compare.

Microsoft / Azure comparison

You know (Microsoft/Azure)AWS equivalentWhat's different
Azure Container AppsECS on Fargate (or ECS Express Mode for the "just give me a URL" experience)ECS exposes every piece (target groups, security groups, roles); Container Apps hides more
AKSAmazon EKSECS is not Kubernetes: no YAML manifests, no kubectl, far fewer moving parts
Container App revisionTask definition revisionRevisions are immutable; a service points at exactly one revision at a time (outside blue/green)
Managed identity on a container appTask roleThe SDK finds the task role credentials automatically through the container credentials endpoint
ACR pull permission via managed identityTask execution roleA separate role used by the ECS agent, not by your code
Application Gateway / Front DoorApplication Load Balancer (+ CloudFront and WAF)The ALB lives in your VPC; tasks register with a target group by IP address
Key Vault references in app settingssecrets in the container definitionValues are injected at task start only; rotation needs a new deployment or in-app fetching
Log Analytics via Container Apps environmentawslogs log driver → CloudWatch LogsYou create the log group and set retention yourself
Deployment slots / revision traffic splitRolling update with circuit breaker, or blue/green deploymentsECS now has built-in blue/green; CodeDeploy is no longer required for it ⚠ VERIFY

Architecture

Security-boundary view. Each arrow crosses a control:

The key design decisions:

  • ALB in public subnets, tasks in private subnets with assignPublicIp = DISABLED. The only way in is through the ALB.
  • Security-group referencing: api-sg allows port 8080 only from alb-sg (a security group ID), never from a CIDR.
  • Target type ip. With the awsvpc network mode each task gets its own elastic network interface (ENI) and private IP, so the target group registers IPs, not instances.
  • Outbound path. Tasks need to reach ECR, CloudWatch Logs and Systems Manager. Either a NAT gateway (simple, about USD 0.045 per hour plus data) or interface VPC endpoints (private, about USD 0.01 per endpoint per AZ per hour ⚠ VERIFY). A DynamoDB or S3 gateway endpoint is free.

The two roles — the most important idea in this lesson

Task execution roleTask role
Used byThe ECS/Fargate agent, before and around your containerYour code, through the AWS SDK
Typical permissionsecr:GetAuthorizationToken, ecr:BatchGetImage, ecr:GetDownloadUrlForLayer, logs:CreateLogStream, logs:PutLogEvents; plus secretsmanager:GetSecretValue / ssm:GetParameters / kms:Decrypt only for secrets referenced in the task definitionWhatever the application needs: e.g. dynamodb:GetItem and dynamodb:Query on one table
Trust policy principalecs-tasks.amazonaws.comecs-tasks.amazonaws.com
Visible to the container?NoYes, via AWS_CONTAINER_CREDENTIALS_RELATIVE_URI
Symptom if missingTask stops before starting: CannotPullContainerError, ResourceInitializationErrorTask runs, but SDK calls fail with AccessDeniedException

The AWS managed policy AmazonECSTaskExecutionRolePolicy covers image pulls and log writing. Never attach your application permissions to the execution role, and never attach AmazonECSTaskExecutionRolePolicy to the task role.

Step-by-step implementation

Prepare the image and the network

Push the Accounts API image to ECR with an immutable tag such as v1 (see the ECR lesson). Use a VPC with two public and two private subnets across two AZs. The lab stack in labs/m03-accounts-api/infra/cdk creates exactly this.

Create the log group with a retention period

Create /hl-m03/accounts-api/ecs with 7-day retention. If the awslogs driver creates the group for you (awslogs-create-group), it has no expiry and the execution role needs logs:CreateLogGroup. Create it yourself.

Create the two roles

Execution role with AmazonECSTaskExecutionRolePolicy. Task role with an inline policy scoped to one DynamoDB table and one Parameter Store path. Both trust ecs-tasks.amazonaws.com, ideally with an aws:SourceAccount condition against the confused-deputy problem.

Register the task definition

Choose 256 CPU units and 512 MiB, ARM64, awsvpc, port 8080, read-only root file system with a writable /tmp volume, the log configuration and non-secret environment variables.

Create the cluster, security groups, ALB and target group

Target group: protocol HTTP, port 8080, target type ip, health check path /health/ready, deregistration delay 30 seconds (the default of 300 seconds makes every deployment five minutes slower).

Create the service

Desired count 2, launch type Fargate, private subnets, no public IP, the target group, a health check grace period, the deployment circuit breaker with rollback, minimumHealthyPercent 100 and maximumPercent 200.

Wait, test, then deploy a second revision

aws ecs wait services-stable, call the ALB URL, then register revision 2 with a changed environment variable and update the service. Watch the rollout with the C# watcher below.

Tear down

Delete the service, ALB, target group, cluster, log group, roles and (if you made one) the NAT gateway and VPC. Check nothing tagged Project=cea-lab remains.

AWS Console instructions

  1. Console → Elastic Container Service → Clusters → Create cluster. Name hl-m03-cluster, infrastructure AWS Fargate (serverless) only. Turn on Container Insights with enhanced observability if you want task-level metrics (it costs CloudWatch metric charges; skip it for a one-hour lab).
  2. Console → Elastic Container Service → Task definitions → Create new task definition. Launch type AWS Fargate, OS/architecture Linux/ARM64, CPU .25 vCPU, memory .5 GB. Choose your Task role and Task execution role. Under Container – 1: name api, image URI from ECR, container port 8080, protocol HTTP, app protocol HTTP. Tick Read only root file system. Under Log collection, choose Amazon CloudWatch and the log group you created. Add environment variables. Add a bind mount volume named tmp and mount it at /tmp.
  3. Console → EC2 → Target Groups → Create target group. Type IP addresses, protocol HTTP, port 8080, your VPC, health check path /health/ready. Skip registering targets; ECS does that.
  4. Console → EC2 → Load Balancers → Create → Application Load Balancer. Internet-facing, two public subnets, security group alb-sg, listener forwarding to the target group. (Production: HTTPS 443 with an ACM certificate and a WAF web ACL.)
  5. Console → Elastic Container Service → Clusters → hl-m03-cluster → Services → Create. Compute Launch type: FARGATE, task definition family and revision, desired tasks 2. Networking: your VPC, the two private subnets, security group api-sg, Public IP turned off. Load balancing: existing ALB, existing target group, container api:8080. Deployment options: rolling update, min 100%, max 200%, Deployment circuit breaker: Use the Amazon ECS deployment circuit breaker with Rollback on failures ticked. Health check grace period 60 seconds.
  6. Open Services → hl-m03-accounts-api → Deployments and Events to watch tasks start and register.

AWS CLI

The commands below assume you already have a VPC, two public subnets, two private subnets, alb-sg and api-sg. Replace the subnet-…, sg-… and account values.

# Common variables (Git Bash). PROFILE and REGION follow CONVENTIONS.md.
export AWS_PROFILE=learn-sandbox AWS_REGION=us-east-1
ACCOUNT=$(aws sts get-caller-identity --query Account --output text)
TAGS='key=Project,value=cea-lab key=Lab,value=m03-fargate key=Environment,value=learn key=Owner,value=you'

# 1. Log group with retention (never infinite).
aws logs create-log-group --log-group-name /hl-m03/accounts-api/ecs \
  --tags Project=cea-lab,Lab=m03-fargate,Environment=learn,Owner=you
aws logs put-retention-policy --log-group-name /hl-m03/accounts-api/ecs --retention-in-days 7

# 2. Trust policy shared by both roles. aws:SourceAccount limits who can make ECS assume it.
cat > ecs-trust.json <<EOF
{ "Version": "2012-10-17",
  "Statement": [{ "Effect": "Allow",
    "Principal": { "Service": "ecs-tasks.amazonaws.com" },
    "Action": "sts:AssumeRole",
    "Condition": { "StringEquals": { "aws:SourceAccount": "$ACCOUNT" } } }] }
EOF

# 3. Execution role: AWS managed policy for ECR pulls and log writes.
aws iam create-role --role-name hl-m03-accounts-api-exec \
  --assume-role-policy-document file://ecs-trust.json
aws iam attach-role-policy --role-name hl-m03-accounts-api-exec \
  --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy

# 4. Task role: only what the application needs.
aws iam create-role --role-name hl-m03-accounts-api-task \
  --assume-role-policy-document file://ecs-trust.json
cat > task-policy.json <<EOF
{ "Version": "2012-10-17",
  "Statement": [
    { "Sid": "ReadAccountsTable", "Effect": "Allow",
      "Action": ["dynamodb:GetItem", "dynamodb:Query", "dynamodb:DescribeTable"],
      "Resource": ["arn:aws:dynamodb:us-east-1:$ACCOUNT:table/hl-m03-accounts",
                   "arn:aws:dynamodb:us-east-1:$ACCOUNT:table/hl-m03-accounts/index/*"] },
    { "Sid": "ReadOwnParameters", "Effect": "Allow",
      "Action": ["ssm:GetParametersByPath"],
      "Resource": "arn:aws:ssm:us-east-1:$ACCOUNT:parameter/hl-m03/accounts-api" } ] }
EOF
aws iam put-role-policy --role-name hl-m03-accounts-api-task \
  --policy-name accounts-api-runtime --policy-document file://task-policy.json

# 5. Register the task definition (file shown in "Complete code").
aws ecs register-task-definition --cli-input-json file://task-definition.json

# 6. Cluster.
aws ecs create-cluster --cluster-name hl-m03-cluster --tags $TAGS

# 7. Target group (type ip) and listener. Deregistration delay 30 s instead of 300 s.
TG_ARN=$(aws elbv2 create-target-group --name hl-m03-api-tg --protocol HTTP --port 8080 \
  --target-type ip --vpc-id vpc-0123456789abcdef0 \
  --health-check-path /health/ready --health-check-interval-seconds 15 \
  --healthy-threshold-count 2 --unhealthy-threshold-count 3 --matcher HttpCode=200 \
  --query 'TargetGroups[0].TargetGroupArn' --output text)
aws elbv2 modify-target-group-attributes --target-group-arn "$TG_ARN" \
  --attributes Key=deregistration_delay.timeout_seconds,Value=30

ALB_ARN=$(aws elbv2 create-load-balancer --name hl-m03-api-alb --type application \
  --scheme internet-facing --subnets subnet-pub-a subnet-pub-b --security-groups sg-alb \
  --query 'LoadBalancers[0].LoadBalancerArn' --output text)
# ⚠ Learning shortcut: HTTP 80. Production uses HTTPS 443 with an ACM certificate.
aws elbv2 create-listener --load-balancer-arn "$ALB_ARN" --protocol HTTP --port 80 \
  --default-actions Type=forward,TargetGroupArn="$TG_ARN"

# 8. Service: private subnets, no public IP, circuit breaker with rollback.
aws ecs create-service --cluster hl-m03-cluster --service-name hl-m03-accounts-api \
  --task-definition hl-m03-accounts-api --desired-count 2 --launch-type FARGATE \
  --platform-version LATEST \
  --network-configuration 'awsvpcConfiguration={subnets=[subnet-priv-a,subnet-priv-b],securityGroups=[sg-api],assignPublicIp=DISABLED}' \
  --load-balancers "targetGroupArn=$TG_ARN,containerName=api,containerPort=8080" \
  --health-check-grace-period-seconds 60 \
  --deployment-configuration 'deploymentCircuitBreaker={enable=true,rollback=true},minimumHealthyPercent=100,maximumPercent=200' \
  --enable-ecs-managed-tags --propagate-tags SERVICE --tags $TAGS

# 9. Block until the service reaches a steady state (polls every 15 s, gives up after 40 tries).
aws ecs wait services-stable --cluster hl-m03-cluster --services hl-m03-accounts-api

# 10. Test through the ALB.
DNS=$(aws elbv2 describe-load-balancers --load-balancer-arns "$ALB_ARN" \
  --query 'LoadBalancers[0].DNSName' --output text)
curl -s "http://$DNS/health/ready"

# 11. Roll out a new revision (after re-registering the task definition with a change).
aws ecs update-service --cluster hl-m03-cluster --service hl-m03-accounts-api \
  --task-definition hl-m03-accounts-api

# 12. See why tasks stopped (the single most useful debugging command).
aws ecs list-tasks --cluster hl-m03-cluster --desired-status STOPPED --query 'taskArns' --output text \
  | xargs -n 10 aws ecs describe-tasks --cluster hl-m03-cluster \
      --query 'tasks[].{task:taskArn,reason:stoppedReason,exit:containers[0].exitCode}' --tasks

Infrastructure as Code

The lab project labs/m03-accounts-api/infra/cdk/Harbourline.Accounts.Infra/FargateStack.cs contains the full stack (VPC, flow logs, DynamoDB gateway endpoint, cluster, task role, ALB, health check, auto scaling and cdk-nag). The essentials look like this:

FargateService.cs (excerpt)
using Amazon.CDK;
using Amazon.CDK.AWS.EC2;
using Amazon.CDK.AWS.ECR;
using Amazon.CDK.AWS.ECS;
using Amazon.CDK.AWS.ECS.Patterns;
using Amazon.CDK.AWS.IAM;
using Amazon.CDK.AWS.Logs;
using Constructs;

public sealed class AccountsFargateStack : Stack
{
    public AccountsFargateStack(Construct scope, string id, IStackProps props) : base(scope, id, props)
    {
        var vpc = new Vpc(this, "Vpc", new VpcProps { MaxAzs = 2, NatGateways = 1 });
        var cluster = new Cluster(this, "Cluster", new ClusterProps { Vpc = vpc, ClusterName = "hl-m03-cluster" });

        // Task role: what the CODE may do.
        var taskRole = new Role(this, "TaskRole", new RoleProps
        {
            AssumedBy = new ServicePrincipal("ecs-tasks.amazonaws.com"),
        });
        taskRole.AddToPolicy(new PolicyStatement(new PolicyStatementProps
        {
            Actions = ["dynamodb:GetItem", "dynamodb:Query", "dynamodb:DescribeTable"],
            Resources = [$"arn:aws:dynamodb:{Region}:{Account}:table/hl-m03-accounts"],
        }));

        var logGroup = new LogGroup(this, "Logs", new LogGroupProps
        {
            LogGroupName = "/hl-m03/accounts-api/ecs",
            Retention = RetentionDays.ONE_WEEK,
            RemovalPolicy = RemovalPolicy.DESTROY,
        });

        var service = new ApplicationLoadBalancedFargateService(this, "Api", new ApplicationLoadBalancedFargateServiceProps
        {
            Cluster = cluster,
            Cpu = 256,
            MemoryLimitMiB = 512,
            DesiredCount = 2,
            RuntimePlatform = new RuntimePlatform
            {
                CpuArchitecture = CpuArchitecture.ARM64,
                OperatingSystemFamily = OperatingSystemFamily.LINUX,
            },
            TaskImageOptions = new ApplicationLoadBalancedTaskImageOptions
            {
                Image = ContainerImage.FromEcrRepository(
                    Repository.FromRepositoryName(this, "Repo", "hl-m03-accounts-api"), "v1"),
                ContainerPort = 8080,
                TaskRole = taskRole, // the execution role is generated by CDK with minimal pull/log rights
                LogDriver = LogDrivers.AwsLogs(new AwsLogDriverProps { StreamPrefix = "api", LogGroup = logGroup }),
                Environment = new Dictionary<string, string> { ["ASPNETCORE_ENVIRONMENT"] = "Production" },
            },
            TaskSubnets = new SubnetSelection { SubnetType = SubnetType.PRIVATE_WITH_EGRESS },
            AssignPublicIp = false,
            CircuitBreaker = new DeploymentCircuitBreaker { Enable = true, Rollback = true },
            HealthCheckGracePeriod = Duration.Seconds(60),
            MinHealthyPercent = 100,
            MaxHealthyPercent = 200,
        });

        service.TargetGroup.ConfigureHealthCheck(new Amazon.CDK.AWS.ElasticLoadBalancingV2.HealthCheck
        {
            Path = "/health/ready",
            Interval = Duration.Seconds(15),
            HealthyThresholdCount = 2,
            UnhealthyThresholdCount = 3,
        });
        service.TargetGroup.SetAttribute("deregistration_delay.timeout_seconds", "30");
    }
}

ApplicationLoadBalancedFargateService is an L3 pattern: one construct creates the ALB, listener, target group, security groups (with the ALB-to-task rule), execution role, task definition and service.

Complete code

The task definition

task-definition.json
{
  "family": "hl-m03-accounts-api",
  "requiresCompatibilities": ["FARGATE"],
  "networkMode": "awsvpc",
  "cpu": "256",
  "memory": "512",
  "runtimePlatform": { "cpuArchitecture": "ARM64", "operatingSystemFamily": "LINUX" },
  "executionRoleArn": "arn:aws:iam::111122223333:role/hl-m03-accounts-api-exec",
  "taskRoleArn": "arn:aws:iam::111122223333:role/hl-m03-accounts-api-task",
  "volumes": [{ "name": "tmp" }],
  "containerDefinitions": [
    {
      "name": "api",
      "image": "111122223333.dkr.ecr.us-east-1.amazonaws.com/hl-m03-accounts-api:v1",
      "essential": true,
      "readonlyRootFilesystem": true,
      "portMappings": [{ "containerPort": 8080, "protocol": "tcp", "name": "http", "appProtocol": "http" }],
      "mountPoints": [{ "sourceVolume": "tmp", "containerPath": "/tmp", "readOnly": false }],
      "environment": [
        { "name": "ASPNETCORE_ENVIRONMENT", "value": "Production" },
        { "name": "Repository__Provider", "value": "DynamoDb" },
        { "name": "Repository__TableName", "value": "hl-m03-accounts" },
        { "name": "ParameterStore__Path", "value": "/hl-m03/accounts-api/" },
        { "name": "AWS__Region", "value": "us-east-1" }
      ],
      "stopTimeout": 30,
      "logConfiguration": {
        "logDriver": "awslogs",
        "options": {
          "awslogs-group": "/hl-m03/accounts-api/ecs",
          "awslogs-region": "us-east-1",
          "awslogs-stream-prefix": "api",
          "mode": "non-blocking",
          "max-buffer-size": "4m"
        }
      }
    }
  ],
  "tags": [
    { "key": "Project", "value": "cea-lab" },
    { "key": "Lab", "value": "m03-fargate" },
    { "key": "Environment", "value": "learn" },
    { "key": "Owner", "value": "you" }
  ]
}

A C# rollout watcher

Pipelines often fire update-service and walk away. This small console app waits for the new deployment to complete or fail, prints the service events, and returns a non-zero exit code on failure, so a GitHub Actions job goes red when ECS rolls back.

Harbourline.Deploy.Watcher.csproj
<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net10.0</TargetFramework>
    <Nullable>enable</Nullable>
    <ImplicitUsings>enable</ImplicitUsings>
    <TreatWarningsAsErrors>true</TreatWarningsAsErrors>
    <RootNamespace>Harbourline.Deploy.Watcher</RootNamespace>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="AWSSDK.ECS" Version="4.1.1.1" />
    <PackageReference Include="AWSSDK.SSO" Version="4.1.0.2" />
    <PackageReference Include="AWSSDK.SSOOIDC" Version="4.1.0.2" />
  </ItemGroup>
</Project>
Program.cs
using Amazon.ECS;
using Amazon.ECS.Model;
using Task = System.Threading.Tasks.Task; // Amazon.ECS.Model also defines a Task type (an ECS task)

// Usage: dotnet run -- <cluster> <service> [timeoutMinutes]
if (args.Length < 2)
{
    Console.Error.WriteLine("Usage: Harbourline.Deploy.Watcher <cluster> <service> [timeoutMinutes]");
    return 2;
}

var cluster = args[0];
var serviceName = args[1];
var timeout = TimeSpan.FromMinutes(args.Length > 2 && int.TryParse(args[2], out var m) ? m : 15);

// Region and credentials come from the default chain: AWS_PROFILE=learn-sandbox locally,
// the runner's OIDC role in CI. No keys in code.
using var ecs = new AmazonECSClient();
using var cts = new CancellationTokenSource(timeout);
var seenEvents = new HashSet<string>();

try
{
    while (true)
    {
        var response = await ecs.DescribeServicesAsync(new DescribeServicesRequest
        {
            Cluster = cluster,
            Services = [serviceName],
        }, cts.Token);

        // SDK v4: collections can be null and value types are nullable. Guard every access.
        var service = response.Services?.FirstOrDefault()
            ?? throw new InvalidOperationException($"Service {serviceName} not found in {cluster}.");

        // Print new service events oldest-first (ECS returns newest first).
        foreach (var e in (service.Events ?? []).AsEnumerable().Reverse())
        {
            if (e.Id is not null && seenEvents.Add(e.Id))
            {
                Console.WriteLine($"{e.CreatedAt:HH:mm:ss} {e.Message}");
            }
        }

        var primary = (service.Deployments ?? []).FirstOrDefault(d => d.Status == "PRIMARY");
        if (primary is null)
        {
            Console.Error.WriteLine("No PRIMARY deployment found.");
            return 1;
        }

        Console.WriteLine(
            $"[{DateTime.UtcNow:HH:mm:ss}] {primary.TaskDefinition?.Split('/').Last()} " +
            $"rollout={primary.RolloutState} running={primary.RunningCount ?? 0}/{primary.DesiredCount ?? 0} " +
            $"failed={primary.FailedTasks ?? 0}");

        if (primary.RolloutState == DeploymentRolloutState.COMPLETED)
        {
            Console.WriteLine("Deployment completed.");
            return 0;
        }

        if (primary.RolloutState == DeploymentRolloutState.FAILED)
        {
            Console.Error.WriteLine($"Deployment FAILED: {primary.RolloutStateReason}");
            return 1;
        }

        await Task.Delay(TimeSpan.FromSeconds(15), cts.Token);
    }
}
catch (OperationCanceledException)
{
    Console.Error.WriteLine($"Timed out after {timeout.TotalMinutes} minutes waiting for the rollout.");
    return 1;
}

Run it after an update:

aws ecs update-service --cluster hl-m03-cluster --service hl-m03-accounts-api --task-definition hl-m03-accounts-api
AWS_PROFILE=learn-sandbox AWS_REGION=us-east-1 dotnet run -- hl-m03-cluster hl-m03-accounts-api 15

Code explained line by line

Task definition

  1. "family" — the name; each register-task-definition call creates the next revision (hl-m03-accounts-api:1, :2 …). Revisions are immutable, which gives you an audit trail of exactly what ran.
  2. "requiresCompatibilities": ["FARGATE"] — ECS validates the definition against Fargate rules at registration time, so mistakes fail early.
  3. "networkMode": "awsvpc" — required on Fargate. Each task gets its own ENI, private IP and security groups. That is why the target group type is ip.
  4. "cpu": "256", "memory": "512" — 0.25 vCPU and 512 MiB, the smallest Fargate size. Fine for a small API; measure before going bigger.
  5. "runtimePlatform" with ARM64 — Graviton. Your image must be built for linux/arm64 (the module's Dockerfile cross-compiles). An x64-only image fails with exec format error.
  6. "executionRoleArn" / "taskRoleArn" — the two roles discussed above. Registering a task definition with a role requires iam:PassRole on that role.
  7. "volumes": [{ "name": "tmp" }] plus "mountPoints" — an ephemeral bind mount so /tmp stays writable while the rest of the file system is read-only. .NET writes some temporary files (and diagnostics) there.
  8. "readonlyRootFilesystem": true — an attacker who gets code execution cannot drop files into the application folder. This is a common control in CIS-style container benchmarks.
  9. "portMappings" — container port 8080, the default ASPNETCORE_HTTP_PORTS in the official .NET 8+ images. With awsvpc there is no host port to choose.
  10. "environment" — non-secret settings only. Double underscores map to configuration sections: Repository__TableName becomes Repository:TableName.
  11. "stopTimeout": 30 — after ECS sends SIGTERM, it waits up to 30 seconds (the maximum on Fargate is 120) before SIGKILL. The .NET host's default shutdown timeout is also 30 seconds, so in-flight requests can finish.
  12. "logConfiguration" with "mode": "non-blocking" — if CloudWatch Logs is slow, log lines are buffered (up to max-buffer-size) rather than blocking your application's stdout. The trade-off is that you may lose lines under extreme pressure; for an audit trail, write audit events to a durable store, not only to logs.

Watcher

  1. new AmazonECSClient() — resolves Region and credentials from the default chain; no keys in code.
  2. DescribeServicesAsync — one call returns the service, its deployments and its last 100 events.
  3. response.Services?.FirstOrDefault() — in AWS SDK for .NET v4, list properties are null (not empty) when absent and numeric properties are nullable (int?). Code written for v3 often throws NullReferenceException after upgrading.
  4. service.Events ... .Reverse() — ECS events are newest-first; reversing gives a readable timeline.
  5. primary.RolloutState — IN_PROGRESS, COMPLETED or FAILED. With the circuit breaker on, FAILED means ECS gave up and (with rollback) started redeploying the last good revision.
  6. Exit codes 0/1/2 — CI systems understand these without parsing output.

Security considerations

⚠ Learning shortcut — not for production

The lab ALB listens on HTTP port 80 with no WAF, and a single NAT gateway gives the private tasks outbound internet access. In production: an HTTPS 443 listener with an ACM certificate and a modern TLS policy, an HTTP→HTTPS redirect, an AWS WAF web ACL (see WAF and Shield), interface VPC endpoints for ECR (ecr.api, ecr.dkr), CloudWatch Logs, Systems Manager and Secrets Manager plus the S3 gateway endpoint (ECR image layers live in S3), and either no NAT gateway at all or an egress firewall (see segmentation and egress control).

  • Least privilege on both roles. Execution role: managed policy plus, if you inject secrets, secretsmanager:GetSecretValue on those specific secret ARNs. Task role: specific actions on specific resources.
  • Confused-deputy protection. Add aws:SourceAccount (and optionally aws:SourceArn) conditions to the trust policy.
  • No public IPs on tasks. Enforce it with an SCP or AWS Config rule (ecs-task-definition-… and ecs-service-… managed rules exist ⚠ VERIFY exact names) so a hurried engineer cannot turn it on.
  • Immutable image tags and digest pinning. With ECR tag immutability, v1 can never be overwritten. For the strongest guarantee, reference the image by digest (@sha256:…).
  • Non-root, read-only, no shell. The chiseled .NET image runs as UID 1654 with no shell. ECS Exec is therefore of little use; that is a feature, not a bug.
  • ECS Exec governance. If you enable enableExecuteCommand on a service, every session is an interactive shell into production. Log sessions to CloudWatch or S3 with KMS encryption, and restrict ecs:ExecuteCommand to a break-glass role.
  • Logs contain personal data. Even with pseudonymised customer IDs, logs are personal data under UK GDPR. Set retention, encrypt the log group with a KMS key in production, and restrict logs:GetLogEvents.
  • Shared responsibility. AWS patches the Fargate host and micro-VM; you patch the image. Rebuild and redeploy at least monthly, and on every critical CVE.

Banking use case

Harbourline runs the Accounts API on Fargate in two AZs in eu-west-2 (London) for UK customers. The platform team publishes a CDK construct library that every team uses, so all services get the same baseline: private subnets, alb-sg → api-sg referencing, VPC endpoints instead of NAT, read-only root file systems, KMS-encrypted log groups with 90-day retention and a circuit breaker with rollback.

A release pipeline registers a new task definition revision, updates the service and runs the rollout watcher. If the rollout fails, the circuit breaker rolls back automatically, the pipeline goes red, and the change ticket is automatically updated with the failure reason from rolloutStateReason. The change advisory board gets evidence of "release attempted, auto-rolled back, no customer impact" without anyone writing it by hand.

Each task definition revision is retained, so in a regulatory inquiry the bank can answer "which image digest, which environment variables and which IAM role served customer requests at 14:32 on 3 March?" by matching the service's deployment history to the revision.

Common mistakes

  • Putting application permissions on the execution role → "it worked when I added DynamoDB to that role" → the code does not use the execution role at all, so the change either did nothing or (worse) you also merged the roles. Keep them separate and name them …-exec and …-task.
  • assignPublicIp = ENABLED to fix image pulls → tasks in private subnets cannot reach ECR without NAT or endpoints → add a NAT gateway or the ECR, S3 and Logs endpoints instead.
  • Target type instance → registration fails for awsvpc tasks → use ip.
  • Health check on / → the API returns 404 or requires authentication → the ALB marks every task unhealthy. Use an anonymous /health/ready.
  • Using latest as the image tag → a deployment of "the same" task definition pulls a different image → use immutable version tags or digests.
  • Forgetting the deregistration delay → each rolling deployment waits 300 seconds per batch → set 30–60 seconds for short API requests.
  • x64 image on an ARM64 task → exec format error in the logs → build with --platform linux/arm64 or a multi-arch manifest.
  • No log retention → logs kept forever, growing cost and GDPR storage-limitation problems → set retention on every log group.

Troubleshooting

SymptomLikely causeFix
Task stops with CannotPullContainerError ... i/o timeoutPrivate subnet with no route to ECR (no NAT, no endpoints)Add NAT or the ecr.api, ecr.dkr, S3 gateway and logs endpoints
CannotPullContainerError ... not foundWrong tag, wrong Region or the image was never pushedaws ecr describe-images --repository-name … --image-ids imageTag=v1
ResourceInitializationError: unable to pull secrets or registry authExecution role lacks secretsmanager:GetSecretValue / ssm:GetParameters / kms:Decrypt, or no network path to those servicesAdd the permission for that ARN; add endpoints or NAT
exec format error in logsImage architecture does not match runtimePlatformBuild for linux/arm64 or switch the task to X86_64
Tasks start, then stop with "failed ELB health checks"Wrong health check path or port, app not listening on 8080, security group blocks ALB → taskCheck the target group's health check, ASPNETCORE_HTTP_PORTS, and api-sg inbound rule from alb-sg
Tasks keep restarting during slow start-upHealth check grace period too shortRaise healthCheckGracePeriodSeconds; make start-up faster
ALB returns 503No healthy targets registeredaws elbv2 describe-target-health --target-group-arn …
App logs AccessDeniedException for DynamoDBTask role missing the action or resource ARNFix the task role policy; check the ARN includes /index/* for GSIs
No logs at allExecution role cannot write, wrong log group name or RegionCheck logConfiguration and the execution role
services-stable waiter times outDeployment still cyclingaws ecs describe-services → events; then look at stopped tasks' stoppedReason

Cost and cleanup

What this lesson costs (us-east-1, checked 2026-11-18; prices change, confirm on the pricing pages):

ResourceRateOne hour
2 Fargate tasks, ARM64, 0.25 vCPU + 0.5 GBabout USD 0.0099 per task-hourabout USD 0.02
Application Load Balancerabout USD 0.0225 per hour + LCU usage ⚠ VERIFYabout USD 0.03
NAT gatewayabout USD 0.045 per hour + USD 0.045 per GB processedabout USD 0.05
Public IPv4 addresses (ALB nodes, NAT)USD 0.005 per IP per hourabout USD 0.02
CloudWatch LogsUSD 0.50 per GB ingested (first 5 GB/month free tier) ⚠ VERIFYcents

Total: roughly USD 0.10–0.15 for one hour. Leaving it running for a month costs about USD 70–100, mostly ALB and NAT. Tear down the same day.

Free alternative: run the same image locally with docker compose (previous lessons) and do this lesson's CLI and IaC sections as a design-only exercise, using cdk synth or terraform plan without deploying.

Teardown:

export AWS_PROFILE=learn-sandbox AWS_REGION=us-east-1
# Scale to zero first so the service can be deleted cleanly.
aws ecs update-service --cluster hl-m03-cluster --service hl-m03-accounts-api --desired-count 0
aws ecs delete-service --cluster hl-m03-cluster --service hl-m03-accounts-api --force
aws ecs wait services-inactive --cluster hl-m03-cluster --services hl-m03-accounts-api

aws elbv2 delete-load-balancer --load-balancer-arn "$ALB_ARN"
aws elbv2 wait load-balancers-deleted --load-balancer-arns "$ALB_ARN"
aws elbv2 delete-target-group --target-group-arn "$TG_ARN"
aws ecs delete-cluster --cluster hl-m03-cluster

# Deregister every revision of the task definition (they cost nothing, but keep the account tidy).
for td in $(aws ecs list-task-definitions --family-prefix hl-m03-accounts-api --query 'taskDefinitionArns' --output text); do
  aws ecs deregister-task-definition --task-definition "$td" > /dev/null
done

aws logs delete-log-group --log-group-name /hl-m03/accounts-api/ecs
aws iam detach-role-policy --role-name hl-m03-accounts-api-exec \
  --policy-arn arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
aws iam delete-role --role-name hl-m03-accounts-api-exec
aws iam delete-role-policy --role-name hl-m03-accounts-api-task --policy-name accounts-api-runtime
aws iam delete-role --role-name hl-m03-accounts-api-task

# If you used the CDK stack instead:  cdk destroy HlM03-Fargate
# Delete the NAT gateway and release its Elastic IP if you created them by hand.

# Final check: nothing tagged for this lab remains.
aws resourcegroupstaggingapi get-resources --tag-filters Key=Project,Values=cea-lab \
  --query 'ResourceTagMappingList[].ResourceARN'

Exercises

  • 🟢 Guided: Deploy the Accounts API image with the CLI steps above, call /health/ready and one account endpoint through the ALB, then register revision 2 with Features__IncludePendingTransactions=true and roll it out while the C# watcher runs.
  • 🟡 Challenge: Deliberately break revision 3 (set the container port to 9090 in the port mapping only, so the app still listens on 8080). Observe the circuit breaker roll back. Record how long the rollback took and which events ECS emitted.
  • 🔴 Live problem: Ticket INC-4471: "After moving the Accounts API to new private subnets, tasks fail to start with ResourceInitializationError. The old subnets worked. No code changed." Find the cause and propose two fixes with their costs.

Assignment

Create a deploy/ folder in your portfolio repository with: the task definition JSON (role ARNs replaced by placeholders), a Terraform or CDK stack that deploys it, the rollout watcher, and a short README.md that explains the two roles and the network path for image pulls. Include a screenshot-free "evidence" section: the CLI output of describe-services showing rolloutState: COMPLETED.

Solutions

Knowledge check

Interview questions

  • "Explain task role vs execution role." — The execution role is used by the ECS agent to set the task up: pull the image, write logs, fetch secrets referenced in the task definition. The task role is assumed by the application code through the container credentials endpoint. Separate them so a compromised application cannot read every injected secret or pull other images, and so permissions map to responsibilities.
  • "Why Fargate over ECS on EC2 for a bank API?" — No hosts to patch, harden or evidence; per-task isolation in a micro-VM; per-second billing; simpler scaling. Choose EC2 when you need GPUs, very high steady utilisation where reserved EC2 is cheaper, special kernel settings or daemon-style agents per host.
  • "How do you make deployments safe on ECS?" — Immutable images, rolling updates with minimumHealthyPercent 100, a readiness-based ALB health check, a sensible grace period, the deployment circuit breaker with rollback, CloudWatch alarms on 5xx and latency, and a pipeline step that waits for rolloutState. For high-risk changes, blue/green with a test listener.
  • "A deployment has been in progress for an hour. What do you check?" — Service events, stopped tasks' stoppedReason and exit codes, target health reason codes, application logs for start-up exceptions, and whether the circuit breaker is enabled.

Certification practice

Completion checklist

  • I can explain cluster, task definition, task and service to a colleague in two sentences
  • I can say which role pulls the image and which role my code uses
  • I deployed the API to private subnets behind an ALB (or completed the design-only alternative)
  • I watched a rollout complete and a broken rollout roll back
  • I ran the teardown and the tag check returned an empty list

Key takeaways

  • ECS is the orchestrator; Fargate removes the servers. You still own the image, roles, network and alarms.
  • Execution role = the agent's permissions (pull, logs, injected secrets). Task role = your code's permissions. Never merge them.
  • Tasks go in private subnets without public IPs, behind an ALB with an ip target group; reach AWS services through NAT or (better) VPC endpoints.
  • Safe deployments: minimumHealthyPercent 100, readiness health checks, a sensible grace period, a short deregistration delay and the circuit breaker with rollback.
  • An idle ALB and NAT gateway cost more than the tasks. Deploy, test and destroy within the hour.

Further reading

Independent learning resource — not affiliated with, endorsed or sponsored by Amazon Web Services. AWS and related marks are trademarks of Amazon.com, Inc. or its affiliates. All banking data in examples is fictional. Sample architectures are for learning and are not automatically compliant with GDPR, PCI DSS or any banking regulation.

On this page