Amazon ECS (Elastic Container Service)

Article.

AWS’s managed container orchestrator. Simpler than EKS (Kubernetes), more proprietary. Two launch types:

• EC2 launch type — you manage EC2 nodes; ECS schedules tasks onto them. More config, lower cost per CPU/RAM.
• Fargate launch type — serverless containers; AWS runs the nodes. Pay per task vCPU/RAM. Less ops.

For Fargate specifics see AWS Fargate.

“Your ECS deployment is stuck half rolled out. What do you check?” is the question this note prepares for. ECS cannot be run locally, so the claims are checked against AWS documentation as of October 2026; the container-level facts were checked with Docker. Two recent changes matter: App Runner stopped accepting new customers on 2026-04-30, and AWS points to ECS Express Mode (one API call creates a Fargate service, an ALB, scaling and networking) as its replacement.

Core objects

Task definition

{
  "family": "orders",
  "networkMode": "awsvpc",
  "requiresCompatibilities": ["FARGATE"],
  "cpu": "512",
  "memory": "1024",
  "executionRoleArn": "arn:aws:iam::...:role/ecsTaskExecution",
  "taskRoleArn":      "arn:aws:iam::...:role/ordersTaskRole",
  "containerDefinitions": [{
    "name": "app",
    "image": "1234.dkr.ecr.us-east-1.amazonaws.com/orders:1.0.0",
    "portMappings": [{"containerPort": 8000}],
    "environment": [{"name": "ENV", "value": "prod"}],
    "secrets": [
      {"name": "DB_PASSWORD",
       "valueFrom": "arn:aws:secretsmanager:...:secret:db-password"}
    ],
    "healthCheck": {
      "command": ["CMD", "python", "-c",
        "import urllib.request as u; u.urlopen('http://localhost:8000/health')"],
      "interval": 30, "timeout": 5, "retries": 3, "startPeriod": 30
    },
    "logConfiguration": {
      "logDriver": "awslogs",
      "options": {
        "awslogs-group": "/ecs/orders",
        "awslogs-region": "us-east-1",
        "awslogs-stream-prefix": "orders"
      }
    }
  }]
}

executionRoleArn vs taskRoleArn

• Execution role — used by ECS itself to pull the image (ECR auth), fetch secrets, write logs to CloudWatch. Boring, granted once.
• Task role — used by your application code inside the container (boto3 calls AWS APIs as this role). This is your app’s IAM identity. Equivalent to IRSA on EKS.

# Inside the container — boto3 picks up the task role automatically
import boto3
s3 = boto3.client("s3")
# uses task role
s3.put_object(Bucket="...", Key="...", Body=...)

No access keys in env, no instance profile lookups, no static creds. This is the right pattern.

The credentials come from the container credentials endpoint (169.254.170.2, via AWS_CONTAINER_CREDENTIALS_RELATIVE_URI), which boto3 checks before the instance metadata service. On the EC2 launch type a task can still reach the instance’s own role through instance metadata unless it is blocked (ECS_AWSVPC_BLOCK_IMDS for awsvpc tasks, an IMDSv2 hop limit of 1 for bridge mode); otherwise every task quietly gets the node’s permissions too.

Service

{
  "serviceName": "orders",
  "cluster": "prod",
  "taskDefinition": "orders:5",
  "desiredCount": 3,
  "launchType": "FARGATE",
  "networkConfiguration": {
    "awsvpcConfiguration": {
      "subnets": ["subnet-a", "subnet-b"],
      "securityGroups": ["sg-orders"],
      "assignPublicIp": "DISABLED"
    }
  },
  "loadBalancers": [{
    "targetGroupArn": "arn:aws:elasticloadbalancing:...:targetgroup/orders/...",
    "containerName": "app",
    "containerPort": 8000
  }],
  "deploymentConfiguration": {
    "maximumPercent": 200,
    "minimumHealthyPercent": 100,
    "deploymentCircuitBreaker": {"enable": true, "rollback": true}
  }
}

Service keeps desiredCount tasks running; rolling updates respect maximumPercent / minimumHealthyPercent. Circuit breaker auto-rolls-back failed deploys. Use it. It trips after a number of failed task launches derived from the desired count (half of it, at least 3, at most 200), and healthCheckGracePeriodSeconds must cover the app’s start-up, or slow starters are killed by the load balancer’s health check before they are ready.

Service Discovery

Two patterns:

Service Connect

The current default for service-to-service calls inside ECS: an Envoy proxy beside each task, with retries, outlier detection and per-service metrics, addressed by a short name. Cloud Map DNS (below) is simpler but clients cache DNS answers, so during a deploy they keep calling task IPs that are gone.

Via ALB

Service registers tasks as targets in a Target Group. Clients hit the ALB; ALB routes to a healthy task.

client → ALB → target group → task IP

AWS Cloud Map (private DNS)

Service creates a DNS record orders.prod.local that resolves to task IPs. For pure service-to-service east-west without ALB hop.

# In another service in the same cluster
import httpx
r = httpx.get("http://orders.prod.local/v1/orders")

Autoscaling

Service autoscaling targets:

• CPU / memory utilization.
• ALB request count per target.
• SQS queue depth (custom metric).
• Any CloudWatch metric via target tracking.

aws application-autoscaling register-scalable-target \
  --service-namespace ecs \
  --resource-id service/prod/orders \
  --scalable-dimension ecs:service:DesiredCount \
  --min-capacity 3 --max-capacity 50

aws application-autoscaling put-scaling-policy \
  --service-namespace ecs \
  --resource-id service/prod/orders \
  --scalable-dimension ecs:service:DesiredCount \
  --policy-name cpu70 \
  --policy-type TargetTrackingScaling \
  --target-tracking-scaling-policy-configuration \
    'TargetValue=70,PredefinedMetricSpecification={
       PredefinedMetricType=ECSServiceAverageCPUUtilization}'

Celery worker autoscaling on SQS depth

The classic “scale workers on backlog”. Target tracking on the raw queue depth is the common mistake: it scales until the whole queue holds 50 messages, regardless of how many tasks are running. The metric to track is backlog per task: queue depth divided by running tasks, which target tracking accepts as metric math:

{"TargetValue": 50,
 "CustomizedMetricSpecification": {"Metrics": [
   {"Id": "depth", "ReturnData": false, "MetricStat": {"Stat": "Sum",
     "Metric": {"Namespace": "AWS/SQS",
                "MetricName": "ApproximateNumberOfMessagesVisible",
                "Dimensions": [{"Name": "QueueName", "Value": "orders-tasks"}]}}},
   {"Id": "tasks", "ReturnData": false, "MetricStat": {"Stat": "Average",
     "Metric": {"Namespace": "ECS/ContainerInsights", "MetricName": "RunningTaskCount",
                "Dimensions": [{"Name": "ClusterName", "Value": "prod"},
                               {"Name": "ServiceName", "Value": "workers"}]}}},
   {"Id": "backlog_per_task", "Expression": "depth / tasks", "ReturnData": true}]}}

Choose the target from latency: if a task handles 10 messages a second and a message may wait 5 seconds, 50 per task is the right target.

Scaling to zero

A service can run with desiredCount: 0 and a minimum capacity of 0. Target tracking on CPU cannot wake it (no task, no CPU metric), so scale from zero with a step-scaling policy on an alarm on queue depth; the first message then waits for a task to start.

Networking modes

Use awsvpc. The others are EC2-only and constrain you.

Logging

awslogs driver writes container stdout/stderr to a CloudWatch Logs group. Each task gets a stream. Pair with CloudWatch Logs Insights for querying.

Since 2025-06-25 the default mode is non-blocking. Before, blocking was the default: when CloudWatch Logs throttled, the app’s writes to stdout blocked and requests stalled. Non-blocking keeps the app running and buffers logs in memory (max-buffer-size, 10m by default); when the buffer fills, logs are dropped silently. Choose per service: non-blocking where availability matters more than every log line, blocking ("mode": "blocking") for audit logs that must not be lost.

For structured / centralized logging, FireLens routes logs through a Fluent Bit sidecar to Firehose, S3 or OpenSearch. The app container uses the awsfirelens driver, and the task needs the log-router container itself:

{"name": "log_router", "essential": true,
 "image": "public.ecr.aws/aws-observability/aws-for-fluent-bit:stable",
 "firelensConfiguration": {"type": "fluentbit"}},
{"name": "app", "logConfiguration": {"logDriver": "awsfirelens",
   "options": {"Name": "firehose", "region": "us-east-1", "delivery_stream": "logs"}}}

Deployments

Rolling (default)

Service launches new tasks, drains old; respects minimumHealthyPercent. Use the deployment circuit breaker to auto-rollback on failed rollouts.

Blue/green, linear and canary (built in)

Since July 2025 ECS does blue/green itself, without CodeDeploy: it starts the new revision beside the old one, shifts test traffic, then production traffic (all at once, or linearly or as a canary since late 2025), waits a bake time, and rolls back automatically on CloudWatch alarms. Lifecycle hooks are Lambda functions run at each stage, for automated tests or a manual approval. It works behind an ALB, an NLB or Service Connect. CodeDeploy-based blue/green (deploymentController: CODE_DEPLOY) still exists, mainly for setups built before.

Common gotchas

• awsvpc and ENI limits per EC2 instance. Each task = an ENI; instance types have ENI caps. Mostly a Fargate-irrelevant issue.
• Fargate ephemeral storage default is 20 GB. Raise it for tasks that download large files.
• Cold start of Fargate tasks ~30-60s. Every task pulls its image; there is no image cache between tasks. Smaller images and SOCI lazy loading (the image starts before it is fully pulled) shorten it; scaling from 3 to 30 still takes about a minute.
• CloudWatch Logs costs at scale. Ingestion is the killer; log retention can be cheap. Use awsfirelens to ship to cheaper storage if volume is high.
• Stopping vs draining tasks. For graceful shutdown your container must handle SIGTERM (ECS sends it, then SIGKILL after stopTimeout: 30 s by default, 120 s at most on Fargate). A Python process running as PID 1 ignores SIGTERM unless it installs a handler, and is killed at the timeout (Docker image vs container).

Where it breaks in production

A deployment stuck half rolled out

New tasks start, fail their health check, are replaced, and fail again. The usual causes: a container health check that cannot run in the image (curl in python:3.14-slim, which ships neither curl nor wget; checked with Docker); a grace period shorter than the app’s start-up; the app listening on 127.0.0.1 instead of 0.0.0.0; a missing secret or IAM permission at start-up. aws ecs describe-services shows the deployment events, and the stopped tasks’ stoppedReason names the cause. With the circuit breaker on, the deploy rolls back instead of looping forever.

Logs missing exactly when they matter

Under a traffic spike the app logs heavily, the awslogs non-blocking buffer fills, and the lines explaining the incident are dropped. Size the buffer for the peak, sample debug logs, or use blocking mode where loss is unacceptable.

Workers that never scale

Queue workers scaled by CPU target tracking: the workers wait on I/O, CPU stays low, the backlog grows to millions, and nothing scales. Scale workers on backlog per task.

The trade not taken

Kubernetes (EKS) is the alternative: portable, a vast ecosystem, and the skills transfer between clouds, at the cost of a control plane, cluster upgrades and far more configuration. ECS gives up portability for an AWS-native model with fewer moving parts; for a team running only on AWS without Kubernetes expertise, that is usually the better trade.

ECS vs EKS

For a team committed to AWS without K8s expertise, ECS wins on simplicity. For teams with K8s skill or multi-cloud requirements, EKS.