AWS Elastic Container Service (ECS) Workflow Steps

Available in PagerDuty Runbook Automation Commercial products.

Amazon ECS is a fully managed container orchestration service that makes it easy for you to deploy, manage, and scale containerized applications.

The following plugins are available for PagerDuty Runbook Automation:

AWS Elastic Container Service (ECS) Workflow Steps

Authentication

Follow the instructions outlined in the AWS Plugins Overview for Runbook Automation to authenticate with AWS.

When defining the IAM Role for Runbook Automation, be sure to include the permissions required by the specific plugins in the Policy associated with the role:

Run Command:
- ssmmessages:CreateControlChannel
- ssmmessages:CreateDataChannel
- ssmmessages:OpenControlChannel
- ssmmessages:OpenDataChannel
Stopped Task Details:
- ecs:ListTasks
- ecs:DescribeTasks
Stop ECS Task:
- ecs:StopTask

Send Commands to ECS Containers

The AWS / ECS / Run Command plugin sends commands to specified ECS containers:

Sending commands to ECS containers makes it easier to collect diagnostic information and quickly troubleshoot errors. For example, in development,
this allows you to easily interact with various processes in your containers and troubleshoot your applications. In production scenarios,
this enables you to gain break-glass access to your containers to debug issues.

Prerequisites

This feature requires a Task IAM role to grant containers the permissions needed for communication between the managed SSM agent (execute-command agent) and the SSM service.
Use the following policy and attach it to your task definition as explained here.

{
   "Version": "2012-10-17",
   "Statement": [
       {
       "Effect": "Allow",
       "Action": [
            "ssmmessages:CreateControlChannel",
            "ssmmessages:CreateDataChannel",
            "ssmmessages:OpenControlChannel",
            "ssmmessages:OpenDataChannel"
       ],
      "Resource": "*"
      }
   ]
}

Enable remote commands by adding the --enable-execute-command flag to one of the following AWS CLI Commands: create-service, update-service, start-task, or run-task.
For example, with an existing Service:
```
 aws ecs create-service \
  --cluster cluster-name \
  --task-definition task-definition-name \
  --enable-execute-command \
  --service-name service-name \
  --desired-count 1
```

[Optional] If you set the task definition parameter initProcessEnabled to true, this starts the init process inside the container,
which removes any zombie SSM agent child processes found. The following provides an example:

{
    "taskRoleArn": "ecsTaskRole",
    "networkMode": "awsvpc",
    "requiresCompatibilities": [
        "EC2",
        "FARGATE"
    ],
    "executionRoleArn": "ecsTaskExecutionRole",
    "memory": ".5 gb",
    "cpu": ".25 vcpu",
    "containerDefinitions": [
        {
            "name": "amazon-linux",
            "image": "amazonlinux:latest",
            "essential": true,
            "command": ["sleep","3600"],
            "linuxParameters": {
                "initProcessEnabled": true
            }
        }
    ],
    "family": "ecs-exec-task"
}

Configuration

The following fields are used for configuration the Workflow Step plugin to send commands to ECS containers:

Container Name: The name of the ECS container to run the command on.
Cluster Name: The name of the cluster where the container resides.
Task ID: The ID the task associated with the specified container.
Command: The shell command you wish to run on the specified container.
Container Region: The region where the container's cluster is located.
Access Key ID: AWS Access Key. This can be set in the Project configuration properties with project.aws.access_key or in Configuration Management.
Secret Key: AWS Secret Key. Click the Select button to choose your AWS Secret from Key Storage. This can be set in the Project configuration properties with project.aws.secret_key_path.

Executing Job Step

Once the Job step is configured, commands can be sent to the specified container. Start by hard-coding a specific command into the Job step as shown here:

If this executes successfully, you would see the following output in the Job output logs:

To use this Job step as part of a larger workflow, or to create a "self service interface", you will likely want to use a Job Option (shown below) or a Data Variable
as the input for the command. See here for more details on Job Options and Data Variables:

Stopped ECS Tasks Error Messages

The AWS / ECS / Stopped Task Details plugin checks a specified cluster for any Stopped ECS Tasks and provides their associated error messages:

In order to use this plugin, you mused specify the ECS Cluster Name.

The IAM Policies required to use this plugin are:

ecs:ListTasks
ecs:DescribeTasks

See here for more details on the Stopped ECS Tasks error message meanings.

Stop an ECS Task

The AWS / ECS / Stop plugin stops a specific ECS Task. This can assist with auto-remediation of issues with ECS containers. There is both a Workflow and a Node step of this plugin.

In order to use the Workflow step plugin, you will need to specify the Cluster-ID (or ARN) and the Task-ID (or ARN):

The IAM Policies required to use this plugin are:

ecs:StopTask

Assume Role ARN

There are cases when it is beneficial to use a different IAM Role than the one used for the AWS PluginGroup for the integration to retrieve the ECS Tasks and add them to the node inventory.
For example, to target ECS Tasks from across multiple AWS Accounts, then a different IAM Role needs to be "assumed" in order to retrieve tasks from each AWS Account.

The Assume Role ARN field can be populated with an IAM Role ARN that Runbook Automation can assume after an authentication method has been provided through the PluginGroup.

By default, this plugin will use the assume-role-arn node-attribute from the ECS tasks added to the inventory through the ECS Node Source.
This can be overridden by placing an IAM role ARN into the field.