Simplifying Amazon EC2 Management with AWS Systems Manager

Overview

Amazon EC2 Systems Manager is a management service offered by Amazon Web Services (AWS) that simplifies resource and application management in Amazon EC2 instances.

It provides a unified user interface to view and control virtual machines, enabling efficient resource tracking, software inventory management, and system configuration automation. Amazon EC2 Systems Manager offers features such as centralized log collection, patch management, and automated task execution, which enhance security and reduce the operational overhead of managing Amazon EC2 instances.

This service streamlines the management process and allows users to maintain and operate their infrastructure deployed on Amazon EC2 easily.

Introduction

Remedial steps can be taken using Amazon EC2 Systems Manager Automation in reaction to events that might affect your AWS resources. To demonstrate this idea, this post walks you through setting up automated remedial activities when an Amazon EC2 instance backed by Amazon EBS is about to retire.

When AWS determines that the underlying hardware supporting an instance has irreversibly failed, the instance is scheduled for retirement. You can stop and restart the instance before retirement if the instance root device is an Amazon EBS volume.

AWS-hosted Amazon EC2 Systems Manager (SSM) Automation makes routine instance and system management and deployment chores easier and comes with no additional cost.

You can always see how your AWS accounts, services, and resources are doing using AWS Health. The service informs you about resource performance or availability issues impacting your AWS-powered apps and provides repair recommendations.

Due to the integration of both services with Amazon CloudWatch Events, SSM Automation documents can be triggered by AWS Health events.

SSM Automation also provides an Approval action that stops an Automation execution temporarily until your approved principals (i.e., AWS IAM user) approve or deny the action. For more information on SSM automated actions, please see Systems Manager Automation actions.

flow2

We will go over the 4 steps to set up Stop and Start Amazon EC2 instances with SSM Automation to respond to Amazon EC2 retirement events from AWS Health.

To run the solution in the US East-1 region via AWS CloudFormation, click here. Please update the region as needed.

We encourage you to review the manual steps below before deploying the AWS CloudFormation stack to understand the solution better.

Pioneers in Cloud Consulting & Migration Services

Reduced infrastructural costs
Accelerated application deployment

Get Started

Step-by-Step Guide

Step 1: Establish the necessary AWS IAM role

The first step is setting up the AWS IAM permissions for the Amazon CloudWatch Events. To do this, you’ll need to create an AWS IAM policy. You must also create an associated AWS IAM role for Amazon CloudWatch. For the sake of simplicity, we’ll refer to the AWS IAM role as “AutomationCWRole”. Here’s a sample AWS IAM policy you can use:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "events:PutEvents",
                "events:PutRule",
                "events:PutTargets",
                "events:DescribeRule",
                "logs:CreateLogGroup",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        }
    ]
}

{

"Version": "2012-10-17",

"Statement": [

{

"Effect": "Allow",

"Action": [

"events:PutEvents",

"events:PutRule",

"events:PutTargets",

"events:DescribeRule",

"logs:CreateLogGroup",

"logs:CreateLogStream",

"logs:PutLogEvents"

"Resource": "*"

}

]

}

Then, Creating the AWS IAM Role:-

aws iam create-role --role-name AutomationCWRole --assume-role-policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"events.amazonaws.com"},"Action":"sts:AssumeRole"}]}'

1	aws iam create-role --role-name AutomationCWRole --assume-role-policy-document '{"Version":"2012-10-17","Statement":[{"Effect":"Allow","Principal":{"Service":"events.amazonaws.com"},"Action":"sts:AssumeRole"}]}'

Attaching the AWS IAM Policy to Role:-

aws iam put-role-policy --role-name AutomationCWRole --policy-name CloudWatchEventsPolicy --policy-document file://path/to/your/policy.json

1	aws iam put-role-policy --role-name AutomationCWRole --policy-name CloudWatchEventsPolicy --policy-document file://path/to/your/policy.json

Please ensure the role name and account ID are updated in the role ARN. It is necessary to confirm that events.amazonaws.com and ssm.amazonaws.com are set up as trusted entities for the AWS IAM role, as indicated by the following:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "ssm.amazonaws.com",
          "events.amazonaws.com"
        ]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

{

"Version": "2012-10-17",

"Statement": [

{

"Effect": "Allow",

"Principal": {

"Service": [

"ssm.amazonaws.com",

"events.amazonaws.com"

]

"Action": "sts:AssumeRole"

}

]

}

Step 2: Establish the Amazon SNS Topic if not already created

To post the approval notification for Automation Approval operations, we must either create a new Amazon SNS topic or use an existing one. Additionally, the approvers must be subscribed to that Amazon SNS topic.

For this example, we’ll utilize the Amazon SNS topic name AutomationStopStart. Note that the prefix: Automation must appear at the beginning of the SNS Topic name.

Step 3: Configure the Amazon CloudWatch Events rule using the Automation document

To begin, we must generate an SSM Automation document (in JSON format) using the designated editor, named “StopStartEC2InstancewithApproval.json”:

{
   "description":"Stop and Start EC2 instances(s) with Approval",
   "schemaVersion":"0.3",
   "assumeRole":"{{ AutomationAssumedRole }}",
   "parameters":{
      "AutomationAssumeRole":{
         "type":"String",
         "description":"The ARN of the role that allows Automation to perform the actions on your behalf.",
         "default":"arn:aws:iam::{{global:ACCOUNT_ID}}:role/AutomationServiceRole"
      },
      "InstanceIds":{
         "type":"String",
         "description":"EC2 Instance(s) to Stop and Start"
      },
      "Approvers":{
         "type":"StringList",
         "description":"IAM user or user arn of approvers for the automation action"
      },
      "SNSTopicArn":{
         "type":"String",
         "description":"The SNS topic ARN that you are using to get notifications on about EC2 retirement notifications. The SNS topic name must start with Automation."
      }
   },
   "mainSteps":[
      {
         "name":"approve",
         "action":"aws:approve",
         "timeoutSeconds":999999,
         "onFailure":"Abort",
         "inputs":{
            "NotificationArn":"{{ SNSTopicArn }}", 
            "Message": "Your approval is required to proceed with the stop and start of an EC2 instance using the EC2 systems manager automation document that is scheduled for retirement.",
            "MinRequiredApprovals":1,
            "Approvers":[
               "{{Approvers}}"
            ]
         }
      },
      {
         "name":"stopInstance",
         "action":"aws:changeInstanceState",
         "maxAttempts":2,
         "timeoutSeconds":120,
         "onFailure":"Continue",
         "inputs":{
            "InstanceIds":[
               "{{ InstanceIds }}"
            ],
            "DesiredState":"stopped"
         }
      },
      {
         "name":"forceStopInstance",
         "action":"aws:changeInstanceState",
         "maxAttempts":1,
         "timeoutSeconds":60,
         "onFailure":"Continue",
         "inputs":{
            "InstanceIds":[
               "{{ InstanceIds }}"
            ],
            "Force":true,
            "DesiredState":"stopped"
         }
      },
      {
         "name":"startInstance",
         "action":"aws:changeInstanceState",
         "maxAttempts":3,
         "timeoutSeconds":120,
         "onFailure":"Continue",
         "inputs":{
            "InstanceIds":[
               "{{ InstanceIds }}"
            ],
            "DesiredState":"running"
         }
      }
   ]
}

{

"description":"Stop and Start EC2 instances(s) with Approval",

"schemaVersion":"0.3",

"assumeRole":"{{ AutomationAssumedRole }}",

"parameters":{

"AutomationAssumeRole":{

"type":"String",

"description":"The ARN of the role that allows Automation to perform the actions on your behalf.",

"default":"arn:aws:iam::{{global:ACCOUNT_ID}}:role/AutomationServiceRole"

"InstanceIds":{

"type":"String",

"description":"EC2 Instance(s) to Stop and Start"

"Approvers":{

"type":"StringList",

"description":"IAM user or user arn of approvers for the automation action"

"SNSTopicArn":{

"type":"String",

"description":"The SNS topic ARN that you are using to get notifications on about EC2 retirement notifications. The SNS topic name must start with Automation."

}

"mainSteps":[

{

"name":"approve",

"action":"aws:approve",

"timeoutSeconds":999999,

"onFailure":"Abort",

"inputs":{

"NotificationArn":"{{ SNSTopicArn }}",

"Message": "Your approval is required to proceed with the stop and start of an EC2 instance using the EC2 systems manager automation document that is scheduled for retirement.",

"MinRequiredApprovals":1,

"Approvers":[

"{{Approvers}}"

]

}

{

"name":"stopInstance",

"action":"aws:changeInstanceState",

"maxAttempts":2,

"timeoutSeconds":120,

"onFailure":"Continue",

"inputs":{

"InstanceIds":[

"{{ InstanceIds }}"

"DesiredState":"stopped"

}

{

"name":"forceStopInstance",

"action":"aws:changeInstanceState",

"maxAttempts":1,

"timeoutSeconds":60,

"onFailure":"Continue",

"inputs":{

"InstanceIds":[

"{{ InstanceIds }}"

"Force":true,

"DesiredState":"stopped"

}

{

"name":"startInstance",

"action":"aws:changeInstanceState",

"maxAttempts":3,

"timeoutSeconds":120,

"onFailure":"Continue",

"inputs":{

"InstanceIds":[

"{{ InstanceIds }}"

"DesiredState":"running"

}

]

}

Next, we’ll utilize the file mentioned above to generate an SSM Automation document using the provided JSON file:

[ aws ssm create-document --content file://StopStartEC2InstancewithApproval.json --name " StopStartEC2InstancewithApproval" --document-type "Automation" ]

1	[ aws ssm create-document --content file://StopStartEC2InstancewithApproval.json --name " StopStartEC2InstancewithApproval" --document-type "Automation" ]

Step 4: Validate and authorize the Automation by conducting a test

Perform a test against the document using the provided inputs:

Execute the following command:

aws ssm start-automation-execution --document-name AmazonEC2InstanceStopStartwithApproval --parameters AutomationAssumedRole="aws:iam::<AccountId>:role/AutomationCWRole",Approvers=<IAMusername>,SNSTopicArn="arn:aws:sns:us-east-1:<AccountId>:AutomationStopStart",InstanceIds=<InstanceId>

1	aws ssm start-automation-execution --document-name AmazonEC2InstanceStopStartwithApproval --parameters AutomationAssumedRole="aws:iam::<AccountId>:role/AutomationCWRole",Approvers=<IAMusername>,SNSTopicArn="arn:aws:sns:us-east-1:<AccountId>:AutomationStopStart",InstanceIds=<InstanceId>

To check the execution status, retrieve the AutomationExecutionId from the preceding command’s output and use it in the following command:

aws ssm get-automation-execution --automation-execution-id <value>

1	aws ssm get-automation-execution --automation-execution-id <value>

Once the approval is published to the subscribers of the SNS topic, you can decide to approve or reject the action.

Send the approval signal using the following command:

aws ssm send-automation-signal --automation-execution-id <automation-execution-id> --signal-type Approve --payload Comment=Replace_This_With_Approve_Comment

1	aws ssm send-automation-signal --automation-execution-id <automation-execution-id> --signal-type Approve --payload Comment=Replace_This_With_Approve_Comment

To approve the automation, we can also investigate the Amazon EC2 console in the Automation section:

flow

Conclusion

Utilizing Amazon EC2 Systems Manager Automation to respond to potentially impactful events by performing remediation actions on your AWS resources. We can use this example to apply to other scheduled changes in Amazon EC2 (like maintenance for system reboots) or any event involving any AWS resource that might be useful to us. The provided document can also stop and start Amazon EC2 instances automatically. It is advised to modify and test it for our use case before implementing it in a live setting.

Drop a query if you have any questions regarding Amazon EC2 Systems Manager Automation and we will get back to you quickly.

Making IT Networks Enterprise-ready – Cloud Management Services

Accelerated cloud migration
End-to-end view of the cloud environment

Get Started

About CloudThat

CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.

CloudThat is recognized as a top-tier partner with AWS and Microsoft, including the prestigious ‘Think Big’ partner award from AWS and the Microsoft Superstars FY 2023 award in Asia & India. Having trained 650k+ professionals in 500+ cloud certifications and completed 300+ consulting projects globally, CloudThat is an official AWS Advanced Consulting Partner, AWS Training Partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, Amazon QuickSight Service Delivery Partner, Amazon EKS Service Delivery Partner, Microsoft Gold Partner, AWS Microsoft Workload Partners, Amazon EC2 Service Delivery Partner, and many more.

To get started, go through our Consultancy page and Managed Services Package, CloudThat’s offerings.

FAQs

1. What is Amazon EC2 Systems Manager Automation?

ANS: – Amazon EC2 Systems Manager Automation is a service provided by AWS that allows you to automate routine maintenance and operations on Amazon EC2 instances. It provides a shortcut to performance tasks such as patching, backup, and scaling.

2. What types of Amazon EC2 reports can this solution handle?

ANS: – You can handle Amazon EC2 notifications, including instance failures, performance degradation, scheduled maintenance issues, and other health issues identified by AWS Health.

WRITTEN BY Guru Bhajan Singh

Guru Bhajan Singh is currently working as a Software Engineer - PHP at CloudThat and has 6+ years of experience in PHP. He holds a Master's degree in Computer Applications and enjoys coding, problem-solving, learning new things, and writing technical blogs.