Using AWS Systems Manager to Monitor Disk Usage on Amazon EC2 Instances

Introduction

AWS Session Manager is a capability of AWS System Manager, which does not require opening inbound ports and doesn’t require public IP addresses for Amazon EC2 instances. It also removes the need for a bastion host. To get the disk utilization without needing to RDP/SSH for each instance, we can use System Manager by following the steps below.

Pioneers in Cloud Consulting & Migration Services

Reduced infrastructural costs
Accelerated application deployment

Get Started

Pre-requisites

AWS Account: Ensure you can access an AWS account with appropriate permissions.
AWS IAM Role: Create an AWS IAM role with the AmazonSSMManagedInstanceCore policy attached. This role should be assigned to the Amazon EC2 instances.
SSM Agent: Ensure that the SSM Agent is installed and running on your Amazon EC2 instances. By default, SSM Agent is pre-installed on Amazon Linux, Amazon Linux 2, and Ubuntu 16.04, 18.04, and 20.04 instances.

Steps to Get Disk Utilization for an Amazon EC2 Instance Using AWS Systems Manager Agent

Go to the “Run Command” under Node Management in Session Manager. Select “AWS-RunShellScript” as shown below. The ‘aws:runShellScript’ command in AWS Systems Manager runs shell scripts on managed instances.

step1

Put the commands in the ‘Command Parameters’. In this case, “df -h” command in Linux displays the disk space usage in a human-readable format.

step1b

Select the Target Instances for which we need to know the disk utilization. We can select multiple instances simultaneously.

step1c

Click on ‘Run Command’. We can view the output once the Command status is successful.

step1d

Components in the Process

Amazon EC2 Instances: Virtual servers running in the AWS cloud.
AWS IAM Role: Provides necessary permissions for the SSM Agent to interact with the AWS Systems Manager.
SSM Agent: An agent that runs on Amazon EC2 instances and enables AWS Systems Manager to manage these instances.
AWS Systems Manager: A service that provides operational data and automation for your infrastructure.
Run Command: A feature of AWS Systems Manager that allows you to remotely and securely manage the configuration of your managed instances.

step2

Benefits and Limitations of Using this Approach

Benefits:

No open inbound ports and no need to manage bastion hosts or SSH keys- Session Manager helps you improve your security posture by letting you close these inbound ports, freeing you from managing SSH keys and certificates, bastion hosts, and jump boxes.
Scalability: Efficiently run commands across a fleet of instances.
Enhances Security: Ensures only authenticated users can access specific resources. Also, it manages session timeouts and invalidation to prevent unauthorized access.
Logging and Auditing: Integrated with services like AWS CloudTrail, Amazon Simple Storage Service, Amazon CloudWatch Logs, and Amazon Simple Notification Service for recording connections made to the managed nodes and also receive notifications when a user in the organization starts or ends session activity.

Limitations:

Initial Setup Complexity: Requires proper AWS IAM role configuration and ensures SSM Agent is installed and running in the Amazon EC2 Instances. This may lead to security or access control issues if access is not provided properly.
Execution Delays: Command execution might take time depending on the number of instances and network latency.

Alternate Way of Getting the Disk Utilization

Alternate Option 1: Amazon CloudWatch Agent and Amazon CloudWatch Logs

Install CloudWatch Agent on Amazon EC2 instances. For Amazon Linux:

cd /tmp
wget https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm

1 2	cd /tmp wget https://s3.amazonaws.com/amazoncloudwatch-agent/amazon_linux/amd64/latest/amazon-cloudwatch-agent.rpm

Create and edit agent’s config file:

sudo vi /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json

1	sudo vi /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json

{
  "agent": {
    "metrics_collection_interval": 60,
    "run_as_user": "cwagent"
  },
  "metrics": {
    "append_dimensions": {
        "InstanceId": "${aws:InstanceId}"
    },
    "metrics_collected": {
      "disk": {
        "measurement": [
          "used_percent"
        ],
        "metrics_collection_interval": 60,
        "resources": [
          "/"
        ]
      }
    }
  }
}

{

"agent": {

"metrics_collection_interval": 60,

"run_as_user": "cwagent"

"metrics": {

"append_dimensions": {

"InstanceId": "${aws:InstanceId}"

"metrics_collected": {

"disk": {

"measurement": [

"used_percent"

"metrics_collection_interval": 60,

"resources": [

"/"

]

}

Restart the agent and check the agent’s logs:

sudo systemctl restart amazon-cloudwatch-agent

1	sudo systemctl restart amazon-cloudwatch-agent

Go to the Amazon CloudWatch Console and check the Metrics Section. CWAgent installed on the instance is visible on the screen.

step4

Click the instance, and the disk utilization of the EBS volume will be visible on the screen.

step4b

Alternate Option 2: AWS Session Manager

Go to the AWS Session Manager, and click ‘Start Session’.

step5

The instance in the target instances will only be visible if the role has been attached to the instance.

step5b

Click on ‘Start Session’. Here, the user initiates an SSH session through Session Manager and sends API calls to the Amazon EC2 instance.

step5c

The AWS Systems Manager SSM Agent, installed on the Amazon EC2 instance, connects to Session Manager and runs the command ‘df -h’.

‘df’ command displays the information about total and available space on a file system.

step5d

Conclusion

Using AWS Systems Manager Session Manager to monitor disk utilization on Amazon EC2 instances offers a secure, scalable, and efficient solution. You enhance your security posture while simplifying the management process by eliminating the need for inbound ports, bastion hosts, and SSH/RDP access.

The ability to run commands across multiple instances simultaneously further streamlines your operations, making it easier to maintain optimal performance and resource allocation.

Drop a query if you have any questions regarding AWS Systems Manager and we will get back to you quickly.

Making IT Networks Enterprise-ready – Cloud Management Services

Accelerated cloud migration
End-to-end view of the cloud environment

Get Started

About CloudThat

CloudThat is an award-winning company and the first in India to offer cloud training and consulting services worldwide. As a Microsoft Solutions Partner, AWS Advanced Tier Training Partner, and Google Cloud Platform Partner, CloudThat has empowered over 850,000 professionals through 600+ cloud certifications winning global recognition for its training excellence including 20 MCT Trainers in Microsoft’s Global Top 100 and an impressive 12 awards in the last 8 years. CloudThat specializes in Cloud Migration, Data Platforms, DevOps, IoT, and cutting-edge technologies like Gen AI & AI/ML. It has delivered over 500 consulting projects for 250+ organizations in 30+ countries as it continues to empower professionals and enterprises to thrive in the digital-first world.

FAQs

1. How do I install the SSM Agent on my Amazon EC2 instances?

ANS: – The SSM Agent is pre-installed on Amazon Linux, Amazon Linux 2, and Ubuntu 16.04, 18.04, and 20.04 instances. For other instances, you can install it manually by following the instructions in the SSM Agent documentation.

2. Can I monitor disk utilization on multiple instances simultaneously?

ANS: – Yes, you can select multiple target instances when running the command in AWS Systems Manager, allowing you to monitor disk utilization across your fleet of instances efficiently.

WRITTEN BY Aniket Kumar Ambasta

Aniket Ambasta works as a Subject Matter Expert at CloudThat. He is an AWS Solutions Architect – Associate certified professional, skilled in cloud migration, infrastructure, security, and cloud FinOps. Apart from his professional interests, he loves exploring and learning new technologies.