Login
August 30, 2024|
Voiced by Amazon Polly |
Introduction
In dynamic cloud environments, it’s crucial to have efficient and automated processes to manage the lifecycle of your Amazon EC2 instances, especially when leveraging warm pools for cost efficiency.
In this blog post, we’ll walk you through a comprehensive approach to automating code deployment to Amazon EC2 instances within a warm pool using AWS CodeDeploy. We’ll use AWS Lambda to manage the lifecycle of these instances, ensuring that they are in the correct state during deployment and return to a hibernated state afterward. This automation simplifies the deployment process and ensures that your applications are always up-to-date with minimal manual intervention.
Pioneers in Cloud Consulting & Migration Services
- Reduced infrastructural costs
- Accelerated application deployment
Objectives
- Create resources using the AWS CloudFormation template.
- Set up AWS Lambda function that:
- Identifies Amazon EC2 instances based on their tags.
- Manages the lifecycle of instances in a warm pool.
- Triggers deployments using AWS CodeDeploy.
- Ensure instances run during deployment and return to a stopped state after deployment.
Prerequisites
- Basic knowledge of AWS services such as Amazon EC2, AWS Lambda, AWS CloudFormation, and AWS CodeDeploy.
- AWS CLI and necessary permissions to create and manage AWS resources.
Step-by-Step Guide
Step 1: AWS CloudFormation Template
We will start by creating the necessary resources using AWS CloudFormation template. This template sets up:
- An Amazon EventBridge rule to trigger the Lambda function when an Amazon EC2 instance changes state.
- Necessary AWS IAM roles and permissions.
- The AWS Lambda function itself.
AWS CloudFormation Template
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 |
{ "AWSTemplateFormatVersion": "2010-09-09", "Description": "This CloudFormation template creates resources required for syncing new instances with the latest deployment from AWS CodeDeploy based on the CodeDeployAppname, DeploymentGroup tags on the instance", "Resources": { "EventRule": { "Type": "AWS::Events::Rule", "Properties": { "Description": "EventRule", "EventPattern": { "source": [ "aws.ec2" ], "detail-type": [ "EC2 Instance State-change Notification" ], "detail": { "state": [ "running" ] } }, "State": "ENABLED", "Targets": [ { "Arn": { "Fn::GetAtt": [ "CodeDeploySyncNewInstance", "Arn" ] }, "Id": "TargetFunctionV1" } ] } }, "PermissionForEventsToInvokeLambda": { "Type": "AWS::Lambda::Permission", "Properties": { "FunctionName": { "Ref": "CodeDeploySyncNewInstance" }, "Action": "lambda:InvokeFunction", "Principal": "events.amazonaws.com", "SourceArn": { "Fn::GetAtt": [ "EventRule", "Arn" ] } } }, "CodeDeployServiceRole": { "Type": "AWS::IAM::Role", "Properties": { "ManagedPolicyArns": [ "arn:aws:iam::aws:policy/service-role/AWSCodeDeployRole" ], "AssumeRolePolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": [ "codedeploy.amazonaws.com" ] }, "Action": [ "sts:AssumeRole" ] } ] } } }, "CodeDeployLambdaRole": { "Type": "AWS::IAM::Role", "Properties": { "ManagedPolicyArns": [ "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole", "arn:aws:iam::aws:policy/AmazonEC2ReadOnlyAccess" ], "AssumeRolePolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": [ "ec2.amazonaws.com", "lambda.amazonaws.com" ] ], "Action": [ "sts:AssumeRole" ] } ] }, "Path": "/", "Policies": [ { "PolicyName": "Lambda-codedeploy", "PolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Sid": "StartContinuousAssessmentLambdaPolicyStmt", "Effect": "Allow", "Action": [ "codedeploy:Get*", "codedeploy:UpdateDeploymentGroup", "codedeploy:List*", "codedeploy:CreateDeployment" ], "Resource": [ "*" ] } ] } }, { "PolicyName": "IamPassRole", "PolicyDocument": { "Version": "2012-10-17", "Statement": [ { "Sid": "StartContinuousAssessmentLambdaPolicyStmt", "Effect": "Allow", "Action": [ "iam:PassRole" ], "Resource": [ { "Fn::GetAtt": [ "CodeDeployServiceRole", "Arn" ] } ] } ] } } ] } }, "CodeDeploySyncNewInstance": { "Type": "AWS::Lambda::Function", "Properties": { "Role": { "Fn::GetAtt": [ "CodeDeployLambdaRole", "Arn" ] }, "Environment": { "Variables": { "CodeDeployRole": { "Fn::GetAtt": [ "CodeDeployServiceRole", "Arn" ] } } }, "Code": { "ZipFile": { "Fn::Join": [ "\n", [ "def handler(event, context):", " return 'Hello from Lambda'" ] ] } }, "Runtime": "python3.8", "Timeout": 300, "Handler": "new-instance-code-sync.lambda_handler", "MemorySize": 512 } } }, "Outputs": { "NewInstanceLambdaFunction": { "Description": "The Lambda function that creates Resources for syncing new instances", "Value": { "Ref": "CodeDeploySyncNewInstance" } } } } |
Explanation
This template sets up the following:
- An Amazon EventBridge rule to trigger the Lambda function when an EC2 instance state changes to running.
- The necessary AWS IAM roles with permissions to allow the Lambda function to interact with Amazon EC2 and AWS CodeDeploy.
- The AWS Lambda function itself will handle the deployment logic.
Step 2: AWS Lambda Function for Deployment
Next, we will set up the AWS Lambda function. This function will:
- Detect when an instance in the warm pool changes to the running state.
- Use the instance’s tags to identify the application and deployment group.
- Trigger the AWS CodeDeploy deployment.
- Return the instance to the stopped state once deployment is complete.
AWS Lambda Function Code
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 |
from __future__ import print_function from dateutil.parser import parse from datetime import datetime import json import time import os import boto3 ec2client = boto3.client('ec2') client = boto3.client('codedeploy') codedeployrole = os.environ.get("CodeDeployRole") def lambda_handler(event, context): print("Received event: " + json.dumps(event, indent=2, default=str)) instance_id = event['detail']['instance-id'] try: # Ensure the instance is running start_instance(instance_id) wait_for_instance_running(instance_id) tags = get_instance_info(ec2client, instance_id) name = tags.get('Name') if not name: raise CustomException("Name tag not found on the instance") codedeploygroup = f"{name}-deployment-group-asg" appname = f"{name}-application" depresponse = deployment_group_tag(appname, codedeploygroup) print("Deployment group response: " + json.dumps(depresponse, indent=2, default=str)) ec2TagFilters = depresponse['deploymentGroupInfo'].get('ec2TagFilters', []) if not ec2TagFilters: print("Warning: No ec2TagFilters found in deployment group. Adding default tag filter based on Name.") ec2TagFilters = [tag for tag in ec2TagFilters if not (tag['Key'] == 'Name' and tag['Value'] == name)] ec2TagFilters.append({ 'Key': 'Name', 'Value': name, 'Type': 'KEY_AND_VALUE' }) update_response = deployment_group_update(appname, codedeploygroup, ec2TagFilters) print("Deployment group update response: " + json.dumps(update_response, indent=2, default=str)) deploymentId = depresponse['deploymentGroupInfo']['lastSuccessfulDeployment']['deploymentId'] print("DeploymentId is " + deploymentId) depIdResponse = get_dep(deploymentId) bucket = depIdResponse['s3Location']['bucket'] key = depIdResponse['s3Location']['key'] syncresp = deploy_now(appname, codedeploygroup, bucket, key) print(syncresp) # Stop the instance after deployment stop_instance(instance_id) return 'Done' except CustomException as ce: print("Custom exception: " + str(ce)) return 'Done' except KeyError as ke: print("Key error: " + str(ke)) return 'Done' except Exception as e: print("Unhandled exception: " + str(e)) return 'Done' def start_instance(instance_id): ec2client.start_instances(InstanceIds=[instance_id]) def stop_instance(instance_id): ec2client.stop_instances(InstanceIds=[instance_id]) def wait_for_instance_running(instance_id): waiter = ec2client.get_waiter('instance_running') waiter.wait(InstanceIds=[instance_id]) def get_instance_info(ec2client, instance_id): response = ec2client.describe_tags( Filters=[ { 'Name': 'resource-id', 'Values': [instance_id] } ] ) tagdict = {} for tag in response['Tags']: tagdict[tag['Key']] = tag['Value'] return tagdict def deployment_group_tag(appname, codedeploygroup): response = client.get_deployment_group( applicationName=appname, deploymentGroupName=codedeploygroup ) return response def deployment_group_update(appname, codedeploygroup, ec2TagFilters): response = client.update_deployment_group( applicationName=appname, currentDeploymentGroupName=codedeploygroup, ec2TagFilters=ec2TagFilters, serviceRoleArn=codedeployrole ) return response def get_dep(deploymentId): response = client.get_deployment(deploymentId=deploymentId) return response['deploymentInfo']['revision'] def deploy_now(appname, codedeploygroup, bucket, key): response = client.create_deployment( applicationName=appname, deploymentGroupName=codedeploygroup, deploymentConfigName='CodeDeployDefault.OneAtATime', description='Test', ignoreApplicationStopFailures=True, updateOutdatedInstancesOnly=True, fileExistsBehavior='OVERWRITE' ) return response class CustomException(Exception): """Raise for my specific kind of exception""" |
Explanation
- AWS Lambda Handler: Main function that processes the event, extracts the instance ID, and triggers the deployment process.
- Instance Management: Functions to start, stop, and wait for the instance state changes.
- AWS CodeDeploy Management: Functions interacting with AWS CodeDeploy, including retrieving and updating deployment groups and triggering deployments.
Conclusion
Using this setup, you can automate code deployment to instances in a warm pool, ensuring they run during the deployment process and return to a stopped state afterward. This approach leverages the power of AWS services like AWS Lambda, AWS CloudFormation, and AWS CodeDeploy to create and scalable deployment pipeline.
You can easily manage and trigger deployments by tagging instances with appropriate metadata, ensuring your applications are always up-to-date with minimal manual intervention.
Drop a query if you have any questions regarding AWS CodeDeploy and we will get back to you quickly.
Making IT Networks Enterprise-ready – Cloud Management Services
- Accelerated cloud migration
- End-to-end view of the cloud environment
About CloudThat
CloudThat is an award-winning company and the first in India to offer cloud training and consulting services worldwide. As a Microsoft Solutions Partner, AWS Advanced Tier Training Partner, and Google Cloud Platform Partner, CloudThat has empowered over 850,000 professionals through 600+ cloud certifications winning global recognition for its training excellence including 20 MCT Trainers in Microsoft’s Global Top 100 and an impressive 12 awards in the last 8 years. CloudThat specializes in Cloud Migration, Data Platforms, DevOps, IoT, and cutting-edge technologies like Gen AI & AI/ML. It has delivered over 500 consulting projects for 250+ organizations in 30+ countries as it continues to empower professionals and enterprises to thrive in the digital-first world.
FAQs
1. What are the benefits of using a warm pool in an Auto Scaling Group?
ANS: – Warm pools allow you to maintain a set of pre-initialized Amazon EC2 instances that can be quickly brought into service, reducing the time it takes to scale your application. By keeping instances in a stopped state, you save on costs, as you’re only billed for storage rather than compute. This approach is particularly beneficial for applications that experience sudden spikes in traffic and need to scale rapidly.
2. How does AWS CodeDeploy interact with instances in a warm pool?
ANS: – AWS CodeDeploy typically deploys code to instances as they enter an Auto Scaling Group. In warm pools, instances are brought from a stopped state to a running state before deployment. The AWS Lambda function handles this transition, ensuring that the instance is ready for deployment and returning it to a stopped state afterward. This integration allows you to maintain consistency in your deployments without manual intervention.
WRITTEN BY Deepak S
Deepak S is a Senior Research Associate at CloudThat, specializing in AWS services. He is passionate about exploring new technologies in cloud and is also an automobile enthusiast.
PREV
Comments