Orchestrating AWS Lambda Invocations with AWS Step Functions for Seamless Workflows

Introduction

AWS Step Functions is a fully-managed AWS service that lets us coordinate distributed microservices and applications using visual workflows. With Step Functions, we can build applications that react to events and execute business logic serverless.

AWS Step Functions provide a way to describe the workflow of your application as a state machine, which is made up of a set of states executed in a specific order. Each state in the state machine performs some specific action or task and can transition to other states based on conditions.

AWS Step Functions support a wide range of AWS services, including AWS Lambda, Amazon SNS, Amazon SQS, Amazon ECS, and more. This makes it easy to build and orchestrate complex workflows that span multiple AWS services.

AWS Step Functions also provide a range of features that make it easy to build and manage your workflows, including:

Visual Workflow Designer: a drag-and-drop interface that lets you create and modify your state machine.
State Machine Logging and Monitoring: Step Functions provide a complete history of all state transitions, making debugging issues and troubleshooting errors easy.
Error Handling: Step Functions provide built-in error handling and retries, so you can easily handle errors and recover from failures.

Step Functions also integrate with other AWS services such as Amazon CloudWatch, AWS X-Ray, and AWS EventBridge, providing additional visibility and control over your workflows.

Why must we use AWS Step Function to invoke AWS Lambda with another?

AWS Step Functions are important for calling one Lambda function from another because it provides a powerful and scalable way to orchestrate the flow of your application. While it’s possible to call one AWS Lambda function from another directly, this approach can quickly become complex and difficult to manage as your application grows.

By using AWS Step Functions to orchestrate the flow of your application, you can easily build complex workflows that involve multiple AWS Lambda functions and other AWS services. This provides a scalable and reliable way to manage the flow of your application, ensuring that each Lambda function is executed in the correct order and with the correct inputs and outputs.

AWS Step Functions also provide a range of features that make it easy to manage and monitor your workflows, including visual workflow design, state machine logging and monitoring, and built-in error handling and retries. This makes it easy to troubleshoot issues and ensure your application runs smoothly.

Pioneers in Cloud Consulting & Migration Services

Reduced infrastructural costs
Accelerated application deployment

Get Started

Steps to invoke AWS Lambda with another AWS Lambda by using AWS Step Function

Step 1: Create the first-lambda

step1

Add the following sample code in first-lambda

import json
import boto3 
import os 
account_number = os.environ["acc_num"]
statemachine_name = os.environ["state_machine"]
aws_region = os.environ["region"] 
def start_state_machine(state_machine_arn:str, msg:str):
    client = boto3.client('stepfunctions')
    response = client.start_execution(
        stateMachineArn=state_machine_arn,
        input=msg)
    return response
def lambda_handler(event, context):
    print(event)
    print(type(event))
    msg = json.dumps(event)
    print(type(msg))
    print(msg)
    start_state_machine(f"arn:aws:states:{aws_region}:{account_number}:stateMachine:{statemachine_name}",msg)
    return event

import json

import boto3

import os

account_number = os.environ["acc_num"]

statemachine_name = os.environ["state_machine"]

aws_region = os.environ["region"]

def start_state_machine(state_machine_arn:str, msg:str):

client = boto3.client('stepfunctions')

response = client.start_execution(

stateMachineArn=state_machine_arn,

input=msg)

return response

def lambda_handler(event, context):

print(event)

print(type(event))

msg = json.dumps(event)

print(type(msg))

print(msg)

start_state_machine(f"arn:aws:states:{aws_region}:{account_number}:stateMachine:{statemachine_name}",msg)

return event

Create Environment Variables on first-lambda

step1b

Use the name of the AWS Step function in advance, which you will create in the next steps, and give the region where you are creating the resources.
Now create an event as the following.

step1c

Add the following policy to the lambda role – AWSStepFunctionFullAccess

step1d

Step 2: Create the second-lambda

step2

Add the following sample code

import json
def lambda_handler(event, context):
    print(event)
    return event

import json

def lambda_handler(event, context):

print(event)

return event

Create a step function.

step2b

Drag and drop the AWS Lambda invocation icon on the left and select the second lambda on the right under the function name.

Note – Don’t select the first-lambda in this step because if you select that, then it will go in an infinite loop which will not stop execution until you stop the execution, so that it might give you cost on the resources.

step2c

Click on next

step2d

Click on Create after giving the name and role for it.

step2e

After the successful execution of the AWS Step function.

step2f

Execution input and output

step2g

The result can be seen in Amazon CloudWatch logs of first-lambda

step2h

The result can be seen in CloudWatch logs of second-lambda

step2i

Conclusion

Using AWS Step Functions to orchestrate the flow of your application provides a powerful and scalable way to manage the interaction between AWS Lambda functions and can help to ensure that your application is reliable and easy to maintain.

Making IT Networks Enterprise-ready – Cloud Management Services

Accelerated cloud migration
End-to-end view of the cloud environment

Get Started

About CloudThat

CloudThat is an official AWS (Amazon Web Services) Advanced Consulting Partner and Training partner and Microsoft Gold Partner, helping people develop knowledge of the cloud and help their businesses aim for higher goals using best-in-industry cloud computing practices and expertise. We are on a mission to build a robust cloud computing ecosystem by disseminating knowledge on technological intricacies within the cloud space. Our blogs, webinars, case studies, and white papers enable all the stakeholders in the cloud computing sphere.

Drop a query if you have any questions regarding AWS Lambda, AWS Step Function, I will get back to you quickly.

To get started, go through our Consultancy page and Managed Services Package, CloudThat’s offerings.

FAQs

1. Which is better, AWS Lambda calling AWS Lambda or with AWS Step Functions?

ANS: – Calling one AWS Lambda function from another directly can be a simple approach for small-scale applications, where the workflow is relatively simple and easy to manage. However, as the application grows in complexity, this approach can quickly become unwieldy and difficult to maintain, as many AWS Lambda functions may need to be called in a specific order, with specific inputs and outputs.

2. Does AWS Step Function have an additional cost for this process?

ANS: – Yes, using AWS Step Functions does incur an additional cost for your application. AWS Step Functions charges for the number of state transitions and the total duration of your state machine execution. Check for more details on AWS service pricing.

WRITTEN BY Suresh Kumar Reddy

Yerraballi Suresh Kumar Reddy is working as a Research Associate - Data and AI/ML at CloudThat. He is a self-motivated and hard-working Cloud Data Science aspirant who is adept at using analytical tools for analyzing and extracting meaningful insights from data.