Leverage AWS S3, Lambda & SNS to Setup Event-driven Messaging Service

Introduction

Broadcasting messages to an extensive use group is time-consuming and requires immense effort when performed manually. AWS provides various services that can be leveraged to set up event-driven messaging.

This blog combines S3 Bucket, Lambda, and SNS topics. Any S3 bucket upload, update or remove event will start an AWS Lambda function. After retrieving the file details, a lambda function will cause an AWS SNS topic to deliver email notifications to the subscribers with the file details.

To send emails to the subscribers, we will develop a lambda function that will take events from an S3 bucket and then trigger SNS topics. When a file is uploaded, updated, or removed from an S3 bucket, the AWS lambda function should be triggered to retrieve the file’s information and start an AWS SNS topic that will send email notifications to the topic’s subscribers.

Prerequisites

• AWS account
• AWS IAM permissions for required Services
• AWS CLI

Architecture Diagram

The architecture diagram above shows how an event is triggered from S3, captured by a lambda function, and then passed to Amazon SNS, which sends emails to the customers.

Step-by-Step Guide for Event Driven Messaging Service

Summary

1. First, create an S3 bucket
2. Create an SNS subscription and confirm the email subscription
3. Create a policy and role
4. Create a Lambda Function

Step I: Creating Amazon S3 bucket

Step 1: Sign into the AWS console and search for S3 in the search bar and select the service

Step 2: Click on Create bucket

Step 3: Give the bucket information as follows:

• Give the bucket a suitable name (the name must be globally unique)
• Choose a region,
• Enable the public ACL, and 
• Next, click on create a bucket

Step 4: As soon as a bucket is created, it appears on the list of buckets. To see additional information, select the bucket.

Step II: Creating the topic:

Step 1: Sign into the AWS console and search for Simple Notification Service (SNS) in the search bar and select the service

Step 2: Provide the proper name of the topic and click on create next step

Step 3: The steps mentioned below will help in creating the topic

• Select the type as standard
• The topic’s name will be presented as specified in the previous step
• The remaining steps are optional
• Click on create a topic

Step 4: The topic is pooped up on top after it has been created. To see additional information, select the created topic.

Step 5: Create a Subscription

• Once the topic has been properly created, we must create a subscription for the customer to receive emails.
• Select “create a subscription,” and the topic name will appear
• From the selection, choose the HTTP protocol
• Substitute a valid email-id for the endpoint
• Click on create a subscription.

Step 6: Confirm the subscription

• When the subscription is made, the status will show as “waiting for confirmation.”
• The confirmation email will be sent to the email address provided in the previous stage when subscribing to the topic. 
• Click the “click here to subscribe” link in the email to complete the subscription process
  
  
• After the mail confirmation, we can see the subscription status as confirmed.
  
  

Step III: Creating IAM Role:

• An IAM role needs to be created for lambda execution. The actions listed below will assist in creating a role.
• Start up the IAM console.

1. In the left navigation pane, select role.
2. Select Create Role.
3. Select Lambda as the everyday use case and the trusted entity type as an AWS service.
4. Select Next, go to the Add Permissions page, and select Create Policy.
5. The page for creating policies to access AWS resources appears in a new tab.
6. Enter the name of your choice and select Create Policy on the review page.
7. Select the newly generated policy and attach the authorization policy page.
8. Choose a name for the role on the name, review, and create the page
   
   
   
   

Step IV: Creating the Lambda function:

Step 1: Create a lambda Function to trigger the upload event in S3 Bucket.

• Choose the Author from scratch.
• Provide the name of the function.
• In the position of runtime, we must choose the language that we want. Here, I am choosing the most recent Python version.
• Choose whether to create a new or existing role as the executing role in the following step. I’m choosing the role that I already created in the previous phase.
• Rest everything. We can keep it optional
• Select Create a Function.
  
  

Step 2: Adding triggers to the lambda function:

• Choose an S3 bucket as the source.
• Next, choose the bucket we want to add as a trigger.
• The remaining steps are optional.
• Click on Add

Step 3: Add the SNS destination to the Lambda

• The final step is to add a destination.
• Choose Asynchronous invocation for the source.
• Choose On Success for the condition
• Choose SNS Topic for the destination type
• And the topic we defined in the previous phase.
• Click on add.
  
  

Result:

Lambda is triggered when the image is uploaded to the S3 bucket and sends the SNS notification.

Conclusion

We can conclude how to cause Lambda to run asynchronously when new S3 bucket events (uploads, deletions, and updates) occur. Additionally, we have understood how to establish SNS as a destination in the lambda function so that the email notification is sent to every subscriber. Because of this, monitoring the S3 bucket to observe what events are happening is made more accessible by this idea.

About CloudThat