Building Real-Time Amazon S3 to AWS Lambda Triggers Using Amazon EventBridge

Overview

When working with Amazon S3, a common use case is to trigger an AWS Lambda function as soon as a file is uploaded. This works fine using Amazon S3 Event Notifications, but there’s a major limitation, you can’t send two events with the same prefix to different AWS Lambdas. That becomes a problem when you want to process the same uploaded file in different ways, such as one AWS Lambda validating the file while another stores metadata in a database. This is where Amazon EventBridge comes in. It helps you overcome this limitation and build a scalable, real-time, event-driven workflow.

The Problem with Native Amazon S3 Event Notifications

Amazon S3 allows you to configure event notifications that trigger an AWS Lambda, Amazon SNS topic, or Amazon SQS queue when an event like s3:ObjectCreated:* occurs. However, there are some practical issues:

• Single destination per prefix

You can’t configure two AWS Lambda triggers with the same prefix. For example, if you want both lambdaA and lambdaB to be triggered when files are uploaded to input/data/, Amazon S3 won’t allow it.

• Limited filtering

Amazon S3 notifications only support prefix and suffix filtering. You can’t create complex conditions like “trigger only for CSV files uploaded today” or “trigger only for files above 1MB.”

• Hard to manage at scale

If you have multiple prefixes or workflows, the number of Amazon S3 rules grows fast. There’s no easy way to manage or fan out events centrally.

• No advanced routing or transformation

You can’t modify or enrich event data before it reaches AWS Lambda.

Solving Amazon S3 Notification Limitations with Amazon EventBridge

Amazon EventBridge is a serverless event bus that receives events from AWS services and routes them based on patterns you define. By connecting Amazon S3 to Amazon EventBridge, you can route file upload events to multiple AWS Lambdas, with filtering, transformation, and fan-out built-in.

Benefits of using Amazon EventBridge with Amazon S3

• Trigger multiple AWS Lambdas for the same event
• Add rich filtering logic (bucket, key pattern, file type, size, etc.)
• Centralize all event rules in one place
• Extend easily to future workflows, no change in Amazon S3 config needed
• Get better monitoring and retry mechanisms

Step-by-Step Setup

Step 1: Enable Amazon EventBridge for your Amazon S3 bucket

• By default, Amazon S3 doesn’t send events to Amazon EventBridge.
• You can enable it using the AWS Console:
• Go to your Amazon S3 bucket → Properties → Event notifications
• Scroll to the “EventBridge” section and click Enable EventBridge.
• Once enabled, all Amazon S3 events (like object create, delete, and restore) will be sent to your account’s default event bus.

Step 2: Create an Amazon EventBridge rule

• Now create a rule that listens to Amazon S3 events and triggers AWS Lambda. Example event pattern:

This rule listens for all object creation events in the specified prefix and triggers the target AWS Lambda function.

Step 3: Add AWS Lambda as the target

• In the Amazon EventBridge rule, choose AWS Lambda Function as the target.
• You can add multiple targets if you want to process the same event in different Lambdas.
• Each AWS Lambda will receive an event. Then your AWS Lambda can parse the bucket name and key to read the file using the boto3 Amazon S3 client.

Step 4: Test the setup

• Upload a file to your bucket and check:
• Amazon EventBridge → Monitoring → “Matched events”
• AWS Lambda → Amazon CloudWatch logs → “Invocation received”
• You should see your AWS Lambda triggered instantly.

Best Practices while using Amazon EventBridge

• Use filtering wisely – Amazon EventBridge lets you filter based on size, prefix, or even tags to reduce unnecessary AWS Lambda invocations.
• Keep Lambdas idempotent – Events may be delivered more than once. Always handle duplicates safely.
• Monitor DLQs (Dead Letter Queues) – Configure DLQs for failed event deliveries.
• Limit payload size – Amazon EventBridge event size limit is 256 KB. If you need to send large content, store it in Amazon S3 and send only the path.

Conclusion

Using Amazon EventBridge to route Amazon S3 events to Lambda provides a simple yet powerful way to overcome the limitations of native Amazon S3 event notifications.

By decoupling Amazon S3 as the producer from your consumers, you can add new processing logic without touching bucket configurations, making Amazon EventBridge an essential tool for real-time, event-driven applications.

Drop a query if you have any questions regarding Amazon S3 and we will get back to you quickly.

About CloudThat