Efficient Large Video Uploads from Browser to Amazon S3 with IndexedDB

Overview

Uploading large video files directly from the browser to Amazon S3 is a common requirement in modern web applications, especially in domains like virtual interviews, e-learning, or video messaging. However, traditional browser storage mechanisms like localStorage or sessionStorage are limited, typically allowing only up to 5MB of storage per origin.

This limitation makes handling large media files a challenge. To overcome this, we used IndexedDB, a low-level API available in all modern browsers that allows structured storage of large amounts of data, including binary files like videos. Combined with multipart upload to Amazon S3, this approach provides a robust, resumable, and efficient way to upload large video files directly from the browser.

This blog will explore how IndexedDB fits into this workflow and how it helped us solve a real-world challenge in a browser-based video recording application.

Introduction

While developing a browser-based video recording and submission tool, we faced a major roadblock: the browser cache couldn’t handle large video files reliably. We needed a persistent client-side storage option that was resilient to page reloads and capable of handling large binary objects.

This blog explains the architecture and implementation of this solution.

Why IndexedDB?

Challenges with Default Storage Options

• localStorage and sessionStorage are limited to ~5MB.
• These are synchronous APIs that are unsuitable for storing binary data like videos.
• Temporary memory storage is volatile and does not survive page reloads.

Benefits of IndexedDB

• It supports large binary data (Blobs, Files) in hundreds of MBs or GBs.
• Asynchronous API, non-blocking, and reliable.
• Data remains persistent across sessions even after a browser refresh.
• Well-supported across all major browsers (Chrome, Firefox, Safari, Edge).

How It Works: Architecture and Flow

1. Store Video in IndexedD

After the user records or selects a video, we store it in IndexedDB using the idb library to simplify API calls. Each video is stored as a Blob with a unique identifier:

2. Upload in Chunks Using Amazon S3 Multipart Upload

Once stored, we:

• Read the video from IndexedDB
• Slice it into smaller chunks (e.g., 5MB each)
• Fetch presigned URLs from our backend for each chunk
• Upload each chunk to Amazon S3 individually
• Complete the multipart upload via a final API call

Work Flow Diagram

Figure 1: Architecture for IndexedDB-Based Amazon S3 Multipart Upload

Real-World Use Case

In our interview bot platform, candidates record video responses that must be uploaded instantly without backend compression or processing. Here’s what we achieved using IndexedDB:

• Video survives accidental refresh or reload.
• Upload progress can be resumed if interrupted.
• Amazon S3 upload completed securely using presigned URLs.
• No server-side storage or processing was required.

This approach worked reliably for videos as large as 500MB and provided seamless UX.

Prompt Engineering Tips (Optional for Devs)

If you’re prompting users to interact with this feature:

• Notify them of potential large file sizes.
• Provide real-time upload progress indicators.
• Allow retries using the IndexedDB-stored file.

Getting Started: Key Implementation Steps

1. Use a wrapper like IDB to work with IndexedDB easily.
2. Use Blob.slice() to divide videos into 5MB chunks.
3. Request presigned URLs from your backend using video metadata.
4. Use fetch() or axios.put() to upload each chunk.
5. Track ETag values for each part.
6. Complete a multipart upload using a final call to your backend.

Conclusion

IndexedDB, combined with Amazon S3’s multipart upload capability, offers a powerful client-side solution for uploading large files directly from the browser. It removes the need for intermediate servers, reduces memory issues, and enhances the user experience in web applications involving video content.

This approach is ideal for use cases like browser-based recording tools, asynchronous interviews, media sharing apps, and more, where large file handling on the client side is critical.

Drop a query if you have any questions regarding IndexedDB and we will get back to you quickly.

About CloudThat