E-Commerce Firm Achieves 70% Performance Increase and Centralized Storage with CloudThat

Solutions

• The goal of achieving a centralized data store for analytics is being achieved by using AWS Glue, AWS Lambda and Amazon S3. 
• AWS Lambda functions are used to fetch sales data from Shopify, Zoho, Unicommerce, and Google Drive and store it in an Amazon S3 bucket as raw data in 9 different partitions.  
• The data is stored in CSV format. 
• AWS Lambda is triggered every day once, and it fetches the data sales data, including cancelled orders from the different APIs provided. 
• The received JSON payloads are converted to CSV at an AWS Lambda level to enrich the data and store it. 
• The AWS Glue job is created to process the raw csv data from the 9 partitions and to create a sales register from the last month of sales data. This job is scheduled and runs five times every month. 
• This AWS Glue job maps the fields among the files from each partition and appends the data to create one single file. 
• Another AWS Glue job is also created to extract the dates and details of the cancelled orders of the last two days. This job is scheduled to run daily.  
• This job also maps the field among the files from each partition and appends the data to create a single file. 
• The AWS Glue jobs store the two files in a centralized location to create the datasets for Amazon Quicksight visualization.