A Social Media Company Overcomes Language Barriers and Expands its User Base with Amazon API Gateway

Solutions

• The solution depicts a centralized solution for data analysis and AI/ML solutions such as Audio Transcription, Text Translation, Sentiment and Key-phrase analysis with Amazon DynamoDB playing a crucial role in various stages of the data processing pipeline and acting as a Primary Database. 
• The raw data is initially transferred from the primary AWS account to the Data Engineering account using Amazon SQS, AWS Lambda and the data is stored in Amazon DynamoDB, leveraging its scalable and high-performance capabilities.  
• From Amazon DynamoDB streams, the data is streamed to Kinesis, which acts as a data pipeline for distributing the information to multiple consumers. AWS Lambda functions and Amazon S3 are the recipients of the data from Kinesis. 
• The AWS Lambda functions perform various tasks, such as calling Amazon Transcribe to convert audio into text. The resulting text is translated into SRT format using Amazon Translate, and metadata is stored in Amazon DynamoDB with proper Partition key and Sort keys. 
• The translated text is sent to Comprehend AWS Lambda functions for Key Phrase and Sentiment analysis, with the results stored in Amazon DynamoDB and Amazon S3.  
• AWS Glue Crawler performs ETL (Extract, Transform, Load) jobs on the data stored in the Amazon S3 bucket. This process transforms the data and prepares it for further analysis.  
• Using Amazon Athena, queries are performed on the transformed data and valuable insights like trending hashtags are derived and stored in Amazon DynamoDB.  
• Insights related to User posts, Failed tasks, and Processed tasks are extracted from Amazon DynamoDB with the help of AWS Lambda and Amazon API Gateway and served to the client application.