Migration of MongoDB from On-premise to AWS
About Client
Empower Network hosts one of the largest blogging communities online and provides educational training products and services to online marketers, entrepreneurs, start-ups, home-based and small businesses.
Problem Statement
Clients MongoDB sharded cluster configuration needs to be upgraded with best practices. As the CPU of main shard was reaching 100% at times leading to interruption in services, degrading the end-user experience, effecting the whole business landscape.
Business Objectives
- To achieve a highly available and faster results time for the blogging platform
- A highly scalable solution for high-customer traffic for the website
- Provide continuous service to customers without any interruptions and lag
Technical Objectives
- To be able to move an on-premises MongoDB cluster to AWS cloud.
- Setup the environment with high availability and low latency
- Secure the MongoDB cluster
- Backward compatibility with on-premises application
Design Factors
- The client wanted high availability, but also strong security.
- We have created separate private subnets for all the MongoDB Nodes.
- We have followed all the best practices provided by the AWS and MongoDB team, like using private static IP addresses; so that a down host would not cause long downtime and can be quickly brought up with the right IP Address.
- We have integrated the nodes with new relic for monitoring.
Amazon Services Used
- Amazon EC2
- VPN
- Elastic Load Balancer
- Auto Scaling
- Amazon Route 53
- Elastic Container Service
- IAM
- CloudWatch
- CloudTrail
- Elastic search service
- Elastic Kubernetes service
Architecture Diagram and Designs
Outcomes
- All the best practices specified by MongoDB have been implemented.
- Detailed log analysis was performed on logs during the 100% CPU incident and the issue was narrowed down to a slow query.
- The query was modified, and the incident has not repeated since.
- Automated backups and alarms configured on CloudWatch, Mongo Monitoring Service.
Lessons Learned
- Hourly backed up data improves point in-time recovery.
- Service interruptions have been mitigated through implementation of AWS best practices.