Login
September 15, 2023|
Voiced by Amazon Polly |
Overview
In-memory caching, powered by tools like Redis, optimizes application performance by storing frequently accessed data in fast RAM, reducing data retrieval latency. This approach lightens the database load, enhancing overall system performance and scalability. Redis also ensures data consistency and supports advanced features like session management and real-time data distribution through Pub-Sub, making applications more responsive, efficient, and versatile.
In this blog, we’ll guide you through creating a 3-master and 3-slave node Redis Cluster on Amazon Web Services (AWS) Elastic Compute Cloud (EC2) instances.
Introduction
Redis is an open-source in-memory data store with versatile uses, including caching and message brokering. Redis Cluster is its distributed version, offering high availability and data partitioning.
Pioneers in Cloud Consulting & Migration Services
- Reduced infrastructural costs
- Accelerated application deployment
Pre-requisites
- AWS Account: You need an AWS account to create Amazon EC2 instances and manage resources.
- Amazon EC2 Instances: Launch six Amazon EC2 instances – three for master nodes and three for slave nodes. Ensure these instances are in the same Amazon Virtual Private Cloud (VPC) and subnet.
- Security Groups: Configure security groups to allow necessary inbound and outbound traffic for Redis. You should configure your security group to permit TCP access for port numbers 6379, 6378, 16379, and 16378. These ports serve distinct purposes, with lower ports facilitating user communication and higher ports facilitating node-to-node communication through the “Cluster Bus” or “Gossip port”.
Architecture Diagram
Please be aware that for the cluster to function correctly, it must have a minimum of three master nodes. Especially when conducting initial tests, it’s highly recommended to set up a cluster consisting of six nodes, evenly split between three master nodes and three slave nodes (Redis.io, 2019).
Today, we will set up a minimal cluster per Redis’ guidance. The architecture we’ll be implementing is depicted below:
Explanation:
There are three Redis master nodes (master 1, master 2, master 3), each responsible for a specific subset of data.
Each master node has two corresponding Redis slave nodes (slave 1, slave 2, slave 3) that replicate the data from the master node they are associated with.
The Redis Cluster spans Amazon EC2 instances, ensuring high availability. A failed master node can be replaced by one of its slaves.
Clients connect to any node, and the cluster routes requests to relevant master or slave nodes based on data distribution and replication.
Note: This diagram simplifies Redis Cluster architecture, excluding network, security group, and AWS-specific details.
Step-by-Step Guide
Step 1 – Creating Amazon EC2 Instance
Step 2 – Redis installation
SSH to the created machines to install the Redis packages. Here, I am going to install Redis on master-1.
Before installing the Redis, we need to update the package manager.
|
1 2 |
# sudo apt-get update # sudo apt-get install software-properties-common |
Add Redis server repository
|
1 2 3 4 5 |
# sudo add-apt-repository ppa:chris-lea/redis-server # sudo apt-get update Install redis # sudo apt-get install redis-server |
You can check whether the server is properly set with Redis by using redis-cli command and ping like below:
Type exit to quit the redis-cli mode.
By following the above steps, Install Redis on all instances (both master and slave nodes).
Step 3 – Configure Redis Master Nodes
|
1 |
# sudo nano /etc/redis/redis.conf |
bind 172.31.50.50 127.0.0.1 ::1 # bind private-ip
protected-mode no
cluster-enabled yes
cluster-node-timeout 15000
Now, we will start the redis-server service and check whether it will give a running status. You can do all these steps simultaneously in all 3 servers we built in AWS.
|
1 2 |
# sudo service redis-server restart # sudo service redis-server status |
If you followed this blog, you would end up with something like below.
If you are successful up to this point, let’s build the cluster like below:
We are going to build a cluster using instance private IP,
|
1 |
# redis-cli --cluster create 172.31.50.50:6379 172.31.64.198:6379 172.31.80.124:6379 |
This will ask you to confirm the sharding, and you must type “yes” for it.
If everything is correct, especially the configuration part, you will end up with a success message:
We can run the following command to check the master node status
|
1 |
# redis-cli -p 6379 cluster nodes |
If you are facing an issue in the above step, check the security group attached to the instance and verify that the ports below are opened 6379, 6378, 16379, and 16378.
Step 4 – Configure Redis Slave Nodes
In all the Redis slave Amazon EC2 instances, we already installed Redis, and now we have to make the following changes in Redis configuration.
|
1 |
# vi /etc/redis/redis.conf |
Change the Redis configuration like below (port is the only change between master and cluster configuration)
|
1 2 3 4 5 6 7 8 9 |
bind 172.31.63.231 127.0.0.1 ::1 # bind private-ip protected-mode no port 6378 cluster-enabled yes cluster-config-file nodes-6378.conf cluster-node-timeout 15000 # sudo service redis-server restart # sudo service redis-server status |
You can check whether the slave is working properly by a simple ping-pong like using the below command.
|
1 |
# redis-cli -p 6378 |
Step 5 – Adding slave with master
Here, we are adding slave-1 with master-1 by using the below command
Syntax:
|
1 2 |
# redis-cli --cluster add-node <slave-1 privateIP>:6378 <Master-1 privateIP>:6379 --cluster-slave # redis-cli --cluster add-node 172.31.63.231:6378 172.31.50.50:6379 --cluster-slave |
Do the same thing in the other 2 servers to add slaves.
Now check the cluster node status. You can see the master and slave connected below.
|
1 |
# redis-cli -p 6379 cluster nodes |
Now we will check the master slave replication. I am going to set the name in master-2 and retrieve the name in slave-2
Conclusion
By following the above steps, you have successfully established a Redis Cluster that offers high availability, fault tolerance, and the capability to handle increased workloads. This setup ensures that your Redis-based applications can deliver consistent and responsive services to your users while remaining resilient to potential disruptions.
Deploying a highly available Redis Cluster on Amazon EC2 with three master nodes and three slave nodes is a powerful way to ensure the reliability, scalability, and performance of your Redis-based applications. By distributing your data across multiple nodes and enabling replication, you can mitigate the risk of data loss and downtime in the event of node failures.
Drop a query if you have any questions regarding Redis Cluster on Amazon EC2 and we will get back to you quickly.
Making IT Networks Enterprise-ready – Cloud Management Services
- Accelerated cloud migration
- End-to-end view of the cloud environment
About CloudThat
CloudThat is an official AWS (Amazon Web Services) Advanced Consulting Partner and Training partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, Amazon QuickSight Service Delivery Partner, AWS EKS Service Delivery Partner, and Microsoft Gold Partner, helping people develop knowledge of the cloud and help their businesses aim for higher goals using best-in-industry cloud computing practices and expertise. We are on a mission to build a robust cloud computing ecosystem by disseminating knowledge on technological intricacies within the cloud space. Our blogs, webinars, case studies, and white papers enable all the stakeholders in the cloud computing sphere.
To get started, go through our Consultancy page and Managed Services Package, CloudThat’s offerings.
FAQs
1. Why is Redis Cluster a better choice than a single Redis instance?
ANS: – Redis Cluster offers several benefits, including high availability, fault tolerance, and scalability. It distributes data across multiple nodes, allowing your application to continue functioning even if some nodes fail. It also enables horizontal scaling as data and traffic grow, making it a better choice for production environments.
2. How can I monitor the Redis Cluster's performance and health?
ANS: – You can leverage various monitoring tools like redis-cli, redis-stat, or dedicated Redis monitoring solutions. These tools provide insights into the cluster’s resource utilization and key performance metrics and help you promptly identify and address any issues.
3. After the initial setup, can I add more nodes to the Redis Cluster?
ANS: – Yes, Redis Cluster is designed to be scalable. You can easily expand your cluster by adding more nodes (both master and slave) to accommodate increased data storage and traffic. Redis automatically redistributes data to the new nodes, ensuring even load distribution.
WRITTEN BY Harikrishnan S
Harikrishnan Seetharaman is a Research Associate (DevOps) at CloudThat. He completed his Bachelor of Engineering degree in Electronics and Communication, and he achieved AWS solution architect-Associate certification. His area of interest is implementing a cloud-native solution for customers and helping them by proving robust and reliable solutions for their complex problems, DevOps, and SaaS. Apart from his professional interest he likes to spend time in farming and learning new DevOps tools.
PREV
Comments