A Guide for Creating a Highly Available Redis Cluster on Amazon EC2

Overview

In-memory caching, powered by tools like Redis, optimizes application performance by storing frequently accessed data in fast RAM, reducing data retrieval latency. This approach lightens the database load, enhancing overall system performance and scalability. Redis also ensures data consistency and supports advanced features like session management and real-time data distribution through Pub-Sub, making applications more responsive, efficient, and versatile.

In this blog, we’ll guide you through creating a 3-master and 3-slave node Redis Cluster on Amazon Web Services (AWS) Elastic Compute Cloud (EC2) instances.

Introduction

Redis is an open-source in-memory data store with versatile uses, including caching and message brokering. Redis Cluster is its distributed version, offering high availability and data partitioning.

Pre-requisites

• AWS Account: You need an AWS account to create Amazon EC2 instances and manage resources.
• Amazon EC2 Instances: Launch six Amazon EC2 instances – three for master nodes and three for slave nodes. Ensure these instances are in the same Amazon Virtual Private Cloud (VPC) and subnet.
• Security Groups: Configure security groups to allow necessary inbound and outbound traffic for Redis. You should configure your security group to permit TCP access for port numbers 6379, 6378, 16379, and 16378. These ports serve distinct purposes, with lower ports facilitating user communication and higher ports facilitating node-to-node communication through the “Cluster Bus” or “Gossip port”.

Architecture Diagram

Please be aware that for the cluster to function correctly, it must have a minimum of three master nodes. Especially when conducting initial tests, it’s highly recommended to set up a cluster consisting of six nodes, evenly split between three master nodes and three slave nodes (Redis.io, 2019).

Today, we will set up a minimal cluster per Redis’ guidance. The architecture we’ll be implementing is depicted below:

Explanation:

There are three Redis master nodes (master 1, master 2, master 3), each responsible for a specific subset of data.

Each master node has two corresponding Redis slave nodes (slave 1, slave 2, slave 3) that replicate the data from the master node they are associated with.

The Redis Cluster spans Amazon EC2 instances, ensuring high availability. A failed master node can be replaced by one of its slaves.

Clients connect to any node, and the cluster routes requests to relevant master or slave nodes based on data distribution and replication.

Note: This diagram simplifies Redis Cluster architecture, excluding network, security group, and AWS-specific details.

Step-by-Step Guide

Step 1 – Creating Amazon EC2 Instance

Step 2 – Redis installation

SSH to the created machines to install the Redis packages. Here, I am going to install Redis on master-1.

Before installing the Redis, we need to update the package manager.

Add Redis server repository

You can check whether the server is properly set with Redis by using redis-cli command and ping like below:

Type exit to quit the redis-cli mode.

By following the above steps, Install Redis on all instances (both master and slave nodes).

Step 3 – Configure Redis Master Nodes

bind 172.31.50.50 127.0.0.1 ::1    # bind private-ip
protected-mode no

cluster-enabled yes

cluster-node-timeout 15000

Now, we will start the redis-server service and check whether it will give a running status. You can do all these steps simultaneously in all 3 servers we built in AWS.

If you followed this blog, you would end up with something like below.

If you are successful up to this point, let’s build the cluster like below:

We are going to build a cluster using instance private IP,

This will ask you to confirm the sharding, and you must type “yes” for it.

If everything is correct, especially the configuration part, you will end up with a success message:

We can run the following command to check the master node status

If you are facing an issue in the above step, check the security group attached to the instance and verify that the ports below are opened 6379, 6378, 16379, and 16378.

Step 4 – Configure Redis Slave Nodes

In all the Redis slave Amazon EC2 instances, we already installed Redis, and now we have to make the following changes in Redis configuration.

Change the Redis configuration like below (port is the only change between master and cluster configuration)

You can check whether the slave is working properly by a simple ping-pong like using the below command.

Step 5 – Adding slave with master

Here, we are adding slave-1 with master-1 by using the below command

Syntax:

Do the same thing in the other 2 servers to add slaves.

Now check the cluster node status. You can see the master and slave connected below.

Now we will check the master slave replication. I am going to set the name in master-2 and retrieve the name in slave-2

Conclusion

By following the above steps, you have successfully established a Redis Cluster that offers high availability, fault tolerance, and the capability to handle increased workloads. This setup ensures that your Redis-based applications can deliver consistent and responsive services to your users while remaining resilient to potential disruptions.
Deploying a highly available Redis Cluster on Amazon EC2 with three master nodes and three slave nodes is a powerful way to ensure the reliability, scalability, and performance of your Redis-based applications. By distributing your data across multiple nodes and enabling replication, you can mitigate the risk of data loss and downtime in the event of node failures.

Drop a query if you have any questions regarding Redis Cluster on Amazon EC2 and we will get back to you quickly.

About CloudThat

CloudThat is an official AWS (Amazon Web Services) Advanced Consulting Partner and Training partner, AWS Migration Partner, AWS Data and Analytics Partner, AWS DevOps Competency Partner, Amazon QuickSight Service Delivery Partner, AWS EKS Service Delivery Partner, and Microsoft Gold Partner, helping people develop knowledge of the cloud and help their businesses aim for higher goals using best-in-industry cloud computing practices and expertise. We are on a mission to build a robust cloud computing ecosystem by disseminating knowledge on technological intricacies within the cloud space. Our blogs, webinars, case studies, and white papers enable all the stakeholders in the cloud computing sphere.

To get started, go through our Consultancy page and Managed Services Package, CloudThat’s offerings.