AWS, Cloud Computing

5 Mins Read

A Guide for Creating a Highly Available Redis Cluster on Amazon EC2

Voiced by Amazon Polly

Overview

In-memory caching, powered by tools like Redis, optimizes application performance by storing frequently accessed data in fast RAM, reducing data retrieval latency. This approach lightens the database load, enhancing overall system performance and scalability. Redis also ensures data consistency and supports advanced features like session management and real-time data distribution through Pub-Sub, making applications more responsive, efficient, and versatile.

In this blog, we’ll guide you through creating a 3-master and 3-slave node Redis Cluster on Amazon Web Services (AWS) Elastic Compute Cloud (EC2) instances.

Pioneers in Cloud Consulting & Migration Services

  • Reduced infrastructural costs
  • Accelerated application deployment
Get Started

Introduction

Redis is an open-source in-memory data store with versatile uses, including caching and message brokering. Redis Cluster is its distributed version, offering high availability and data partitioning.

Developers and system administrators need to build resilient and scalable Redis-dependent applications. You can effortlessly scale your cluster by adding nodes or replicas to meet growing data and traffic needs, all while ensuring user expectations for fault tolerance and availability.

Pre-requisites

  • AWS Account: You need an AWS account to create Amazon EC2 instances and manage resources.
  • Amazon EC2 Instances: Launch six Amazon EC2 instances – three for master nodes and three for slave nodes. Ensure these instances are in the same Amazon Virtual Private Cloud (VPC) and subnet.
  • Security Groups: Configure security groups to allow necessary inbound and outbound traffic for Redis. You should configure your security group to permit TCP access for port numbers 6379, 6378, 16379, and 16378. These ports serve distinct purposes, with lower ports facilitating user communication and higher ports facilitating node-to-node communication through the “Cluster Bus” or “Gossip port”.

Architecture Diagram

Please be aware that for the cluster to function correctly, it must have a minimum of three master nodes. Especially when conducting initial tests, it’s highly recommended to set up a cluster consisting of six nodes, evenly split between three master nodes and three slave nodes (Redis.io, 2019).

Today, we will set up a minimal cluster per Redis’ guidance. The architecture we’ll be implementing is depicted below:

AD

Explanation:

There are three Redis master nodes (master 1, master 2, master 3), each responsible for a specific subset of data.

Each master node has two corresponding Redis slave nodes (slave 1, slave 2, slave 3) that replicate the data from the master node they are associated with.

The Redis Cluster spans Amazon EC2 instances, ensuring high availability. A failed master node can be replaced by one of its slaves.

Clients connect to any node, and the cluster routes requests to relevant master or slave nodes based on data distribution and replication.

Note: This diagram simplifies Redis Cluster architecture, excluding network, security group, and AWS-specific details.

Step-by-Step Guide

Step 1 – Creating Amazon EC2 Instance

step1

Step 2 – Redis installation

SSH to the created machines to install the Redis packages. Here, I am going to install Redis on master-1.

Before installing the Redis, we need to update the package manager.

step2

Add Redis server repository

You can check whether the server is properly set with Redis by using redis-cli command and ping like below:

step2b

Type exit to quit the redis-cli mode.

By following the above steps, Install Redis on all instances (both master and slave nodes).

Step 3 – Configure Redis Master Nodes

bind 172.31.50.50 127.0.0.1 ::1    # bind private-ip
protected-mode no

cluster-enabled yes

cluster-node-timeout 15000

Now, we will start the redis-server service and check whether it will give a running status. You can do all these steps simultaneously in all 3 servers we built in AWS.

step3

If you followed this blog, you would end up with something like below.

step3b

step3c

If you are successful up to this point, let’s build the cluster like below:

We are going to build a cluster using instance private IP,

This will ask you to confirm the sharding, and you must type “yes” for it.

If everything is correct, especially the configuration part, you will end up with a success message:

 

step3d

We can run the following command to check the master node status

step3e

If you are facing an issue in the above step, check the security group attached to the instance and verify that the ports below are opened 6379, 6378, 16379, and 16378.

Step 4 – Configure Redis Slave Nodes

In all the Redis slave Amazon EC2 instances, we already installed Redis, and now we have to make the following changes in Redis configuration.

Change the Redis configuration like below (port is the only change between master and cluster configuration)

You can check whether the slave is working properly by a simple ping-pong like using the below command.

step4

Step 5 – Adding slave with master

Here, we are adding slave-1 with master-1 by using the below command

Syntax:

step5

Do the same thing in the other 2 servers to add slaves.

Now check the cluster node status. You can see the master and slave connected below.

step5b

Now we will check the master slave replication. I am going to set the name in master-2 and retrieve the name in slave-2

step5c

step5d

Conclusion

By following the above steps, you have successfully established a Redis Cluster that offers high availability, fault tolerance, and the capability to handle increased workloads. This setup ensures that your Redis-based applications can deliver consistent and responsive services to your users while remaining resilient to potential disruptions.
Deploying a highly available Redis Cluster on Amazon EC2 with three master nodes and three slave nodes is a powerful way to ensure the reliability, scalability, and performance of your Redis-based applications. By distributing your data across multiple nodes and enabling replication, you can mitigate the risk of data loss and downtime in the event of node failures.

Drop a query if you have any questions regarding Redis Cluster on Amazon EC2 and we will get back to you quickly.

Making IT Networks Enterprise-ready – Cloud Management Services

  • Accelerated cloud migration
  • End-to-end view of the cloud environment
Get Started

About CloudThat

CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.

CloudThat is the first Indian Company to win the prestigious Microsoft Partner 2024 Award and is recognized as a top-tier partner with AWS and Microsoft, including the prestigious ‘Think Big’ partner award from AWS and the Microsoft Superstars FY 2023 award in Asia & India. Having trained 850k+ professionals in 600+ cloud certifications and completed 500+ consulting projects globally, CloudThat is an official AWS Advanced Consulting Partner, Microsoft Gold Partner, AWS Training PartnerAWS Migration PartnerAWS Data and Analytics PartnerAWS DevOps Competency PartnerAWS GenAI Competency PartnerAmazon QuickSight Service Delivery PartnerAmazon EKS Service Delivery Partner AWS Microsoft Workload PartnersAmazon EC2 Service Delivery PartnerAmazon ECS Service Delivery PartnerAWS Glue Service Delivery PartnerAmazon Redshift Service Delivery PartnerAWS Control Tower Service Delivery PartnerAWS WAF Service Delivery PartnerAmazon CloudFront Service Delivery PartnerAmazon OpenSearch Service Delivery PartnerAWS DMS Service Delivery PartnerAWS Systems Manager Service Delivery PartnerAmazon RDS Service Delivery PartnerAWS CloudFormation Service Delivery PartnerAWS ConfigAmazon EMR and many more.

FAQs

1. Why is Redis Cluster a better choice than a single Redis instance?

ANS: – Redis Cluster offers several benefits, including high availability, fault tolerance, and scalability. It distributes data across multiple nodes, allowing your application to continue functioning even if some nodes fail. It also enables horizontal scaling as data and traffic grow, making it a better choice for production environments.

2. How can I monitor the Redis Cluster's performance and health?

ANS: – You can leverage various monitoring tools like redis-cli, redis-stat, or dedicated Redis monitoring solutions. These tools provide insights into the cluster’s resource utilization and key performance metrics and help you promptly identify and address any issues.

3. After the initial setup, can I add more nodes to the Redis Cluster?

ANS: – Yes, Redis Cluster is designed to be scalable. You can easily expand your cluster by adding more nodes (both master and slave) to accommodate increased data storage and traffic. Redis automatically redistributes data to the new nodes, ensuring even load distribution.

WRITTEN BY Harikrishnan S

Harikrishnan Seetharaman is a Research Associate (DevOps) at CloudThat. He completed his Bachelor of Engineering degree in Electronics and Communication, and he achieved AWS solution architect-Associate certification. His area of interest is implementing a cloud-native solution for customers and helping them by proving robust and reliable solutions for their complex problems, DevOps, and SaaS. Apart from his professional interest he likes to spend time in farming and learning new DevOps tools.

Share

Comments

    Click to Comment

Get The Most Out Of Us

Our support doesn't end here. We have monthly newsletters, study guides, practice questions, and more to assist you in upgrading your cloud career. Subscribe to get them all!