AWS Redshift Resize: A Guide to Scaling Your Data Warehouse

Introduction

Amazon Redshift is a massively parallel processing data warehousing solution that allows organisations to process petabyte-scale of data quickly and efficiently. As your data grows and business demands increase, you might need to scale your Redshift cluster to optimize performance and accommodate changing workloads. AWS provides Resize options to help you adjust your cluster size and type to better align with your performance, storage, and cost requirements. This comprehensive blog dives deep into Redshift Resize, exploring its types, use cases, procedures, and alternatives to resize.

What is AWS Redshift Resize?

Redshift Resize is the process of changing the node configuration (node type or number of nodes) of a Redshift cluster. You can use it to increase storage or performance, migrate to a different node type, consolidate or expand your cluster based on workload. Amazon Redshift offers two types of resizing techniques,

• Elastic resize
• Classic resize

Elastic Resize: Nodes in your cluster can be added or removed. Additionally, you can switch the node type, for example, from DC2 to RA3 nodes. The average time to accomplish an elastic resize is ten minutes. AWS advise it as a first choice because of resize. Data slices, or partitions that are allotted memory and disk space in each node, are redistributed when you do an elastic resize. An elastic resize operation, where you add or remove nodes of the same kind, has the following stages:

1. Elastic resize takes a cluster snapshot.
2. The operation moves cluster metadata. The cluster will be offline for a few minutes. The majority of queries are temporarily pause, while connections remain open. It is possible that some queries will be dropped. This is a short stage.
3. Session connections are restored, and queries resume.
4. Elastic resize redistributes data among node slices in the background. The cluster is open for read and write operations; however, some queries may take longer to run.

When you use elastic resize to alter the node type, it behaves similarly to adding or removing nodes of the same type. Initially, a snapshot is created. A new target cluster is created with the most recent data from the snapshot, and data is moved to it in the background. During this time, all data is read only. When the resizing is nearly complete, Amazon Redshift modifies the endpoint to link to the new cluster, and all connections to the source cluster are terminated.

Elastic resize has the following constraints:

1. When you add or remove nodes from a cluster that is a producer for data sharing, you cannot connect to it from consumers while Amazon Redshift migrates the cluster metadata.
2. Elastic resize has limits on the nodes that you can add to a cluster. For example, a dc2 cluster supports elastic resize up to double the number of nodes. Some ra3 node types allow you to increase the number of nodes by up to four times the current count. Assume that your cluster comprises of ra3.4xlarge or ra3.16xlarge nodes. You can then use elastic resize to boost the number of nodes in an 8-node cluster to 32.

The following table lists growth limits for each node type that supports elastic resize.

Classic resize : It solves scenarios where elastic resize does not support a change in cluster size or node type.  Classic resize takes more time to complete. When you do a classic resize, Amazon Redshift creates a target cluster and moves your data and metadata from the source cluster.

Step-by-Step: How to Resize a Redshift Cluster

1. 1. Elastic Resize
   1. Go to the Amazon Redshift Console.
   2. Choose your cluster > Actions > Resize cluster
   3. Select Elastic Resize
   4. Enter the new number of nodes
   5. Click Resize

   Takes a few minutes. You can monitor status from the “Cluster Status” page.

   1. Classic Resize (Node Type Change or Large Change)
   1. Go to the Redshift Console
   2. Select the cluster > Actions > Resize cluster
   3. Choose Classic Resize
   4. Pick the new node type and number of nodes
   5. Initiate resize

   Takes longer – Redshift creates a new cluster in the background.

   Alternatives to Resize

   • Concurrency Scaling: Adds temporary capacity during spikes.
   • Redshift Spectrum: Offload cold data to S3.
   • Materialized Views: Reduce query time without resizing.
   • Data Sharing: Share datasets across clusters without duplication.

References

https://repost.aws/knowledge-center/redshift-elastic-resize

https://docs.aws.amazon.com/redshift/latest/mgmt/managing-cluster-operations.html#classic-resize-faster

About CloudThat