A Guide to GCP Cloud Bigtable

Introduction

Cloud Bigtable is a fully managed, wide-column NoSQL database that offers single-digit millisecond latency and replication for high availability. It is best suited for massive analytical and operational workloads. You can store terabytes or even petabytes of data using Cloud Bigtable, a sparsely populated table that scales to billions of rows and thousands of columns. Every row has a single indexed value, which is referred to as the row key. Bigtable integrates easily with big data tools like Hadoop, Dataflow, and Dataproc and supports HBase API.

Components in Bigtable

Instances:  To make use of Bigtable, you first need to create an instance. Your data is contained in a Bigtable instance. A cluster can be found in an instance, and they can be in different zones or regions. Each cluster contains at least one node. A table is connected to the instance, not to the cluster or node.

An instance has a few more properties:

1. Storage type: – The storage, for instance, can be SSD(Solid-state drive) or HDD(hard disk drive). Make sure you select the appropriate storage type for your use case when choosing between SSD and HDD, as this decision is permanent, and each cluster in your instance needs to use the same kind of storage.
2. Node Scaling mode: – You can scale the nodes manually or automatically.
3. Cluster configuration: – You can define your cluster’s cluster ID, region, and zone.

Cluster: The Bigtable service is represented as a cluster in a particular place. A Bigtable instance can have clusters in up to eight regions, each associated with a single instance. One of the clusters in the instance handles requests sent by your application to a Bigtable instance. Every cluster is found in a specific zone. There can only be one cluster per zone in a region. For example, if an instance has a cluster in us-east1-b, you can create one more cluster in us-east1-c and one cluster in sone another region like asia-east1-a.

Replication is not used by Bigtable instances that have just one cluster. Bigtable automatically replicates your data when you add a second cluster to an instance. To do this, it maintains distinct copies of your data in each cluster zone and synchronizes updates between the copies.

Node: Each cluster in an instance has one or more nodes, which are compute resources that Bigtable uses to manage your data. Each node in a cluster handles the subset of requests to the cluster. Bigtable separates the compute from the storage. Data is never stored in a node. When a Bigtable node fails, no data is lost. Recovery from the failure of a Bigtable node is very fast and automatic. The cluster can be scaled out by adding nodes, increasing the number of simultaneous requests the cluster can handle. Bigtable has a limit of 1000 tables per instance.

Bigtable storage model

Each massively scalable table in Bigtable is a sorted key/value map that holds data. The table comprises columns with unique values for every row and rows that usually describe a single item. Typically, related columns are arranged together into column families. A column qualifier, a special name inside the column family, and the column family together identify each column. Bigtable tables are sparse, meaning a column takes up no space if not used in each row.

Steps to create a Bigtable instance:

1. Search for Bigtable service on the console and select

2. Select Create instance.

3. It’s a 3-step process. The first step is to name your instance. For this, give your name and ID to your instance and then click

4. The next step is to select your storage type. Here, select SSD and click on

5. The third step is to configure the cluster. Here, the cluster ID will come automatically. Select a region and zone as per your requirements.

6. Scroll down and define the node scaling mode to

7. For replication of the cluster, you can select the show advance option, select add a cluster, and define the replication region, AZ, for the replicated cluster. (Note: This step is optional, after step 6, you can directly follow step 8.)

8. Click on Create.

Bigtable Pricing example

Pricing example: Single cluster with one node

Assume you have a single cluster on your Bigtable instance with one node. Your application server is in the same region as Bigtable. In 30 days of the billing cycle, let’s consider you using the following Bigtable resources.

• One instance in us-east1 with a single cluster and one node throughout the month
• Average of 20 GB of data stored on SSD drives in us-east1 (South Carolina)
• No network ingress
• No network egress

Bigtable nodes cost

1 cluster * 1 node * 30 days * 24 hours/day * $0.65 per node per hour in us-east1: $468.00

Storage

1 cluster * 20 GB * $0.17 per GB in us-east1(South Carolina): $3.4

Network

No network ingress

No network egress

Monthly total

In this example, the total monthly bill for Bigtable is $471.4

About CloudThat