Automate Instance Labelling by Creator in GCP: Simplify GCE Instance Management

Overview

Managing many instances efficiently in Google Compute Engine (GCE) and track who created it can be complex. However, by leveraging the powerful feature of auto-labeling based on the creator attribute, you can simplify instance management and enhance resource visibility in your cloud environment. This blog will explore auto-labeling in GCE and demonstrate how to automate the labeling process to categorize and track instances based on their creators. Discover the benefits of auto-labeling by the creator, learn implementation techniques, and uncover best practices for designing a logical labeling structure.

Pioneers in Cloud Consulting & Migration Services

Reduced infrastructural costs
Accelerated application deployment

Get Started

Introduction

In the world of cloud computing, managing a large number of instances efficiently is a common challenge.

Google Compute Engine (GCE) offers a powerful infrastructure for creating virtual machines (VMs) to run applications and services. To simplify instance management and improve resource tracking, GCE provides an auto-labeling feature, allowing you to assign labels to instances based on the creator automatically.

By implementing auto-labeling based on the creator, you can streamline instance organization, enhance resource visibility, and automate operations effectively. This blog will explore auto-labeling in GCE and demonstrate how to leverage this feature to simplify instance management based on the creator.

Architecture

arch

Steps to setup Auto-Labeler

Create a service account

Create a Role with the following permission:

compute.instances.get
compute.instances.setLabels

1 2	compute.instances.get compute.instances.setLabels

This role would be later attached to the service account of Cloud Function for labeling the GCE Instances.

step1

step1b

Create a service account with the previously created role assigned.

step1b

2. Create a Pub/Sub topic:

This Pub/Sub topic would be the destination of the Log Router. It will receive every filtered log, such as the instance creation logs.

step2

3. Create a Log Router:

Create a log Router that will filter the logs, for instance, creation and route it to the Pub/Sub topic created earlier.
Put the sink details like name, destination, etc.

step3

In the “Choose logs to include in sink”, fill this. Make sure to put your project ID.

logName="projects/<project_id>/logs/cloudaudit.googleapis.com%2Factivity"
protoPayload.methodName: "compute.instances.insert"

1 2	logName="projects/<project_id>/logs/cloudaudit.googleapis.com%2Factivity" protoPayload.methodName: "compute.instances.insert"

step3b

Then click on “Create Sink”

4. Create Cloud Function:

This Cloud Function would get triggered whenever there is any log of instance creation.
The trigger for the Cloud Function should be “Pub/Sub” and have the service account attached to it.
The codes for Cloud Function are:

package.json

{
	"name": "auto-labeller-function",
	"version": "1.0.0",
	"description": "Cloud Function for auto-labeller",
	"main": "index.js",
	"scripts": {
		"start": "node index.js"
	},
	"dependencies": {
		"@google-cloud/compute": "^3.9.1"
	}
}

{

"name": "auto-labeller-function",

"version": "1.0.0",

"description": "Cloud Function for auto-labeller",

"main": "index.js",

"scripts": {

"start": "node index.js"

"dependencies": {

"@google-cloud/compute": "^3.9.1"

}

index.js

const { labelComputeEngineInstance } = require('./gce');

exports.labelResource = async (event, context) => {
	const logData = JSON.parse(Buffer.from(event.data, 'base64').toString());
	console.log(JSON.stringify(logData));
	// const logData = event.data;
	if (
	String(logData.protoPayload.methodName).includes('compute.instances.insert')
	) {
	console.log('Labelling Compute Engine Instance...');
	await labelComputeEngineInstance(logData);
     } else {
	console.log(logData);
     }
};

const { labelComputeEngineInstance } = require('./gce');

exports.labelResource = async (event, context) => {

const logData = JSON.parse(Buffer.from(event.data, 'base64').toString());

console.log(JSON.stringify(logData));

// const logData = event.data;

if (

String(logData.protoPayload.methodName).includes('compute.instances.insert')

) {

console.log('Labelling Compute Engine Instance...');

await labelComputeEngineInstance(logData);

} else {

console.log(logData);

}

};

gcs.js

exports.labelComputeEngineInstance = async (log) => {
	// Start preparing the labels
	const resourceNameArray = String(log.protoPayload.resourceName).split('/');
	const projectId = resourceNameArray[1];
	const zone = resourceNameArray[3];
	const instanceName = resourceNameArray[5];
	const createdBy = String(
		log.protoPayload.authenticationInfo.principalEmail
	).replace(/[^a-z0-9_-]/g, '-');
	const labels = {
		'created-by': createdBy,
	};
	// End preparing the labels

	try {
		// Imports the Compute library
		const { InstancesClient } = require('@google-cloud/compute').v1;
		// Instantiates a client
		const computeClient = new InstancesClient();

		// Construct get request
		const getRequest = {
			instance: instanceName,
			project: projectId,
			zone,
		};

		// Run  get request
		const getResponse = await computeClient.get(getRequest);

		if (getResponse[0].labels['created-by']) {
			console.log("Already has 'created-by' label. Exiting...");
			return;
		}

		// Construct set label request
		const setLabelRequest = {
			instance: instanceName,
			project: projectId,
			zone,
			instancesSetLabelsRequestResource: {
				labelFingerprint: String(getResponse[0].labelFingerprint),
				labels: labels,
			},
		};

		// Run set label request
		const setLabelResponse = await computeClient.setLabels(setLabelRequest);
		console.log(JSON.stringify(labels));
		console.log(
			`Labels set for ${instanceName} instance in project ${projectId}`
		);
	} catch (error) {
		console.log(
			`Error setting lables to ${instanceName} VM in project ${projectId}.`
		);
		console.log(error);
	}
};

exports.labelComputeEngineInstance = async (log) => {

// Start preparing the labels

const resourceNameArray = String(log.protoPayload.resourceName).split('/');

const projectId = resourceNameArray[1];

const zone = resourceNameArray[3];

const instanceName = resourceNameArray[5];

const createdBy = String(

log.protoPayload.authenticationInfo.principalEmail

).replace(/[^a-z0-9_-]/g, '-');

const labels = {

'created-by': createdBy,

};

// End preparing the labels

try {

// Imports the Compute library

const { InstancesClient } = require('@google-cloud/compute').v1;

// Instantiates a client

const computeClient = new InstancesClient();

// Construct get request

const getRequest = {

instance: instanceName,

project: projectId,

zone,

};

// Run get request

const getResponse = await computeClient.get(getRequest);

if (getResponse[0].labels['created-by']) {

console.log("Already has 'created-by' label. Exiting...");

return;

}

// Construct set label request

const setLabelRequest = {

instance: instanceName,

project: projectId,

zone,

instancesSetLabelsRequestResource: {

labelFingerprint: String(getResponse[0].labelFingerprint),

labels: labels,

};

// Run set label request

const setLabelResponse = await computeClient.setLabels(setLabelRequest);

console.log(JSON.stringify(labels));

console.log(

`Labels set for ${instanceName} instance in project ${projectId}`

);

} catch (error) {

console.log(

`Error setting lables to ${instanceName} VM in project ${projectId}.`

);

console.log(error);

}

};

step4

step4b

step4c

step4d

5. After the Cloud Function is created, your auto-labeller setup is complete. Try creating any GCE Instance, and the label ‘created-by’ will be applied to the instance with the creator’s email ID value.

Conclusion

Auto-labeling instances based on the creator attribute in Google Compute Engine brings a significant advantage to instance management and resource tracking. By automating the labeling process, you can effortlessly organize your instances, gain better visibility into resource usage, and streamline operations specific to individual creators. Leveraging auto-labeling by the creator empowers you to optimize your cloud infrastructure, improve accountability, and enhance collaboration among teams or individuals responsible for instance creation. With a well-designed labeling structure and automated workflows, you can effectively manage your GCE instances, maximize resource utilization, and achieve operational excellence in your cloud environment. Embrace auto-labeling by the creator in GCE and unlock the potential to streamline instance management, simplify resource tracking, and drive efficiency in your cloud-based applications and services.

Making IT Networks Enterprise-ready – Cloud Management Services

Accelerated cloud migration
End-to-end view of the cloud environment

Get Started

About CloudThat

CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.

FAQs

1. What is GCE Instance Labelling in Google Cloud Platform (GCP)?

ANS: – GCE Instance Labelling in GCP is a feature that allows users to assign custom metadata labels to their Google Compute Engine (GCE) instances. These labels serve as key-value pairs that provide additional information and context about the instances.

2. Why is labeling GCE instances important for GCP users?

ANS: – Labelling GCE instances is important for GCP users because it enables better organization, management, and categorization of resources. With labels, users can easily identify and group instances based on specific criteria such as project, environment, owner, or purpose.

3. Are there any limitations or considerations when automating instance labeling in GCP?

ANS: – When automating instance labeling in GCP, it’s important to consider a few limitations. For example, label changes may not be immediately reflected in all GCP services or interfaces. It’s also crucial to ensure appropriate permissions and access controls are in place to prevent unauthorized modification of labels. Additionally, careful planning and testing are necessary to avoid unintended consequences or conflicts with existing labeling conventions.

WRITTEN BY Avinash Kumar

Avinash Kumar is a Senior Research Associate at CloudThat, specializing in Cloud Engineering, NodeJS development, and Google Cloud Platform. With his skills, he creates innovative solutions that meet the complex needs of today's digital landscape. He's dedicated to staying at the forefront of emerging cloud technologies.