Unleashing the Power of GCP Stackdriver Logging & Monitoring

Introduction

In the dynamic world of cloud computing, Google Cloud Platform (GCP) offers a powerful duo: Stackdriver Logging and Monitoring. These robust tools provide essential capabilities to manage, monitor, and optimize your applications and infrastructure. This blog post will take a closer look at Stackdriver Logging and Monitoring, focusing on creating custom metrics. By diving into code snippets and practical examples, we’ll unveil how custom metrics can empower you to achieve unparalleled observability.

Pioneers in Cloud Consulting & Migration Services

Reduced infrastructural costs
Accelerated application deployment

Get Started

Understanding Stackdriver Logging

Stackdriver Logging consolidates logs from diverse sources, allowing you to collect, analyze, and store them effortlessly.
You can search, filter, and view logs from various GCP services and custom sources through a unified interface.
Code snippet for logging a message using the Stackdriver Logging client library in Python:

from google.cloud import logging
 	Client = logging.Client()
Logger = client.loggez( my-log')
 	logger. log_text( Hello, Stackdriver Logging!')

from google.cloud import logging

Client = logging.Client()

Logger = client.loggez( my-log')

logger. log_text( Hello, Stackdriver Logging!')

Importance of GCP Stackdriver Logging & Monitoring

GCP (Google Cloud Platform) Stackdriver Logging & Monitoring is a crucial component of the GCP ecosystem that provides important benefits for monitoring and troubleshooting applications and infrastructure deployed on Google Cloud.

Here are some key reasons why Stackdriver Logging & Monitoring is important:

Centralized Logging: Stackdriver Logging allows you to collect, search, analyze, and store logs from various services and resources across your Google Cloud environment. It provides a centralized location for all your logs, making monitoring and troubleshooting your applications and infrastructure easier.
Real-time Monitoring: Stackdriver Monitoring lets you monitor your applications and infrastructure’s performance and health in real-time. It offers various metrics, including CPU usage, memory usage, network traffic, and custom application metrics. With real-time monitoring, you can detect issues promptly and take appropriate actions to maintain system stability.
Alerting and Notification: Stackdriver Monitoring allows you to set up custom alerts based on predefined conditions or metrics thresholds. When an alert is triggered, you can receive notifications through various channels such as email and SMS or even trigger automated actions using Google Cloud Pub/Sub. This proactive alerting mechanism helps you address issues before they impact your users or services.
Dashboards and Visualization: Stackdriver Monitoring provides customizable dashboards and visualization tools that allow you to create intuitive and informative displays of your metrics and logs. You can build interactive charts, graphs, and widgets to gain insights into the performance and behavior of your applications. This helps you understand trends, identify anomalies, and make informed decisions based on data.
Diagnostics and Troubleshooting: Stackdriver Logging & Monitoring offers advanced diagnostic capabilities to investigate and troubleshoot issues. You can analyze logs and metrics together, correlate events, and identify patterns or anomalies that may indicate the root cause of problems. This helps reduce the time to diagnose and resolve issues, improving system reliability and minimizing downtime.
Integration with Google Cloud Services: Stackdriver Logging & Monitoring seamlessly integrates with other Google Cloud services, such as Google Kubernetes Engine (GKE), Compute Engine, and Cloud Functions. It provides native support for these services, enabling you to gain deeper insights and monitor their specific metrics and logs without additional configuration.

Exploring Stackdriver Monitoring

Stackdriver Monitoring complements Logging by providing comprehensive monitoring capabilities for your GCP resources.
It enables real-time visualization, analysis, and alerting of various metrics.
Dashboards offer a customizable view of your system’s health and performance.
Code snippet for creating a custom metric and submitting data points using the Stackdriver Monitoring API:

from google.cloud import monitoring v3
 	client = monitoring v3.MetricServiceClient()
project_name = client.project_path(“your-project-id")
 	descriptor = monitoring_v3.MetricDescriptor()
 	descriptor. type = ‘custom. googleapis. com/my_custom metric’
descriptor.metric_kind = monitoring v3.MetricDescriptor.Metrickind. GAUGE
descriptor.value type = monitoring v3.MetricDescriptor.valueType.DOUBLE
descriptor.description = ‘My-custom-metric’
descriptor = client.create_metric_descriptor(project_name, descriptor)
 	series = monitoring_v3.TimeSeries()
 	series.metric.type = ‘custom.googleapis.com/my_custom metric’
 	series. resource. type = ‘global’
 	point = series.points.add()
 	Point.value.double_value = 123.45
 	now = time. time()
 	point.interval.end_time.seconds = int (now)
 	point.interval.end_time.nanos = int((now - point.interval.end_time.seconds) * 10**9)
 	client.create_time_series(project name, [series])

from google.cloud import monitoring v3

client = monitoring v3.MetricServiceClient()

project_name = client.project_path(“your-project-id")

descriptor = monitoring_v3.MetricDescriptor()

descriptor. type = ‘custom. googleapis. com/my_custom metric’

descriptor.metric_kind = monitoring v3.MetricDescriptor.Metrickind. GAUGE

descriptor.value type = monitoring v3.MetricDescriptor.valueType.DOUBLE

descriptor.description = ‘My-custom-metric’

descriptor = client.create_metric_descriptor(project_name, descriptor)

series = monitoring_v3.TimeSeries()

series.metric.type = ‘custom.googleapis.com/my_custom metric’

series. resource. type = ‘global’

point = series.points.add()

Point.value.double_value = 123.45

now = time. time()

point.interval.end_time.seconds = int (now)

point.interval.end_time.nanos = int((now - point.interval.end_time.seconds) * 10**9)

client.create_time_series(project name, [series])

Creating Custom Metrics

Custom metrics offer flexibility to monitor specific aspects of your applications.
They allow you to track application-specific error rates, user engagement metrics, or custom resource utilization.
Code snippet for creating a custom metric in Prometheus format using the Stackdriver Monitoring API:

from google.cloud import monitoring_v3 
client = monitoring_v3.MetricServiceClient()
project_name = client.project_path(‘your-project-id') 
descriptor = monitoring_v3.MetricDescriptor()
descriptor. type = ‘custom. googleapis.com/my_prometheus_metric’
descriptor.metric_kind = monitoring_v3.MetricDescriptor.Metrickind. GAUGE
descriptor.value type = monitoring_v3.MetricDescriptor.valueType.DOUBLE
descriptor.description = ‘My custom Prometheus-style metric’
descriptor. labels. append( 
Monitoring_v3.LabelDescriptor(key=’label_key’, value type='STRING')
) 
descriptor = client.create_metric_descriptor(project_name, descriptor)

from google.cloud import monitoring_v3

client = monitoring_v3.MetricServiceClient()

project_name = client.project_path(‘your-project-id')

descriptor = monitoring_v3.MetricDescriptor()

descriptor. type = ‘custom. googleapis.com/my_prometheus_metric’

descriptor.metric_kind = monitoring_v3.MetricDescriptor.Metrickind. GAUGE

descriptor.value type = monitoring_v3.MetricDescriptor.valueType.DOUBLE

descriptor.description = ‘My custom Prometheus-style metric’

descriptor. labels. append(

Monitoring_v3.LabelDescriptor(key=’label_key’, value type='STRING')

)

descriptor = client.create_metric_descriptor(project_name, descriptor)

Conclusion

GCP’s Stackdriver Logging and Monitoring provide indispensable tools for efficient cloud management. Creating custom metrics allows you to gain deeper insights and tailor your monitoring strategy to meet specific needs.

With Stackdriver Logging, you can easily centralize and analyze logs, while Stackdriver Monitoring offers real-time visualization and analysis of metrics. By integrating code snippets and practical examples, we’ve highlighted the power of custom metrics in enhancing observability.

Empowering organizations to become ‘data driven’ enterprises with our Cloud experts.

Reduced infrastructure costs
Timely data-driven decisions

Get Started

About CloudThat

CloudThat is a leading provider of Cloud Training and Consulting services with a global presence in India, the USA, Asia, Europe, and Africa. Specializing in AWS, Microsoft Azure, GCP, VMware, Databricks, and more, the company serves mid-market and enterprise clients, offering comprehensive expertise in Cloud Migration, Data Platforms, DevOps, IoT, AI/ML, and more.

FAQs

1. How can I view and analyze logs using Stackdriver Logging?

ANS: – Stackdriver Logging provides a user-friendly interface to search, filter, and analyze logs from various sources. You can access the logs through the GCP Console, the Stackdriver Logging API, or client libraries. Additionally, you can create log-based metrics and alerts based on specific log entries or patterns to gain deeper insights into your applications.

2. Can I integrate Stackdriver Monitoring with my existing monitoring tools?

ANS: – Yes, Stackdriver Monitoring supports integration with popular monitoring tools such as Prometheus and Grafana. You can leverage their advanced visualization and alerting capabilities by exporting metrics from Stackdriver Monitoring to these tools. This enables you to have a unified monitoring solution that combines the strengths of Stackdriver with the flexibility of other monitoring ecosystems.

3. Is it possible to automate the creation of custom metrics in Stackdriver Monitoring?

ANS: – Yes, you can automate creating custom metrics using the Stackdriver Monitoring API. Using the client libraries or making direct API calls, you can programmatically create metric descriptors and define the metric type, value type, and associated labels. This allows you to dynamically create and manage custom metrics to suit your evolving monitoring requirements, streamlining the metric creation process across your applications and infrastructure.

WRITTEN BY Sahil Kumar

Sahil Kumar works as a Subject Matter Expert - Data and AI/ML at CloudThat. He is a certified Google Cloud Professional Data Engineer. He has a great enthusiasm for cloud computing and a strong desire to learn new technologies continuously.