Learn Installation and Configuration of Prometheus On a Linux Server

What Is Prometheus?

Prometheus is an open-source tool with a metrics-based monitoring system. It has a simple yet powerful data model and a query language that lets you analyze how your applications and infrastructure are performing. It is very lightweight and has a good alerting mechanism. It is a cloud-agnostic tool so we can use it with any cloud provider. Many organizations prefer this tool because of its no cost, wide capabilities, and integrations with many third-party tools. In this blog, I have explained how to implement it on an AWS EC2 instance.

Architecture

This diagram shows the architecture of Prometheus and certain elements of its ecosystem:

Here you can understand the connectivity of different components of Prometheus with other exporters and tools better. These are the most commonly used exporters here, but to view a list of other exporters which can be integrated by Prometheus, click here.

Prometheus scrapes metrics from instrumented jobs directly. It keeps all scraped samples locally in its time-series database (TSDB) and runs rules over this data to either query and record new time series from existing data or generate alerts. Grafana or other API consumers can be used to visualize the collected data using Prometheus as a data source.

Find below the server level architecture of Prometheus and Grafana to be set up in a cloud infrastructure for monitoring and alerting purposes.

Here we have used two different servers for Prometheus and Grafana for optimized performance, but you can install both on the same server as well. You can install exporters like CloudWatch exporter on different servers also depending upon your requirement, and you just have to add that target into the configuration file of Prometheus under the exporter’s job.

Install and Configure Prometheus

Here we will learn how the latest Prometheus version can be installed and configured on a Linux machine.

Prerequisites

Make sure you have sudo access to the Linux server because it requires high privileges for the commands used in this blog.
To download the Prometheus binary, the server must have access to the internet.
The inbound firewall rules must be opened to the server to access Prometheus port 9090.

Setup Prometheus Binaries

Step 1: To update the yum package repositories, use the below command.

sudo yum update -y

1	sudo yum update -y

Step 2: Go to the official release page of Prometheus, and get the Linux binary download link.

Step 3: Create a user for Prometheus and required directories for Prometheus files. Then assign ownership of these directories to Prometheus user.

sudo useradd --no-create-home --shell /bin/false prometheus
sudo mkdir /etc/prometheus
sudo mkdir /var/lib/prometheus
sudo chown prometheus:prometheus /etc/prometheus
sudo chown prometheus:prometheus /var/lib/prometheus

sudo useradd --no-create-home --shell /bin/false prometheus

sudo mkdir /etc/prometheus

sudo mkdir /var/lib/prometheus

sudo chown prometheus:prometheus /etc/prometheus

sudo chown prometheus:prometheus /var/lib/prometheus

Step 4: Using curl, download the Prometheus binary, un-tar it and change the extracted folder name to Prometheus-files.

curl -LO https://github.com/prometheus/prometheus/releases/download/v2.17.1/prometheus-2.17.1.linux-amd64.tar.gz
tar -xvf prometheus-2.17.1.linux-amd64.tar.gz
mv prometheus-2.17.1.linux-amd64 prometheus-files

curl -LO https://github.com/prometheus/prometheus/releases/download/v2.17.1/prometheus-2.17.1.linux-amd64.tar.gz

tar -xvf prometheus-2.17.1.linux-amd64.tar.gz

mv prometheus-2.17.1.linux-amd64 prometheus-files

Step 5: Now from Prometheus-files folder, copy Prometheus and promtool binary to /usr/local/bin path and assign the ownership of copied Prometheus file to Prometheus user.

sudo cp prometheus-files/prometheus /usr/local/bin/
sudo cp prometheus-files/promtool /usr/local/bin/
sudo chown prometheus:prometheus /usr/local/bin/prometheus

sudo cp prometheus-files/prometheus /usr/local/bin/

sudo cp prometheus-files/promtool /usr/local/bin/

sudo chown prometheus:prometheus /usr/local/bin/prometheus

Step 6: From Prometheus-files, move the consoles and console_libraries directories to /etc/prometheus folder and change the ownership of these directories as well to Prometheus user.

sudo cp -r prometheus-files/consoles /etc/prometheus
sudo cp -r prometheus-files/console_libraries /etc/prometheus
sudo chown -R prometheus:prometheus /etc/prometheus/consoles
sudo chown -R prometheus:prometheus /etc/prometheus/console_libraries

sudo cp -r prometheus-files/consoles /etc/prometheus

sudo cp -r prometheus-files/console_libraries /etc/prometheus

sudo chown -R prometheus:prometheus /etc/prometheus/consoles

sudo chown -R prometheus:prometheus /etc/prometheus/console_libraries

Setup Prometheus Configuration

As we know that all the configuration files are in /etc directory in Linux. Therefore, all the Prometheus configurations should be present in /etc/prometheus/prometheus.yml file.

Step 1: Create the configuration file as prometheus.yml and open in the editor.

sudo vi /etc/prometheus/prometheus.yml

1	sudo vi /etc/prometheus/prometheus.yml

Step 2: Copy the following contents to the prometheus.yml file. Here, we are creating a job name as Prometheus which will scrape metrics of Prometheus from the localhost. Then save the file.

global:
  scrape_interval: 10s

scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 5s
    static_configs:
      - targets: ['localhost:9090']

global:

scrape_interval: 10s

scrape_configs:

- job_name: 'prometheus'

scrape_interval: 5s

static_configs:

- targets: ['localhost:9090']

Step 3: Now, change the ownership of the prometheus.yml to Prometheus user.

sudo chown prometheus:prometheus /etc/prometheus/prometheus.yml

1	sudo chown prometheus:prometheus /etc/prometheus/prometheus.yml

Setup Prometheus Service File

Step 1: Create a Prometheus service file.

sudo vi /etc/systemd/system/prometheus.service

1	sudo vi /etc/systemd/system/prometheus.service

Step 2: Edit the service file and add the below content in it.

[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target

 [Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus \
    --config.file /etc/prometheus/prometheus.yml \
    --storage.tsdb.path /var/lib/prometheus/ \
    --web.console.templates=/etc/prometheus/consoles \
    --web.console.libraries=/etc/prometheus/console_libraries

 [Install]
WantedBy=multi-user.target

[Unit]

Description=Prometheus

Wants=network-online.target

After=network-online.target

[Service]

User=prometheus

Group=prometheus

Type=simple

ExecStart=/usr/local/bin/prometheus \

--config.file /etc/prometheus/prometheus.yml \

--storage.tsdb.path /var/lib/prometheus/ \

--web.console.templates=/etc/prometheus/consoles \

--web.console.libraries=/etc/prometheus/console_libraries

[Install]

WantedBy=multi-user.target

Step 3: Register the Prometheus service be reloading the system service then start the Prometheus service.

sudo systemctl daemon-reload
sudo systemctl start prometheus

1 2	sudo systemctl daemon-reload sudo systemctl start prometheus

Confirm if Prometheus service status is active using the below command.

sudo systemctl status prometheus

1	sudo systemctl status prometheus

The output should be as below.

Access Prometheus Web UI

You can now connect the Prometheus UI on the Prometheus server IP on port 9090.

http://<prometheus-ip>:9090/graph

1	http://<prometheus-ip>:9090/graph

The following UI should be visible as shown below.

We only have installed Prometheus for now. The next step will be to register the targets for the exporters which you are going to use in the prometheus.yml file to get the metrics from the source systems.

If you want to learn more about Prometheus go to our website and check more courses on DevOps.

Please comment if you have any questions.

WRITTEN BY Saurabh Jain

Saurabh Kumar Jain is a Subject Matter Expert working with CloudThat. He has experience in AWS, Microsoft Azure, and DevOps technologies. He is specialized in cloud security and architecture design. He also holds experience on various cloud and DevOps projects based on the cloud maturity model. He is an AWS Certified Security Speciality, AWS Certified Solutions Architect - Associate, Microsoft certified Azure Administrator and a certified Terraform Associate.