Build A Monitoring Dashboard with Grafana and Prometheus

👨‍💼 Nowadays we most of us are working of lot many important things such as: building the infrastructure of cloud for an organization and then deploying multi tier micro services and applications and so on. This looks great when you see application or website running on your Phone or PC browsers, right ? Having said that what if when browser doesn’t open the site or load it very slow ?

Certainly it Needs attention for admins. So , it is equally important to monitor your applications and set alerting in case anything goes wrong. One of such Open source tool which performs Analysis or logs , errors from various sources and centrally manages monitoring of all of it & notifying to admins is GRAFANA 📊

The combination of Prometheus and Grafana is becoming a more and more common monitoring stack used by DevOps teams for storing and visualizing time series data. Prometheus acts as the storage backend and Grafana as the interface for analysis and visualization.

Table of Content

  1. What is Grafana?
  2. How to Install Grafana on Ubuntu Machine
  3. How to Log in to Grafana?
  4. What is Prometheus?
  5. How to Install Prometheus?
  6. What is node exporter ?
  7. How to install node exporter?
  8. Configure a Prometheus Monitoring Server with a Grafana Dashboard

What is Grafana?

Its an open source tool which allows you to query , visualize , analyze and alert on metrics and logs no matter where they are stored. Grafana comes bundled with rich support for many time series databases like Graphite , Prometheus, Elasticsearch , Influx DB. It also supports cloud monitoring vendors such as Amazon cloud watch , Microsoft Azure and SQL Db’s such as Postgres and MySQL. It is written in GO Language.

How to Install Grafana on Ubuntu Machine ?

  • Download the Grafana GPG Key
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -    # Adds the key to your Apt installation's list of trusted key which allows you to download and verify GPG Signed Packages. 
  • Add Grafana repository to your APT Sources
sudo add-apt-repository "deb https://packages.grafana.com/oss/deb stable main"
  • Refresh your APT Cache to update your package list
sudo apt update
  • Install Grafana package
sudo apt install grafana
  • Start the Grafana server and verify the service status
sudo systemctl start grafana-server

How to Log in to Grafana?

  1. Open your browser and enter  localhost:3000 ( IF you installed on your EC2 instance type ec2_ip:3000 )
  2. In email or username, enter admin.
  3. In password, enter admin.
  4. Click Log In.

What is Prometheus ?

Prometheus is a powerful, open-source monitoring system that collects metrics from services and stores them in a time series database. It records real-time metrics and alerts. It is written in Go Language. It allows powerful queries and great visualization. Prometheus works very well with Grafana when it comes to Dashboards and alerting notifications.

How to Install Prometheus on Ubuntu machine ?

  • Add users one for prometheus and other for node_exporter
sudo useradd --no-create-home --shell /bin/false prometheus

sudo useradd --no-create-home --shell /bin/false node_exporter
  • Create two directories which will be used later by Prometheus
sudo mkdir /etc/prometheus   # Some libraries will be copied in this directory

sudo mkdir /var/lib/prometheus   # logs will be stored here
  • Set user and group permissions on new directories
sudo chown prometheus:prometheus /etc/prometheus

sudo chown prometheus:prometheus /var/lib/prometheus
  • Downloading Prometheus and Generate checksum in home directory
cd ~

curl -LO https://github.com/prometheus/prometheus/releases/download/v2.0.0/prometheus-2.0.0.linux-amd64.tar.gz

sha256sum prometheus-2.0.0.linux-amd64.tar.gz
  • unpack the downloaded archive and copy to /usr/local/bin so that it can run from this location.
tar xvf prometheus-2.0.0.linux-amd64.tar.gz
  • Extraction will generate two binaries in the directory i.e prometheus and promtool
sudo cp prometheus-2.0.0.linux-amd64/prometheus /usr/local/bin/
sudo cp prometheus-2.0.0.linux-amd64/promtool /usr/local/bin/
  • Set user and group permissions on copied directories
sudo chown prometheus:prometheus /usr/local/bin/prometheus
sudo chown prometheus:prometheus /usr/local/bin/promtool
  • Copy both the files consoles and console_libraries under /etc directory (consoles and console_libraries contains the web interface, configuration files examples and the license)
sudo cp -r prometheus-2.0.0.linux-amd64/consoles /etc/prometheus
sudo cp -r prometheus-2.0.0.linux-amd64/console_libraries /etc/prometheus
  • Set user and group permissions on copied directories
sudo chown -R prometheus:prometheus /etc/prometheus/consoles
sudo chown -R prometheus:prometheus /etc/prometheus/console_libraries
  • Remove the leftover files from your home directory as they are no longer needed.
rm -rf prometheus-2.0.0.linux-amd64.tar.gz prometheus-2.0.0.linux-amd64
  • Now update the config file of the prometheus
sudo vi /etc/prometheus/prometheus.yml
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'prometheus'
    scrape_interval: 5s
    static_configs:
      - targets: ['localhost:9090']
  • Now edit the permission of config file
sudo chown prometheus:prometheus /etc/prometheus/prometheus.yml

We should be set now with all files and permissions , lets Lets Start the Prometheus by running the command below

sudo -u prometheus /usr/local/bin/prometheus \
--config.file /etc/prometheus/prometheus.yml \
--storage.tsdb.path /var/lib/prometheus/ \
--web.console.templates=/etc/prometheus/consoles \
--web.console.libraries=/etc/prometheus/console_libraries
  • Verify if Prometheus is up and running by visiting your browser and type IP followed by 3000 as port

What is Node exporter ?

Prometheus has capability to collect metrics but it can extend it capabilities by adding Node Exporter. Node exporter can collect system information such as CPU, Disk, and memory utilization.

How to install node exporter?

  • Download the latest version of Node Exporter and unpack the achieve
wget https://github.com/prometheus/node_exporter/releases/download/v0.16.0/node_exporter-0.16.0.linux-amd64.tar.gz
tar xvf node_exporter-0.16.0.linux-amd64.tar.gz
  • Copy the Binary file and set the ownership to the user.
sudo cp node_exporter-0.16.0.linux-amd64/node_exporter /usr/local/bin
sudo chown node_exporter:node_exporter /usr/local/bin/node_exporter
  • Create Node exporter service
sudo vi /etc/systemd/system/node_exporter.service
  • Create Node exporter service
[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

[Service]
User=node_exporter
Group=node_exporter
Type=simple
ExecStart=/usr/local/bin/node_exporter

[Install]
WantedBy=multi-user.target
  • . Reload Systemd to use the newly defined service:
sudo systemctl daemon-reload
  • Run Node Exporter by typing the following command:
sudo systemctl start node_exporter
sudo systemctl enable node_exporter
  • Verify the node exporter by again browsing the IP address followed by 9100

Configuring a Prometheus Monitoring Server with a Grafana Dashboard

  • Prerequisite:
    • Install Prometheus, Grafana and Node exporter which we already installed earlier in this tutorial.
  • Change the Prometheus configuration: Include the node exporter this time by making changes in /etc/prometheus/prometheus.yml
global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'prometheus'              # Prometheus will scrape 
    scrape_interval: 5s
    static_configs:
      - targets: ['localhost:9090']
  - job_name: 'node_exporter_metrics'  # we are including Node Expoter
    scrape_interval: 5s
    static_configs:
      - targets: ['localhost:9100']
  • Start Prometheus by running the command below
sudo -u prometheus /usr/local/bin/prometheus --config.file /etc/prometheus/prometheus.yml --storage.tsdb.path /var/lib/prometheus/ --web.console.templates=/etc/prometheus/consoles --web.console.libraries=/etc/prometheus/console_libraries
You can collect data and CPU , memory stats as node exporter is also installed.
  • Now you are connected to Prometheus , it will begin collecting data such as CPU , memory utilization.

Great , now we can see Grafana Dashboard is showing us required data visualization which Prometheus node exporter collected .

Conclusion

In this tutorial, we were able to Install all components such as Prometheus , Grafana , Node exporter and configured a Prometheus server with node exporter data collectors that are scraped by our Prometheus server which provides the data to build Dashboards with Grafana. Hope this Tutorial will be helpful and please go though it again.

Please Like and share with your friends

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s