Network & Services monitoring with Prometheus

If you have already installed both Prometheus & Grafana you are ready to start creating data collectors and utilising them to store status of your systems.

Scraping of ICMP-Ping statuses

Now, that both the Prometheus and Blackbox_exporter are running, we can instruct prometheus to nudge the blackbox_exporter for Pings of devices we are interested in.

Everything around prometheus (not grafana) is configured through config files, so.
We need to open up the prometheus/prometheus.yml file and add a new job at the end of file:

    - job_name: "Ping statuses"
        metrics_path: /probe
        params:
            module: [icmp]
        static_configs:
            - targets:
                - 192.168.1.1
                - 192.168.1.2
        relabel_configs:
            - source_labels: [__address__]
                target_label: __param_target
            - source_labels: [__param_target]
                target_label: instance
            - target_label: __address__
                replacement: 127.0.0.1:9115 # This is your blackbox exporter.

Now if we save this file and restart prometheus, we can see in Status>Targets, that the blackbox_exporter is pinging the devices (this menu does not show ping status. It shows status of execution of blackbox_exporter jobs).

If we click on the devices address in the Status>Targets menu, we can view latest output of the exporter and get to know all the data that is available to query.

These are the basics, you are now ready to explore other exporters or read about #Data queries or go straight to Creating Dashboards

Data queries

Everything we are going to get from data collected in Prometheus has to be queried with it's own query 'language' called PromQL. As stated previously, variables which we can ask Prometheus for, can be found through Status>Targets.
The functions we can use to manipulate the queried data can be learned from Guides in Prometheus docs