Crush Your Competition with Prometheus at Your Side!

As software engineers, we want to design and develop robust applications where everything is under control. How we can be sure that everything is under control and that our application is functioning in its perfect state? One of the sources of the truth here is metrics that we can gather within the application.

Measuring and monitoring the metrics of an application is fundamental for optimizing performance, ensuring reliability, planning for future growth, detecting and responding to issues, gaining insights into user behaviour and business performance, ensuring compliance and security, and driving continuous improvement. By monitoring and analyzing relevant metrics, teams can effectively manage and maintain their applications to deliver value to users and stakeholders.

In this article, I’m discovering to you very powerful tool when it comes to gathering metrics — Prometheus. Prometheus, the renowned open-source monitoring solution, provides a robust metric system integral for gathering insights into the performance and health of applications. We'll explore the different types of metrics in Prometheus with examples of use cases for each of them to give you an idea of what metrics you can gather and also what type to use for your use cases.

Prometheus metrics come in various types, each tailored to specific monitoring needs. Choosing the right metric type is decisive for gaining meaningful insights into the performance and behaviour of your system.

By understanding the characteristics of counters, gauges, histograms, summaries, and untyped metrics, you can design a comprehensive monitoring strategy.

Counter Metrics

Counters represent cumulative values that only increase over time.

package main

import (
	"net/http"
	"github.com/prometheus/client_golang/prometheus"
	"github.com/prometheus/client_golang/prometheus/promhttp"
)

var requestCount = prometheus.NewCounterVec(
	prometheus.CounterOpts{
		Name: "http_requests_total",
		Help: "Total number of HTTP requests.",
	},
	[]string{"method"},
)

func main() {
	prometheus.MustRegister(requestCount)

	http.Handle("/metrics", promhttp.Handler())
	http.HandleFunc("/", handler)

	http.ListenAndServe(":8080", nil)
}

func handler(w http.ResponseWriter, r *http.Request) {
	requestCount.WithLabelValues(r.Method).Inc()
	w.Write([]byte("Hello, world!"))
}

Use Cases:

• Tracking the total number of items processed.
• Monitoring request counts: Counters are ideal for tracking the number of requests a service receives.
• Measuring resource utilization: Counters can represent the total amount of resources consumed over time.

Gauge Metrics

Gauges represent instantaneous values that can increase or decrease.

package main

import (
	"net/http"
	"github.com/prometheus/client_golang/prometheus"
	"github.com/prometheus/client_golang/prometheus/promhttp"
)

var activeUsers = prometheus.NewGauge(
	prometheus.GaugeOpts{
		Name: "active_users",
		Help: "Number of active users.",
	},
)

func main() {
	prometheus.MustRegister(activeUsers)

	http.Handle("/metrics", promhttp.Handler())
	http.HandleFunc("/", handler)

	http.ListenAndServe(":8080", nil)
}

func handler(w http.ResponseWriter, r *http.Request) {
	// Simulate user activity
	activeUsers.Inc()
	w.Write([]byte("Hello, world!"))
}

Use Cases:

• Tracking system resource levels: Gauges are suitable for monitoring variables like CPU usage or memory consumption.
• Reporting current state: Gauges can indicate the real-time status of various components, such as the number of active connections.
• Tracking the number of active users.

Histogram Metrics

Histograms provide a way to track the distribution of observed values over time.

package main

import (
	"net/http"
	"github.com/prometheus/client_golang/prometheus"
	"github.com/prometheus/client_golang/prometheus/promhttp"
)

var requestDuration = prometheus.NewHistogram(
	prometheus.HistogramOpts{
		Name:    "http_request_duration_seconds",
		Help:    "Histogram of request durations.",
		Buckets: prometheus.DefBuckets,
	},
)

func main() {
	prometheus.MustRegister(requestDuration)

	http.Handle("/metrics", promhttp.Handler())
	http.HandleFunc("/", handler)

	http.ListenAndServe(":8080", nil)
}

func handler(w http.ResponseWriter, r *http.Request) {
	start := time.Now()

	// Simulate processing time
	time.Sleep(100 * time.Millisecond)

	duration := time.Since(start).Seconds()
	requestDuration.Observe(duration)

	w.Write([]byte("Hello, world!"))
}

Use Cases:

• Latency analysis: Histograms can help analyze the time it takes to complete operations, revealing performance outliers.
• Response time profiling: Understanding the distribution of response times aids in identifying and resolving performance bottlenecks.

Summary Metrics

Similar to histograms, summaries provide quantiles, but with a predefined set of quantiles (e.g., 0.5, 0.9, 0.99).

package main

import (
	"net/http"
	"github.com/prometheus/client_golang/prometheus"
	"github.com/prometheus/client_golang/prometheus/promhttp"
)

var responseTimeSummary = prometheus.NewSummaryVec(
	prometheus.SummaryOpts{
		Name:       "http_response_time_seconds",
		Help:       "Summary of response times.",
		Objectives: map[float64]float64{0.5: 0.05, 0.9: 0.01, 0.99: 0.001},
	},
	[]string{"endpoint"},
)

func main() {
	prometheus.MustRegister(responseTimeSummary)

	http.Handle("/metrics", promhttp.Handler())
	http.HandleFunc("/", handler)

	http.ListenAndServe(":8080", nil)
}

func handler(w http.ResponseWriter, r *http.Request) {
	start := time.Now()

	// Simulate processing time
	time.Sleep(100 * time.Millisecond)

	duration := time.Since(start).Seconds()
	responseTimeSummary.WithLabelValues("/").Observe(duration)

	w.Write([]byte("Hello, world!"))
}

Use Cases:

• Monitoring percentiles: Summaries are useful for assessing the performance of critical operations by examining different percentiles.
• Identifying outliers: Tracking quantiles helps pinpoint outliers and anomalies in the system.

Untyped Metrics

Untyped metrics represent a single value without any specific constraints on its behaviour.

package main

import (
	"net/http"
	"github.com/prometheus/client_golang/prometheus"
	"github.com/prometheus/client_golang/prometheus/promhttp"
)

var customMetric = prometheus.NewUntyped(
	prometheus.UntypedOpts{
		Name: "custom_metric",
		Help: "A custom, unconstrained metric.",
	},
)

func main() {
	prometheus.MustRegister(customMetric)

	http.Handle("/metrics", promhttp.Handler())
	http.HandleFunc("/", handler)

	http.ListenAndServe(":8080", nil)
}

func handler(w http.ResponseWriter, r *http.Request) {
	// Simulate some custom metric value
	customMetric.Set(42)

	w.Write([]byte("Hello, world!"))
}

Use Cases:

• Miscellaneous metrics: Untyped metrics are versatile and can be used for any situation where a single, unconstrained value needs to be monitored.
• Custom metrics: For situations where none of the other metric types fit, untyped metrics provide a flexible option.

In my opinion, Prometheus is an excellent choice for monitoring modern, distributed systems, offering scalability, flexibility, and robust alerting capabilities and is a must-have tool in every software/DevOps engineer skill set. The extensive and dynamic community of Prometheus enthusiasts significantly enhances its appeal as a preferred tool for monitoring and managing complex infrastructures.