Monitoring RabbitMQ: Native Nodes vs. Independent Agents

by Ayanda Dube

This blog post is an excerpt from RabbitMQ Operations & Maintenance using WombatOAM by Ayanda Dube. Download the full guide here.

RabbitMQ users are accustomed to monitoring and keeping track of the status of their RabbitMQ installations using the native RabbitMQ Management plugin, or, alternatively, using third party monitoring tools such as Sensu, which internally make use of the RabbitMQ Management API for metrics acquisition.

Regardless of which tool users prefer, a common aspect which cuts across most of these off-the-shelf tools is full dependency on the RabbitMQ Management plugin. In other words, for you to monitor and manage your RabbitMQ installation via a web interface, the RabbitMQ Management plugin has to be enabled at all times on the RabbitMQ nodes.

The downside of this approach lies mainly on the overhead the RabbitMQ Management plugin introduces per node on which it is enabled. The following image depicts the accompanying and required applications introduced on a node when the RabbitMQ Management plugin is enabled;

Fig. 1: Applications introduced per node when the RabbitMQ Management plugin is enabled

A total of 13 additional applications are required by the RabbitMQ Management plugin, which aren’t related to, or required to run any of the AMQP operations.

Internally, the RabbitMQ Management plugin creates multiple Erlang ETS tables, which are RAM based, in order to store, aggregate and compute statistics and various RabbitMQ specific node metrics. Unless the hardware has been dimensioned to take this into account, it can place a huge demand on the node’s memory, and could potentially contribute to a number of unknown side effects when traffic load patterns vary from peak to peak.

In an ideal world RabbitMQ nodes should be dedicated to delivery of the AMQP protocol i.e. queueing and message interchange logic between connected clients. All potential burdensome operations like UI monitoring and management should ideally be taken care of on a completely independant node; deployable on a separate physical machine from the RabbitMQ nodes, for all OAM functions. This is how the telecoms systems have addressed monitoring and operations for decades.

This inflexibility of the RabbitMQ Management plugin and other monitoring tools dependant on its API, bring to light the main strengths and advantages of using WombatOAM. Illustrated below, is the application overhead WombatOAM introduces on a RabbitMQ node;

Fig 2: Application overhead WombatOAM introduces on a RabbitMQ node

The WombatOAM approach to monitoring RabbitMQ

So what is different about the WombatOAM approach of monitoring RabbitMQ?

Read the rest of this blog post on www.erlang-solutions.com.

This blog post is an excerpt from RabbitMQ Operations & Maintenance using WombatOAM by Ayanda Dube. Download the full guide here.

Originally published at www.erlang-solutions.com.