Simple Framework for Alerting anomalies, spikes and other patterns from data in elasticsearch. We use ElasticSearch, Log stash, Kibana for managing over increasing amount of data and logs, where kibana is great visualizing & querying data, but it need companion tool for alerting when inconsistencies in our data — where Elastalert is created. Overview — Reliable , Modular ,Ease Set Up & Configure. — in event of restart elastic search or in case of unavailable elasticsearch. Why is it Reliable? ElastAlert— (when it started & resume previously stopped). save its state to elasticsearch If Elasticsearch is , ElastAlert waits until it recovers before continuing. unresponsive ElastAlert (alerts) which throws Errors may be . automatically retried for Period of time Why is it Modular? — Main due to 3 Components. :  Responsible for Processing Data returned from elasticsearch(basically by initializing the rule configuration & parse data -return data by querying elasticsearch). Rule Types : Responsible for Taking action based on match. Alerts : Global Configuration file (config.yaml) — defines several aspects of its operation.(Ex: es_host,es_port,type) Configuration with two types of components, rule types and alerts. How it works? — by combining elasticsearch Elasticsearch is periodically queried and the data is passed to the rule type, which determines when a match is found. When a match occurs, it is given to one or more alerts, which take action based on the match. This is configured by a set of rules, each of which defines a query, a rule type, and a set of alerts. Several Rule types with common monitoring are included with ElastAlert: Match when there are X events in Y time — Frequency Match when the rate of events increase/decrease — Spike Match when there are less than X events in Y time — flatline Match when there a certain field matches a blacklist/whitelist — blacklist/whitelist Match on any event matching a given a filter — any Match when a field has two different values within some time — Change — Jira ,email, Google Chat,the Hive,Telegram,Slack… Support In Built are In Addition to this basic usage elastalert supports : Alert links to Kibana Dashboard Aggregate Counts for arbitrary fields Combine alerts into periodic reports Separate Alerts by a unique field Intercept/enhance match data. : Installation Clone Repo : git clone or pip install elastalert https://github.com/Yelp/elastalert.git See Requirements in Cloned Repo and install using pip. Elasticsearch Python3 python3 setup.py install Yes! we have installed ElastAlert. In the next part, we'll talk about configuring elasticsearch to elastalert and configuring built in type queries. In case of queries please feel free to comment if any issues!. Thanks for reading Read Part II Here

Google

Slack

Read My Stories

Too Long; Didn't Read

Get free API security automated scan in minutes

A 101 on ElastAlert & How To Set It Up

Too Long; Didn't Read

Companies Mentioned

@prabhu_r

RELATED STORIES