Tutorial: collecting and analyzing Docker container logs with Sumo Logic (for free)

When it comes to powerful log analytics options for Docker, there are many commercial options out there, but they can be quite expensive. I’m going to share one which offers a free tier without compromising on the features.

Docker has a built-in logging driver called [json-file](https://docs.docker.com/config/containers/logging/json-file/) . Container logs are formatted as JSON and written to a text file per container. You use the [docker logs](https://docs.docker.com/engine/reference/commandline/logs/) command to view the logs as plain text. The only filtering options available are since and until. You may pipe the output to grep to do keyword search, same as with plain text file logs.

This is good enough when you want to view logs for a single container and debug issues ad-hock, right now. For anything more complex (e.g. parsing, filtering and sorting) you‘ll want a real log collection and analytics tool.

There are multiple options out there. Some are paid. Some are free and open source, but DIY. There is one, though, which combines the power and convenience of a commercial SaaS platform with a free entry level plan for small projects and hobbyists — Sumo Logic.

Sumo Logic offers a free plan with 0.5GB/day (15GB/mo) log data ingestion. If you are smart about what goes into your container logs and how often, you’ll be able to stretch the data allowance and get the power of a professional SaaS log collection and analytics platform for free.

Let’s dive in!

Setup

Sign-up for a free trial with Sumo Logic. You will have to use your “work” email address. Consumer mailboxes like Gmail won’t work.

Under Administration > Security > Access Keys click the + icon in the top right corner to add a new key:

Sumo Logic Access Keys screen

Pick the Access Key Label and click Generate Key. Copy and store your keys somewhere secure. You will use these when starting the Sumo Logic Docker Collector container.

Sumo Logic Access ID and Access Key screen

Start the collector container on your Docker host

$ docker run -d -v /var/run/docker.sock:/var/run/docker.sock --name=sumo-logic-collector --restart=always sumologic/collector:latest <AccessID> <AccessKey>

Replace <AccessID> and <AccessKey> with the values you recorded previously.

Give the collector a minute to initialize, then go to Manage Data > Collection to confirm it shows up in your account and logs data.

There will be two data sources streamed by the collector container:

• Docker-stats — CPU/memory/network/etc. container stats
• Docker-logs — actual container logs

If the collector does not show up in Sumo Logic, then check the collector container logs for clues:

$ docker logs sumologic-collector

Running SumoLogic Collector...

wrapper | --> Wrapper Started as Consolewrapper | Java Service Wrapper Standard Edition 64-bit 3.5.13wrapper | Copyright (C) 1999-2011 Tanuki Software, Ltd. All Rights Reserved.wrapper | http://wrapper.tanukisoftware.comwrapper | Licensed to Sumo Logic Inc. for Collectorwrapper |wrapper | Launching a JVM...jvm 1 | WrapperManager: Initializing...jvm 1 | . . . . . . . . .jvm 1 | .+'|=|`+. .+'| |`+. .+'|=|`+.=|`+. .+'|=|`+.jvm 1 | | | `+.| | | | | | | `+ | `+ | | | | |jvm 1 | | | . | | | | | | | | | | | | | |jvm 1 | `+.|=|`+. | | | | | | | | | | | | | |jvm 1 | . | | | | | | | | | | | | | | | |jvm 1 | |`+. | | | | | | | | | | | | | | | |jvm 1 | `+.|=|.+' `+.|=|.+' `+.| |.| |+' `+.|=|.+'jvm 1 | Sumo Logic Collector Version 19.209-26jvm 1 | Sumo Logic Build Hash fa2afe3jvm 1 | current folder:/opt/SumoCollectorjvm 1 | * See /opt/SumoCollector/./logs for more details.jvm 1 | * Connecting to https://collectors.sumologic.com.**jvm 1 | * ERROR: Registration failed: Your Sumo Logic credentials could not be verified. Make sure the token or accessKey/ID is valid and your user account has permissions to manage Collectors. (error key: collectors.unauthorized)**jvm 1 | Collector exiting...wrapper | <-- Wrapper Stopped

UTC timezone is used by default for logs timestamps. You may want to adjust that in the settings (Edit link) as necessary.

Switching collector timezone

See the official sumologic-collector-docker repo for additional configuration options and documentation.

Docker stats dashboard in Sumo Logic

Let’s start with some nice dashboards and graphs available out of the box.

Sumo Logic has a “Docker App” available in the App Catalog. Go ahead and add it to your Library.

Set the Source Category for logs to docker

Set the Source Category for logs to docker

Once the app is installed and some data is collected, you will see some nice graphs pulled for the containers stats.

Docker Overview dashboard

Docker CPU Performance dashboard

Searching and analyzing container logs

Now let’s see how we can get to the actual container logs in Sumo Logic.

Create a new Log Search using the “+ New” button in the top right (you can also use the alt+s keyboard shortcut for this)

Create a new Log Search

To only view container logs use the _source=Docker-logs filter. You can also narrow the search down by collector name, source host, etc. Sumo Logic will automatically suggest filter options and available values.

Search container logs

To view logs for a specific container — add the _sourcename filter, e.g.

_source=Docker-logsAND _sourcename = "plex-server"

To search for a specific log message or keyword, either add it manually in the search query as AND <keyword> filter or highlight it and use the options from the dropdown.

Filter logs by keyword

Easy! Now we are only viewing the specific messages we are interested in.

Filter logs by keyword: results

Once we got down to the instances of the messages we are interested in, we may want to see what else was captured in the logs around that time. Sumo Logic makes this extremely easy as well. Click on any of the filters under a message and select Surrounding Messages and a timeframe:

Finding surrounding messages for a particular log event

Surrounding log messages within a specific timeframe

There is so much more that can be done with Sumo Logic. Checkout the docs and video tutorials available on YouTube and in the app.

Learning materials in Sumo Logic

Disclaimer: I’m not affiliated in any way with Sumo Logic and did not receive any form of compensation from them or anyone else to write this article. I was looking for a solution for my own needs and found Sumo to fit it very nicely.

I hope you enjoyed reading and found this tutorial useful.Clap all 50 times if so! This helps others discover content on Medium.