Logentries Uses Docker for a New Logging Container Service – InApps Technology 2022

Logging and analytics service Logentries has launched a logging container service specifically built using new features made available through Docker’s 1.5 release, including the availability of the new Docker Stats API.

While Docker users can access Stats API endpoints directly, Logentries believes their new products, including the Logentries Container and a Community Pack of pre-configured (and customizable) searches, tags and visualizations, will make it easier for DevOps teams to more quickly orchestrate logging functionalities into their overall Docker application deployment environments.

“Our Docker Logentries Container will listen to all your other Docker containers,” says Trevor Parsons, Cofounder and Chief Scientist at Logentries. “There are things we want to do:

• Make logging easier in a production environment, and
• Take advantage of the Stats API to monitor and understand what is happening across your container environment.”

Parsons sees an increasing number of Docker users looking for logging solutions that go beyond just monitoring and troubleshooting, who also use the stats to provide deeper insights into an application’s use case.

“Very often people will log things like response time or value of a price plan, application performance, some timing activity, and then they will correlate that with CPU and memory to troubleshoot and understand the user experience of their application. Generally you need numerous sources of information: from the application and from the Docker containers themselves.

“But we are seeing more and more use of logs for analytical methods and for business metrics, to understand user activity and move beyond IT troubleshooter use cases. Application developers are wanting to keep track of what happens as the dynamics shift.”

The Frustration of Logging in Docker

The lack of logging capabilities is a frustration for many Docker users. At the end of 2013, Jérôme Petazzoni from Docker flagged the issue in a Google Groups discussion. He noted the need to find potential solutions to improve capturing more log sources and to improve how log entries are consumed. Since mid-July last year, the issue was moved to an open proposal on GitHub, with close to 20,000 interested in the issue, and over 1600 actively monitoring updates to the discussion of how best to log activity within Docker environments.

Just a couple of weeks ago, developer Brendan Jerwin sought answers on Server Fault to better log his Docker environment. “I’m setting up Neo4j in Docker for use in a CoreOS cluster. The container is running Neo4j console and I want all the logging to go to the console… so I can use systemd journaling to handle it from there.”

Six Million Ways to Log in Docker?

Parsons says the problem for developers like Jerwin is that “there are no standards and defined ways to log in Docker. We have come across a lot of people who are starting to use Docker in production and there are different approaches.” Parsons points to three different ways to log from Docker:

• Log from the Docker container itself
• Log from the host (an approach that isn’t really aligned with Docker’s containerization principles)
• Create a standardized Docker container that does the logging for you (the approach taken by Logentries, open source projects like Fluentd and other services like Loggly).

In December last year, Christian Beedgen, CTO and Cofounder of Sumo Logic — a cloud native log analytics service — shared six million ways to log in Docker (okay, actually ten are listed in the slides). Mirroring Parsons comment that there are no standards, Beedgen is seeing customers want to use more sophisticated logging practices to match a variety of use cases, including where more than one process is being run in a container, when customers want to write logs to stdout or when they are running Sumo Logic’s collector on the host, and logging straight from the application.

Beedgen explains: “So basically the trick is – how do you get to the data so that you can collect it? If you have a file-based collector, you need to first dump it to a file. This could be scripted, but the script would need to track every container coming and going, call the Stats API for each container, and write to file. It is doable, but a bit too involved in my mind for scripting.

“If your collector supports syslog, you could also netcat to a local syslog port that the collector is listening on. Not much better, but saves writing and tracking files.

“The better option is if the collection mechanism natively understands the Docker API. This is actually what I understand the Logentries approach to be. Basically, have a program [in their case it is Node.js based] that knows how to track containers as they come and go, and for each new container, call the Stats API, and then stream the results to the collection point.

“This approach also works for container logs emitted via stdout – this is essentially what Logspout does. Personally, I would just expand Logspout to be aware of the Stats API on top of the stdout API that it already supports.”

Beedgen is impressed with the availability of the Stats API in Docker 1.5 and is considering how to add its functionality to the Sumo Logic images already available on the Docker Hub. “We basically have to add the notion of a ‘Docker source’ to the collectors. We already have source types for local files, remote files, syslog, Windows event logs, etc – so the collector itself is modular in that sense. We ‘just’ need to add a Docker module that will basically talk to the Docker daemon, track the containers that are coming and going, attach to their stdout to shuttle that away as logs, and then use the Stats API to get the metrics, and finally, consume the Docker daemon event stream itself.”

It is this sort of hands-on configuration that Parsons from Logentries is confident will make their community package, with common tools and preconfigured visualizations and searches, appealing to Docker customers who want to be able to quickly apply logging capabilities in their Docker production environments. “Docker has allowed their system to become very dynamic with a lot of ephemeral servers that spin up and down very quickly. Having a centralized place to collect that information is becoming massively important to people.”

Image via Flickr Creative Commons.

InApps Technology is a wholly owned subsidiary of Insight Partners, an investor in the following companies mentioned in this article: Docker.