Docker Logs: Guide to Managing Docker Container Logs

Introduction

Logging helps users identify and track patterns in system behavior, troubleshoot issues, and fix bugs more efficiently. Docker offers built-in logging features and solutions for effective log management when working with containerized applications.

This article introduces Docker container logs, the associated Docker CLI commands, and best practices for developers.

About Docker Logs

Docker logs provide information about the processes inside running containers. Read the following sections to learn about the essential concepts of Docker logging.

What Are Docker Container Logs?

Container logs are records of events and output generated by Docker containers on runtime. Each log captures stdout and stderr streams of a single container, offering valuable insights into the container's behavior, performance, and potential issues.

By default, Docker logs are JSON files located in the /var/lib/docker/containers directory of the host system. Each file follows the template below, replacing [container_id] with the ID of the relevant container:

[container_id]-json.log

A Docker log features the following elements:

• Messages, warnings, and errors generated by the application running inside the container.
• Container OS data, such as kernel messages and system errors.
• CPU, memory, and disk usage metrics.
• Container lifecycle data, including start and stop times, configuration details, and errors.
• Information on network requests and responses, such as IP addresses, ports, and connection status.
• Unauthorized access attempts, security vulnerabilities, and other security events.

What Is Logging Driver?

A logging driver is a tool for capturing, storing, and processing Docker container logs. The most common logging drivers are:

• json-file (default). Writes logs in JSON format and stores them locally on the host machine.
• syslog. Sends logs to a syslog server, supporting centralized log management and analysis.
• journald. Writes logs to the systemd journal.
• gelf. Offers advanced log analysis and visualization by forwarding Graylog Extended Log Format (GELF) logs to a Graylog server.
• fluentd. Sends logs to a Fluentd agent for flexible log processing and plugin-based routing.

How to Configure Logging Driver

Users can configure logging drivers on the Docker daemon level and the container level.

The daemon-level type of configuration ensures all the containers on the host use the same logging driver. The configuration is performed by editing the /etc/docker/daemon.json file.

For example, follow the steps below to change the driver from json-file to syslog:

1. Create or open daemon.json with a text editor such as Nano:

sudo nano /etc/docker/daemon.json

2. Add the following lines to the file:

{
  "log-driver": "syslog",
  "log-opts": {
    "syslog-address": "udp://localhost:514"
  }
}

3. Save the file and exit.

4. Restart the Docker daemon:

sudo systemctl restart docker

Setting up the logging driver on the container level involves using the docker run command to override the default logging driver and specify driver-related options. This action can be achieved by using the --log-driver and --log-opt options, as in the example below:

docker run -it --log-driver=syslog --log-opt syslog-address=udp://localhost:514 [container_image]

Where Are Docker Container Logs Stored?

During the container operation, Docker forwards all output and error messages to a file, a log collector, or a log management service. For example, the default json-file driver stores logs in the following host-system directory:

/var/lib/docker/containers/[container_id]/[container_id]-json.log

Replace [container_id] in the path above with the actual ID of the container.

Why View Docker Logs

Docker logs are necessary for understanding and troubleshooting the behavior of containerized applications. By analyzing logs, users can perform the following key actions:

• Identify errors, debug specific issues, and trace application execution flow.
• Gain insights into resource usage, response times, and other performance metrics.
• Detect suspicious activity, unauthorized access attempts, and security vulnerabilities.
• Demonstrate compliance with security standards and regulations.

Docker Command for Checking Container Logs

The docker logs command instructs Docker to fetch the logs for a running container at execution time. It works only with containers utilizing the json-file or journald logging driver.

The command syntax for retrieving container logs is:

docker logs [option] [container_name_or_id]

Replace [container_name_or_id] with the name or ID number of the container you want to inspect. To find the necessary information, use the following command to list running containers:

docker ps

The container IDs and names of the running containers are in their respective columns.

In the example above, Docker is running an Apache httpd container named apache-web. To display the container's logs, run:

docker logs apache-web

Alternatively, use the container ID:

docker logs 85532d36367f

The output lists the event logs for the container.

docker logs Command Options

Docker allows adding options to the docker logs command to customize its output. The available command options include:

Command Option	Description
--details	Display additional log details.
--follow, -f	Follow log output.
--since	Display logs created after a specified timestamp (e.g., 2022-05-06T14:48:33Z). The option also accepts durations (e.g., 20m).
--tail, -n	Specify the number of lines to show, starting from the last line in the logs.
--timestamps, -t	Show timestamps.
--until	Display logs created before a specified timestamp.

Docker Logging Best Practices

Docker provides multiple logging mechanisms for tracking and managing logs, some of which are built-in and set up by default. The following recommended approaches and best practices assist in keeping container logs accurate, up-to-date, and secure.

Choose Docker Logging Driver

Docker provides built-in logging drivers within containers to serve as log management systems. The drivers read the container output (the data broadcast by the stdout and stderr streams), format the logs, and store them in a file on the host machine or a defined endpoint.

The type of driver determines the log format and storage location. By default, Docker uses the json-file driver, which writes JSON-formatted logs on the host machine. Use other built-in drivers to forward records to logging services, log shippers, or centralized management services.

Docker also allows the creation of a custom logging driver. Users can add custom drivers as plugins to containers and even distribute them through a Docker registry.

Select Delivery Mode

Delivery modes determine how Docker prioritizes and delivers messages from the container to the log driver.

Docker has two delivery modes:

• Direct, blocking delivery. Blocking mode sends all messages directly to the driver, interrupting the application when a new message occurs. This option ensures Docker immediately logs all output, but it can affect application performance and cause latency if the logging driver requires more time to block.
• Non-blocking delivery. Non-blocking is a delivery mode with an intermediate ring buffer within the container where logs are stored until the logging driver processes them. If the logging driver is busy, the logs are in the in-memory ring buffer and passed on only when the driver is ready to process them.

Docker utilizes the blocking delivery mode by default. To log messages in the non-blocking mode, add the mode=non-blocking attribute to the docker run command when spinning up the container.

Logging via Application

Application-based Docker logging involves managing and analyzing logging events using the application's framework. In this type of logging, Docker stores logs inside the application's containers.

The main advantage of logging via application is that developers gain more control over the logging events. On the other hand, saving logs inside a container is not recommended because of its transient nature. If a container shuts down, its filesystem is destroyed.

Configure persistent storage or transmit logs to a remote management solution to ensure persistency.

Logging Using Data Volumes

Keeping Docker logs within a container may result in data loss if the container shuts down. One way of solving this issue is to use data volumes as persistent storage for log events.

Docker volumes are file systems stored on the host and mounted on Docker containers to preserve data generated by the running container. Volumes are independent of the container life cycle, which means the data is secure even if the container fails. Furthermore, users can easily copy, backup, and share file systems between containers.

The diagram below illustrates the difference between volumes, which mount a part of the local filesystem, and bind mounts, which connect to the filesystem directly.

While bind mounts are sufficient for testing purposes, volumes are the recommended persistent log storage option.

Use Dedicated Logging Container

Collect and manage Docker logs using a dedicated logging container independent of the host machine. This container collects the log files from the Docker environment and monitors and inspects the logs before sending them to a centralized location.

As the logging container is an independent unit, users can quickly move it between different environments. Moreover, scaling the logging infrastructure is straightforward, as new logging containers can be added to the existing setup.

Another advantage of utilizing dedicated logging containers is that they do not require installing a configuration code. On the other hand, the application and logging containers must be correctly defined to ensure smooth automated logging.

Apply Sidecar Approach

The sidecar approach is one of the most popular methods of managing microservice architecture for more complex deployments.

Sidecars are services deployed alongside the parent application within a container as a secondary process. They share the volume and network with the main container, ultimately increasing the app's functionality.

Each application has a dedicated log service container customized for the program. The logging container saves log files to the volume, labels them, and ships them to a third-party log management solution.

Managing Docker logs using the sidecar approach simplifies identifying and tracking each log's origin through custom tags. On the other hand, the sidecar method is more complex and requires more resources than the dedicated logging solution.

Docker Logging Tools and Software

Below is the list of tools commonly used for Docker log aggregation and analysis:

• ELK Stack. Elasticsearch, Logstash, and Kibana combined in a stack for log aggregation, processing, and visualization.  
• Fluentd. A robust log collector and processor for collecting logs from various sources, including Docker containers.  
• Graylog. An open-source log management platform with advanced features like search, alerting, and visualization.  
• Loki. A highly available, horizontally scalable, multi-tenant log aggregation system.  
• Splunk. An enterprise-grade log management and analysis platform.  

When it comes to log monitoring, popular tools include:

• Prometheus. An open-source monitoring system that collects metrics from various sources, including Docker containers.  
• Grafana. A popular visualization and monitoring tool for creating dashboards to visualize log data.
• Datadog. A cloud-based monitoring and analytics platform that provides a unified view of infrastructure and applications.  
• Sysdig. A security and monitoring tool for analyzing system and container activity.

Conclusion

After reading this article, you should better understand how Docker logs work. The article also explained how to use the basic commands for working with logs and listed best practices for log management.

To learn more about the commands for working with Docker containers, check out our list of Docker Commands.