TechTarget.com/searchitoperations

https://www.techtarget.com/searchitoperations/tip/Your-starter-guide-to-Docker-troubleshooting

Your starter guide to Docker troubleshooting

By Stephen J. Bigelow

Containers start quickly, run reliably and use few resources compared to VMs. But they aren't perfect, and container engines, such as Docker, have their own problems. When containerized application troubles arise, those countless containers that run across the infrastructure can pose a daunting troubleshooting challenge.

Docker provides a series of tools and features that help IT administrators isolate and understand a broad assortment of common container problems. Admins can rely on Docker logs, commands such as pause, inspect, info, stats, events and exec, as well as an entrypoint override process to troubleshoot troublesome containers.

However, effective container troubleshooting takes time and experience. Use the documentation that accompanies Docker to get the full range of available commands, and take time to experiment with options in a test environment to discover the most useful commands for Docker troubleshooting.

Use Docker logs

Developers are often careful to incorporate logging features that log key events during operation -- especially errors and alerts when operations don't go as intended. Logs can be some of the most useful and insightful troubleshooting information available to admins.

Docker provides the logs command, which displays the container's history file captured to the standard output (stdout) device on the Docker host.

$ docker logs [options] <container name>

Use the logs command, and follow it with the container's name. Admins can tailor the log output with several options, including the following:

• --details includes additional attributes to the log output, such as environment variables.
• --follow or -f shows new log output as the container runs.
• --since shows the log entries after the desired timestamp.
• --tail or -n shows some number (n) of lines from the end (tail) of the log.
• --timestamp or -t shows the log output for the desired point in time.
• --until shows all log entries until the desired timestamp.

Editor's note: Docker syntax uses two dashes preceding a full option name and one dash preceding an abbreviated option.

All logs command options help limit displayed log data volume so the admin can quickly locate any necessary log information. To see the entire log for a container named stevetest, the Docker console command is the following:

$ docker logs stevetest

If an admin must follow (-f) log entries three seconds before (--until) the desired point in time, add the -f and --until options, such as the following:

$ docker logs -f --until=3s stevetest

Logs are available even after the container exits until an admin removes its file system.

Commands to gain container control

Admins can stop, start, restart or remove troubled containers with a series of associated Docker console commands. The pause command halts all processes in the desired containers:

$ docker pause <container 1 name> <container 2 name…>

To pause the activity of containers named test1 and test2 in order to check their respective logs, the Docker pause command is the following:

$ docker pause test1 test2

The unpause command resumes the processes in those specified containers:

$ docker unpause <container 1 name> <container 2 name…>

To resume the activity in both test containers, the Docker unpause command is the following:

$ docker unpause test1 test2

Docker also has a restart command, which enables admins to kill and restart a desired container. This is a handy way to fix crashed or hung containers. The general syntax is the following:

$ docker restart [options] <container 1 name> <container 2 name…>

The only option available for the restart command is the --time or -t argument, which designates the number of seconds to wait before the restart. An admin can restart the container called stevetest with the following:

$ docker restart stevetest

But, if the admin must invoke a 30-second pause before the restart, the command would look like the following:

$ docker restart –t30 stevetest

The Docker remove (rm) command can remove one or more Docker containers. Its general syntax is the following:

$ docker rm [options] <container 1 name> <container 2 name…>

There are several options to force or help remove associated container components:

• --force or -f can force the removal of a running container.
• --link or -l removes specific links or associations between containers.
• --volumes or -v removes volumes associated with the specified containers.

Use the inspect command

The Docker inspect command returns an assortment of detailed Docker container information. The information can include the container state, network port mappings, environment variable values and path to history log files.

The basic syntax of the inspect command is the following:

$ docker inspect [options] <container 1 name or ID> <container 2 name or ID…>

There are several options that control the information displayed in the inspect command, including the following:

• --format or -f formats the output according to a template.
• --size or -s displays container file sizes.

To inspect the container named stevetest, use the inspect command alone with no options to display all available details about the container, such as the following:

$ docker inspect stevetest

Docker responds with a complete list of container details, including the container's ID, creation date and time, its path, any environmental variables and values, and network details. Admins can employ inspect to quickly determine the container's IP address, media access control address, log path or port bindings.

Use the info command

The inspect command delivers specific container details, but the Docker info command provides detailed information about the broader Docker installation.

This information can include the number of unique containers and images, kernel data and storage data.

The info command is particularly handy when an issue affects all containers at the server level. The basic syntax of the info command is the following:

$ docker info [options]

The only available option is the --format or -f option, which outputs the details in a selected Go format. Omitting the formatting option simply produces all details as a simple list.

Docker info command output may provide details that include the number of running, paused and stopped containers; number of available images; what storage driver is in use; storage use and availability metrics; specifics for plugins, OS and the kernel; CPU and memory details; the current Docker server version; and execution and logging drivers in use.

The info command can be handy when an issue affects all containers at the server level. It's a wealth of details about the Docker installation and its operating environment. Admins can use that information to discern incorrect paths, incompatible component versions and undesirable drivers.

Use the stats command

Container troubleshooting can require a close resource use examination. The Docker stats command provides a live data stream of resource use for running containers. The basic syntax for the stats command is the following:

$ docker stats [options] <container 1 name> <container 2 name…>

The stats command automatically streams usage data for all running containers. Admins can declutter this information by using options and specifying desired container names. The options associated with the stats command include the following:

• --all or -a shows stats for all running and stopped containers.
• --format displays data using Go formatting.
• --no-stream snapshots the first result and does not provide ongoing streams.
• --no-trunc does not truncate output.

Admins can employ the --format option to narrow any returned data placed into specific columns. The default (unformatted) streaming output provides eight distinct columns of details for admins to study: container ID, container name, CPU cycles the container uses, container memory space percentage, actual memory consumption and memory limits, exchanged network data, amount of data on block storage and number of processes on the container.

Resource use metrics can be a powerful troubleshooting tool. Containers that use excessive resources, exchange excessive network or storage I/O, or generate an unusually high number of threads can indicate potential problems with the instance.

Use the events command

The Docker events command enables admins to see up to 1,000 server-based events in real time. Reported events can come from a variety of sources:

• Containers report events such as attach, destroy, copy, start, stop, pause and rename.
• Images report events such as load, delete, save, tag, push and import.
• Plugins report events including enable, disable, install and remove.
• Volumes report events such as create, mount, unmount and destroy.
• Networks report events including create, connect, disconnect and destroy.
• Services, configs, secrets and nodes report events such as create, update and remove.
• Daemons report events.

The basic syntax for the Docker events command is the following:

$ docker events [options]

The options available for the events command include the following:

• --filter or -f filters the output to narrow the amount of output data.
• --format formats the output with desired Go templates.
• --since shows events generated since the desired timestamp.
• --until shows events leading up to the desired timestamp.

To look for specific events, use the --filter option to narrow the output by specific container or event type. For example, to look for any stop events, use a --filter, such as the following:

$ docker events --filter 'event=stop'

Use the exec command

The Docker exec command enables admins to execute new commands within the default directory of a running container. For exec to function, the command must be executable, and admins must run the target container.

Admins routinely use the exec command to check or set environment variables or command privileges. The basic syntax for the exec command is the following:

$ docker exec [options] <container command> <command arguments>

Some options for the exec command include the following:

• --detach or -d runs the command in the background (detached mode).
• --detach-keys overrides the required sequence to detach a container.
• --env or -e sets environment variables.
• --env-file reads the state of environment variables.
• --interactive or -i enables standard input (interactive mode).
• --privileged enables extended privileges to the specified command.
• --tty or -t creates a TTY mode for output.
• --workdir or -w sets up the working directory for the new command to execute into.

To execute an interactive/TTY-enabled Bash shell on a container called stevetest, admins could use the following:

$ docker exec -it stevetest bash

Override the entrypoint

During container creation, admins make an associated Dockerfile that includes any arguments and parameters upon container launch. At container launch, the container uses this specified entrypoint to run the container.

Admins can use powerful entrypoint commands and Docker run commands with the --entrypoint flag to change how the container launches and runs. There are two forms of entrypoint: the executable form, which may appear in a Dockerfile, and the shell form, which can run like any other Docker command.

The executable (Dockerfile) form uses the following syntax:

entrypoint ["executable path/name", "parameter 1", "parameter 2…"]

The shell form is similar but omits brackets and quotes to look like the following:

entrypoint command, parameter 1, parameter 2…

The following Dockerfile line illustrates how entrypoint can run an Apache application in a container as a foreground process:

entrypoint ["/usr/sbin/apache2", "-D", "foreground"]

If admins change entrypoint parameters, they can alter how containers launch and run. Beyond the Dockerfile, they can use run –entrypoint to change entrypoints on demand and overwrite the default Docker entrypoint.

Admins may run a container and set the entrypoint to a background shell before they load the Apache application. This helps admins execute shell commands to further troubleshoot the troubled app:

$ docker run -d --entrypoint /bin/sh /usr/sbin/apache2

Editor's note: Docker entrypoint features can be extremely powerful and complex. The examples and discussion here are only a highlight of its potential capability. Refer to Docker documentation for detailed entrypoint examples and uses.