Joshua's Docs - Docker Cheatsheet, notes, and more
Light

Resources

What Type Link
Base Commands - Quick Ref Reference docs.docker
Compose - getting started Tutorial docs.docker
Compose - getting started Tutorial fedoraproject
Compose Guide (Heroku) Guide Heroku
Example of a YAML config for a complex backend/frontend docker setup Tutorial freecodecamp
Fireship: "Docker in 100 Seconds" YT Video YouTube

Docker CLI - Basic Commands

What Command Helpful Flags
Build an image docker build {path}
{path} is
--tag / -t Specify image name (+ optional tag). Format is name:tag, but :tag is optional.
     - Example: -t joshua/custom-image:3.2
     - You can use -t multiple times, for multiple tags
--build-arg Specify environment variable values only for build-time.
List active containers docker ps --all
Restart a container docker restart {containerName}
Stop a container docker kill {containerName}
Inspect something docker inspect {thingIdOrName}
View logs docker logs {?:containerName} -f follow
-tail
--details show extra detail
Execute a command inside a container docker exec {containerName} -i interactive
-t allocate TTY
-d detach
Create volume docker volume create {options} {volumeName} --driver (see notes on volume types)
--label
--name
--opt
List volumes docker volume ls
Drop volume docker volume rm {volumeName} -f or --force
Print version docker version
List images docker image ls
Pull an image docker pull {imageName}
Run an image docker run {imageName}

Example: docker run hello-world
--name (give container a short name)
List downloaded images docker images OR docker image ls --filter

Note: For most of the commands that take an image name, you can optionally request a specific tag or digest version of an image by using a special suffix. For example, {imageName}:{tag} or {imageName}@sha256:{digestSha}

docker-compose is listed below, in its own section

Docker Compose (aka docker-compose)

Docker Compose - Commands

There is an online official compose command overview, or use docker-compose --help

Many of these commands are in common with compose-less environments, just using docker instead of docker-compose

There are common flags that can be applied with any subcommand. For example:

-f or --file lets you specify a docker .yml file other than the default of docker-compose.yml.

If using -f / --file, it should come before the command, like docker-compose -f ./my-compose-file.yml up -d

What Command Helpful Flags
Test out a config docker-compose config
Stop containers, without removing docker-compose stop
Stop and remove containers docker-compose down -v: remove named volumes
Build and launch container docker-compose up -d (starts in detached mode)
--force-recreate
--build
Start a specific service docker-compose run {serviceName}
Removed stopped containers docker-compose rm If you don't pass a name, it removes all (like rm *)
-v removes the volumes as well. Useful if something is persisting when you don't want it to and need a hard reset.
-f : Force
Validate a config without running it docker-compose config -q - Quiet mode, only validates true/false without verbose output
Show logs docker-compose logs -f follow
--tail={#} Tail lines
You can also pass the container name - docker-compose logs {containerName}
Run a command inside the container docker-compose exec -it {serviceName} {command} -it (interactive, tty)

Docker-Compose File syntax and options

Docker-Compose: Extending Configurations via the CLI

Let's say that you have an existing docker-compose config file, like docker-compose.yml, but you want to change some of the settings without actually modifying the file (maybe you can't?).

One option is to create another config file (B) that extends the original (A), and run them together, with docker-compose -f ${PATH_TO_FILE_A} -f ${PATH_TO_FILE_B} ${REST_OF_CMD}

  • Example: docker-compose -f docker-compose.yml -f docker-compose.prod.yml up webapp

Docker-Compose: Random Tips and Notes

  • You can use environment values within your docker-compose file (variable substitution)
    • Syntax is ${VARIABLE_NAME} (just like JS template literal), or $VARIABLE_NAME
    • The .env file has to be in the same directory as the docker-compose.yml file (see 1, 2)
      • Depending on your version of docker-compose, you might be able to get around this with either --project-directory (ref) or --env-file (ref), however, I have not had success with either so far
  • You can pass environment values to the container itself
    • WARNING: These have to be passed explicitly - Docker will not just pass all values through
    • Full details
    • You can use the env_file option to pass an entire .env file contents to the container
      • Put under environment: but leave off value
      • Pass via docker-compose run -e {VAR}={VAL} {imageName}
  • You can use comments (prefix with #, like shell scripts)
  • Specifying ports in YAML:
    • Compose syntax is host_port:container_port
      • Example: 3000:80 would mean that you would go to localhost:3000 on your computer (host) to access a web app that is serving on 80 within the container
    • If both host and container are using the same port, you still have to specify both
      • This is because a ports value of 80 does not convert to 80:80 - instead it turns into {RANDOM_port}:80
  • You can start an individual service within a compose file with docker-compose up {SERVICE}

Dockerfile

Dockerfile syntax and options

📄 Cheatsheet: Dockerfile Syntax Cheatsheet

💡 Doc: Best Practices

Dockerfile Gotchas and Tips

There are some interesting "gotchas" with Dockerfiles - usually related to order-of-operations and combining or splitting up commands.

  • You never want a command like apt-get update to run as its own command - it should always be combined with an install command to make sure the results are not cached.
  • RUN only accepts a single shell string at a time. However, there are some workarounds, which are really just the same as multi-line shell tricks:
    • Use \ at the end of lines to break up long commands
    • Use the heredoc syntax to run true multi-line command strings:
      RUN <<EOF
      echo "Line 1"
      echo "Line 2"
      EOF

Docker Build and Managing Docker Images

Docker Build vs Builder vs Buildx vs Compose Build

If you come to the Docker docs and look for sections on building, you will find docker build, docker builder, and docker buildx. Plus docker compose build.

In short:

  • docker builder is an alias for the default behavior of docker build
  • buildx (docker buildx) is a more powerful builder that can be used for multi-platform builds

Finally, docker compose build is really just a wrapper around docker build - it runs docker build over the services in your docker-compose.yml file to build the individual image for each service.

Networking

Main resource: Docker - Networking Overview

Ports vs Expose

📄 Helpful StackOverflow: "What is the difference between docker-compose ports vs expose"

In newer versions of docker, EXPOSE doesn't even do anything; its purpose is informational, as a way to explicitly advertise (to readers of the code) that the image (or service) will be working with the port(s) referenced, and other services should be able to talk to it through that port. Expose does not expose them to the host machine.

The reason why this is only informational is that cross-service communication is enabled by Docker out-of-the-box, regardless of whether or not you use expose.

Port settings (e.g. docker-compose {service}.ports or --publish) actually do expose container ports to the host.

The syntax for port settings are host_port:container_port, or just container_port (which uses a random host port)

Volume types options / drivers

See: Docs: Docker storage drivers

Type Description
tmpfs Temp data, stored in memory, not persisted between sessions
btrfs Advanced, persisted storage. Requires a bunch of pre-reqs, including Linux Kernel support.
nfs Shared, persisted storage

How do i...

How do I...

  • List active containers
    • docker ps
  • Restart a container with changes after a .yml / yaml config change?
    • Bring up, and force image rebuild:
      • docker-compose up -d --force-recreate --build
      • If you don't need to bring up: docker-compose build --no-cache
      • NOTE: neither of the above options rebuilds volumes, they only rebuild images. You can use --renew-anon-volumes to recreate anonymous volumes, but this won't work for named volumes.
    • If you run into issues (changes don't seem to take affect, passwords not working, etc)...
      • Sometimes the volume data will persist, even if you ran docker-compose down -v first.
      • Try finding the volume data directory and running rm -rf on it, then rebuild and start back up
  • Rebuild volumes?
    • If it is an anonymous volume, you can use --renew-anon-volumes with docker-compose up
    • If it is a named volume:
      • docker-compose down -v
      • Stop container, then run docker volume rm {VOLUME_REF}
  • Get a container IP address
    • docker inspect {containerName}
    • Just IP address:
      • docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' container_name_or_id
      • Use double quotes if you have issues.
      • Credit: S/O
  • Debug a container not starting correctly (die event), or view logs
    • Multiple options for log viewing:
      • See this S/O
        • Summary:
          • Start event logger with docker events& - this kind of does tail
          • Do "thing" - e.g. up, that triggers error
          • You should see error appear in event log
          • Copy error instance ID from event log and use with docker logs {eventId}
      • docker-compose -f {composeFile} logs
        • Use -f to follow
        • Use --tail={#} to tail # of lines
      • docker logs {?:containerName}
  • Access BASH inside container
    • docker exec -ti {containerName} bash
      • Use exit to stop
    • You can also try docker attach {containerName} to link console STDIN/STDOUT, but this seems less optimal than exec
  • Remove a stubborn network ("error while removing network ... has active endpoints")
    1. Find out what containers are using the network
      • docker inspect {networkName}
    2. Either bring down those containers, or break the connection
      • To break connection(s), use docker network disconnect {networkName} {containerName}
    3. Once you have broken linkage, you should be able to bring down
      • Either re-run down, or manually remove network with docker network rm {networkName}
  • Kill *all containers
    • docker ps -q | xargs docker kill
  • Find the size of an image before downloading?
    • Use the "Tags" tab on a Docker image listing page, e.g. Alpine
  • Bind / expose a port for an already running container?

Issues / Troubleshooting

  • Linux / macOS: Commands only work when prefaced with sudo
    • Add your user to the Docker usergroup: sudo usermod -aG docker $USER
      • You must either log out and back in, or use su -s ${USER} to refresh group membership
    • See Docker docs: Linux: Manager Docker as a non-root user
  • PostgreSQL issues with Windows - usually due to permissions issue
  • Can't remove volume (even with -f or --force): "Error response from daemon: remove {volume}: volume is in use"
    • Try this first:
      • docker-compose down -v (or docker-compose rm -f -v) (WARNING: This will remove all volumes)
    • If the volume is being locked by a container, you can find what is using it:
      • docker ps -a --filter volume={volume} to find container, and then docker rm {CONTAINER} to remove container
    • Try docker system prune, and then remove volumes again
    • As a last ditch effort, you can try restarting the entire Docker service (either via GUI, or CLI), then try removing container
    • Lots of tips in the responses to this S/O question
  • Can't stop or kill a running container (maybe with a message like An HTTP request took too long...)
    • Sometimes Docker just gets in a really strange state. AFAIK, there are a bunch of open issues for this, but the only real fix is to just restart the Docker service entirely. There should even be a menu option to restart Docker Desktop!
    • You could try getting in through a shell (e.g. with docker exec), but that is likely to fail if the container is hosed anyways.
  • Docker keeps pulling from the wrong node_modules, or stale modules, etc.
    • Make sure you aren't actually mounting your local node_modules directory directly, or else you are going to run into version issues
    • Try explicitly using a named volume for node_modules
  • For NodeJS projects, keep getting a unexpected token "export" error
    • Likely issue with node_modules resolution; make sure that dependencies can be found within the container, and try to avoid bind-mounting node_modules
      • Don't use node_modules as a volume name!
    • If node_modules is not bind-mounted and everything is set up correctly, but you are still getting this error, make sure that node_modules is actually up-to-date - if package.json changed, you likely need to rebuild your container image, as it probably has npm install as a step
      • Example reset: docker-compose down -v && docker-compose build && docker-compose up
      • Also, make sure both package.json and package-lock.json are copied to the image before running npm install
Markdown Source Last Updated:
Mon Feb 06 2023 02:02:24 GMT+0000 (Coordinated Universal Time)
Markdown Source Created:
Thu Dec 05 2019 01:06:19 GMT+0000 (Coordinated Universal Time)
© 2023 Joshua Tzucker, Built with Gatsby
Feedback