Docker from Scratch, Part 4: Compose and Volumes
In the last post, we’ve created a repeatable web server container using a single file, the Dockerfile. While this is great, there’s one big limitation to a Dockerfile: It can only create one container. Docker containers are also only intended to run one process. What do we do if we want to run a web server, and a database, and a Solr instance, and a…
We’d need to create a Dockerfile for each server process. Given that we’d need to call the Docker “run” command for each container, building and starting containers would get rather tedious. Thankfully, we don’t need to do this thanks to an additional utility, Docker Compose
Docker Compose -- or, just “Compose” -- is typically installed with Docker on Mac OS X and Windows systems. On Linux systems, Compose is typically available as a separate package. Check the installation page on the Compose documentation site for details.
Compose wasn’t originally a part of Docker, but rather a completely separate project called Docker Fig. You may still find references for Fig out on the Internet. Thankfully, the idea behind Fig and file formats are mostly (if not completely) identical to Compose.
The Compose File
Just like Docker has a Dockerfile, Compose also has it’s own file, docker-compose.yml. There are several big differences between the Compose file and the Dockerfile. First of all, the Compose file is descriptive, rather than instructive. Dockerfiles tend to be made mostly of statements that are executed to build a container. The Compose file describes the container in its running state, leaving the details on how to build the container to Dockerfiles.
The Compose file is also based on an existing file format, YAML. YAML is often used to describe hierarchical structures while being light, and human readable. Think about it like XML without the “closing tag tax”.
A simple Compose file for our web server container would look like this:
web: build: .
The Compose file would be saved as docker-compose.yml in the same directory as our Dockerfile. The first line defines a service, basically a container. Every statement under that describes the container. In YAML, indentation is significant, like it is in Python.
Typically, you see one of two statements immediately following the service name in a Compose file: a “build” statement, or an “image” statement. Both are analogous to the FROM statement in the Dockerfile, with an important difference. The “build” statement is always a path to a Dockerfile on the local system. The “image” statement always refers to an image on the Docker Hub.
When building services in Compose, there are times you might want to change some of the defaults that are specified in the Dockerfile. If you’re using the “build” statement, it’s pretty easy to just modify the Dockerfile you have locally. What about when using “image” and you’re referring to a remote image on the Hub? It turns out, you can easily override many things in your Compose file.
For our Apache Web Server container, we may wish to specify a different port than that in the Dockerfile. In fact, you can do more than that:
web: build: . ports: - “80:80”
The “ports” statement takes a list of network ports to open between the host system and the container. Each pair is on a separate line, preceded by a hyphen -- the normal YAML list format. The pair first specifies the port as seen by the host system, with the port after the full colon is the port the container expects. This makes it very easy to start our Apache container on a non-default port such as 8080. We only need to change the “ports” statement in the Compose file.
YAML has a bit of a quirk where numbers lower than 60 are concerned. YAML may try to parse those as base-60 rather than base-10. For this reason, we specify the port pair as a string by enclosing it in double quotes.
The “ports” statement isn’t the only thing we can override in the Dockerfile. There are “command” and “entrypoint” statements that correspond to “CMD” and “ENTRYPOINT” in the Dockerfile. You can find a complete list on the Docker Compose Reference site.
Managing Containers with Compose
Using Compose instead of basic Dockerfiles also has a small but much needed advantage. Until now, whenever we ran, killed, or removed a container, we had to specify the container ID. We were always looking up these IDs using the docker “ps” command. This can be both tedious and error prone; we wouldn’t want to kill or destroy the wrong container on a production server!
Starting a container with Compose is pretty similar to when you used the Docker “run” command. Instead of “run”, however, you use “up”:
$ docker-compose up -d
Similar to the Vagrant “up” command, the Compose “up” command builds the containers specified by docker-compose.yml, and then runs them. The “-d” switch runs the containers in the background. That’s right, containers. You can actually build multiple related containers in just one Compose file!
Stopping and destroying all the containers described in the Compose file is even easier:
$ docker-compose kill $ docker-compose rm
The “kill” command will stop all the containers in our Compose file, while “rm” will remove and delete them. Notice we didn’t need to specify the container IDs at all. Yay! But how did it do this? Well, if you run the Docker “ps” command after “up”ing your services, you’ll notice something interesting:
$ docker ps CONTAINER ID COMMAND STATUS NAMES 245fbb0bf255 "apachectl -D FOREGR Up 18 secs dockerthingy_web_1 $ pwd /home/tess/dockerthingy
Ah-ha! Compose created the container with a special name. It used the parent directory of the docker-compose.yml file, and then appended the service name with an instance number. This way, Compose can always identify the containers to stop or destroy.
This format also carries with it another advantage. Typically, the docker-compose.yml file is placed in the root directory of your project or git repository. This not only makes it easier to “up” your development environment, the directory will tend to have a more unique name to avoid potential conflicts on the same Docker host.
Getting Files into the Container
So far we’ve been concerned with just building our container and getting it into a usable state; and there’s been a lot to learn! Now, however, we can start putting our container to use. For a web server container, we want to get our application files into the server docroot so that Apache may serve them.
Dockerfiles support a COPY statement that will -- you guessed it -- copy the specified file from the host system into the container:
COPY path/on/host.txt /target/path/on/container.txt
There’s also an ADD command that builds upon COPY in two key ways: ADD can take a remote URL as a source path and download the file over the Internet. ADD may also take an archive file in *.tar.gz or *.zip format and extract its contents in the destination directory. While this sounds great, it’s best practice to always use COPY if you are copying a local file into the container.
Now that we know how to get files into the container, let’s update our Dockerfile to copy an HTML file into our container’s web docroot:
FROM debian:wheezy MAINTAINER email@example.com RUN apt-get update RUN apt-get install -y apache2 COPY index.html /var/www CMD ["-D", "FOREGROUND"] ENTRYPOINT ["apachectl"]
We’ve added the COPY statement immediately before the CMD and ENTRYPOINT statements. Even though this is a simple change, we need to rebuild our container. Normally, we’d do this with the Docker “build” command. There’s also a “build” command for Compose. This will force the contains to be rebuilt:
$ docker-compose build
Now we can up the Compose service again. This time, when look at the container’s webserver port, we should see our updated index file.
$ docker-compose up -d $ curl docker.dev <html> <body> <h1>Behold, my amazing page!</h1> <p>Okay, yeah, it sucks. Sorry.</p> </body> </html>
The problem with both COPY and ADD is that it only works during the build phase, not the run phase. If we ever need to change the web files we’re developing, we’d have to stop the container, destroy it, and build it again. While that process is much faster with Docker than it is with a VM, we’d rather just work on the same files as in our project directory, and have them updated in the container whenever we save changes on the host’s filesystem.
The solution is volumes. Volumes allow us to mount a directory on our local system as a directory in the container. We can do this with the “volumes” statement in our Compose file:
web: build: . ports: - “80:80” volumes: - ./docroot:/var/www
Like “ports”, “volumes” is a YAML list that takes host:container pairings. Instead of ports, we specify the path to the directory on the host system, with the path to the directory to mount inside the container. It can take both relative and absolute paths. Typically, you use a relative path for the host system and an absolute path for the container.
For our example Compose file, we’ve created a “docroot” directory within our project directory. We can then add our application files -- including index.html -- to it:
dockerthingy/ ├── docker-compose.yml ├── Dockerfile └── docroot/ └── index.html
Make sure to remove the COPY from our Dockerfile! We won’t need it any longer since we’re using volumes. When we “up” the container, Docker will mount our docroot/ directory in the container at the Apache Web Server default document root, /var/www. If we make any changes to our index.html file, they will be synced to the container automatically.
Compose is an awesome addition to Docker. It allows you to define multiple containers as a whole. We’ve used COPY and ADD to get custom files into the container from the host OS. Then, we used volumes to sync a directory on our host’s filesystem to the container. Next time, we’ll add a second container to Compose file for running a database.