Docker Circular Import Hell


Lately I've been stuck on a particular problem with my Docker container projects. Generally, I need three containers to run Drupal: A web server container, a database container, and a container to host any CLI tools. The latter isn't necessary, but it's very, very useful when working with Drupal core and module code.

Drupal has two primary CLI tools, drush, and Drupal Console. These tools allow you to reinitialize the site, clear internal caches, install modules, and generate module code. You don't need either, but it makes developing on Drupal much, much easier it's considered essential. Drupal 8 also has some reliance on Composer when initializing and updating the site. Drush in particular, needs to know the site web address as well as the have a connection to the database in order to function. Drush, like Drupal, is written in PHP. Both behave best when they have the same kind of PHP extensions and configurations. 

In order to support both the web and CLI containers, I have PHP installed in each. The web container relies (currently) on mod_php and Apache, while the CLI container has a standalone PHP executable. It would be great if instead of having two installations of PHP, I only had one. The idea seems simple enough; with one instance of PHP, there'd be certainty that the same version and extensions would be available. 

I started toying with this idea over a week ago. I decided to use nginx instead of Apache mostly for my own education. I'm familiar with Apache, so why not learn something new in the process? An nginx containtainer already exists on Docker Hub, making it easy to get started. I modified my CLI container to install PHP-FPM and set it up as a background process in supervisord. With that complete, I modified the nginx configuration file to point all PHP traffic to the CLI container. 

And that's where I ran into a problem. Remember that my CLI container needs to be able to access the website as Drush requires access to the site. In my docker-compose.yml, I created a links entry:

 build: .docker/cli
   - ./docroot:/var/www/docroot
   - db
   - web

That seems sensible enough. For the web container to know the hostname of the CLI container, however, I needed to add it to the links entries for the web container:

  build: .docker/web
    - "80:80"
    - "443:443"
    - ./docroot:/var/www/docroot
    - db
    - cli

Seems sensible enough at first blush, but that's when I learned something about Docker's links. When I tried to up the container set, I got the following error:

$ docker-compose up -d               
ERROR: Circular import between web and cli

What. The. Heck? It turns out that links in Compose are intended to be one-way. You can't configure two containers that point at each other. I haven't been able to uncover a technical reason for this as of yet. You'd think that when you list a container as a link, it would only add host file entries for the listed container. There's nothing about hostnames that need to be one-directional to my knowledge, so this utterly flummoxed me. Perhaps there is a reason deeper in Docker's engine I didn't know about. 

It also appears this isn't an uncommon problem. While still experimental, the suggested solution is to leverage Docker's new networking system by using the --x-networking switch with Compose. According to the documentation, this creates a private network shared by the container set, rather like a virtual switch in vSphere. The network is named after the parent directory of the docker-compose.yml file, or what is specified in the net directive in the file itself. Sounds great! Only it's still experimental. There's no way to know how soon this will become default behavior in Compose. Docker has been moving astonishingly quickly lately, but you shouldn't depend on anything that isn't finalized.

The obvious solution for now is to create a separate FPM container. We'd use DockerFile directives to ensure the version and extensions remain in sync between FPM and CLI containers. It would work, but I can't say I'm happy with that option. I'd still be left with two instances of PHP, an unnecessary duplication. For now, however, it might be the easiest way forward.