In our previous article we have covered propagating proxy environment variables into nested docker-in-docker containers. But how can you run a container inside another container? Are there performance costs? How can you start docker daemon inside of a container?
There are two approaches with their own pros and cons. Docker and all of the other container engines are built around the concept of kernel namespaces, so “running” a container is nothing more than instructing the kernel to isolate some processes from the rest of the world (from the root namespace). The “amount” of isolation, however is configurable, this is where the infamous --capabilities option plays its role, which in turn is again a wrapper around kernel capabilities. If you are interested – read the list and see if you can spot a capability that allows us to create more containers (hint: containers are namespaces!). But if you are too busy, read on!
Allow containers to create sub-containers
In order to be able to run docker-engine inside of a container you need to enable CAP_SYS_ADMIN capability. Specifically, the bullet that reads
call setns(2) (requires CAP_SYS_ADMIN in the target
namespace);
Notice the target namespace emphasis. In order to be able to use docker-engine from inside of a container, that container needs to be run with --capabilities CAP_SYS_ADMIN along with the rest of the arguments you need to run a container. The benefit of such method is that you can completely isolate your containers and they won’t care if they are run as top-level containers or sub-containers. All the networking, for example, will be configured accordingly and even if you have same docker subnets inside of multiple containers it will still work fine due to proper namespacing. The down-side of such an approach is that you still need docker-engine installed in every container, which will take some extra storage, although it will only require marginally higher usage of other computing resources. There is, however a different approach.
Pass-through of docker interface
Docker is split into several codebases that communicate via a UNIX socket. You are probably familiar with starting docker service first before you are able to call docker commands, otherwise it will report
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
All of the docker run, and docker exec, and docker build commands communicate via this docker.sock file, transmitting everything to the docker backend that actually creates containers for you. Even docker build context (all of the files in the folder with Dockerfile) gets transmitted through this socket first. However, thanks to UNIX – it’s just a file. And files can be mounted as volumes inside of the docker interface. To pass this file to your containers simply run them with
docker run -v /var/run/docker.sock:/var/run/docker.sock $REST_OF_COMMAND_ARGSAnd don’t forget to install docker-ce-cli into your secondary containers – no docker engine is required. More over if you are trying to run something like Jenkins in a container – you don’t even need to install docker-ce-cli because Jenkins can talk directly to docker.sock. Docker actually provides an SDK in several languages to communicate with docker over docker.sock so if you ever find a need to call docker run from within your python or go application – using the SDK is a much more reliable way of communication. There are also unofficial libraries for C, C++, C#, Haskell, Java, NodeJS, Perl, PHP, Ruby, Rust, Swift, and Scala.
