Design pattern for lean docker images and fast develop / build iterations
Problems to be solved
Long build time
Long upload to registry
Ways solving the problems
Avoid installing dependencies on high frequency builds (i.e install dependencies on base image)
Reduce image size by using small base image, install dependencies for compile time and runtime, remove all compile time dependencies when build finished
Schematics
Explanation
Base layer
Depending on what we are trying to achieve, we will select a relatively small image. For comparison, the suggested Alpine image weight only 4MB
Middle layer
This layer will be maintained in the service / team level, this layer will be responsible of having preinstalled python / nginx / jvm whatever dependency that the application layer will need, without the application layer need to install / reinstall it
Application layer
This layer will be responsible of the installation of the projects code to the local venv, creating shared volumes / users. This layer will not install any dependencies
Dockerfile for middle layer
FROM alpine:3.7
MAINTAINER GalCo
ADD requirements.txt /
RUN apk add --update \
python \
python-dev \
py-pip \
build-base \
postgresql-dev \
gcc \
musl-dev \
linux-headers \
nginx \
bash \
&& rm -r -f /var/cache/apk/* \
&& pip install -r /requirements.txt \
&& apk del gcc \
python-dev \
build-base \
musl-dev \
linux-headers
As you can see, we install all the compile-time and run-time dependencies, and after installation of /requirements.txt, we remove all compile-time dependencies. This produces image in ~60MB of size
Dockerfile for application layer
FROM galco/base:alpine-python2.7
MAINTAINER galco
RUN [ -d svc ] || mkdir /svc
COPY . /svc
WORKDIR /svc
RUN pip install -e . --no-deps
COPY ops/etc/nginx.conf /etc/nginx/
COPY $SUPERVISORD /etc/supervisord.conf
COPY ops/etc/uwsgi.ini /etc/uwsgi.ini
EXPOSE 8040
As you can see, nothing is being installed in this layer, the previous layer is pushed to galco/base repository, and project is installed without its dependencies (--no-deps ) - the dependencies are pre-installed in previous layer
This layer is lightwieghted, and should not pass 2MB of size. This layer will be built frequently
Some docker shenanigans
$ docker history <image-id>
The above command will show you the different layers that the image is compund from, and each layer size and command that created it
The command below will let you persisting your image to your local storage, and let you share it with other machines. For instance if you have CI machines, you can share the cache by using shared volume as AWS EFS
$ docker save <images-ids> > svc-cache.tar
The following command will let you load the persisted image to your local docker cache, and might save you time when building / uploading new layers
$ docker load < svc-cache.tar
Sum things up
What this post is all about is making the dev team making more efficient work, by building images faster, in shorter iterative manner.
This was accomplished by creating smaller images, and using base image that all different dependencies are pre-installed on
Thanks for reading