README.md

GitLab CI template for Docker

This project implements a generic GitLab CI template Docker based projects.

Usage

In order to include this template in your project, add the following to your .gitlab-ci.yml :

include:
  - project: 'to-be-continuous/docker'
    ref: '3.2.0'
    file: '/templates/gitlab-ci-docker.yml'

Understanding the Docker template

The template supports two ways of building your Docker images:

1. The former Docker-in-Docker technique, that was widely used for years because of no other alternative, but that is now commonly recognized to have significant security issues (read this post for more info),
2. Or using kaniko, an open-source tool from Google for building Docker images, and that solves Docker-in-Docker security issues (and also speeds-up build times).

By default, the template uses the kaniko way, but you may activate the Docker-in-Docker build at your own risks by setting DOCKER_DIND_BUILD to true (see below). ⚠️ In that case, make sure your runner has required privileges to run Docker-in-Docker (see GitLab doc).

Global variables

The Docker template uses some global configuration used throughout all jobs.

Name	Description	Default value
DOCKER_DIND_BUILD	Set to enable Docker-in-Docker build (⚠️ unsecured, requires privileged runners).	(none) (kaniko build by default)
DOCKER_KANIKO_IMAGE	The Docker image used to run kaniko - for kaniko build only	gcr.io/kaniko-project/executor:debug (use debug images for GitLab)
DOCKER_IMAGE	The Docker image used to run the docker client (see full list) - for Docker-in-Docker build only	docker:latest
DOCKER_DIND_IMAGE	The Docker image used to run the Docker daemon (see full list) - for Docker-in-Docker build only	docker:dind
DOCKER_FILE	The path to your Dockerfile	./Dockerfile
DOCKER_CONTEXT_PATH	The Docker context path (working directory)	none only set if you want a context path different from the Dockerfile location

In addition to this, the template supports standard Linux proxy variables:

Name	Description	Default value
http_proxy	Proxy used for http requests	none
https_proxy	Proxy used for https requests	none
no_proxy	List of comma-separated hosts/host suffixes	none

Images

For each Dockerfile, the template builds an image that may be pushed as two distinct images, depending on a certain workflow:

1. snapshot: the image is first built from the Dockerfile and then pushed to some Docker registry as the snapshot image. It can be seen as the raw result of the build, but still untested and unreliable.
2. release: once the snapshot image has been thoroughly tested (both by package-test stage jobs and/or acceptance stage jobs after being deployed to some server), then the image is pushed one more time as the release image. This second push can be seen as the promotion of the snapshot image being now tested and reliable.

In practice:

• the snapshot image is always pushed by the template (pipeline triggered by a Git tag or commit on any branch),
• the release image is only pushed:
  • on a pipeline triggered by a Git tag,
  • on a pipeline triggered by a Git commit on master.

The snapshot and release images are defined by the following variables:

Name	Description	Default value
DOCKER_SNAPSHOT_IMAGE	Docker snapshot image	$CI_REGISTRY_IMAGE/snapshot:$CI_COMMIT_REF_SLUG
DOCKER_RELEASE_IMAGE	Docker release image	$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME

As you can see, the Docker template is configured by default to use the GitLab container registry. You may perfectly override this and use another Docker registry, but be aware of a few things:

• the DOCKER_SNAPSHOT_IMAGE requires a Docker registry that allows tag overwrite,
• the DOCKER_RELEASE_IMAGE may use a Docker registry that doesn't allow tag overwrite, but:
  1. you should avoid overwriting a Git tag (at it will obviously fail while trying to (re)push the Docker image),
  2. you have to deactivate publish on master branch by setting the $PUBLISH_ON_PROD variable to false (as it would lead to the master tag being overwritten).

Registries and credentials

As seen in the previous chapter, the Docker template uses by default the GitLab registry to push snapshot and release images. Thus it makes use of credentials provided by GitLab itself to login (CI_REGISTRY_USER / CI_REGISTRY_PASSWORD).

But when using other registry(ies), you'll have also to configure appropriate Docker credentials.

Using the same registry for snapshot and release

If you use the same registry for both snapshot and release images, you shall use the following configuration

variables:

Name	Description
🔒 DOCKER_REGISTRY_USER	Docker registry username for image registry
🔒 DOCKER_REGISTRY_PASSWORD	Docker registry password for image registry

Using different registries for snapshot and release

If you use different registries for snapshot and release images, you shall use separate configuration variables:

Name	Description
🔒 DOCKER_REGISTRY_SNAPSHOT_USER	Docker registry username for snapshot image registry
🔒 DOCKER_REGISTRY_SNAPSHOT_PASSWORD	Docker registry password for snapshot image registry
🔒 DOCKER_REGISTRY_RELEASE_USER	Docker registry username for release image registry
🔒 DOCKER_REGISTRY_RELEASE_PASSWORD	Docker registry password for release image registry

Setting your own Docker configuration file (advanced)

There might be cases where you need to provide the complete Docker configuration file:

• need to declare authentication credentials for other registries than the 2 predefined ones (snapshot & release),
• need to declare a credentials store (ex: in order to publish to Amazon ECR with Kaniko for instance),
• need to declare proxies,
• ...

If you are in one of those cases, you will need to use the DOCKER_CONFIG_FILE variable, expected to declare the path to your custom Docker configuration file (JSON). You may:

• leave the default value (.docker/config.json) or override it to some alternate location in your project repository and create the file without any secret in it using our dynamic variables replacement (see below),
• or override it as a GitLab project variable of type File, possibly inlining your secret credentials in it.

Name	Description	Default value
DOCKER_CONFIG_FILE	Path to the Docker configuration file (JSON)	.docker/config.json

Moreover, this file supports dynamic environment variables replacement. That means it may contain references to other environment variables (in the format ${variable_name}) that will be dynamically replaced by the template before evaluation. In addition to you own defined variables, you may use the following variables (provided and managed by the template):

• ${docker_snapshot_authent_token}: the authentication token required by the snapshot registry (computed from configured DOCKER_REGISTRY_SNAPSHOT_USER / DOCKER_REGISTRY_SNAPSHOT_PASSWORD variables)
• ${docker_snapshot_registry_host}: the snapshot registry host (based on the configured DOCKER_SNAPSHOT_IMAGE variable)
• ${docker_release_authent_token}: the authentication token required by the release registry (computed from configured DOCKER_REGISTRY_RELEASE_USER / DOCKER_REGISTRY_RELEASE_PASSWORD variables)
• ${docker_release_registry_host}: the release registry host (based on the configured DOCKER_RELEASE_IMAGE variable)

Example 1: Docker configuration file inlined in the project repository (.docker/config.json) with dynamic variables replacement:

{
    "auths": {
        "${docker_snapshot_registry_host}": {
            "auth": "${docker_release_authent_token}"
        },
        "${docker_release_registry_host}": {
            "auth": "${docker_snapshot_authent_token}"
        },
        "my-readonly-repo-to-pull": {
            "auth": "${MY_OWN_REGISTRY_TOKEN}"
        }
    }
}

This file uses:

• template-managed ${docker_snapshot_authent_token}, ${docker_snapshot_registry_host}, ${docker_release_authent_token} and ${docker_release_registry_host} variables,
• the user-defined ${MY_OWN_REGISTRY_TOKEN} (ℹ️ an authentication token can be obtained with command echo "user:password" | base64 and then be stored as a masked GitLab CI/CD project variable).

Example 2: Docker configuration file declared as a GitLab project variable of type File with dynamic variables replacement:

{
    "auths": {
        "$${docker_snapshot_registry_host}": {
            "auth": "$${docker_release_authent_token}"
        },
        "$${docker_release_registry_host}": {
            "auth": "$${docker_snapshot_authent_token}"
        },
        "my-readonly-repo-to-pull": {
            "auth": "ZG9ja2VyZHVkZTpnb3RjaGEh"
        }
    }
}

This file uses:

• template-managed ${docker_snapshot_authent_token}, ${docker_snapshot_registry_host}, ${docker_release_authent_token} and ${docker_release_registry_host} variables (⚠️ mind the double $$ to prevent GitLab from trying to evaluate the variable),
• the user-defined authentication may be inlined as a GitLab project variable is a place safe enough to store secrets.

Multi Dockerfile support

This template supports building multiple Docker images from a single Git repository.

You can define the images to build using the parallel matrix jobs pattern inside the .docker-base job (this is the top parent job of all Docker template jobs).

Since each job in the template extends this base job, the pipeline will produce one job instance per image to build. You can independently configure each instance of these jobs by redefining the variables described throughout this documentation.

For example, if you want to build two Docker images, you must specify where the Dockerfiles are located and where the resulting images will be stored. You can do so by adding a patch to the .docker-base job in your .gitlab-ci.yml file so that it looks like this:

.docker-base:
  parallel:
    matrix:
    - DOCKER_FILE: "front/Dockerfile"
      DOCKER_SNAPSHOT_IMAGE: "$CI_REGISTRY/$CI_PROJECT_PATH/front:$CI_COMMIT_REF_SLUG"
      DOCKER_RELEASE_IMAGE: "$CI_REGISTRY/$CI_PROJECT_PATH/front:$CI_COMMIT_REF_NAME"
    - DOCKER_FILE: "back/Dockerfile"
      DOCKER_SNAPSHOT_IMAGE: "$CI_REGISTRY/$CI_PROJECT_PATH/back:$CI_COMMIT_REF_SLUG"
      DOCKER_RELEASE_IMAGE: "$CI_REGISTRY/$CI_PROJECT_PATH/back:$CI_COMMIT_REF_NAME"

If you need to redefine a variable with the same value for all your Dockerfiles, you can just declare this variable as a global variable. For example, if you want to build all your images using Docker-in-Docker, you can simply define the DOCKER_DIND_BUILD variable as a global variable:

variables:
  DOCKER_DIND_BUILD: "true"

Secrets management

Here are some advices about your secrets (variables marked with a 🔒):

1. Manage them as project or group CI/CD variables:
   • masked to prevent them from being inadvertently displayed in your job logs,
   • protected if you want to secure some secrets you don't want everyone in the project to have access to (for instance production secrets).
2. In case a secret contains characters that prevent it from being masked, simply define its value as the Base64 encoded value prefixed with @b64@: it will then be possible to mask it and the template will automatically decode it prior to using it.
3. Don't forget to escape special characters (ex: $ -> $$).

Jobs

docker-lint job

This job performs a Lint on your Dockerfile.

It is bound to the build stage, and uses the following variables:

Name	Description	Default value
DOCKER_LINT_IMAGE	The dockerlint image	projectatomic/dockerfile-lint:latest
DOCKER_LINT_ARGS	Additional dockerfile_lint arguments	(none)

In case you have to disable some rules, copy and edit the rules into mycustomdockerlint.yml and set DOCKER_LINT_ARGS: '-r mycustomdockerlint.yml'

docker-hadolint job

This job performs a Lint on your Dockerfile.

It is bound to the build stage, and uses the following variables:

Name	Description	Default value
DOCKER_HADOLINT_IMAGE	The Hadolint image	hadolint/hadolint:latest-alpine
DOCKER_HADOLINT_ARGS	Additional hadolint arguments	(none)