feat: add buildah support for building images

BREAKING CHANGE: $DOCKER_DIND_BUILD no longer supported (replaced by $`DOCKER_BUILD_TOOL`)

feat: add buildah support for building images
f8de5636 · Kiran Patel · Pierre Smeyers · 9799ef7a · f8de5636 · f8de5636
Commit f8de5636 authored 1 year ago by Kiran Patel Committed by Pierre Smeyers 1 year ago
--- a/README.md
+++ b/README.md
@@ -15,16 +15,19 @@ include:

 ## Understanding the Docker template

-The template supports two ways of building your Docker images:
+The template supports following ways of building container images:

-1. The former **Docker-in-Docker** technique, that was widely used for years because of no other alternative, but that
+1. The former **Docker-in-Docker (DinD)** technique, that was widely used for years because of no other alternative, but that
  is now commonly recognized to have **significant security issues** ([read this post](https://jpetazzo.github.io/2015/09/03/do-not-use-docker-in-docker-for-ci/) for more info),
-2. Or using [kaniko](https://github.com/GoogleContainerTools/kaniko), an open-source tool from Google for building Docker
+2. Or using [kaniko](https://github.com/GoogleContainerTools/kaniko), an open-source, daemonless tool from Google for building Docker
  images, and that solves Docker-in-Docker security issues (and also speeds-up build times).
+3. Or using [buildah](https://buildah.io/), an open-source, daemonless tool backed by RedHat for building Docker
+  images, and that solves Docker-in-Docker security issues (and also speeds-up build times), and can also be configured to run rootless.

 By default, the template uses the [kaniko](https://docs.gitlab.com/ee/ci/docker/using_kaniko.html) way, but you may
-activate the Docker-in-Docker build at your own risks by setting `DOCKER_DIND_BUILD` to `true` (see below).
-:warning: In that case, make sure your runner has required privileges to run Docker-in-Docker ([see GitLab doc](https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-in-docker-workflow-with-docker-executor)).
+select an alternate build tool by using the `DOCKER_BUILD_TOOL` variable (see below).
+
+:warning: If you choose to use 'Docker-in-Docker' option considering the associated security risks, make sure your runner has required privileges to run Docker-in-Docker ([see GitLab doc](https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-docker-in-docker-workflow-with-docker-executor)).

 ### Global variables

@@ -32,8 +35,9 @@ The Docker template uses some global configuration used throughout all jobs.

 | Name                  | Description                            | Default value     |
 | --------------------- | -------------------------------------- | ----------------- |
-| `DOCKER_DIND_BUILD`   | Set to enable Docker-in-Docker build (:warning: unsecured, requires privileged runners). | _(none)_ (kaniko build by default) |
-| `DOCKER_KANIKO_IMAGE` | The Docker image used to run kaniko - _for kaniko build only_ | `gcr.io/kaniko-project/executor:debug` (use `debug` images for GitLab) |
+| `DOCKER_BUILD_TOOL`   | The build tool to use for building container image, possible values are `kaniko`, `buildah` or `dind` | `kaniko` |
+| `DOCKER_KANIKO_IMAGE` | The image used to run `kaniko` - _for kaniko build only_ | `gcr.io/kaniko-project/executor:debug` (use `debug` images for GitLab) |
+| `DOCKER_BUILDAH_IMAGE` | The image used to run `buildah` - _for buildah build only_ | `quay.io/buildah/stable` |
 | `DOCKER_IMAGE`        | The Docker image used to run the docker client (see [full list](https://hub.docker.com/r/library/docker/)) - _for Docker-in-Docker build only_ | `registry.hub.docker.com/library/docker:latest`  |
 | `DOCKER_DIND_IMAGE`   | The Docker image used to run the Docker daemon (see [full list](https://hub.docker.com/r/library/docker/)) - _for Docker-in-Docker build only_ | `registry.hub.docker.com/library/docker:dind`    |
 | `DOCKER_FILE`         | The path to your `Dockerfile`          | `./Dockerfile`    |
@@ -216,11 +220,11 @@ You can do so by adding a patch to the `.docker-base` job in your `.gitlab-ci.ym
      DOCKER_RELEASE_IMAGE: "$CI_REGISTRY/$CI_PROJECT_PATH/back:$CI_COMMIT_REF_NAME"
 ```

-If you need to redefine a variable with the same value for all your Dockerfiles, you can just declare this variable as a global variable. For example, if you want to build all your images using Docker-in-Docker, you can simply define the `DOCKER_DIND_BUILD` variable as a global variable:
+If you need to redefine a variable with the same value for all your Dockerfiles, you can just declare this variable as a global variable. For example, if you want to build all your images using `buildah`, you can simply define the `DOCKER_BUILD_TOOL` variable as a global variable with value `buildah`:

 ```yaml
 variables:
-  DOCKER_DIND_BUILD: "true"
+  DOCKER_BUILD_TOOL: "buildah"
 ```

 ### Secrets management
@@ -282,7 +286,7 @@ In addition to a textual report in the console, this job produces the following
 | `reports/docker-hadolint-*.native.json`      | native hadolint test report (json) | [DefectDojo integration](https://defectdojo.github.io/django-DefectDojo/integrations/parsers/#hadolint)<br/>_This report is generated only if DefectDojo template is detected_ |
 | `reports/docker-hadolint-*.codeclimate.json` | hadolint (GitLab) codeclimate format | [GitLab integration](https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html#artifactsreportscodequality) |

-### `docker-build` job
+### `docker-*-build` jobs

 This job builds the image and publishes it to the _snapshot_ repository.

@@ -290,8 +294,9 @@ It is bound to the `package-build` stage, and uses the following variables:

 | Name                            | Description                                                                                                   | Default value                  |
 | ------------------------------- | ------------------------------------------------------------------------------------------------------------- | ------------------------------ |
-| `DOCKER_BUILD_ARGS`             | Additional `docker build`/`kaniko` arguments                                                                  | _(none)_                       |
-| `DOCKER_REGISTRY_MIRROR`        | URL of a Docker registry mirror to use during the image build (instead of default `https://index.docker.io`)  | _(none)_                       |
+| `DOCKER_BUILD_ARGS`             | Additional `docker/kaniko/buildah` `build` arguments                                                                  | _(none)_                       |
+| `DOCKER_REGISTRY_MIRROR`        | URL of a Docker registry mirror to use during the image build (instead of default `https://index.docker.io`) <br>:warning: Used by the `kaniko` and `dind` options only  | _(none)_                       |
+| `CONTAINER_REGISTRIES_CONFIG_FILE` | The [`registries.conf`](https://www.redhat.com/sysadmin/manage-container-registries) configuration to be used<br>:warning: Used by the `buildah` build only | _(none)_ |
 | `DOCKER_METADATA`               | Additional `docker build`/`kaniko` arguments to set label                                                     | OCI Image Format Specification |

 This job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ee/ci/pipelines/job_artifacts.html#artifactsreportsdotenv)):

--- a/kicker.json
+++ b/kicker.json
@@ -4,24 +4,36 @@
  "template_path": "templates/gitlab-ci-docker.yml",
  "kind": "package",
  "variables": [
+    {
+      "name": "DOCKER_BUILD_TOOL",
+      "type": "enum",
+      "values": ["kaniko", "buildah", "dind"],
+      "description": "The build tool to use for building container image",
+      "default": "kaniko"
+    },
    {
      "name": "DOCKER_KANIKO_IMAGE",
-      "description": "The Docker image used to run kaniko\n\n_for kaniko build only_",
+      "description": "The image used to run kaniko\n\n_for kaniko build only_",
      "default": "gcr.io/kaniko-project/executor:debug"
    },
+    {
+      "name": "DOCKER_BUILDAH_IMAGE",
+      "description": "The image used to run buildah\n\n_for buildah build only_",
+      "default": "quay.io/buildah/stable:latest"
+    },
    {
      "name": "DOCKER_IMAGE",
-      "description": "The Docker image used to run the docker client\n\n_for Docker-in-Docker build only_",
+      "description": "The image used to run the docker client\n\n_for Docker-in-Docker(dind) build only_",
      "default": "registry.hub.docker.com/library/docker:latest"
    },
    {
      "name": "DOCKER_DIND_IMAGE",
-      "description": "The Docker image used to run the Docker daemon\n\n_for Docker-in-Docker build only_",
+      "description": "The image used to run the Docker daemon\n\n_for Docker-in-Docker(dind) build only_",
      "default": "registry.hub.docker.com/library/docker:dind"
    },
    {
      "name": "DOCKER_SKOPEO_IMAGE",
-      "description": "The docker image used to publish docker image with Skopeo",
+      "description": "The image used to publish docker image with Skopeo",
      "default": "quay.io/skopeo/stable:latest"
    },
    {
@@ -52,11 +64,11 @@
    },
    {
      "name": "DOCKER_BUILD_ARGS",
-      "description": "Additional docker build/kaniko arguments"
+      "description": "Additional docker/kaniko/buildah build arguments"
    },
    {
      "name": "DOCKER_METADATA",
-      "description": "Additional docker build/kaniko arguments to set labels",
+      "description": "Additional metadata to set as labels",
      "default": "--label org.opencontainers.image.url=${CI_PROJECT_URL} --label org.opencontainers.image.source=${CI_PROJECT_URL} --label org.opencontainers.image.title=${CI_PROJECT_PATH} --label org.opencontainers.image.ref.name=${CI_COMMIT_REF_NAME} --label org.opencontainers.image.revision=${CI_COMMIT_SHA} --label org.opencontainers.image.created=${CI_JOB_STARTED_AT}",
      "advanced": true
    },
@@ -78,7 +90,12 @@
    },
    {
      "name": "DOCKER_REGISTRY_MIRROR",
-      "description": "URL of a Docker registry mirror to use instead of default `https://index.docker.io`"
+      "description": "URL of a Docker registry mirror to use instead of default `https://index.docker.io`\n\n_Used by `kaniko` and `dind` builds only_"
+    },
+    {
+      "name": "CONTAINER_REGISTRIES_CONFIG_FILE",
+      "description": "The [registries.conf](https://www.redhat.com/sysadmin/manage-container-registries) configuration to be used\n\n_Used by the `buildah` build only_",
+      "advanced": true
    }
  ],
  "features": [
@@ -118,12 +135,6 @@
        }
      ]
    },
-    {
-      "id": "dind",
-      "name": "Docker-in-Docker",
-      "description": "Use Docker-in-Docker to build the image (instead of Kaniko)\n\n_Warning: unsecured, requires privileged runners_",
-      "enable_with": "DOCKER_DIND_BUILD"
-    },
    {
      "id": "healthcheck",
      "name": "Health Check",

--- a/templates/gitlab-ci-docker.yml
+++ b/templates/gitlab-ci-docker.yml
@@ -20,6 +20,7 @@ workflow:
    - if: '$CI_COMMIT_BRANCH && $CI_OPEN_MERGE_REQUESTS'
      when: never
    - when: always
+    

 # test job prototype: implement adaptive pipeline rules
 .test-policy:
@@ -50,6 +51,7 @@ variables:
  DOCKER_DIND_IMAGE: "registry.hub.docker.com/library/docker:dind"
  DOCKER_KANIKO_IMAGE: "gcr.io/kaniko-project/executor:debug"
  DOCKER_SKOPEO_IMAGE: "quay.io/skopeo/stable:latest"
+  DOCKER_BUILDAH_IMAGE: "quay.io/buildah/stable:latest"

  # for retro-compatibility (deprecated & undocumented)
  DOCKER_DOCKERFILE_PATH: "."
@@ -90,6 +92,15 @@ variables:
    --label org.opencontainers.image.revision=${CI_COMMIT_SHA}
    --label org.opencontainers.image.created=${CI_JOB_STARTED_AT}

+  # default to kaniko, possible options : kaniko|buildah|dind
+  DOCKER_BUILD_TOOL:
+    value: "kaniko"
+    options:
+      - "kaniko"
+      - "buildah"
+      - "dind"
+    description: "The build tool to use for building container image"
+
 # ==================================================
 # Stages definition
 # ==================================================
@@ -327,15 +338,12 @@ stages:
    docker_release_config_json=$(echo -n "{\"auths\":{\"$docker_release_registry_host\":{\"auth\":\"$docker_release_authent_token\"},\"HttpHeaders\":{\"User-Agent\":\"$USER_AGENT\"}}}")

    # Create the configuration file for Docker and Kaniko
-    mkdir -p /root/.docker
-    mkdir -p /kaniko/.docker
+    mkdir -p "$HOME/.docker"
    if [ -f "${DOCKER_CONFIG_FILE}" ]
    then
-      awkenvsubst < "${DOCKER_CONFIG_FILE}" > /root/.docker/config.json
-      awkenvsubst < "${DOCKER_CONFIG_FILE}" > /kaniko/.docker/config.json
+      awkenvsubst < "${DOCKER_CONFIG_FILE}" > "$HOME/.docker/config.json"
    else
-      echo "${docker_snapshot_config_json}" > /root/.docker/config.json
-      echo "${docker_snapshot_config_json}" > /kaniko/.docker/config.json
+      echo "${docker_snapshot_config_json}" > "$HOME/.docker/config.json"
    fi

    # Create the configuration file for Skopeo
@@ -405,6 +413,16 @@ stages:
    /kaniko/executor --context "$(docker_context_path)" --dockerfile "$DOCKER_FILE" --destination "$docker_image" --cache --cache-dir="$KANIKO_CACHE_DIR" --verbosity $DOCKER_KANIKO_VERBOSITY $kaniko_registry_mirror_option $DOCKER_METADATA $DOCKER_BUILD_ARGS "$@"
  }

+  # Used by containers tools like buildah, skopeo.
+  function configure_containers_registries() {
+    if [[ -n "$CONTAINER_REGISTRIES_CONFIG_FILE" ]]
+    then
+      mkdir -p "$HOME/.config/containers"
+      echo "${CONTAINER_REGISTRIES_CONFIG_FILE}" > "$HOME/.config/containers/registries.conf"
+      log_info "Configured $HOME/.config/containers/registries.conf"
+    fi
+  }
+
  init_workspace

  # ENDSCRIPT
@@ -453,7 +471,7 @@ stages:
          if [[ -n "${DOCKER_REGISTRY_MIRROR}" ]]; then dockerd-entrypoint.sh --registry-mirror ${DOCKER_REGISTRY_MIRROR}; else dockerd-entrypoint.sh; fi || exit
  before_script:
    - *docker-scripts
-    - if ! is_runner_dind_capable; then fail "Docker-in-Docker is not enabled on this runner. Either use a Docker-in-Docker capable runner, or disable this job by unsetting \$DOCKER_DIND_BUILD"; fi
+    - if ! is_runner_dind_capable; then fail "Docker-in-Docker is not enabled on this runner. Either use a Docker-in-Docker capable runner, or disable this job by setting \$DOCKER_BUILD_TOOL to a different value"; fi

 # ==================================================
 # Stage: build
@@ -530,8 +548,7 @@ docker-kaniko-build:
      dotenv:
        - docker.env
  rules:
-    # execute if $DOCKER_DIND_BUILD not set
-    - if: '$DOCKER_DIND_BUILD == null || $DOCKER_DIND_BUILD == ""'
+    - if: '$DOCKER_BUILD_TOOL == "kaniko"'

 docker-dind-build:
  extends: .docker-dind-base
@@ -560,7 +577,36 @@ docker-dind-build:
      dotenv:
        - docker.env
  rules:
-    - if: $DOCKER_DIND_BUILD
+    - if: '$DOCKER_BUILD_TOOL == "dind"'
+
+docker-buildah-build:
+  extends: .docker-base
+  stage: package-build
+  image: "$DOCKER_BUILDAH_IMAGE"
+  variables:
+    BUILDAH_BUILD_CACHE: "$CI_REGISTRY_IMAGE/snapshot/cache"
+  script:
+    - configure_containers_registries
+    # build and push image
+    - buildah build --file "$DOCKER_FILE" --tag $DOCKER_SNAPSHOT_IMAGE --layers --cache-from $BUILDAH_BUILD_CACHE --cache-to $BUILDAH_BUILD_CACHE --build-arg http_proxy="$http_proxy" --build-arg https_proxy="$https_proxy" --build-arg no_proxy="$no_proxy" $DOCKER_METADATA $DOCKER_BUILD_ARGS "$(docker_context_path)"
+    - buildah push --digestfile .img-digest.txt "$DOCKER_SNAPSHOT_IMAGE"
+    # display digest of the resulting image
+    - cat .img-digest.txt 
+    # create dotenv file
+    - docker_digest=$(cat .img-digest.txt)
+    - docker_repository=${DOCKER_SNAPSHOT_IMAGE%:*}
+    - docker_tag=${DOCKER_SNAPSHOT_IMAGE##*:}
+    - echo "docker_image=$DOCKER_SNAPSHOT_IMAGE" > docker.env
+    - echo "docker_image_digest=$docker_repository@$docker_digest" >> docker.env
+    - echo "docker_repository=$docker_repository" >> docker.env
+    - echo "docker_tag=$docker_tag" >> docker.env
+    - echo "docker_digest=$docker_digest" >> docker.env
+  artifacts:
+    reports:
+      dotenv:
+        - docker.env
+  rules:
+    - if: '$DOCKER_BUILD_TOOL == "buildah"'

 # ==================================================
 # Stage: package-test
@@ -626,7 +672,7 @@ docker-healthcheck:
  rules:
    - if: '$DOCKER_HEALTHCHECK_DISABLED == "true"'
      when: never
-    - if: '$DOCKER_DIND_BUILD == null || $DOCKER_DIND_BUILD == ""'
+    - if: '$DOCKER_BUILD_TOOL != "dind"'
      when: never
    - !reference [.test-policy, rules]