Merge branch 'feat/add-aws-variant' into 'master'

feat(ecr): add Amazon ECR variant

See merge request to-be-continuous/docker!87

README.md

@@ -77,7 +77,6 @@ The **snapshot** and **release** images are defined by the following variables:
 | `DOCKER_SNAPSHOT_IMAGE`   | Docker snapshot image | `$CI_REGISTRY_IMAGE/snapshot:$CI_COMMIT_REF_SLUG` |
 | `DOCKER_RELEASE_IMAGE`    | Docker release image  | `$CI_REGISTRY_IMAGE:$CI_COMMIT_REF_NAME`          |
 
-
 As you can see, the Docker template is configured by default to use the GitLab container registry.
 You may perfectly override this and use another Docker registry, but be aware of a few things:
 
@@ -104,7 +103,6 @@ variables:
 | :lock: `DOCKER_REGISTRY_USER`    | Docker registry username for image registry |
 | :lock: `DOCKER_REGISTRY_PASSWORD`| Docker registry password for image registry  |
 
-
 #### Using different registries for snapshot and release
 
 If you use **different registries** for snapshot and release images, you shall use separate configuration variables:
@@ -116,8 +114,6 @@ If you use **different registries** for snapshot and release images, you shall u
 | :lock: `DOCKER_REGISTRY_RELEASE_USER`    | Docker registry username for release image registry |
 | :lock: `DOCKER_REGISTRY_RELEASE_PASSWORD`| Docker registry password for release image registry |
 
-
-
 #### Setting your own Docker configuration file (advanced)
 
 There might be cases where you need to provide the complete [Docker configuration file](https://docs.docker.com/engine/reference/commandline/cli/#configuration-files):
@@ -641,6 +637,9 @@ List of requirements before using this variant for publishing your container ima
 | `GCP_RELEASE_OIDC_PROVIDER`  | Workload Identity Provider to push the release image _(only define if different from default)_ | _none_ |
 | `GCP_RELEASE_OIDC_ACCOUNT`   | Service Account to use to push the release image _(only define if different from default)_ | _none_ |
 
+:warning: if using Kaniko, don't forget to either create the cache repository (snapshot image repository + `/cache`) or override `$KANIKO_SNAPSHOT_IMAGE_CACHE`
+to use the snapshot image repository (will host your snapshot image as well as cached layers).
+
 #### Example
 
 ```yaml
@@ -654,9 +653,10 @@ include:
 
 variables:
   # untested & unverified container image
-  DOCKER_SNAPSHOT_IMAGE: "{GCP_REGION}-docker.pkg.dev/{GCP_PROJECT_ID}/{YOUR_REPOSITORY}/{YOUR_IMAGE_NAME}/snapshot"
+  DOCKER_SNAPSHOT_IMAGE: "{GCP_REGION}-docker.pkg.dev/{GCP_PROJECT_ID}/{YOUR_REPOSITORY}/{YOUR_IMAGE_NAME}/snapshot:$CI_COMMIT_REF_SLUG"
+  # ⚠ don't forget to create the '{GCP_REGION}-docker.pkg.dev/{GCP_PROJECT_ID}/{YOUR_REPOSITORY}/{YOUR_IMAGE_NAME}/snapshot/cache' repo for Kaniko
   # validated container image (published)
-  DOCKER_RELEASE_IMAGE: "{GCP_REGION}-docker.pkg.dev/{GCP_PROJECT_ID}/{YOUR_REPOSITORY}/{YOUR_IMAGE_NAME}"
+  DOCKER_RELEASE_IMAGE: "{GCP_REGION}-docker.pkg.dev/{GCP_PROJECT_ID}/{YOUR_REPOSITORY}/{YOUR_IMAGE_NAME}:$CI_COMMIT_REF_NAME"
   # default WIF provider
   GCP_OIDC_PROVIDER: "projects/{GCP_PROJECT_NUMBER}/locations/global/workloadIdentityPools/{YOUR_WIF_POOL_NAME}/providers/gitlab-diod"
   # default GCP Service Account
@@ -667,3 +667,74 @@ variables:
   GCP_SNAPSHOT_OIDC_ACCOUNT: "{YOUR_REGISTRY_SA}@{GCP_PROJECT_ID}.iam.gserviceaccount.com"
   DOCKER_BUILD_TOOL: "kaniko" # Only Kaniko has been proved to work for this use case YET
 ```
+
+### Amazon Elastic Container Registry
+
+This variant allows publishing your container images to Amazon's [Elastic Container Registry](https://docs.aws.amazon.com/ecr/).
+
+It takes care of retrieving an [ECR authorization token](https://docs.aws.amazon.com/AmazonECR/latest/APIReference/API_GetAuthorizationToken.html)
+that will be used as a temporary credential to login to the ECR registry.
+
+In order to use the AWS APIs, the variant supports two authentication methods:
+
+1. [federated authentication using OpenID Connect](https://docs.gitlab.com/ee/ci/cloud_services/aws/) (**recommended method**),
+2. or basic authentication with AWS access key ID & secret access key.
+
+:warning: when using this variant, you must have created the ECR repositories to push the snapshot and/or the release images.
+
+#### Configuration
+
+| Name                     | description                            | default value     |
+| ------------------------ | -------------------------------------- | ----------------- |
+| `TBC_AWS_PROVIDER_IMAGE` | The [AWS Auth Provider](https://gitlab.com/to-be-continuous/tools/aws-auth-provider) image to use (can be overridden) | `$CI_REGISTRY/to-be-continuous/tools/aws-auth-provider:master` |
+| `AWS_REGION`             | Default region (where the ECR registry is located) | _none_ |
+| `AWS_SNAPSHOT_REGION`    | Region of the ECR registry for the snapshot image _(only define if different from default)_ | _none_ |
+| `AWS_RELEASE_REGION`     | Region of the ECR registry for the release image _(only define if different from default)_ | _none_ |
+
+:warning: if using Kaniko, don't forget to either create the cache repository (snapshot image repository + `/cache`) or override `$KANIKO_SNAPSHOT_IMAGE_CACHE`
+to use the snapshot image repository (will host your snapshot image as well as cached layers).
+
+##### OIDC authentication config
+
+This is the recommended authentication method. In order to use it, first carefuly follow [GitLab's documentation](https://docs.gitlab.com/ee/ci/cloud_services/aws/),
+then set the required configuration.
+
+| Name                     | description                            | default value     |
+| ------------------------ | -------------------------------------- | ----------------- |
+| `AWS_OIDC_AUD`           | The `aud` claim for the JWT token      | `$CI_SERVER_URL` |
+| `AWS_OIDC_ROLE_ARN`      | Default IAM Role ARN associated with GitLab | _none_ |
+| `AWS_SNAPSHOT_OIDC_ROLE_ARN`| IAM Role ARN associated with GitLab for the snapshot image _(only define if different from default)_| _none_ |
+| `AWS_RELEASE_OIDC_ROLE_ARN`| IAM Role ARN associated with GitLab for the release image _(only define if different from default)_| _none_ |
+
+##### Basic authentication config
+
+| Name                     | description                            | default value     |
+| ------------------------ | -------------------------------------- | ----------------- |
+| `AWS_ACCESS_KEY_ID`      | Default access key ID | _none_ (disabled) |
+| `AWS_SECRET_ACCESS_KEY`  | Default secret access key | _none_ (disabled) |
+| `AWS_SNAPSHOT_ACCESS_KEY_ID`| Access key ID for the snapshot image _(only define if different from default)_ | _none_ |
+| `AWS_SNAPSHOT_SECRET_ACCESS_KEY`| Secret access key for the snapshot image _(only define if different from default)_ | _none_ |
+| `AWS_RELEASE_ACCESS_KEY_ID`| Access key ID for the release image _(only define if different from default)_ | _none_ |
+| `AWS_RELEASE_SECRET_ACCESS_KEY`| Secret access key for the release image _(only define if different from default)_ | _none_ |
+
+#### Example
+
+```yaml
+include:
+  - project: 'to-be-continuous/docker'
+    ref: "5.2.0"
+    file: '/templates/gitlab-ci-docker.yml'
+  - project: 'to-be-continuous/docker'
+    ref: "5.2.0"
+    file: '/templates/gitlab-ci-docker-ecr.yml'
+
+variables:
+  AWS_REGION: "us-east-1"
+  # untested & unverified container image
+  DOCKER_SNAPSHOT_IMAGE: "123456789012.dkr.ecr.us-east-1.amazonaws.com/$CI_PROJECT_PATH/snapshot:$CI_COMMIT_REF_SLUG"
+  # ⚠ don't forget to create the '123456789012.dkr.ecr.us-east-1.amazonaws.com/$CI_PROJECT_PATH/snapshot/cache' repo for Kaniko
+  # validated container image (published)
+  DOCKER_RELEASE_IMAGE: "123456789012.dkr.ecr.us-east-1.amazonaws.com/$CI_PROJECT_PATH:$CI_COMMIT_REF_NAME"
+  # default Role ARN (using OIDC authentication method)
+  AWS_OIDC_ROLE_ARN: "arn:aws:iam::123456789012:role/gitlab-ci"
+```

kicker.json

@@ -269,7 +269,7 @@
         {
           "name": "TBC_GCP_PROVIDER_IMAGE",
           "description": "The [GCP Auth Provider](https://gitlab.com/to-be-continuous/tools/gcp-auth-provider) image to use",
-          "default": "$CI_REGISTRY/to-be-continuous/tools/vault-secrets-provider:master",
+          "default": "$CI_REGISTRY/to-be-continuous/tools/gcp-auth-provider:main",
           "advanced": true
         },
         {
@@ -301,6 +301,90 @@
           "advanced": true
         }
       ]
+    },
+    {
+      "id": "ecr",
+      "name": "Amazon ECR",
+      "description": "Retrieves a registry authentication for the Amazon's [Elastic Container Registry](https://docs.aws.amazon.com/ecr/)",
+      "template_path": "templates/gitlab-ci-docker-ecr.yml",
+      "variables": [
+        {
+          "name": "TBC_AWS_PROVIDER_IMAGE",
+          "description": "The [AWS Auth Provider](https://gitlab.com/to-be-continuous/tools/aws-auth-provider) image to use",
+          "default": "$CI_REGISTRY/to-be-continuous/tools/aws-auth-provider:master",
+          "advanced": true
+        },
+        {
+          "name": "AWS_REGION",
+          "description": "Default region (where the ECR registry is located)"
+        },
+        {
+          "name": "AWS_SNAPSHOT_REGION",
+          "description": "Region of the ECR registry for the snapshot image _(only define if different from default)_",
+          "advanced": true
+        },
+        {
+          "name": "AWS_RELEASE_REGION",
+          "description": "Region of the ECR registry for the release image _(only define if different from default)_",
+          "advanced": true
+        },
+        {
+          "name": "AWS_OIDC_AUD",
+          "description": "The `aud` claim for the JWT token _(only required for [OIDC authentication](https://docs.gitlab.com/ee/ci/cloud_services/aws/))_",
+          "default": "$CI_SERVER_URL",
+          "advanced": true
+        },
+        {
+          "name": "AWS_OIDC_ROLE_ARN",
+          "description": "Default IAM Role ARN associated with GitLab _(only required for [OIDC authentication](https://docs.gitlab.com/ee/ci/cloud_services/aws/))_"
+        },
+        {
+          "name": "AWS_SNAPSHOT_OIDC_ROLE_ARN",
+          "description": "IAM Role ARN associated with GitLab for the snapshot image _(only required for [OIDC authentication](https://docs.gitlab.com/ee/ci/cloud_services/aws/) and if different from default)_",
+          "advanced": true
+        },
+        {
+          "name": "AWS_RELEASE_OIDC_ROLE_ARN",
+          "description": "IAM Role ARN associated with GitLab for the release image _(only required for [OIDC authentication](https://docs.gitlab.com/ee/ci/cloud_services/aws/) and if different from default)_",
+          "advanced": true
+        },
+        {
+          "name": "AWS_ACCESS_KEY_ID",
+          "description": "Default access key ID (only required for basic authentication)",
+          "secret": true,
+          "advanced": true
+        },
+        {
+          "name": "AWS_SECRET_ACCESS_KEY",
+          "description": "Default secret access key (only required for basic authentication)",
+          "secret": true,
+          "advanced": true
+        },
+        {
+          "name": "AWS_SNAPSHOT_ACCESS_KEY_ID",
+          "description": "Access key ID for the snapshot image (only required for basic authentication and if different from default)",
+          "secret": true,
+          "advanced": true
+        },
+        {
+          "name": "AWS_SNAPSHOT_SECRET_ACCESS_KEY",
+          "description": "Secret access key for the snapshot image (only required for basic authentication and if different from default)",
+          "secret": true,
+          "advanced": true
+        },
+        {
+          "name": "AWS_RELEASE_ACCESS_KEY_ID",
+          "description": "Access key ID for the release image (only required for basic authentication and if different from default)",
+          "secret": true,
+          "advanced": true
+        },
+        {
+          "name": "AWS_RELEASE_SECRET_ACCESS_KEY",
+          "description": "Secret access key for the release image (only required for basic authentication and if different from default)",
+          "secret": true,
+          "advanced": true
+        }
+      ]
     }
   ]
 }

templates/gitlab-ci-docker-ecr.yml

+# =====================================================================================================================
+# === AWS Auth template variant
+# =====================================================================================================================
+variables:
+  TBC_AWS_AUTH_PROVIDER: "$CI_REGISTRY/to-be-continuous/tools/aws-auth-provider:master"
+  AWS_OIDC_AUD: "$CI_SERVER_URL"
+
+.docker-base:
+  services:
+    - name: "$TBC_TRACKING_IMAGE"
+      command: ["--service", "docker", "5.4.1"]
+    - name: "$TBC_AWS_AUTH_PROVIDER"
+      alias: "aws-auth-provider"
+  id_tokens:
+    # required for OIDC auth
+    AWS_JWT:
+      aud: "$AWS_OIDC_AUD"
+  variables:
+    # DOCKER_REGISTRY_SNAPSHOT_USER: "@url@http://aws-auth-provider/ecr/auth/username?env_ctx=SNAPSHOT"
+    # DOCKER_REGISTRY_RELEASE_USER: "@url@http://aws-auth-provider/ecr/auth/username?env_ctx=RELEASE"
+    DOCKER_REGISTRY_SNAPSHOT_USER: "AWS" # GetAuthorizationToken API always generate token for user 'AWS'
+    DOCKER_REGISTRY_RELEASE_USER: "AWS" # GetAuthorizationToken API always generate token for user 'AWS'
+    DOCKER_REGISTRY_SNAPSHOT_PASSWORD: "@url@http://aws-auth-provider/ecr/auth/password?env_ctx=SNAPSHOT"
+    DOCKER_REGISTRY_RELEASE_PASSWORD: "@url@http://aws-auth-provider/ecr/auth/password?env_ctx=RELEASE"
+    #  secrets have to be explicitly declared in the YAML to be exported to the service
+    AWS_JWT: "$AWS_JWT"
+    # can't use AWS_ACCESS_KEY_ID as it is read by boto3
+    AWS_DEFAULT_ACCESS_KEY_ID: "$AWS_ACCESS_KEY_ID"
+    # can't use AWS_SECRET_ACCESS_KEY as it is read by boto3
+    AWS_DEFAULT_SECRET_ACCESS_KEY: "$AWS_SECRET_ACCESS_KEY"
+    AWS_SNAPSHOT_ACCESS_KEY_ID: "$AWS_SNAPSHOT_ACCESS_KEY_ID"
+    AWS_SNAPSHOT_SECRET_ACCESS_KEY: "$AWS_SNAPSHOT_SECRET_ACCESS_KEY"
+    AWS_RELEASE_ACCESS_KEY_ID: "$AWS_RELEASE_ACCESS_KEY_ID"
+    AWS_RELEASE_SECRET_ACCESS_KEY: "$AWS_RELEASE_SECRET_ACCESS_KEY"