Skip to content
Snippets Groups Projects
Commit 7fd9f9f9 authored by Benguria Elguezabal, Gorka's avatar Benguria Elguezabal, Gorka
Browse files

Merge remote-tracking branch 'upstream/main'

# Conflicts:
#	.gitlab-ci.yml
#	CHANGELOG.md
parents 9be06f8c 9eddda6b
Branches
Tags
No related merge requests found
Pipeline #177729 passed
...@@ -8,8 +8,8 @@ Closes #999 ...@@ -8,8 +8,8 @@ Closes #999
## Checklist ## Checklist
* General: * General:
* [ ] use [rules](https://docs.gitlab.com/ee/ci/yaml/#rules) instead of [only/except](https://docs.gitlab.com/ee/ci/yaml/#onlyexcept-advanced) * [ ] use [rules](https://docs.gitlab.com/ci/yaml/#rules) instead of [only/except](https://docs.gitlab.com/ci/yaml/#onlyexcept-advanced)
* [ ] optimized [cache](https://docs.gitlab.com/ee/ci/caching/) configuration (wherever applicable) * [ ] optimized [cache](https://docs.gitlab.com/ci/caching/) configuration (wherever applicable)
* Publicly usable: * Publicly usable:
* [ ] untagged runners * [ ] untagged runners
* [ ] no proxy configuration but support `http_proxy`/`https_proxy`/`no_proxy` * [ ] no proxy configuration but support `http_proxy`/`https_proxy`/`no_proxy`
......
# [1.2.0](https://git.code.tecnalia.dev/smartdatalab/public/ci-cd-components/docker-compose/compare/1.1.0...1.2.0) (2025-01-29) ## [1.2.2](https://gitlab.com/to-be-continuous/docker-compose/compare/1.2.1...1.2.2) (2025-04-11)
### Bug Fixes
* **envsubst:** leave lines with '# nosubst' unchanged when substituting (used to be simply dropped) ([392e60d](https://gitlab.com/to-be-continuous/docker-compose/commit/392e60d69d47130f7f1a249ef8001c1fe6b234d0))
## [1.2.1](https://gitlab.com/to-be-continuous/docker-compose/compare/1.2.0...1.2.1) (2025-02-01)
### Bug Fixes
* homogenize new TBC envsubst mechanism ([63bca5c](https://gitlab.com/to-be-continuous/docker-compose/commit/63bca5c391b44614a0e34edcc4dcdc82f55e6c4d))
# [1.2.0](https://gitlab.com/to-be-continuous/docker-compose/compare/1.1.0...1.2.0) (2025-01-27)
### Features ### Features
......
...@@ -61,7 +61,7 @@ To contribute: ...@@ -61,7 +61,7 @@ To contribute:
1. Create an issue describing the bug or enhancement you want to propose (select the right issue template). 1. Create an issue describing the bug or enhancement you want to propose (select the right issue template).
2. Make sure the issue has been reviewed and agreed. 2. Make sure the issue has been reviewed and agreed.
3. Create a Merge Request, from your **own** fork (see [forking workflow](https://docs.gitlab.com/ee/user/project/repository/forking_workflow.html) documentation). 3. Create a Merge Request, from your **own** fork (see [forking workflow](https://docs.gitlab.com/user/project/repository/forking_workflow/) documentation).
Don't hesitate to mark your MR as `Draft` as long as you think it's not ready to be reviewed. Don't hesitate to mark your MR as `Draft` as long as you think it's not ready to be reviewed.
### Git Commit Conventions ### Git Commit Conventions
......
...@@ -4,8 +4,8 @@ This project implements a GitLab CI/CD template to deploy your application with ...@@ -4,8 +4,8 @@ This project implements a GitLab CI/CD template to deploy your application with
## Usage ## Usage
This template can be used both as a [CI/CD component](https://docs.gitlab.com/ee/ci/components/#use-a-component) This template can be used both as a [CI/CD component](https://docs.gitlab.com/ci/components/#use-a-component)
or using the legacy [`include:project`](https://docs.gitlab.com/ee/ci/yaml/index.html#includeproject) syntax. or using the legacy [`include:project`](https://docs.gitlab.com/ci/yaml/#includeproject) syntax.
### Use as a CI/CD component ### Use as a CI/CD component
...@@ -14,7 +14,7 @@ Add the following to your `.gitlab-ci.yml`: ...@@ -14,7 +14,7 @@ Add the following to your `.gitlab-ci.yml`:
```yaml ```yaml
include: include:
# 1: include the component # 1: include the component
- component: $CI_SERVER_FQDN/to-be-continuous/docker-compose/gitlab-ci-docker-compose@1.2.0 - component: $CI_SERVER_FQDN/to-be-continuous/docker-compose/gitlab-ci-docker-compose@1.2.2
# 2: set/override component inputs # 2: set/override component inputs
inputs: inputs:
# ⚠ this is only an example # ⚠ this is only an example
...@@ -32,7 +32,7 @@ Add the following to your `.gitlab-ci.yml`: ...@@ -32,7 +32,7 @@ Add the following to your `.gitlab-ci.yml`:
include: include:
# 1: include the template # 1: include the template
- project: 'to-be-continuous/docker-compose' - project: 'to-be-continuous/docker-compose'
ref: '1.2.0' ref: '1.2.2'
file: '/templates/gitlab-ci-docker-compose.yml' file: '/templates/gitlab-ci-docker-compose.yml'
variables: variables:
...@@ -67,7 +67,7 @@ _ongoing developments_ (a.k.a. _feature_ or _topic_ branches). ...@@ -67,7 +67,7 @@ _ongoing developments_ (a.k.a. _feature_ or _topic_ branches).
When enabled, it deploys the result from upstream build stages to a dedicated and temporary environment. When enabled, it deploys the result from upstream build stages to a dedicated and temporary environment.
It is only active for non-production, non-integration branches. It is only active for non-production, non-integration branches.
It is a strict equivalent of GitLab's [Review Apps](https://docs.gitlab.com/ee/ci/review_apps/) feature. It is a strict equivalent of GitLab's [Review Apps](https://docs.gitlab.com/ci/review_apps/) feature.
It also comes with a _cleanup_ job (accessible either from the _environments_ page, or from the pipeline view). It also comes with a _cleanup_ job (accessible either from the _environments_ page, or from the pipeline view).
...@@ -244,8 +244,8 @@ Part of this complexity can be handled by the lookup strategies described above ...@@ -244,8 +244,8 @@ Part of this complexity can be handled by the lookup strategies described above
* `${environment_type}`: the current environment type (`review`, `integration`, `staging` or `production`) * `${environment_type}`: the current environment type (`review`, `integration`, `staging` or `production`)
* `${environment_name}`: the application name to use for the current environment (ex: `myproject-review-fix-bug-12` or `myproject-staging`) * `${environment_name}`: the application name to use for the current environment (ex: `myproject-review-fix-bug-12` or `myproject-staging`)
* `${hostname}`: the environment hostname, extracted from the current environment url (after late variable expansion - see below) * `${hostname}`: the environment hostname, extracted from the current environment url (after late variable expansion - see below)
2. any [GitLab CI variable](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html) 2. any [GitLab CI variable](https://docs.gitlab.com/ci/variables/predefined_variables/)
3. any [custom variable](https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-docker-host) 3. any [custom variable](https://docs.gitlab.com/ci/variables/#add-a-cicd-variable-to-a-docker-host)
(ex: `${SECRET_TOKEN}` that you have set in your project CI/CD variables) (ex: `${SECRET_TOKEN}` that you have set in your project CI/CD variables)
Be aware that environment variables may be freely used and substituted in [dotenv files](https://docs.docker.com/compose/environment-variables/env-file/) Be aware that environment variables may be freely used and substituted in [dotenv files](https://docs.docker.com/compose/environment-variables/env-file/)
...@@ -256,7 +256,7 @@ using the appropriate [interpolation syntax](https://docs.docker.com/compose/env ...@@ -256,7 +256,7 @@ using the appropriate [interpolation syntax](https://docs.docker.com/compose/env
The Docker Compose template supports two ways of providing your environments url: The Docker Compose template supports two ways of providing your environments url:
* a **static way**: when the environments url can be determined in advance, probably because you're exposing your routes through a DNS you manage, * a **static way**: when the environments url can be determined in advance, probably because you're exposing your routes through a DNS you manage,
* a [**dynamic way**](https://docs.gitlab.com/ee/ci/environments/#set-a-dynamic-environment-url): when the url cannot be known before the * a [**dynamic way**](https://docs.gitlab.com/ci/environments/#set-a-dynamic-environment-url): when the url cannot be known before the
deployment job is executed. deployment job is executed.
The **static way** can be implemented simply by setting the appropriate configuration variable(s) depending on the environment (see environments configuration chapters): The **static way** can be implemented simply by setting the appropriate configuration variable(s) depending on the environment (see environments configuration chapters):
...@@ -285,7 +285,7 @@ the dynamically generated url. When detected by the template, it will use it as ...@@ -285,7 +285,7 @@ the dynamically generated url. When detected by the template, it will use it as
### Deployment output variables ### Deployment output variables
Each deployment job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ee/ci/yaml/artifacts_reports.html#artifactsreportsdotenv)): Each deployment job produces _output variables_ that are propagated to downstream jobs (using [dotenv artifacts](https://docs.gitlab.com/ci/yaml/artifacts_reports/#artifactsreportsdotenv)):
* `$environment_type`: set to the type of environment (`review`, `integration`, `staging` or `production`), * `$environment_type`: set to the type of environment (`review`, `integration`, `staging` or `production`),
* `$environment_name`: the application name (see below), * `$environment_name`: the application name (see below),
...@@ -301,12 +301,12 @@ You may also add and propagate your own custom variables, by pushing them to the ...@@ -301,12 +301,12 @@ You may also add and propagate your own custom variables, by pushing them to the
Here are some advices about your **secrets** (variables marked with a :lock:): Here are some advices about your **secrets** (variables marked with a :lock:):
1. Manage them as [project or group CI/CD variables](https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-docker-host): 1. Manage them as [project or group CI/CD variables](https://docs.gitlab.com/ci/variables/#add-a-cicd-variable-to-a-docker-host):
* [**masked**](https://docs.gitlab.com/ee/ci/variables/#mask-a-cicd-variable) to prevent them from being inadvertently * [**masked**](https://docs.gitlab.com/ci/variables/#mask-a-cicd-variable) to prevent them from being inadvertently
displayed in your job logs, displayed in your job logs,
* [**protected**](https://docs.gitlab.com/ee/ci/variables/#protected-cicd-variables) if you want to secure some secrets * [**protected**](https://docs.gitlab.com/ci/variables/#protected-cicd-variables) if you want to secure some secrets
you don't want everyone in the project to have access to (for instance production secrets). you don't want everyone in the project to have access to (for instance production secrets).
2. In case a secret contains [characters that prevent it from being masked](https://docs.gitlab.com/ee/ci/variables/#mask-a-cicd-variable), 2. In case a secret contains [characters that prevent it from being masked](https://docs.gitlab.com/ci/variables/#mask-a-cicd-variable),
simply define its value as the [Base64](https://en.wikipedia.org/wiki/Base64) encoded value prefixed with `@b64@`: simply define its value as the [Base64](https://en.wikipedia.org/wiki/Base64) encoded value prefixed with `@b64@`:
it will then be possible to mask it and the template will automatically decode it prior to using it. it will then be possible to mask it and the template will automatically decode it prior to using it.
3. Don't forget to escape special characters (ex: `$` -> `$$`). 3. Don't forget to escape special characters (ex: `$` -> `$$`).
...@@ -317,9 +317,9 @@ The Docker Compose template uses some global configuration used throughout all j ...@@ -317,9 +317,9 @@ The Docker Compose template uses some global configuration used throughout all j
| Input / Variable | Description | Default value | | Input / Variable | Description | Default value |
| ------------------------ |-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| ----------------- | | ------------------------ |-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| ----------------- |
| `image` / `DCMP_IMAGE` | The Docker image used to run Docker Compose CLI commands | `registry.hub.docker.com/library/docker:latest` | | `image` / `DCMP_IMAGE` | The Docker image used to run Docker Compose CLI commands | `registry.hub.docker.com/library/docker:latest` <br/>[![Trivy Badge](https://to-be-continuous.gitlab.io/doc/secu/trivy-badge-DCMP_IMAGE.svg)](https://to-be-continuous.gitlab.io/doc/secu/trivy-DCMP_IMAGE) |
| `cmd` / `DCMP_CMD` | The docker compose or stack command (`docker compose`, `docker-compose` or `docker stack`) | _none_ (auto) | | `cmd` / `DCMP_CMD` | The docker compose or stack command (`docker compose`, `docker-compose` or `docker stack`) | _none_ (auto) |
| `base-app-name` / `DCMP_BASE_APP_NAME`| Base application name | `$CI_PROJECT_NAME` ([see GitLab doc](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)) | | `base-app-name` / `DCMP_BASE_APP_NAME`| Base application name | `$CI_PROJECT_NAME` ([see GitLab doc](https://docs.gitlab.com/ci/variables/predefined_variables/)) |
| `environment-url` / `DCMP_ENVIRONMENT_URL`| Default environments url _(only define for static environment URLs declaration)_<br/>_supports late variable expansion (ex: `https://%{environment_name}.docker-compose.acme.com`)_ | _none_ | | `environment-url` / `DCMP_ENVIRONMENT_URL`| Default environments url _(only define for static environment URLs declaration)_<br/>_supports late variable expansion (ex: `https://%{environment_name}.docker-compose.acme.com`)_ | _none_ |
| `scripts-dir` / `DCMP_SCRIPTS_DIR`| Directory where Compose files, dotenv files and hook scripts are located | `.` _(root project dir)_ | | `scripts-dir` / `DCMP_SCRIPTS_DIR`| Directory where Compose files, dotenv files and hook scripts are located | `.` _(root project dir)_ |
| `up-opts` / `DCMP_UP_OPTS` | [`compose up` options](https://docs.docker.com/reference/cli/docker/compose/up/#options) (only when using Docker Compose) | `--no-build --remove-orphans --wait --wait-timeout 180` | | `up-opts` / `DCMP_UP_OPTS` | [`compose up` options](https://docs.docker.com/reference/cli/docker/compose/up/#options) (only when using Docker Compose) | `--no-build --remove-orphans --wait --wait-timeout 180` |
......
...@@ -104,7 +104,7 @@ ...@@ -104,7 +104,7 @@
{ {
"id": "review", "id": "review",
"name": "Review", "name": "Review",
"description": "Dynamic review environments for your topic branches (see GitLab [Review Apps](https://docs.gitlab.com/ee/ci/review_apps/))", "description": "Dynamic review environments for your topic branches (see GitLab [Review Apps](https://docs.gitlab.com/ci/review_apps/))",
"variables": [ "variables": [
{ {
"name": "DCMP_REVIEW_DOCKER_HOST", "name": "DCMP_REVIEW_DOCKER_HOST",
......
...@@ -464,9 +464,82 @@ stages: ...@@ -464,9 +464,82 @@ stages:
echo "$1" | tr '[:lower:]' '[:upper:]' | tr '[:punct:]' '_' echo "$1" | tr '[:lower:]' '[:upper:]' | tr '[:punct:]' '_'
} }
function awkenvsubst() { function tbc_envsubst() {
# escapes '&' char in variables for gsub awk '
awk '{while(match($0,"[$%]{[^}]*}")) {var=substr($0,RSTART+2,RLENGTH-3);val=ENVIRON[var];gsub("&","\\\\&",val);gsub("[$%]{"var"}",val)}}1' BEGIN {
count_replaced_lines = 0
# ASCII codes
for (i=0; i<=255; i++)
char2code[sprintf("%c", i)] = i
}
# determine encoding (from env or from file extension)
function encoding() {
enc = ENVIRON["TBC_ENVSUBST_ENCODING"]
if (enc != "")
return enc
if (match(FILENAME, /\.(json|yaml|yml)$/))
return "jsonstr"
return "raw"
}
# see: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/encodeURIComponent
function uriencode(str) {
len = length(str)
enc = ""
for (i=1; i<=len; i++) {
c = substr(str, i, 1);
if (index("abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789-_.!~*'\''()", c))
enc = enc c
else
enc = enc "%" sprintf("%02X", char2code[c])
}
return enc
}
/# *nosubst/ {
print $0
next
}
{
orig_line = $0
line = $0
count_repl_in_line = 0
# /!\ 3rd arg (match) not supported in BusyBox awk
while (match(line, /[$%]\{([[:alnum:]_]+)\}/)) {
expr_start = RSTART
expr_len = RLENGTH
# get var name
var = substr(line, expr_start+2, expr_len-3)
# get var value (from env)
val = ENVIRON[var]
# check variable is set
if (val == "") {
printf("[\033[1;93mWARN\033[0m] Environment variable \033[33;1m%s\033[0m is not set or empty\n", var) > "/dev/stderr"
} else {
enc = encoding()
if (enc == "jsonstr") {
gsub(/["\\]/, "\\\\&", val)
gsub("\n", "\\n", val)
gsub("\r", "\\r", val)
gsub("\t", "\\t", val)
} else if (enc == "uricomp") {
val = uriencode(val)
} else if (enc == "raw") {
} else {
printf("[\033[1;93mWARN\033[0m] Unsupported encoding \033[33;1m%s\033[0m: ignored\n", enc) > "/dev/stderr"
}
}
# replace expression in line
line = substr(line, 1, expr_start - 1) val substr(line, expr_start + expr_len)
count_repl_in_line++
}
if (count_repl_in_line) {
if (count_replaced_lines == 0)
printf("[\033[1;94mINFO\033[0m] Variable expansion occurred in file \033[33;1m%s\033[0m:\n", FILENAME) > "/dev/stderr"
count_replaced_lines++
printf("> line %s: %s\n", NR, orig_line) > "/dev/stderr"
}
print line
}
' "$@"
} }
function configure_network() { function configure_network() {
...@@ -474,7 +547,7 @@ stages: ...@@ -474,7 +547,7 @@ stages:
if [[ -f ".netrc" ]] if [[ -f ".netrc" ]]
then then
log_info "--- \\e[32m.netrc\\e[0m file found: envsubst and install" log_info "--- \\e[32m.netrc\\e[0m file found: envsubst and install"
awkenvsubst < .netrc > ~/.netrc tbc_envsubst .netrc > ~/.netrc
chmod 0600 ~/.netrc chmod 0600 ~/.netrc
fi fi
...@@ -533,7 +606,7 @@ stages: ...@@ -533,7 +606,7 @@ stages:
# special variable supported # special variable supported
TBC_CI_REGISTRY_TOKEN=$(echo -n "$CI_REGISTRY_USER:$CI_REGISTRY_PASSWORD" | base64 | tr -d '\n') TBC_CI_REGISTRY_TOKEN=$(echo -n "$CI_REGISTRY_USER:$CI_REGISTRY_PASSWORD" | base64 | tr -d '\n')
export TBC_CI_REGISTRY_TOKEN export TBC_CI_REGISTRY_TOKEN
awkenvsubst < .docker/config.json > ~/.docker/config.json tbc_envsubst .docker/config.json > ~/.docker/config.json
else else
log_info "--- \\e[32m.docker/config.json\\e[0m file not found: looking for TBC built images..." log_info "--- \\e[32m.docker/config.json\\e[0m file not found: looking for TBC built images..."
_image_vars=$(env | awk -F '=' "/^[A-Z]+_(SNAPSHOT|RELEASE)_IMAGE=/ {print \$1}" | sort | uniq) _image_vars=$(env | awk -F '=' "/^[A-Z]+_(SNAPSHOT|RELEASE)_IMAGE=/ {print \$1}" | sort | uniq)
...@@ -688,7 +761,7 @@ stages: ...@@ -688,7 +761,7 @@ stages:
function compose_up() { function compose_up() {
environment_url=${ENV_URL:-$DCMP_ENVIRONMENT_URL} environment_url=${ENV_URL:-$DCMP_ENVIRONMENT_URL}
# variables expansion in $environment_url # variables expansion in $environment_url
environment_url=$(echo "$environment_url" | awkenvsubst) environment_url=$(echo "$environment_url" | TBC_ENVSUBST_ENCODING=uricomp tbc_envsubst)
export environment_url export environment_url
# extract hostname from $environment_url # extract hostname from $environment_url
hostname=$(echo "$environment_url" | awk -F[/:] '{print $4}') hostname=$(echo "$environment_url" | awk -F[/:] '{print $4}')
...@@ -801,7 +874,7 @@ stages: ...@@ -801,7 +874,7 @@ stages:
image: $DCMP_IMAGE image: $DCMP_IMAGE
services: services:
- name: "$TBC_TRACKING_IMAGE" - name: "$TBC_TRACKING_IMAGE"
command: ["--service", "docker-compose", "1.2.0"] command: ["--service", "docker-compose", "1.2.2"]
before_script: before_script:
- !reference [.compose-scripts] - !reference [.compose-scripts]
- install_ca_certs "${CUSTOM_CA_CERTS:-$DEFAULT_CA_CERTS}" - install_ca_certs "${CUSTOM_CA_CERTS:-$DEFAULT_CA_CERTS}"
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment