DCMP_PROD_DOCKER_HOST:"ssh://docker@192.168.64.7:2375"# enable production env
```
## Understand
...
...
@@ -93,18 +93,22 @@ You're free to enable whichever or both, and you can also choose your deployment
### Supported authentication methods
The Docker Compose template supports the following authentication methods:
The Docker Compose template supports [deployment on remote Docker hosts](https://www.docker.com/blog/how-to-deploy-on-remote-docker-hosts-with-docker-compose/), using
dedicated variables:
* TODO (document)
*`DCMP_REVIEW_DOCKER_HOST`, `DCMP_INTEG_DOCKER_HOST`, `DCMP_STAGING_DOCKER_HOST` and `DCMP_PROD_DOCKER_HOST` to both enable and configure the target Docker host for each environment (ex: `ssh://docker@192.168.64.5:2375`),
* :lock: `DCMP_SSH_PRIVATE_KEY`, :lock: `DCMP_REVIEW_SSH_PRIVATE_KEY`, :lock: `DCMP_INTEG_SSH_PRIVATE_KEY`, :lock: `DCMP_STAGING_SSH_PRIVATE_KEY` and :lock: `DCMP_PROD_SSH_PRIVATE_KEY` to provide the global or per env SSH private key (in case SSH authentication is required).
### Deployment context variables
In order to manage the various deployment environments, this template provides a couple of **dynamic variables**
that you might use in your hook scripts, deployment manifests and other deployment resources:
that you might use in your hook scripts or [Docker Compose files](https://docs.docker.com/compose/compose-application-model/#the-compose-file):
*`${environment_type}`: the current deployment environment type (`review`, `integration`, `staging` or `production`)
*`${environment_name}`: a generated application name to use for the current deployment environment (ex: `myapp-review-fix-bug-12` or `myapp-staging`) - _details below_
> :information_source: the `${environment_name}` is used by the Docker Compose template as the [Docker Compose project name](https://docs.docker.com/compose/project-name/).
#### Generated environment name
The `${environment_name}` variable is generated to designate each deployment environment with a unique and meaningful application name.
...
...
@@ -131,38 +135,116 @@ Examples (with an application's base name `myapp`):
### Deployment and cleanup
> TODO: explain here the supported techniques to deploy and cleanup the environments.
The Docker Compose template requires you to provide:
* a _required_ [Compose file](https://docs.docker.com/compose/compose-application-model/#the-compose-file) that implements your application deployment and cleanup,
* _optional_ [`.override.` Compose files](https://docs.docker.com/compose/multiple-compose-files/merge/) defining common and/or per-env override,
* _optional_ [dotenv files](https://docs.docker.com/compose/environment-variables/env-file/) defining common and/or per-env configuration,
* _optional_ hook scripts (shell) to implement logic that can't be implemented with Docker Compose.
#### Compose files lookup strategy
Unless you have explicitly set the [`COMPOSE_FILES`](https://docs.docker.com/compose/environment-variables/envvars/#compose_file) variable, the Docker Compose template will handle it and
implement the following [Compose file(s)](https://docs.docker.com/compose/compose-application-model/#the-compose-file) lookup strategy:
Unless you have explicitly set the [`COMPOSE_ENV_FILES`](https://docs.docker.com/compose/environment-variables/envvars/#compose_env_files) variable, the Docker Compose template will handle it
and implement the following [dotenv files](https://docs.docker.com/compose/environment-variables/env-file/) lookup strategy:
1. a `.env` file defining defaults for all environments,
2. a `${environment_type}.env` file that might redefine or override defaults for a specific environment (e.g. `staging.env`).
#### Deployment
The **deployment** is processed as follows by the template:
1. _optionally_ executes the `pre-compose-up.sh` script found in your project to perform pre-deployment stuff (for e.g. create required services),
3. _optionally_ executes the `post-compose-up.sh` script found in your project to perform post-deployment stuff,
> :information_source: Important:
>
> Example:
> * Compose files, dotenv files and hook scripts are searched in the `$DCMP_SCRIPTS_DIR` directory (configurable),
> * Hook scripts need to be executable; you can add the execution flag with `git update-index --chmod=+x pre-compose-up.sh`.
The Docker Compose template requires you to provide a shell script that fully implements your application
deployment and cleanup using the `docker-compose` CLI and all other tools available in the selected Docker image.
#### Cleanup
The deployment script is searched as follows:
The **cleanup** is processed as follows by the template:
1.look for a specific `dcmp-deploy-$environment_type.sh` in the `$DCMP_SCRIPTS_DIR` directory in your project (e.g. `dcmp-deploy-staging.sh` for staging environment),
2.if not found: look for a default `dcmp-deploy.sh` in the `$DCMP_SCRIPTS_DIR` directory in your project,
3.if not found: the deployment job will fail.
1._optionally_ executes the `pre-compose-down.sh` script found in your project to perform pre-cleanup stuff,
3._optionally_ executes the `post-compose-down.sh` script found in your project to perform post-cleanup stuff,
The cleanup script is searched as follows:
> :information_source: Important:
>
> * Compose files, dotenv files and hook scripts are searched in the `$DCMP_SCRIPTS_DIR` directory (configurable),
> * Hook scripts need to be executable; you can add the execution flag with `git update-index --chmod=+x pre-compose-up.sh`.
1. look for a specific `dcmp-cleanup-$environment_type.sh` in the `$DCMP_SCRIPTS_DIR` directory in your project (e.g. `dcmp-cleanup-staging.sh` for staging environment),
2. if not found: look for a default `dcmp-cleanup.sh` in the `$DCMP_SCRIPTS_DIR` directory in your project,
3. if not found: the cleanup job will fail.
#### Using Variables
> :information_source: Your deployment (and cleanup) scripts have to be able to cope with various environments, each with different application names, exposed routes, settings, ...
> Part of this complexity can be handled by the lookup policies described above (ex: one script per env) and also by using available environment variables:
>
> 1. [deployment context variables](#deployment-context-variables) provided by the template:
> * `${environment_type}`: the current environment type (`review`, `integration`, `staging` or `production`)
> * `${environment_name}`: the application name to use for the current environment (ex: `myproject-review-fix-bug-12` or `myproject-staging`)
> * `${hostname}`: the environment hostname, extracted from the current environment url (after late variable expansion - see below)
> 2. any [GitLab CI variable](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)
> 3. any [custom variable](https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-project)
> (ex: `${SECRET_TOKEN}` that you have set in your project CI/CD variables)
Your deployment (and cleanup) scripts have to be able to cope with various environments, each with different application names, exposed routes, settings, ...
Part of this complexity can be handled by the lookup strategies described above (ex: one file per env) and also by using available environment variables:
1.[deployment context variables](#deployment-context-variables) provided by the template:
*`${environment_type}`: the current environment type (`review`, `integration`, `staging` or `production`)
*`${environment_name}`: the application name to use for the current environment (ex: `myproject-review-fix-bug-12` or `myproject-staging`)
*`${hostname}`: the environment hostname, extracted from the current environment url (after late variable expansion - see below)
2. any [GitLab CI variable](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)
3. any [custom variable](https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-docker-host)
(ex: `${SECRET_TOKEN}` that you have set in your project CI/CD variables)
Be aware that environment variables may be freely used and substitued in [dotenv files](https://docs.docker.com/compose/environment-variables/env-file/)
using the appropriate [interpolation syntax](https://docs.docker.com/compose/environment-variables/env-file/#interpolation).
### Environments URL management
...
...
@@ -193,7 +275,7 @@ The **static way** can be implemented simply by setting the appropriate configur
To implement the **dynamic way**, your deployment script shall simply generate a `environment_url.txt` file in the working directory, containing only
To implement the **dynamic way**, your post deployment hook script shall simply generate a `environment_url.txt` file in the working directory, containing only
the dynamically generated url. When detected by the template, it will use it as the newly deployed environment url.
### Deployment output variables
...
...
@@ -206,7 +288,7 @@ Each deployment job produces _output variables_ that are propagated to downstrea
Those variables may be freely used in downstream jobs (for instance to run acceptance tests against the latest deployed environment).
You may also add and propagate your own custom variables, by pushing them to the `docker-compose.env` file in your [deployment scripts or hooks](#deployment-and-cleanup).
You may also add and propagate your own custom variables, by pushing them to the `docker-compose.out.env` file in your [deployment scripts or hooks](#deployment-and-cleanup).
## Configuration reference
...
...
@@ -214,7 +296,7 @@ You may also add and propagate your own custom variables, by pushing them to the
Here are some advices about your **secrets** (variables marked with a :lock:):
1. Manage them as [project or group CI/CD variables](https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-project):
1. Manage them as [project or group CI/CD variables](https://docs.gitlab.com/ee/ci/variables/#add-a-cicd-variable-to-a-docker-host):
*[**masked**](https://docs.gitlab.com/ee/ci/variables/#mask-a-cicd-variable) to prevent them from being inadvertently
displayed in your job logs,
*[**protected**](https://docs.gitlab.com/ee/ci/variables/#protected-cicd-variables) if you want to secure some secrets
...
...
@@ -230,28 +312,30 @@ The Docker Compose template uses some global configuration used throughout all j
| Input / Variable | Description | Default value |
| :lock: `DCMP_REVIEW_SSH_PRIVATE_KEY` | `review` env specific SSH key to use when connecting to Docker Host over SSH | `$DCMP_SSH_PRIVATE_KEY` |
| `review-app-name` / `DCMP_REVIEW_APP_NAME` | Application name for `review` env | `"${DCMP_BASE_APP_NAME}-${CI_ENVIRONMENT_SLUG}"` (ex: `myproject-review-fix-bug-12`) |
| `review-api-url` / `DCMP_REVIEW_API_URL` | API url for `review` env _(only define to override default)_ | `$DCMP_API_URL` |
| :lock: `DCMP_REVIEW_API_TOKEN` | API token for `review` env _(only define to override default)_ | `$DCMP_API_TOKEN` |
| `review-environment-url` / `DCMP_REVIEW_ENVIRONMENT_URL`| The review environments url _(only define for static environment URLs declaration and if different from default)_ | `$DCMP_ENVIRONMENT_URL` |
| `review-autostop-duration` / `DCMP_REVIEW_AUTOSTOP_DURATION`| The amount of time before GitLab will automatically stop `review` environments | `4 hours` |
...
...
@@ -259,16 +343,15 @@ Here are variables supported to configure review environments:
The integration environment is the environment associated to your integration branch (`develop` by default).
It is **disabled by default** and can be enabled by setting the `DCMP_INTEG_PROJECT` variable (see below).
It is **disabled by default** and can be enabled by setting the `DCMP_INTEG_DOCKER_HOST` variable (see below).
Here are variables supported to configure the integration environment:
| Input / Variable | Description | Default value |
| :lock: `DCMP_INTEG_SSH_PRIVATE_KEY` | `integration` env specific SSH key to use when connecting to Docker Host over SSH | `$DCMP_SSH_PRIVATE_KEY` |
| `integ-app-name` / `DCMP_INTEG_APP_NAME` | Application name for `integration` env | `${DCMP_BASE_APP_NAME}-integration` |
| `integ-api-url` / `DCMP_INTEG_API_URL` | API url for `integration` env _(only define to override default)_ | `$DCMP_API_URL` |
| :lock: `DCMP_INTEG_API_TOKEN` | API token for `integration` env _(only define to override default)_ | `$DCMP_API_TOKEN` |
| `integ-environment-url` / `DCMP_INTEG_ENVIRONMENT_URL`| The integration environment url _(only define for static environment URLs declaration and if different from default)_ | `$DCMP_ENVIRONMENT_URL` |
### Staging environment configuration
...
...
@@ -276,31 +359,43 @@ Here are variables supported to configure the integration environment:
The staging environment is an iso-prod environment meant for testing and validation purpose associated to your production
branch (`main` or `master` by default).
It is **disabled by default** and can be enabled by setting the `DCMP_STAGING_PROJECT` variable (see below).
It is **disabled by default** and can be enabled by setting the `DCMP_STAGING_DOCKER_HOST` variable (see below).
Here are variables supported to configure the staging environment:
| Input / Variable | Description | Default value |
| :lock: `DCMP_STAGING_SSH_PRIVATE_KEY` | `staging` env specific SSH key to use when connecting to Docker Host over SSH | `$DCMP_SSH_PRIVATE_KEY` |
| `staging-app-name` / `DCMP_STAGING_APP_NAME` | Application name for `staging` env | `${DCMP_BASE_APP_NAME}-staging` |
| `staging-api-url` / `DCMP_STAGING_API_URL` | API url for `staging` env _(only define to override default)_ | `$DCMP_API_URL` |
| :lock: `DCMP_STAGING_API_TOKEN` | API token for `staging` env _(only define to override default)_ | `$DCMP_API_TOKEN` |
| `staging-environment-url` / `DCMP_STAGING_ENVIRONMENT_URL`| The staging environment url _(only define for static environment URLs declaration and if different from default)_ | `$DCMP_ENVIRONMENT_URL` |
### Production environment configuration
The production environment is the final deployment environment associated with your production branch (`main` or `master` by default).
It is **disabled by default** and can be enabled by setting the `DCMP_PROD_PROJECT` variable (see below).
It is **disabled by default** and can be enabled by setting the `DCMP_PROD_DOCKER_HOST` variable (see below).
Here are variables supported to configure the production environment:
| Input / Variable | Description | Default value |
| :lock: `DCMP_PROD_SSH_PRIVATE_KEY` | `production` env specific SSH key to use when connecting to Docker Host over SSH | `$DCMP_SSH_PRIVATE_KEY` |
| `prod-app-name` / `DCMP_PROD_APP_NAME` | Application name for `production` env | `$DCMP_BASE_APP_NAME` |
| `prod-api-url` / `DCMP_PROD_API_URL` | API url for `production` env _(only define to override default)_ | `$DCMP_API_URL` |
| :lock: `DCMP_PROD_API_TOKEN` | API token for `production` env _(only define to override default)_ | `$DCMP_API_TOKEN` |
| `prod-environment-url` / `DCMP_PROD_ENVIRONMENT_URL`| The production environment url _(only define for static environment URLs declaration and if different from default)_ | `$DCMP_ENVIRONMENT_URL` |
| `prod-deploy-strategy` / `DCMP_PROD_DEPLOY_STRATEGY`| Defines the deployment to production strategy. One of `manual` (i.e. _one-click_) or `auto`. | `manual` |
### Compose Config job
The Docker Compose template enables running [Compose Config](https://docs.docker.com/reference/cli/docker/compose/config/), thus enabling detection of syntax errors in your Compose or dotenv files.
This job is mapped to the `package-test` stage and is **active** by default.
Here are its parameters:
| Input / Variable | Description | Default value |
