Skip to content
Snippets Groups Projects
Commit 0166bd43 authored by Cédric OLIVIER's avatar Cédric OLIVIER Committed by Pierre Smeyers
Browse files

feat: support CI/CD component design 

See: https://docs.gitlab.com/ee/ci/components/
parent b767b8f9
Branches
Tags
No related merge requests found
Show whitespace changes
Inline Side-by-side
README.md
+ 76
− 54
View file @ 0166bd43
...@@ -4,30 +4,55 @@ This project implements a GitLab CI/CD template to build, test and analyse your ...@@ -4,30 +4,55 @@ This project implements a GitLab CI/CD template to build, test and analyse your
## Usage ## Usage
In order to include this template in your project, add the following to your `gitlab-ci.yml`: This template can be used both as a [CI/CD component](https://docs.gitlab.com/ee/ci/components/#use-a-component-in-a-cicd-configuration)
or using the legacy [`include:project`](https://docs.gitlab.com/ee/ci/yaml/index.html#includeproject) syntax.
### Use as a CI/CD component
Add the following to your `gitlab-ci.yml`:
```yaml ```yaml
include: include:
# 1: include the component
- component: gitlab.com/to-be-continuous/python/gitlab-ci-python@6.4.1
# 2: set/override component inputs
inputs:
image: registry.hub.docker.com/library/python:3.10
pytest-enabled: true
```
### Use as a CI/CD template (legacy)
Add the following to your `gitlab-ci.yml`:
```yaml
include:
# 1: include the template
- project: 'to-be-continuous/python' - project: 'to-be-continuous/python'
ref: '6.4.1' ref: '6.4.1'
file: '/templates/gitlab-ci-python.yml' file: '/templates/gitlab-ci-python.yml'
variables:
# 2: set/override template variables
PYTHON_IMAGE: registry.hub.docker.com/library/python:3.10
PYTEST_ENABLED: "true"
``` ```
## Global configuration ## Global configuration
The Python template uses some global configuration used throughout all jobs. The Python template uses some global configuration used throughout all jobs.
| Name | description | default value | | Input / Variable | Description | Default value |
| -------------------- | ------------------------------------------------------------------------------------- | ------------------ | | -------------------- | ------------------------------------------------------------------------------------- | ------------------ |
| `PYTHON_IMAGE` | The Docker image used to run Python <br/>:warning: **set the version required by your project** | `registry.hub.docker.com/library/python:3` | | `image` / `PYTHON_IMAGE` | The Docker image used to run Python <br/>:warning: **set the version required by your project** | `registry.hub.docker.com/library/python:3` |
| `PYTHON_PROJECT_DIR` | Python project root directory | `.` | | `project-dir` / `PYTHON_PROJECT_DIR` | Python project root directory | `.` |
| `PYTHON_BUILD_SYSTEM`| Python build-system to use to install dependencies, build and package the project (see below) | _none_ (auto-detect) | | `build-system` / `PYTHON_BUILD_SYSTEM`| Python build-system to use to install dependencies, build and package the project (see below) | _none_ (auto-detect) |
| `PIP_INDEX_URL` | Python repository url | _none_ | | `PIP_INDEX_URL` | Python repository url | _none_ |
| `PIP_EXTRA_INDEX_URL` | Exra Python repository url | _none_ | | `PIP_EXTRA_INDEX_URL` | Exra Python repository url | _none_ |
| `PIP_OPTS` | pip [extra options](https://pip.pypa.io/en/stable/cli/pip/#general-options) | _none_ | | `pip-opts` / `PIP_OPTS` | pip [extra options](https://pip.pypa.io/en/stable/cli/pip/#general-options) | _none_ |
| `PYTHON_EXTRA_DEPS` | Python extra sets of dependencies to install<br/>For [Setuptools](https://setuptools.pypa.io/en/latest/userguide/dependency_management.html?highlight=extras#optional-dependencies) or [Poetry](https://python-poetry.org/docs/pyproject/#extras) only | _none_ | | `extra-deps` / `PYTHON_EXTRA_DEPS` | Python extra sets of dependencies to install<br/>For [Setuptools](https://setuptools.pypa.io/en/latest/userguide/dependency_management.html?highlight=extras#optional-dependencies) or [Poetry](https://python-poetry.org/docs/pyproject/#extras) only | _none_ |
| `PYTHON_REQS_FILE` | Main requirements file _(relative to `$PYTHON_PROJECT_DIR`)_<br/>For [Requirements Files](https://pip.pypa.io/en/stable/user_guide/#requirements-files) build-system only | `requirements.txt` | | `reqs-file` / `PYTHON_REQS_FILE` | Main requirements file _(relative to `$PYTHON_PROJECT_DIR`)_<br/>For [Requirements Files](https://pip.pypa.io/en/stable/user_guide/#requirements-files) build-system only | `requirements.txt` |
| `PYTHON_EXTRA_REQS_FILES` | Extra dev requirements file(s) to install _(relative to `$PYTHON_PROJECT_DIR`)_ | `requirements-dev.txt` | | `extra-reqs-files` / `PYTHON_EXTRA_REQS_FILES` | Extra dev requirements file(s) to install _(relative to `$PYTHON_PROJECT_DIR`)_ | `requirements-dev.txt` |
The cache policy also makes the necessary to manage pip cache (not to download Python dependencies over and over again). The cache policy also makes the necessary to manage pip cache (not to download Python dependencies over and over again).
...@@ -66,10 +91,10 @@ It is activated by setting `$PYLINT_ENABLED` to `true`. ...@@ -66,10 +91,10 @@ It is activated by setting `$PYLINT_ENABLED` to `true`.
It is bound to the `build` stage, and uses the following variables: It is bound to the `build` stage, and uses the following variables:
| Name | description | default value | | Input / Variable | Description | Default value |
| ------------------------ | ---------------------------------- | ----------------- | | ------------------------ | ---------------------------------- | ----------------- |
| `PYLINT_ARGS` | Additional [pylint CLI options](http://pylint.pycqa.org/en/latest/user_guide/run.html#command-line-options) | _none_ | | `pylint-args` / `PYLINT_ARGS` | Additional [pylint CLI options](http://pylint.pycqa.org/en/latest/user_guide/run.html#command-line-options) | _none_ |
| `PYLINT_FILES` | Files or directories to analyse | _none_ (by default analyses all found python source files) | | `pylint-files` / `PYLINT_FILES` | Files or directories to analyse | _none_ (by default analyses all found python source files) |
In addition to a textual report in the console, this job produces the following reports, kept for one day: In addition to a textual report in the console, this job produces the following reports, kept for one day:
...@@ -96,9 +121,9 @@ In order to produce JUnit test reports, the tests are executed with the [xmlrunn ...@@ -96,9 +121,9 @@ In order to produce JUnit test reports, the tests are executed with the [xmlrunn
It is bound to the `build` stage, and uses the following variables: It is bound to the `build` stage, and uses the following variables:
| Name | description | default value | | Input / Variable | Description | Default value |
| ------------------------ | -------------------------------------------------------------------- | ----------------------- | | ------------------------ | -------------------------------------------------------------------- | ----------------------- |
| `UNITTEST_ARGS` | Additional xmlrunner/unittest CLI options | _none_ | | `unittest-args` / `UNITTEST_ARGS` | Additional xmlrunner/unittest CLI options | _none_ |
:information_source: use a `.coveragerc` file at the root of your Python project to control the coverage settings. :information_source: use a `.coveragerc` file at the root of your Python project to control the coverage settings.
...@@ -128,9 +153,9 @@ It is activated by setting `$PYTEST_ENABLED` to `true`. ...@@ -128,9 +153,9 @@ It is activated by setting `$PYTEST_ENABLED` to `true`.
It is bound to the `build` stage, and uses the following variables: It is bound to the `build` stage, and uses the following variables:
| Name | description | default value | | Input / Variable | Description | Default value |
| ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- | | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- |
| `PYTEST_ARGS` | Additional [pytest](https://docs.pytest.org/en/stable/usage.html) or [pytest-cov](https://github.com/pytest-dev/pytest-cov#usage) CLI options | _none_ | | `pytest-args` / `PYTEST_ARGS` | Additional [pytest](https://docs.pytest.org/en/stable/usage.html) or [pytest-cov](https://github.com/pytest-dev/pytest-cov#usage) CLI options | _none_ |
:information_source: use a `.coveragerc` file at the root of your Python project to control the coverage settings. :information_source: use a `.coveragerc` file at the root of your Python project to control the coverage settings.
...@@ -160,9 +185,9 @@ It is activated by setting `$NOSETESTS_ENABLED` to `true`. ...@@ -160,9 +185,9 @@ It is activated by setting `$NOSETESTS_ENABLED` to `true`.
It is bound to the `build` stage, and uses the following variables: It is bound to the `build` stage, and uses the following variables:
| Name | description | default value | | Input / Variable | Description | Default value |
| ------------------------ | --------------------------------------------------------------------------------------- | ----------------------- | | ------------------------ | --------------------------------------------------------------------------------------- | ----------------------- |
| `NOSETESTS_ARGS` | Additional [nose CLI options](https://nose.readthedocs.io/en/latest/usage.html#options) | _none_ | | `nosetests-args` / `NOSETESTS_ARGS` | Additional [nose CLI options](https://nose.readthedocs.io/en/latest/usage.html#options) | _none_ |
By default coverage will be run on all the project directories. You can restrict it to your packages by setting the `$NOSE_COVER_PACKAGE` variable. By default coverage will be run on all the project directories. You can restrict it to your packages by setting the `$NOSE_COVER_PACKAGE` variable.
More [info](https://nose.readthedocs.io/en/latest/plugins/cover.html) More [info](https://nose.readthedocs.io/en/latest/plugins/cover.html)
...@@ -183,9 +208,9 @@ are not set), and performs a [`compileall`](https://docs.python.org/3/library/co ...@@ -183,9 +208,9 @@ are not set), and performs a [`compileall`](https://docs.python.org/3/library/co
It is bound to the `build` stage, and uses the following variables: It is bound to the `build` stage, and uses the following variables:
| Name | description | default value | | Input / Variable | Description | Default value |
| --------------------- | ----------------------------------------------------------------------------- | ------------- | | --------------------- | ----------------------------------------------------------------------------- | ------------- |
| `PYTHON_COMPILE_ARGS` | [`compileall` CLI options](https://docs.python.org/3/library/compileall.html) | `*` | | `compile-args` / `PYTHON_COMPILE_ARGS` | [`compileall` CLI options](https://docs.python.org/3/library/compileall.html) | `*` |
### `py-bandit` job (SAST) ### `py-bandit` job (SAST)
...@@ -193,10 +218,10 @@ This job is **disabled by default** and performs a [Bandit](https://pypi.org/pro ...@@ -193,10 +218,10 @@ This job is **disabled by default** and performs a [Bandit](https://pypi.org/pro
It is bound to the `test` stage, and uses the following variables: It is bound to the `test` stage, and uses the following variables:
| Name | description | default value | | Input / Variable | Description | Default value |
| ---------------- | ---------------------------------------------------------------------- | ----------------- | | ---------------- | ---------------------------------------------------------------------- | ----------------- |
| `BANDIT_ENABLED` | Set to `true` to enable Bandit analysis | _none_ (disabled) | | `bandit-enabled` / `BANDIT_ENABLED` | Set to `true` to enable Bandit analysis | _none_ (disabled) |
| `BANDIT_ARGS` | Additional [Bandit CLI options](https://github.com/PyCQA/bandit#usage) | `--recursive .` | | `bandit-args` / `BANDIT_ARGS` | Additional [Bandit CLI options](https://github.com/PyCQA/bandit#usage) | `--recursive .` |
In addition to a textual report in the console, this job produces the following reports, kept for one day: In addition to a textual report in the console, this job produces the following reports, kept for one day:
...@@ -211,10 +236,10 @@ This job is **disabled by default** and performs a dependency check analysis usi ...@@ -211,10 +236,10 @@ This job is **disabled by default** and performs a dependency check analysis usi
It is bound to the `test` stage, and uses the following variables: It is bound to the `test` stage, and uses the following variables:
| Name | description | default value | | Input / Variable | Description | Default value |
| ---------------- | ----------------------------------------------------------------------- | ----------------- | | ---------------- | ----------------------------------------------------------------------- | ----------------- |
| `SAFETY_ENABLED` | Set to `true` to enable Safety job | _none_ (disabled) | | `safety-enabled` / `SAFETY_ENABLED` | Set to `true` to enable Safety job | _none_ (disabled) |
| `SAFETY_ARGS` | Additional [Safety CLI options](https://github.com/pyupio/safety#usage) | `--full-report` | | `safety-args` / `SAFETY_ARGS` | Additional [Safety CLI options](https://github.com/pyupio/safety#usage) | `--full-report` |
### `py-trivy` job (dependency check) ### `py-trivy` job (dependency check)
...@@ -222,10 +247,10 @@ This job is **disabled by default** and performs a dependency check analysis usi ...@@ -222,10 +247,10 @@ This job is **disabled by default** and performs a dependency check analysis usi
It is bound to the `test` stage, and uses the following variables: It is bound to the `test` stage, and uses the following variables:
| Name | description | default value | | Input / Variable | Description | Default value |
| ---------------- | ----------------------------------------------------------------------- | ----------------- | | ---------------- | ----------------------------------------------------------------------- | ----------------- |
| `PYTHON_TRIVY_ENABLED` | Set to `true` to enable Trivy job | _none_ (disabled) | | `trivy-enabled` / `PYTHON_TRIVY_ENABLED` | Set to `true` to enable Trivy job | _none_ (disabled) |
| `PYTHON_TRIVY_ARGS` | Additional [Trivy CLI options](https://aquasecurity.github.io/trivy/v0.21.1/getting-started/cli/fs/) | `--vuln-type library` | | `trivy-args` / `PYTHON_TRIVY_ARGS` | Additional [Trivy CLI options](https://aquasecurity.github.io/trivy/v0.21.1/getting-started/cli/fs/) | `--vuln-type library` |
In addition to a textual report in the console, this job produces the following reports, kept for one day: In addition to a textual report in the console, this job produces the following reports, kept for one day:
...@@ -239,12 +264,12 @@ This job generates a [SBOM](https://cyclonedx.org/) file listing all dependencie ...@@ -239,12 +264,12 @@ This job generates a [SBOM](https://cyclonedx.org/) file listing all dependencie
It is bound to the `test` stage, and uses the following variables: It is bound to the `test` stage, and uses the following variables:
| Name | description | default value | | Input / Variable | Description | Default value |
| --------------------- | -------------------------------------- | ----------------- | | --------------------- | -------------------------------------- | ----------------- |
| `PYTHON_SBOM_DISABLED` | Set to `true` to disable this job | _none_ | | `sbom-disabled` / `PYTHON_SBOM_DISABLED` | Set to `true` to disable this job | _none_ |
| `PYTHON_SBOM_SYFT_URL` | Url to the `tar.gz` package for `linux_amd64` of Syft to use (ex: `https://github.com/anchore/syft/releases/download/v0.62.3/syft_0.62.3_linux_amd64.tar.gz`)<br/>_When unset, the latest version will be used_ | _none_ | | `sbom-syft-url` / `PYTHON_SBOM_SYFT_URL` | Url to the `tar.gz` package for `linux_amd64` of Syft to use (ex: `https://github.com/anchore/syft/releases/download/v0.62.3/syft_0.62.3_linux_amd64.tar.gz`)<br/>_When unset, the latest version will be used_ | _none_ |
| `PYTHON_SBOM_NAME` | Component name of the emitted SBOM | `$CI_PROJECT_PATH/$PYTHON_PROJECT_DIR` | | `sbom-name` / `PYTHON_SBOM_NAME` | Component name of the emitted SBOM | `$CI_PROJECT_PATH/$PYTHON_PROJECT_DIR` |
| `PYTHON_SBOM_OPTS` | Options for syft used for SBOM analysis | `--catalogers python-index-cataloger` | | `sbom-opts` / `PYTHON_SBOM_OPTS` | Options for syft used for SBOM analysis | `--catalogers python-index-cataloger` |
In addition to logs in the console, this job produces the following reports, kept for one week: In addition to logs in the console, this job produces the following reports, kept for one week:
...@@ -299,16 +324,16 @@ The Python template supports two packaging systems: ...@@ -299,16 +324,16 @@ The Python template supports two packaging systems:
The release job is bound to the `publish` stage, appears only on production and integration branches and uses the following variables: The release job is bound to the `publish` stage, appears only on production and integration branches and uses the following variables:
| Name | description | default value | | Input / Variable | Description | Default value |
| ----------------------- | ----------------------------------------------------------------------- | ----------------- | | ----------------------- | ----------------------------------------------------------------------- | ----------------- |
| `PYTHON_RELEASE_ENABLED`| Set to `true` to enable the release job | _none_ (disabled) | | `release-enabled` / `PYTHON_RELEASE_ENABLED`| Set to `true` to enable the release job | _none_ (disabled) |
| `PYTHON_RELEASE_NEXT` | The part of the version to increase (one of: `major`, `minor`, `patch`) | `minor` | | `release-next` / `PYTHON_RELEASE_NEXT` | The part of the version to increase (one of: `major`, `minor`, `patch`) | `minor` |
| `PYTHON_SEMREL_RELEASE_DISABLED`| Set to `true` to disable [semantic-release integration](#semantic-release-integration) | _none_ (disabled) | | `semrel-release-disabled` / `PYTHON_SEMREL_RELEASE_DISABLED`| Set to `true` to disable [semantic-release integration](#semantic-release-integration) | _none_ (disabled) |
| `GIT_USERNAME` | Git username for Git push operations (see below) | _none_ | | `GIT_USERNAME` | Git username for Git push operations (see below) | _none_ |
| :lock: `GIT_PASSWORD` | Git password for Git push operations (see below) | _none_ | | :lock: `GIT_PASSWORD` | Git password for Git push operations (see below) | _none_ |
| :lock: `GIT_PRIVATE_KEY`| SSH key for Git push operations (see below) | _none_ | | :lock: `GIT_PRIVATE_KEY`| SSH key for Git push operations (see below) | _none_ |
| `PYTHON_RELEASE_COMMIT_MESSAGE`| The Git commit message to use on the release commit. This is templated using the [Python Format String Syntax](http://docs.python.org/2/library/string.html#format-string-syntax). Available in the template context are current_version and new_version. | `chore(python-release): {current_version} → {new_version}` | | `release-commit-message` / `PYTHON_RELEASE_COMMIT_MESSAGE`| The Git commit message to use on the release commit. This is templated using the [Python Format String Syntax](http://docs.python.org/2/library/string.html#format-string-syntax). Available in the template context are current_version and new_version. | `chore(python-release): {current_version} → {new_version}` |
| `PYTHON_REPOSITORY_URL`| Target PyPI repository to publish packages | _[GitLab project's PyPI packages repository](https://docs.gitlab.com/ee/user/packages/pypi_repository/)_ | | `repository-url` / `PYTHON_REPOSITORY_URL`| Target PyPI repository to publish packages | _[GitLab project's PyPI packages repository](https://docs.gitlab.com/ee/user/packages/pypi_repository/)_ |
| `PYTHON_REPOSITORY_USERNAME`| Target PyPI repository username credential | `gitlab-ci-token` | | `PYTHON_REPOSITORY_USERNAME`| Target PyPI repository username credential | `gitlab-ci-token` |
| :lock: `PYTHON_REPOSITORY_PASSWORD`| Target PyPI repository password credential | `$CI_JOB_TOKEN` | | :lock: `PYTHON_REPOSITORY_PASSWORD`| Target PyPI repository password credential | `$CI_JOB_TOKEN` |
...@@ -359,7 +384,7 @@ You can either use a SSH key or user/password credentials. ...@@ -359,7 +384,7 @@ You can either use a SSH key or user/password credentials.
We recommend you to use a [project deploy key](https://docs.gitlab.com/ee/user/project/deploy_keys/#project-deploy-keys) with write access to your project. We recommend you to use a [project deploy key](https://docs.gitlab.com/ee/user/project/deploy_keys/#project-deploy-keys) with write access to your project.
The key should not have a passphrase (see [how to generate a new SSH key pair](https://docs.gitlab.com/ce/ssh/README.html#generating-a-new-ssh-key-pair)). The key should not have a passphrase (see [how to generate a new SSH key pair](https://docs.gitlab.com/ee/user/ssh.html#generate-an-ssh-key-pair)).
Specify :lock: `$GIT_PRIVATE_KEY` as secret project variable with the private part of the deploy key. Specify :lock: `$GIT_PRIVATE_KEY` as secret project variable with the private part of the deploy key.
...@@ -412,11 +437,11 @@ This variant allows delegating your secrets management to a [Vault](https://www. ...@@ -412,11 +437,11 @@ This variant allows delegating your secrets management to a [Vault](https://www.
In order to be able to communicate with the Vault server, the variant requires the additional configuration parameters: In order to be able to communicate with the Vault server, the variant requires the additional configuration parameters:
| Name | Description | Default value | | Input / Variable | Description | Default value |
| ----------------- | -------------------------------------- | ----------------- | | ----------------- | -------------------------------------- | ----------------- |
| `TBC_VAULT_IMAGE` | The [Vault Secrets Provider](https://gitlab.com/to-be-continuous/tools/vault-secrets-provider) image to use (can be overridden) | `$CI_REGISTRY/to-be-continuous/tools/vault-secrets-provider:master` | | `TBC_VAULT_IMAGE` | The [Vault Secrets Provider](https://gitlab.com/to-be-continuous/tools/vault-secrets-provider) image to use (can be overridden) | `$CI_REGISTRY/to-be-continuous/tools/vault-secrets-provider:master` |
| `VAULT_BASE_URL` | The Vault server base API url | _none_ | | `vault-base-url` / `VAULT_BASE_URL` | The Vault server base API url | _none_ |
| `VAULT_OIDC_AUD` | The `aud` claim for the JWT | `$CI_SERVER_URL` | | `vault-oidc-aud` / `VAULT_OIDC_AUD` | The `aud` claim for the JWT | `$CI_SERVER_URL` |
| :lock: `VAULT_ROLE_ID` | The [AppRole](https://www.vaultproject.io/docs/auth/approle) RoleID | **must be defined** | | :lock: `VAULT_ROLE_ID` | The [AppRole](https://www.vaultproject.io/docs/auth/approle) RoleID | **must be defined** |
| :lock: `VAULT_SECRET_ID` | The [AppRole](https://www.vaultproject.io/docs/auth/approle) SecretID | **must be defined** | | :lock: `VAULT_SECRET_ID` | The [AppRole](https://www.vaultproject.io/docs/auth/approle) SecretID | **must be defined** |
...@@ -439,22 +464,19 @@ With: ...@@ -439,22 +464,19 @@ With:
```yaml ```yaml
include: include:
# main template # main component
- project: 'to-be-continuous/python' - component: gitlab.com/to-be-continuous/python/gitlab-ci-python@6.4.1
ref: '6.4.1'
file: '/templates/gitlab-ci-python.yml'
# Vault variant # Vault variant
- project: 'to-be-continuous/python' - component: gitlab.com/to-be-continuous/python/gitlab-ci-python-vault@6.4.1
ref: '6.4.1' inputs:
file: '/templates/gitlab-ci-python-vault.yml' vault-base-url: "https://vault.acme.host/v1"
# audience claim for JWT
vault-oidc-aud: "https://vault.acme.host"
variables: variables:
# audience claim for JWT
VAULT_OIDC_AUD: "https://vault.acme.host"
# Secrets managed by Vault # Secrets managed by Vault
GIT_PASSWORD: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/git/semantic-release?field=group-access-token" GIT_PASSWORD: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/git/semantic-release?field=group-access-token"
GIT_PRIVATE_KEY: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/git/semantic-release?field=private-key" GIT_PRIVATE_KEY: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/git/semantic-release?field=private-key"
PYTHON_REPOSITORY_PASSWORD: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/pip-repo/repository?field=password" PYTHON_REPOSITORY_PASSWORD: "@url@http://vault-secrets-provider/api/secrets/b7ecb6ebabc231/pip-repo/repository?field=password"
VAULT_BASE_URL: "https://vault.acme.host/v1"
# $VAULT_ROLE_ID and $VAULT_SECRET_ID defined as a secret CI/CD variable # $VAULT_ROLE_ID and $VAULT_SECRET_ID defined as a secret CI/CD variable
``` ```
bumpversion.sh
+ 1
− 1
View file @ 0166bd43
...@@ -27,7 +27,7 @@ if [[ "$curVer" ]]; then ...@@ -27,7 +27,7 @@ if [[ "$curVer" ]]; then
log_info "Bump version from \\e[33;1m${curVer}\\e[0m to \\e[33;1m${nextVer}\\e[0m (release type: $relType)..." log_info "Bump version from \\e[33;1m${curVer}\\e[0m to \\e[33;1m${nextVer}\\e[0m (release type: $relType)..."
# replace in README # replace in README
sed -e "s/ref: '$curVer'/ref: '$nextVer'/" README.md > README.md.next sed -e "s/ref: '$curVer'/ref: '$nextVer'/" -e "s/@$curVer/@$nextVer/" README.md > README.md.next
mv -f README.md.next README.md mv -f README.md.next README.md
# replace in template and variants # replace in template and variants
......
kicker.json
+ 6
− 0
View file @ 0166bd43
...@@ -148,6 +148,12 @@ ...@@ -148,6 +148,12 @@
"description": "Detect security vulnerabilities with [Trivy](https://github.com/aquasecurity/trivy/) (dependencies analysis)", "description": "Detect security vulnerabilities with [Trivy](https://github.com/aquasecurity/trivy/) (dependencies analysis)",
"enable_with": "PYTHON_TRIVY_ENABLED", "enable_with": "PYTHON_TRIVY_ENABLED",
"variables": [ "variables": [
{
"name": "PYTHON_TRIVY_IMAGE",
"description": "The Docker image used to run Trivy",
"default": "registry.hub.docker.com/aquasec/trivy:latest",
"advanced": true
},
{ {
"name": "PYTHON_TRIVY_ARGS", "name": "PYTHON_TRIVY_ARGS",
"description": "Additional [Trivy CLI options](https://aquasecurity.github.io/trivy/v0.21.1/getting-started/cli/fs/)", "description": "Additional [Trivy CLI options](https://aquasecurity.github.io/trivy/v0.21.1/getting-started/cli/fs/)",
......
templates/gitlab-ci-python-vault.yml
+ 12
− 2
View file @ 0166bd43
# ===================================================================================================================== # =====================================================================================================================
# === Vault template variant # === Vault template variant
# ===================================================================================================================== # =====================================================================================================================
spec:
inputs:
vault-base-url:
description: The Vault server base API url
default: '' # null
vault-oidc-aud:
description: The `aud` claim for the JWT
default: $CI_SERVER_URL
---
variables: variables:
# variabilized vault-secrets-provider image # variabilized vault-secrets-provider image
TBC_VAULT_IMAGE: "$CI_REGISTRY/to-be-continuous/tools/vault-secrets-provider:master" TBC_VAULT_IMAGE: $CI_REGISTRY/to-be-continuous/tools/vault-secrets-provider:master
# variables have to be explicitly declared in the YAML to be exported to the service # variables have to be explicitly declared in the YAML to be exported to the service
VAULT_ROLE_ID: "$VAULT_ROLE_ID" VAULT_ROLE_ID: "$VAULT_ROLE_ID"
VAULT_SECRET_ID: "$VAULT_SECRET_ID" VAULT_SECRET_ID: "$VAULT_SECRET_ID"
VAULT_OIDC_AUD: "$CI_SERVER_URL" VAULT_OIDC_AUD: $[[ inputs.vault-oidc-aud ]]
VAULT_BASE_URL: $[[ inputs.vault-base-url ]]
.python-base: .python-base:
services: services:
......
templates/gitlab-ci-python.yml
+ 183
− 15
View file @ 0166bd43
...@@ -13,6 +13,148 @@ ...@@ -13,6 +13,148 @@
# program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth # program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth
# Floor, Boston, MA 02110-1301, USA. # Floor, Boston, MA 02110-1301, USA.
# ========================================================================================= # =========================================================================================
spec:
inputs:
image:
description: The Docker image used to run Python - **set the version required by your project**
default: registry.hub.docker.com/library/python:3
project-dir:
description: Python project root directory
default: .
build-system:
description: Python build-system to use to install dependencies, build and package the project
options:
- auto
- setuptools
- poetry
- pipenv
- reqfile
default: auto
reqs-file:
description: |-
Main requirements file _(relative to `$PYTHON_PROJECT_DIR`)_
For [Requirements Files](https://pip.pypa.io/en/stable/user_guide/#requirements-files) build-system only
default: requirements.txt
extra-reqs-files:
description: |-
Extra dev requirements file(s) to install _(relative to `$PYTHON_PROJECT_DIR`)_
For [Requirements Files](https://pip.pypa.io/en/stable/user_guide/#requirements-files) build-system only
default: requirements-dev.txt
compile-args:
description: '[`compileall` CLI options](https://docs.python.org/3/library/compileall.html)'
default: '*'
pip-opts:
description: pip extra [options](https://pip.pypa.io/en/stable/cli/pip/#general-options)
default: '' # null
extra-deps:
description: |-
Extra sets of dependencies to install
For [Setuptools](https://setuptools.pypa.io/en/latest/userguide/dependency_management.html?highlight=extras#optional-dependencies) or [Poetry](https://python-poetry.org/docs/pyproject/#extras) only
default: '' # null
package-enabled:
description: Enable package
type: boolean
default: false
pylint-enabled:
description: Enable pylint
type: boolean
default: false
pylint-args:
description: Additional [pylint CLI options](http://pylint.pycqa.org/en/latest/user_guide/run.html#command-line-options)
default: '' # null
pylint-files:
description: Files or directories to analyse
default: '' # null
unittest-enabled:
description: Enable unittest
type: boolean
default: false
unittest-args:
description: Additional xmlrunner/unittest CLI options
default: '' # null
pytest-enabled:
description: Enable pytest
type: boolean
default: false
pytest-args:
description: Additional [pytest](https://docs.pytest.org/en/stable/usage.html) or [pytest-cov](https://github.com/pytest-dev/pytest-cov#usage) CLI options
default: '' # null
nosetests-enabled:
description: Enable nosetest
type: boolean
default: false
nosetests-args:
description: Additional [nose CLI options](https://nose.readthedocs.io/en/latest/usage.html#options)
default: '' # null
bandit-enabled:
description: Enable Bandit
type: boolean
default: false
bandit-args:
description: Additional [Bandit CLI options](https://github.com/PyCQA/bandit#usage)
default: --recursive .
safety-enabled:
description: Enable Safety
type: boolean
default: false
safety-args:
description: Additional [Safety CLI options](https://github.com/pyupio/safety#usage)
default: --full-report
trivy-enabled:
description: Enable Trivy
type: boolean
default: false
trivy-image:
description: The Docker image used to run Trivy
default: registry.hub.docker.com/aquasec/trivy:latest
trivy-args:
description: Additional [Trivy CLI options](https://aquasecurity.github.io/trivy/v0.21.1/getting-started/cli/fs/)
default: --vuln-type library
sbom-disabled:
description: Disable Software Bill of Materials
type: boolean
default: false
sbom-syft-url:
description: |-
Url to the `tar.gz` package for `linux_amd64` of Syft to use
_When unset, the latest version will be used_
default: '' # null
sbom-name:
description: Component name of the emitted SBOM
default: $CI_PROJECT_PATH/$PYTHON_PROJECT_DIR
sbom-opts:
description: Options for syft used for SBOM analysis
default: --catalogers python-index-cataloger
release-enabled:
description: Enable Release
type: boolean
default: false
release-next:
description: 'The part of the version to increase (one of: `major`, `minor`, `patch`)'
options:
- ''
- major
- minor
- patch
default: minor
semrel-release-disabled:
description: Disable semantic-release integration
type: boolean
default: false
release-commit-message:
description: The Git commit message to use on the release commit. This is templated using the [Python Format String Syntax](http://docs.python.org/2/library/string.html#format-string-syntax). Available in the template context are current_version and new_version.
default: "chore(python-release): {current_version} \u2192 {new_version}"
repository-url:
description: |-
Target PyPI repository to publish packages.
_defaults to [GitLab project's packages repository](https://docs.gitlab.com/ee/user/packages/pypi_repository/)_
default: ${CI_SERVER_URL}/api/v4/projects/${CI_PROJECT_ID}/packages/pypi
---
# default workflow rules: Merge Request pipelines # default workflow rules: Merge Request pipelines
workflow: workflow:
rules: rules:
...@@ -66,12 +208,13 @@ variables: ...@@ -66,12 +208,13 @@ variables:
POETRY_VIRTUALENVS_IN_PROJECT: "false" POETRY_VIRTUALENVS_IN_PROJECT: "false"
PIPENV_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pipenv" PIPENV_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pipenv"
PYTHON_IMAGE: "registry.hub.docker.com/library/python:3" # PYTHON_IMAGE: "registry.hub.docker.com/library/python:3"
PYTHON_IMAGE: $[[ inputs.image ]]
# Default Python project root directory # Default Python project root directory
PYTHON_PROJECT_DIR: . PYTHON_PROJECT_DIR: $[[ inputs.project-dir ]]
PYTHON_REQS_FILE: requirements.txt PYTHON_REQS_FILE: $[[ inputs.reqs-file ]]
PYTHON_EXTRA_REQS_FILES: "requirements-dev.txt" PYTHON_EXTRA_REQS_FILES: $[[ inputs.extra-reqs-files ]]
# default production ref name (pattern) # default production ref name (pattern)
PROD_REF: '/^(master|main)$/' PROD_REF: '/^(master|main)$/'
...@@ -81,29 +224,54 @@ variables: ...@@ -81,29 +224,54 @@ variables:
RELEASE_REF: '/^v?[0-9]+\.[0-9]+\.[0-9]+$/' RELEASE_REF: '/^v?[0-9]+\.[0-9]+\.[0-9]+$/'
# compileall # compileall
PYTHON_COMPILE_ARGS: "*" PYTHON_COMPILE_ARGS: $[[ inputs.compile-args ]]
BANDIT_ARGS: "--recursive ." BANDIT_ARGS: $[[ inputs.bandit-args ]]
# Safety tool # Safety tool
SAFETY_ARGS: "--full-report" SAFETY_ARGS: $[[ inputs.safety-args ]]
# Trivy tool # Trivy tool
PYTHON_TRIVY_IMAGE: "registry.hub.docker.com/aquasec/trivy:latest" PYTHON_TRIVY_ENABLED: $[[ inputs.trivy-enabled ]]
PYTHON_TRIVY_ARGS: "--vuln-type library" PYTHON_TRIVY_IMAGE: $[[ inputs.trivy-image ]]
PYTHON_TRIVY_ARGS: $[[ inputs.trivy-args ]]
PYTHON_SBOM_NAME: "$CI_PROJECT_PATH/$PYTHON_PROJECT_DIR" PYTHON_SBOM_NAME: $[[ inputs.sbom-name ]]
PYTHON_SBOM_OPTS: "--catalogers python-index-cataloger" PYTHON_SBOM_OPTS: $[[ inputs.sbom-opts ]]
PYTHON_RELEASE_NEXT: "minor" PYTHON_RELEASE_NEXT: $[[ inputs.release-next ]]
PYTHON_RELEASE_COMMIT_MESSAGE: "chore(python-release): {current_version} → {new_version}" PYTHON_RELEASE_COMMIT_MESSAGE: $[[ inputs.release-commit-message ]]
# By default, publish on the Packages registry of the project # By default, publish on the Packages registry of the project
# https://docs.gitlab.com/ee/user/packages/pypi_repository/#authenticate-with-a-ci-job-token # https://docs.gitlab.com/ee/user/packages/pypi_repository/#authenticate-with-a-ci-job-token
PYTHON_REPOSITORY_URL: ${CI_SERVER_URL}/api/v4/projects/${CI_PROJECT_ID}/packages/pypi PYTHON_REPOSITORY_URL: $[[ inputs.repository-url ]]
PYTHON_REPOSITORY_USERNAME: 'gitlab-ci-token' PYTHON_REPOSITORY_USERNAME: gitlab-ci-token
PYTHON_REPOSITORY_PASSWORD: $CI_JOB_TOKEN PYTHON_REPOSITORY_PASSWORD: $CI_JOB_TOKEN
PYTHON_BUILD_SYSTEM: $[[ inputs.build-system ]]
PIP_OPTS: $[[ inputs.pip-opts ]]
PYTHON_EXTRA_DEPS: $[[ inputs.extra-deps ]]
PYTHON_PACKAGE_ENABLED: $[[ inputs.package-enabled ]]
PYLINT_ENABLED: $[[ inputs.pylint-enabled ]]
PYLINT_ARGS: $[[ inputs.pylint-args ]]
PYLINT_FILES: $[[ inputs.pylint-files ]]
UNITTEST_ENABLED: $[[ inputs.unittest-enabled ]]
UNITTEST_ARGS: $[[ inputs.unittest-args ]]
PYTEST_ENABLED: $[[ inputs.pytest-enabled ]]
PYTEST_ARGS: $[[ inputs.pytest-args ]]
NOSETESTS_ARGS: $[[ inputs.nosetests-args ]]
PYTHON_SBOM_SYFT_URL: $[[ inputs.sbom-syft-url ]]
PYTHON_SEMREL_RELEASE_DISABLED: $[[ inputs.semrel-release-disabled ]]
NOSETESTS_ENABLED: $[[ inputs.nosetests-enabled ]]
BANDIT_ENABLED: $[[ inputs.bandit-enabled ]]
SAFETY_ENABLED: $[[ inputs.safety-enabled ]]
PYTHON_SBOM_DISABLED: $[[ inputs.sbom-disabled ]]
PYTHON_RELEASE_ENABLED: $[[ inputs.release-enabled ]]
.python-scripts: &python-scripts | .python-scripts: &python-scripts |
# BEGSCRIPT # BEGSCRIPT
set -e set -e
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment