GitLab CI template for Python

This project implements a generic GitLab CI template for Python.

It provides several features, usable in different modes (by configuration) following those recommendations

Usage

In order to include this template in your project, add the following to your gitlab-ci.yml:

include:
  - project: 'to-be-continuous/python'
    ref: '3.0.0'
    file: '/templates/gitlab-ci-python.yml'

Global configuration

The Python template uses some global configuration used throughout all jobs.

The cache policy also declares the .cache/pip directory as cached (not to download Python dependencies over and over again).

Default configuration follows this Python project structure

Poetry support

The Python template supports Poetry as packaging and dependency management tool.

If a pyproject.toml file is detected at the root of your Python project, requirements will automatically be generated from Poetry. Poetry support can be explicitly disabled by setting PYTHON_POETRY_DISABLED to true.

⚠️ If no poetry.lock file is found, the template will emit a (non-blocking) warning message, to enforce Poetry recommendation:

You should commit the poetry.lock file to your project repo so that all people working on the project are locked to the same versions of dependencies.

Poetry support uses the following variables:

Jobs

Lint jobs

py-pylint job

This job is disabled by default and performs code analysis based on pylint Python lib. It is activated by setting $PYLINT_ENABLED to true.

It is bound to the build stage, and uses the following variables:

This job produces the following artifacts, kept for one day:

• Code quality json report in code climate format.

Test jobs

The Python template features four alternative test jobs:

• py-unittest that performs tests based on unittest Python lib,
• or py-pytest that performs tests based on pytest Python lib,
• or py-nosetest that performs tests based on nose Python lib,
• or py-compile that performs byte code generation to check syntax if not tests are available.

py-unittest job

This job is disabled by default and performs tests based on unittest Python lib. It is activated by setting $UNITTEST_ENABLED to true.

In order to produce JUnit test reports, the tests are executed with the xmlrunner module.

It is bound to the build stage, and uses the following variables:

This job produces the following artifacts, kept for one day:

• JUnit test report (using the xmlrunner module)
• code coverage report (cobertura xml format).

⚠️ create a .coveragerc file at the root of your Python project to control the coverage settings.

Example:

[run]
# enables branch coverage
branch = True
# list of directories/packages to cover
source = 
    module_1
    module_2

py-pytest job

This job is disabled by default and performs tests based on pytest Python lib. It is activated by setting $PYTEST_ENABLED to true.

It is bound to the build stage, and uses the following variables:

This job produces the following artifacts, kept for one day:

• JUnit test report (with the --junit-xml argument)
• code coverage report (cobertura xml format).

⚠️ create a .coveragerc file at the root of your Python project to control the coverage settings.

Example:

[run]
# enables branch coverage
branch = True
# list of directories/packages to cover
source = 
    module_1
    module_2

py-nosetest job

This job is disabled by default and performs tests based on nose Python lib. It is activated by setting $NOSETESTS_ENABLED to true.

It is bound to the build stage, and uses the following variables:

By default coverage will be run on all the directory. You can restrict it to your packages by setting NOSE_COVER_PACKAGE variable. More info

This job produces the following artifacts, kept for one day:

• JUnit test report (with the --with-xunit argument)
• code coverage report (cobertura xml format + html report).

⚠️ create a .coveragerc file at the root of your Python project or use nose CLI options to control the coverage settings.

py-compile job

This job is a fallback if no unit test has been setup ($UNITTEST_ENABLED and $PYTEST_ENABLED and $NOSETEST_ENABLED are not set), and performs a compileall.

It is bound to the build stage, and uses the following variables:

SonarQube analysis

If you're using the SonarQube template to analyse your Python code, here is a sample sonar-project.properties file:

# see: https://docs.sonarqube.org/latest/analysis/languages/python/
# set your source directory(ies) here (relative to the sonar-project.properties file)
sonar.sources=.
# exclude unwanted directories and files from being analysed
sonar.exclusions=**/test_*.py

# set your tests directory(ies) here (relative to the sonar-project.properties file)
sonar.tests=.
sonar.test.inclusions=**/test_*.py

# tests report: generic format
sonar.python.xunit.reportPath=reports/unittest/TEST-*.xml
# coverage report: XUnit format
sonar.python.coverage.reportPaths=reports/coverage.xml

More info:

• Python language support
• test coverage & execution parameters
• third-party issues

py-bandit job (SAST)

This job is disabled by default and performs a Bandit analysis.

It is bound to the test stage, and uses the following variables:

This job outputs a textual report in the console, and in case of failure also exports a JSON report in the reports/ directory (relative to project root dir).

py-safety job (dependency check)

This job is disabled by default and performs a dependency check analysis using Safety.

It is bound to the test stage, and uses the following variables:

This job outputs a textual report in the console, and in case of failure also exports a JSON report in the reports/ directory (relative to project root dir).

Package jobs

py-package job

This job is disabled by default and performs a packaging of your Python code.

It is bound to the package-build stage, applies only on git tags and uses the following variables:

Publish jobs

py-release job

This job is disabled by default and performs an automatic tagging of your Python code.

• Bumpversion Python library is used for version management.
• Looks for an existing .bumpversion.cfg at the project root. If found, it will be the configuration used by bumpversion. If not, the $RELEASE_VERSION_PART variable and setup.py will be used instead.
• Creating a Git tag involves an authenticated and authorized Git user.

Don't use your personal password !!! Use an access token with write_repository rights. If you have a generic account, add it to the project and generate access token from this account.

It is bound to the publish stage, applies only on master branch and uses the following variables:

py-publish job

This job is disabled by default and performs a publication of your Python code.

It is bound to the publish stage, applies only on git tags and uses the following variables:

More info:

• Python Packaging User Guide
• PyPI packages in the Package Registry

If you want to automatically create tag and publish your Python package, please have a look here

py-docs job

This job is no longer supported in this version of the template. It might come back later on with a more generic & configurable implementation.

GitLab compatibility

ℹ️ This template is actually tested and validated on GitLab Community Edition instance version 13.12.11