README.md

GitLab CI template for Python

This project implements a GitLab CI/CD template to build, test and analyse your Python projects.

Usage

This template can be used both as a CI/CD component or using the legacy include:project syntax.

Use as a CI/CD component

Add the following to your gitlab-ci.yml:

include:
  # 1: include the component
  - component: gitlab.com/to-be-continuous/python/gitlab-ci-python@6.7.0
    # 2: set/override component inputs
    inputs:
      image: registry.hub.docker.com/library/python:3.10
      pytest-enabled: true

Use as a CI/CD template (legacy)

Add the following to your gitlab-ci.yml:

include:
  # 1: include the template
  - project: 'to-be-continuous/python'
    ref: '6.7.0'
    file: '/templates/gitlab-ci-python.yml'

variables:
  # 2: set/override template variables
  PYTHON_IMAGE: registry.hub.docker.com/library/python:3.10
  PYTEST_ENABLED: "true"

Global configuration

The Python template uses some global configuration used throughout all jobs.

Input / Variable	Description	Default value
image / PYTHON_IMAGE	The Docker image used to run Python ⚠️ set the version required by your project	registry.hub.docker.com/library/python:3
project-dir / PYTHON_PROJECT_DIR	Python project root directory	.
build-system / PYTHON_BUILD_SYSTEM	Python build-system to use to install dependencies, build and package the project (see below)	auto (auto-detect)
PIP_INDEX_URL	Python repository url	none
PIP_EXTRA_INDEX_URL	Extra Python repository url	none
pip-opts / PIP_OPTS	pip extra options	none
extra-deps / PYTHON_EXTRA_DEPS	Python extra sets of dependencies to install For Setuptools or Poetry only	none
reqs-file / PYTHON_REQS_FILE	Main requirements file (relative to $PYTHON_PROJECT_DIR) For Requirements Files build-system only	requirements.txt
extra-reqs-files / PYTHON_EXTRA_REQS_FILES	Extra dev requirements file(s) to install (relative to $PYTHON_PROJECT_DIR)	requirements-dev.txt

The cache policy also makes the necessary to manage pip cache (not to download Python dependencies over and over again).

Multi build-system support

The Python template supports the most popular dependency management & build systems.

By default it tries to auto-detect the build system used by the project (based on the presence of pyproject.toml and/or setup.py and/or requirements.txt), but the build system might also be set explicitly using the $PYTHON_BUILD_SYSTEM variable:

Value	Build System (scope)
none (default) or auto	The template tries to auto-detect the actual build system
setuptools	Setuptools (dependencies, build & packaging)
poetry	Poetry (dependencies, build, test & packaging)
pipenv	Pipenv (dependencies only)
reqfile	Requirements Files (dependencies only)

⚠️ You can explicitly set the build tool version by setting $PYTHON_BUILD_SYSTEM variable including a version identification. For example PYTHON_BUILD_SYSTEM="poetry==1.1.15"

Jobs

py-package job

This job allows building your Python project distribution packages.

It is bound to the build stage, it is disabled by default and can be enabled by setting $PYTHON_PACKAGE_ENABLED to true.

Lint jobs

py-lint job

This job is disabled by default and performs code analysis based on pylint Python lib. It is activated by setting $PYLINT_ENABLED to true.

It is bound to the build stage, and uses the following variables:

Input / Variable	Description	Default value
pylint-args / PYLINT_ARGS	Additional pylint CLI options	none
pylint-files / PYLINT_FILES	Files or directories to analyse	none (by default analyses all found python source files)

In addition to a textual report in the console, this job produces the following reports, kept for one day:

Report	Format	Usage
$PYTHON_PROJECT_DIR/reports/py-lint.codeclimate.json	Code Climate	GitLab integration
$PYTHON_PROJECT_DIR/reports/py-lint.parseable.txt	parseable	SonarQube integration

Test jobs

The Python template features four alternative test jobs:

• py-unittest that performs tests based on unittest Python lib,
• or py-pytest that performs tests based on pytest Python lib,
• or py-nosetest that performs tests based on nose Python lib,
• or py-compile that performs byte code generation to check syntax if not tests are available.

py-unittest job

This job is disabled by default and performs tests based on unittest Python lib. It is activated by setting $UNITTEST_ENABLED to true.

In order to produce JUnit test reports, the tests are executed with the xmlrunner module.

It is bound to the build stage, and uses the following variables:

Input / Variable	Description	Default value
unittest-args / UNITTEST_ARGS	Additional xmlrunner/unittest CLI options	none

ℹ️ use a .coveragerc file at the root of your Python project to control the coverage settings.

Example:

[run]
# enables branch coverage
branch = True
# list of directories/packages to cover
source =
    module_1
    module_2

In addition to a textual report in the console, this job produces the following reports, kept for one day:

Report	Format	Usage
$PYTHON_PROJECT_DIR/reports/TEST-*.xml	xUnit test report(s)	GitLab integration & SonarQube integration
$PYTHON_PROJECT_DIR/reports/py-coverage.cobertura.xml	Cobertura XML coverage report	GitLab integration & SonarQube integration

py-pytest job

This job is disabled by default and performs tests based on pytest Python lib. It is activated by setting $PYTEST_ENABLED to true.

It is bound to the build stage, and uses the following variables:

Input / Variable	Description	Default value
pytest-args / PYTEST_ARGS	Additional pytest or pytest-cov CLI options	none

ℹ️ use a .coveragerc file at the root of your Python project to control the coverage settings.

Example:

[run]
# enables branch coverage
branch = True
# list of directories/packages to cover
source =
    module_1
    module_2

In addition to a textual report in the console, this job produces the following reports, kept for one day:

Report	Format	Usage
$PYTHON_PROJECT_DIR/reports/TEST-*.xml	xUnit test report(s)	GitLab integration & SonarQube integration
$PYTHON_PROJECT_DIR/reports/py-coverage.cobertura.xml	Cobertura XML coverage report	GitLab integration & SonarQube integration

py-nosetests job

This job is disabled by default and performs tests based on nose Python lib. It is activated by setting $NOSETESTS_ENABLED to true.

It is bound to the build stage, and uses the following variables:

Input / Variable	Description	Default value
nosetests-args / NOSETESTS_ARGS	Additional nose CLI options	none

By default coverage will be run on all the project directories. You can restrict it to your packages by setting the $NOSE_COVER_PACKAGE variable. More info

ℹ️ use a .coveragerc file at the root of your Python project to control the coverage settings.

In addition to a textual report in the console, this job produces the following reports, kept for one day:

Report	Format	Usage
$PYTHON_PROJECT_DIR/reports/TEST-*.xml	xUnit test report(s)	GitLab integration & SonarQube integration
$PYTHON_PROJECT_DIR/reports/py-coverage.cobertura.xml	Cobertura XML coverage report	GitLab integration & SonarQube integration

py-compile job

This job is a fallback if no unit test has been set up ($UNITTEST_ENABLED and $PYTEST_ENABLED and $NOSETEST_ENABLED are not set), and performs a compileall.

It is bound to the build stage, and uses the following variables:

Input / Variable	Description	Default value
compile-args / PYTHON_COMPILE_ARGS	compileall CLI options	*

py-bandit job (SAST)

This job is disabled by default and performs a Bandit analysis.

It is bound to the test stage, and uses the following variables:

Input / Variable	Description	Default value
bandit-enabled / BANDIT_ENABLED	Set to true to enable Bandit analysis	none (disabled)
bandit-args / BANDIT_ARGS	Additional Bandit CLI options	--recursive .

In addition to a textual report in the console, this job produces the following reports, kept for one day:

Report	Format	Usage
$PYTHON_PROJECT_DIR/reports/py-bandit.bandit.csv	CSV	SonarQube integration This report is generated only if SonarQube template is detected
$PYTHON_PROJECT_DIR/reports/py-bandit.bandit.json	JSON	DefectDojo integration This report is generated only if DefectDojo template is detected