README.md

GitLab CI template for Python

This project implements a GitLab CI/CD template to build, test and analyse your Python projects.

Usage

In order to include this template in your project, add the following to your gitlab-ci.yml:

include:
  - project: 'to-be-continuous/python'
    ref: '6.1.3'
    file: '/templates/gitlab-ci-python.yml'

Global configuration

The Python template uses some global configuration used throughout all jobs.

Name	description	default value
PYTHON_IMAGE	The Docker image used to run Python ⚠️ set the version required by your project	registry.hub.docker.com/library/python:3
PYTHON_PROJECT_DIR	Python project root directory	.
PYTHON_BUILD_SYSTEM	Python build-system to use to install dependencies, build and package the project (see below)	none (auto-detect)
PIP_INDEX_URL	Python repository url	none
PIP_EXTRA_INDEX_URL	Exra Python repository url	none
PIP_OPTS	pip extra options	none
PYTHON_EXTRA_DEPS	Python extra sets of dependencies to install For Setuptools or Poetry only	none
PYTHON_REQS_FILE	Main requirements file (relative to $PYTHON_PROJECT_DIR) For Requirements Files build-system only	requirements.txt
PYTHON_EXTRA_REQS_FILES	Extra dev requirements file(s) to install (relative to $PYTHON_PROJECT_DIR)	requirements-dev.txt

The cache policy also makes the necessary to manage pip cache (not to download Python dependencies over and over again).

Multi build-system support

The Python template supports the most popular dependency management & build systems.

By default it tries to auto-detect the build system used by the project (based on the presence of pyproject.toml and/or setup.py and/or requirements.txt), but the build system might also be set explicitly using the $PYTHON_BUILD_SYSTEM variable:

Value	Build System (scope)
none (default) or auto	The template tries to auto-detect the actual build system
setuptools	Setuptools (dependencies, build & packaging)
poetry	Poetry (dependencies, build, test & packaging)
pipenv	Pipenv (dependencies only)
reqfile	Requirements Files (dependencies only)

⚠️ You can explicitly set the build tool version by setting $PYTHON_BUILD_SYSTEM variable including a version identification. For example PYTHON_BUILD_SYSTEM="poetry==1.1.15"

Jobs

py-package job

This job allows building your Python project distribution packages.

It is bound to the build stage, it is disabled by default and can be enabled by setting $PYTHON_PACKAGE_ENABLED to true.

Lint jobs

py-pylint job

This job is disabled by default and performs code analysis based on pylint Python lib. It is activated by setting $PYLINT_ENABLED to true.

It is bound to the build stage, and uses the following variables:

Name	description	default value
PYLINT_ARGS	Additional pylint CLI options	none
PYLINT_FILES	Files or directories to analyse	none (by default analyses all found python source files)

In addition to a textual report in the console, this job produces the following reports, kept for one day:

Report	Format	Usage
$PYTHON_PROJECT_DIR/reports/py-lint.codeclimate.json	Code Climate	GitLab integration
$PYTHON_PROJECT_DIR/reports/py-lint.parseable.txt	parseable	SonarQube integration

Test jobs

The Python template features four alternative test jobs:

• py-unittest that performs tests based on unittest Python lib,
• or py-pytest that performs tests based on pytest Python lib,
• or py-nosetest that performs tests based on nose Python lib,
• or py-compile that performs byte code generation to check syntax if not tests are available.

py-unittest job

This job is disabled by default and performs tests based on unittest Python lib. It is activated by setting $UNITTEST_ENABLED to true.

In order to produce JUnit test reports, the tests are executed with the xmlrunner module.

It is bound to the build stage, and uses the following variables:

Name	description	default value
UNITTEST_ARGS	Additional xmlrunner/unittest CLI options	none

ℹ️ use a .coveragerc file at the root of your Python project to control the coverage settings.

Example:

[run]
# enables branch coverage
branch = True
# list of directories/packages to cover
source =
    module_1
    module_2

In addition to a textual report in the console, this job produces the following reports, kept for one day:

Report	Format	Usage
$PYTHON_PROJECT_DIR/reports/TEST-*.xml	xUnit test report(s)	GitLab integration & SonarQube integration
$PYTHON_PROJECT_DIR/reports/py-coverage.cobertura.xml	Cobertura XML coverage report	GitLab integration & SonarQube integration

py-pytest job

This job is disabled by default and performs tests based on pytest Python lib. It is activated by setting $PYTEST_ENABLED to true.

It is bound to the build stage, and uses the following variables:

Name	description	default value
PYTEST_ARGS	Additional pytest or pytest-cov CLI options	none

ℹ️ use a .coveragerc file at the root of your Python project to control the coverage settings.

Example:

[run]
# enables branch coverage
branch = True
# list of directories/packages to cover
source =
    module_1
    module_2

In addition to a textual report in the console, this job produces the following reports, kept for one day:

Report	Format	Usage
$PYTHON_PROJECT_DIR/reports/TEST-*.xml	xUnit test report(s)	GitLab integration & SonarQube integration
$PYTHON_PROJECT_DIR/reports/py-coverage.cobertura.xml	Cobertura XML coverage report	GitLab integration & SonarQube integration

py-nosetest job

This job is disabled by default and performs tests based on nose Python lib. It is activated by setting $NOSETESTS_ENABLED to true.

It is bound to the build stage, and uses the following variables:

Name	description	default value
NOSETESTS_ARGS	Additional nose CLI options	none

By default coverage will be run on all the project directories. You can restrict it to your packages by setting the $NOSE_COVER_PACKAGE variable. More info

ℹ️ use a .coveragerc file at the root of your Python project to control the coverage settings.

In addition to a textual report in the console, this job produces the following reports, kept for one day:

Report	Format	Usage
$PYTHON_PROJECT_DIR/reports/TEST-*.xml	xUnit test report(s)	GitLab integration & SonarQube integration
$PYTHON_PROJECT_DIR/reports/py-coverage.cobertura.xml	Cobertura XML coverage report	GitLab integration & SonarQube integration

py-compile job

This job is a fallback if no unit test has been setup ($UNITTEST_ENABLED and $PYTEST_ENABLED and $NOSETEST_ENABLED are not set), and performs a compileall.

It is bound to the build stage, and uses the following variables:

Name	description	default value
PYTHON_COMPILE_ARGS	compileall CLI options	*

py-bandit job (SAST)

This job is disabled by default and performs a Bandit analysis.

It is bound to the test stage, and uses the following variables:

Name	description	default value
BANDIT_ENABLED	Set to true to enable Bandit analysis	none (disabled)
BANDIT_ARGS	Additional Bandit CLI options	--recursive .

In addition to a textual report in the console, this job produces the following reports, kept for one day:

Report	Format	Usage
$PYTHON_PROJECT_DIR/reports/py-bandit.bandit.csv	CSV	SonarQube integration This report is generated only if SonarQube template is detected
$PYTHON_PROJECT_DIR/reports/py-bandit.bandit.json	JSON	DefectDojo integration This report is generated only if DefectDojo template is detected

py-safety job (dependency check)

This job is disabled by default and performs a dependency check analysis using Safety.

It is bound to the test stage, and uses the following variables:

Name	description	default value
SAFETY_ENABLED	Set to true to enable Safety job	none (disabled)
SAFETY_ARGS	Additional Safety CLI options	--full-report

py-trivy job (dependency check)