Merge branch 'feat/hatch' into 'master'

feat(system-build): add hatch support as a new build system

Closes #103

See merge request to-be-continuous/python!144

README.md

@@ -50,7 +50,7 @@ The Python template uses some global configuration used throughout all jobs.
 | `PIP_INDEX_URL`      | Python repository url                                                                 | _none_             |
 | `PIP_EXTRA_INDEX_URL` | Extra Python repository url                                                           | _none_             |
 | `pip-opts` / `PIP_OPTS` | pip [extra options](https://pip.pypa.io/en/stable/cli/pip/#general-options)     | _none_             |
-| `extra-deps` / `PYTHON_EXTRA_DEPS` | Python extra sets of dependencies to install<br/>For [Setuptools](https://setuptools.pypa.io/en/latest/userguide/dependency_management.html?highlight=extras#optional-dependencies) or [Poetry](https://python-poetry.org/docs/pyproject/#extras) only | _none_ |
+| `extra-deps` / `PYTHON_EXTRA_DEPS` | Python extra sets of dependencies to install<br/>For [Setuptools](https://setuptools.pypa.io/en/latest/userguide/dependency_management.html?highlight=extras#optional-dependencies) or [Poetry](https://python-poetry.org/docs/pyproject/#extras) or [uv](https://docs.astral.sh/uv/) only | _none_ |
 | `reqs-file` / `PYTHON_REQS_FILE` | Main requirements file _(relative to `$PYTHON_PROJECT_DIR`)_<br/>For [Requirements Files](https://pip.pypa.io/en/stable/user_guide/#requirements-files) build-system only | `requirements.txt` |
 | `extra-reqs-files` / `PYTHON_EXTRA_REQS_FILES` | Extra dev requirements file(s) to install _(relative to `$PYTHON_PROJECT_DIR`)_ | `requirements-dev.txt` |
 
@@ -66,15 +66,28 @@ and/or `setup.py` and/or `requirements.txt`), but the build system might also be
 
 | Value            | Build System (scope)                                                                                   |
 | ---------------- |--------------------------------------------------------------------------------------------------------|
-| _none_ (default) or `auto` | The template tries to **auto-detect** the actual build system                                          |
-| `setuptools`     | [Setuptools](https://setuptools.pypa.io/) (dependencies, build & packaging)                            |
-| `poetry`         | [Poetry](https://python-poetry.org/) (dependencies, build, test & packaging)                           |
-| `uv`             | [uv](https://docs.astral.sh/uv/) (dependencies, build, test & packaging)|
-| `pipenv`         | [Pipenv](https://pipenv.pypa.io/) (dependencies only)                                                  |
-| `reqfile`        | [Requirements Files](https://pip.pypa.io/en/stable/user_guide/#requirements-files) (dependencies only) |
+| _none_ (default) or `auto` | The template tries to **auto-detect** 	:sparkles: the actual build system                                          |
+| `setuptools`     | [![setuptools](https://img.shields.io/badge/setuptools-grey)](https://setuptools.pypa.io/) ![setuptools](https://img.shields.io/badge/dependencies_%7C_build_%7C_packaging-blue)                          |
+| `poetry`         | [![Poetry](https://img.shields.io/endpoint?url=https://python-poetry.org/badge/v0.json)](https://python-poetry.org/) ![Poetry](https://img.shields.io/badge/dependencies_%7C_build_%7C_test_%7C_packaging-blue)                           |
+| `uv`             | [![uv](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/uv/main/assets/badge/v0.json)](https://docs.astral.sh/uv/) ![uv](https://img.shields.io/badge/dependencies_%7C_build_%7C_test_%7C_packaging-blue)  |
+| `hatch`          | [![Hatch project](https://img.shields.io/badge/%F0%9F%A5%9A-Hatch-4051b5.svg)](https://hatch.pypa.io/latest/) ![Hatch](https://img.shields.io/badge/dependencies_%7C_build_%7C_test_%7C_packaging-blue) |
+| `pipenv`         | [![Pipenv](https://img.shields.io/badge/Pipenv-grey)](https://pipenv.pypa.io/) ![Pipenv](https://img.shields.io/badge/dependencies-blue)                                                  |
+| `reqfile`        | [![Requirements Files](https://img.shields.io/badge/Requirements_Files-grey)](https://pip.pypa.io/en/stable/user_guide/#requirements-files) ![Pipenv](https://img.shields.io/badge/dependencies-blue) |
 
 :warning: You can explicitly set the build tool version by setting `$PYTHON_BUILD_SYSTEM` variable including a [version identification](https://peps.python.org/pep-0440/). For example `PYTHON_BUILD_SYSTEM="poetry==1.1.15"`
 
+### Hatch
+
+All template jobs use the `default` Hatch environment. So dev dependencies should defined in the `[tool.hatch.envs.default.dependencies]` section of `pyproject.toml`.
+
+```toml
+[tool.hatch.envs.default]
+dependencies = [
+  "pytest>=8.0.0,<9",
+...
+]
+````
+
 ## Jobs
 
 ### `py-package` job
@@ -396,7 +409,8 @@ This job is **disabled by default** and allows to perform a complete release of
 The Python template supports three packaging systems:
 
 * [Poetry](https://python-poetry.org/): uses Poetry-specific [version](https://python-poetry.org/docs/cli/#version), [build](https://python-poetry.org/docs/cli/#build) and [publish](https://python-poetry.org/docs/cli/#publish) commands.
-* [uv](https://docs.astral.sh/uv/): uses [uv](https://docs.astral.sh/uv/) as version management, [build](https://docs.astral.sh/uv/guides/publish/#building-your-package) as package builder and [publish](https://docs.astral.sh/uv/guides/publish/) to publish.
+* [uv](https://docs.astral.sh/uv/): uses [bump-my-version](https://github.com/callowayproject/bump-my-version) as version management, [build](https://docs.astral.sh/uv/guides/publish/#building-your-package) as package builder and [publish](https://docs.astral.sh/uv/guides/publish/) to publish.
+* [hatch](https://hatch.pypa.io/latest/): uses [bump-my-version](https://github.com/callowayproject/bump-my-version) as version management, [build](https://hatch.pypa.io/latest/build/) as package builder and [publish](https://hatch.pypa.io/latest/publish/) to publish.
 * [Setuptools](https://setuptools.pypa.io/): uses [bump-my-version](https://github.com/callowayproject/bump-my-version) as version management, [build](https://pypa-build.readthedocs.io/) as package builder and [Twine](https://twine.readthedocs.io/) to publish.
 
 The release job is bound to the `publish` stage, appears only on production and integration branches and uses the following variables:
@@ -421,7 +435,8 @@ This job is **disabled by default** and allow to publish the built packages to a
 The Python template supports three packaging systems:
 
 * [Poetry](https://python-poetry.org/): uses Poetry-specific [version](https://python-poetry.org/docs/cli/#version), [build](https://python-poetry.org/docs/cli/#build) and [publish](https://python-poetry.org/docs/cli/#publish) commands.
-* [uv](https://docs.astral.sh/uv/): uses [uv](https://docs.astral.sh/uv/) as version management, [build](https://docs.astral.sh/uv/guides/publish/#building-your-package) as package builder and [publish](https://docs.astral.sh/uv/guides/publish/) to publish.
+* [uv](https://docs.astral.sh/uv/): uses [bump-my-version](https://github.com/callowayproject/bump-my-version) as version management, [build](https://docs.astral.sh/uv/guides/publish/#building-your-package) as package builder and [publish](https://docs.astral.sh/uv/guides/publish/) to publish.
+* [hatch](https://hatch.pypa.io/latest//): uses [bump-my-version](https://github.com/callowayproject/bump-my-version) as version management, [build](https://hatch.pypa.io/latest/build/) as package builder and [publish](https://hatch.pypa.io/latest/publish/) to publish.
 * [Setuptools](https://setuptools.pypa.io/): uses [bump-my-version](https://github.com/callowayproject/bump-my-version) as version management, [build](https://pypa-build.readthedocs.io/) as package builder and [Twine](https://twine.readthedocs.io/) to publish.
 
 The publish job is bound to the `publish` stage, is executed on a Git tag matching [semantic versioning pattern](https://semver.org/) and uses the following variables:

kicker.json

@@ -21,7 +21,7 @@
       "name": "PYTHON_BUILD_SYSTEM",
       "description": "Python build-system to use to install dependencies, build and package the project",
       "type": "enum",
-      "values": ["auto", "setuptools", "poetry", "pipenv", "reqfile", "uv"],
+      "values": ["auto", "setuptools", "poetry", "pipenv", "reqfile", "uv", "hatch"],
       "default": "auto",
       "advanced": true
     },

templates/gitlab-ci-python.yml

@@ -30,6 +30,7 @@ spec:
       - pipenv
       - reqfile
       - uv
+      - hatch
       default: auto
     reqs-file:
       description: |-
@@ -332,6 +333,17 @@ variables:
       echo -e "[\\e[1;91mERROR\\e[0m] $*"
   }
 
+  function log_elapsed_time() {
+    _end_time=$(get_current_ts_ms)
+    _named=$1
+    _start_time=$2
+
+    _diff_ms=$((_end_time - _start_time))
+    _elapsed_sec="$((_diff_ms / 1000)).$(((_diff_ms % 1000) / 100))"
+
+    log_info " *** ${_named} took \\e[32m$_elapsed_sec\\e[0m seconds"  
+  }
+
   function fail() {
     log_error "$*"
     exit 1
@@ -345,6 +357,16 @@ variables:
     fi
   }
 
+  function get_current_ts_ms() {
+    if command -v busybox > /dev/null
+    then
+      # no easy way to retrieve ms in BusyBox
+      echo $(($(date +%s) * 1000))
+    else
+      echo $(($(date +%s%N) / 1000000))
+    fi
+  }
+
   function install_ca_certs() {
     certs=$1
     if [[ -z "$certs" ]]
@@ -586,11 +608,14 @@ variables:
   }
 
   function guess_build_system() {
+    _start_time=$(get_current_ts_ms)
+
     case "${PYTHON_BUILD_SYSTEM:-auto}" in
     auto)
       ;;
-    poetry*|setuptools*|pipenv*|uv*)
-      log_info "--- Build system explicitly declared: ${PYTHON_BUILD_SYSTEM}"
+    poetry*|setuptools*|pipenv*|uv*|hatch*)
+      export PYTHON_BUILD_SYSTEM_CMD="${PYTHON_BUILD_SYSTEM%%[=<>]*}"
+      log_info "--- Build system explicitly declared: ${PYTHON_BUILD_SYSTEM} cmd=\\e[33m$PYTHON_BUILD_SYSTEM_CMD\\e[0m"
       return
       ;;
     reqfile)
@@ -615,6 +640,7 @@ variables:
       then
         log_info "--- Build system auto-detected: uv (uv.lock and pyproject.toml)"
         export PYTHON_BUILD_SYSTEM="uv"
+        export PYTHON_BUILD_SYSTEM_CMD="uv"
         return
       fi
       log_error "--- Build system auto-detected: uv (uv.lock) but no pyproject.toml found: please read template doc"
@@ -632,6 +658,13 @@ variables:
       poetry.core.masonry.api)
         log_info "--- Build system auto-detected: PEP 517 with Poetry backend"
         export PYTHON_BUILD_SYSTEM="poetry"
+        export PYTHON_BUILD_SYSTEM_CMD="poetry"
+        return
+        ;;
+      hatchling.build)
+        log_info "--- Build system auto-detected: PEP 517 with Hatch backend"
+        export PYTHON_BUILD_SYSTEM="hatch"
+        export PYTHON_BUILD_SYSTEM_CMD="hatch"
         return
         ;;
       setuptools.build_meta)
@@ -658,17 +691,12 @@ variables:
       log_error "--- Build system auto-detect failed: please read template doc"
       exit 1
     fi
-  }
 
-  function maybe_install_poetry() {
-    if [[ "$PYTHON_BUILD_SYSTEM" =~ ^poetry ]] && ! command -v poetry > /dev/null
-    then
-      # shellcheck disable=SC2086
-      pip install ${PIP_OPTS} "$PYTHON_BUILD_SYSTEM"
-    fi
+    log_elapsed_time "guess_build_system" "$_start_time"
     }
-  function maybe_install_uv() {
-    if [[ "$PYTHON_BUILD_SYSTEM" =~ ^uv ]] && ! command -v uv > /dev/null
+
+  function maybe_install_build_system() {
+    if ! command -v "$PYTHON_BUILD_SYSTEM_CMD" > /dev/null
     then
       # shellcheck disable=SC2086
       pip install ${PIP_OPTS} "$PYTHON_BUILD_SYSTEM"
@@ -677,23 +705,23 @@ variables:
 
   # install requirements
   function install_requirements() {
+    _start_time=$(get_current_ts_ms)
+
     case "$PYTHON_BUILD_SYSTEM" in
     poetry*)
       if  [[ ! -f "poetry.lock" ]]; then
         log_warn "Using Poetry but \\e[33;1mpoetry.lock\\e[0m file not found: you shall commit it with your project files"
       fi
-      maybe_install_poetry
+      maybe_install_build_system
       poetry install ${PYTHON_EXTRA_DEPS:+--extras "$PYTHON_EXTRA_DEPS"}
       ;;
     setuptools*)
-      # shellcheck disable=SC2086
-      pip install ${PIP_OPTS} "$PYTHON_BUILD_SYSTEM"
+      maybe_install_build_system
       # shellcheck disable=SC2086
       pip install ${PIP_OPTS} ".${PYTHON_EXTRA_DEPS:+[$PYTHON_EXTRA_DEPS]}"
       ;;
     pipenv*)
-      # shellcheck disable=SC2086
-      pip install ${PIP_OPTS} "$PYTHON_BUILD_SYSTEM"
+      maybe_install_build_system
       if  [[ ! -f "Pipfile.lock" ]]; then
         log_warn "Using Pipenv but \\e[33;1mPipfile.lock\\e[0m file not found: you shall commit it with your project files"
         pipenv install --dev --system
@@ -722,21 +750,31 @@ variables:
       if  [[ ! -f "uv.lock" ]]; then
         log_warn "Using uv but \\e[33;1muv.lock\\e[0m file not found: you shall commit it with your project files"
       fi
-      maybe_install_uv
+      maybe_install_build_system
       uv sync --frozen ${PYTHON_EXTRA_DEPS:+--extra "$PYTHON_EXTRA_DEPS"}
       ;;
+    hatch*)
+      maybe_install_build_system
+      hatch env create
+      ;;
     esac
+
+    log_elapsed_time "install_requirements" "$_start_time"
   }
 
   function _run() {
     if [[ "$PYTHON_BUILD_SYSTEM" =~ ^poetry ]]
     then
-      maybe_install_poetry
+      maybe_install_build_system
       poetry run "$@"
     elif [[ "$PYTHON_BUILD_SYSTEM" =~ ^uv ]]
     then
-      maybe_install_uv
+      maybe_install_build_system
       uv run "$@"
+    elif [[ "$PYTHON_BUILD_SYSTEM" =~ ^hatch ]]
+    then
+      maybe_install_build_system
+      $PYTHON_BUILD_SYSTEM_CMD run "$@"
     else
       "$@"
     fi
@@ -752,7 +790,7 @@ variables:
 
     if [[ "$PYTHON_BUILD_SYSTEM" =~ ^uv ]]
     then
-      maybe_install_uv
+      maybe_install_build_system
       # shellcheck disable=SC2086
       uv pip "$cmd" ${PIP_OPTS} "$@"
     else
@@ -762,19 +800,31 @@ variables:
   }
 
   function py_package() {
+    _start_time=$(get_current_ts_ms)
+
     if [[ "$PYTHON_BUILD_SYSTEM" =~ ^poetry ]]
     then
-      maybe_install_poetry
-      poetry build
+      maybe_install_build_system
+      log_info "--- build packages (poetry)..."
+      poetry build ${TRACE+--verbose}
     elif [[ "$PYTHON_BUILD_SYSTEM" =~ ^uv ]]
     then
-      maybe_install_uv
-      uv build
+      maybe_install_build_system
+      log_info "--- build packages (uv)..."
+      uv build ${TRACE+--verbose}
+    elif [[ "$PYTHON_BUILD_SYSTEM" =~ ^hatch ]]
+    then
+      log_info "--- build packages (hatch)..."
+      maybe_install_build_system
+      $PYTHON_BUILD_SYSTEM_CMD build
     else
+      log_info "--- build packages ..."
       # shellcheck disable=SC2086
       pip install ${PIP_OPTS} build
       python -m build
     fi
+
+    log_elapsed_time "py_package" "$_start_time"
   }
 
   function configure_scm_auth() {
@@ -800,7 +850,31 @@ variables:
     fi
   }
 
+  function py_bump_my_version() {
+    mkdir -p -m 777 tbc_tmp >&2
+    py_cur_version="$1"
+    py_release_part="$2"
+    echo "$py_cur_version" > tbc_tmp/version.txt
+    _pip install bump-my-version >&2
+    log_info "[bump-my-version] increase \\e[1;94m${py_release_part}\\e[0m (from current \\e[1;94m${py_cur_version}\\e[0m)" >&2
+    _run bump-my-version bump ${TRACE+--verbose} --current-version "$py_cur_version" "$py_release_part" tbc_tmp/version.txt  >&2
+    cat tbc_tmp/version.txt
+    rm -fr tbc_tmp/version.txt >&2
+  }
+
+  function py_commit_pyproject() {
+    py_cur_version="$1"
+    py_next_version="$2"  
+    # Git commit and tag
+    git add pyproject.toml
+    # emulate bump-my-version to generate commit message
+    py_commit_message=$(python -c "print('$PYTHON_RELEASE_COMMIT_MESSAGE'.format(current_version='$py_cur_version', new_version='$py_next_version'))")
+    git commit -m "$py_commit_message"
+    git tag "$py_next_version"  
+  }
+
   function py_release() {
+  log_info "PYTHON_RELEASE_ENABLED:$PYTHON_RELEASE_ENABLED  PYTHON_PUBLISH_ENABLED:$PYTHON_PUBLISH_ENABLED"
     # 1: retrieve next release info from semantic-release
     if [ "$SEMREL_INFO_ON" ] && [ "$PYTHON_SEMREL_RELEASE_DISABLED" != "true" ]
     then
@@ -819,7 +893,7 @@ variables:
     # 2: bump-my-version (+ Git commit & tag)
     if [[ "$PYTHON_BUILD_SYSTEM" =~ ^poetry ]]
     then
-      maybe_install_poetry
+      maybe_install_build_system
       if [[ -z "$py_next_version" ]]
       then
         py_cur_version=$(poetry version --short)
@@ -837,7 +911,7 @@ variables:
       git tag "$py_next_version"
     elif [[ "$PYTHON_BUILD_SYSTEM" =~ ^uv ]]
     then
-      maybe_install_uv
+      maybe_install_build_system
       if [[ -z "$py_next_version" ]]
       then
         # quick version waiting for uv to manage bump
@@ -862,6 +936,19 @@ variables:
       py_commit_message=$(python -c "print('$PYTHON_RELEASE_COMMIT_MESSAGE'.format(current_version='$py_cur_version', new_version='$py_next_version'))")
       git commit -m "$py_commit_message"
       git tag --force "$py_next_version"
+    elif [[ "$PYTHON_BUILD_SYSTEM" =~ ^hatch ]]
+    then
+      maybe_install_build_system
+      if [[ -z "$py_next_version" ]]
+      then
+        py_cur_version=$(hatch version)
+        py_next_version=$(py_bump_my_version "$py_cur_version" "$PYTHON_RELEASE_NEXT")
+      fi
+
+      log_info "[hatch] change version \\e[1;94m${py_cur_version}\\e[0m → \\e[1;94m${py_next_version}\\e[0m"
+      _pip install toml-cli
+      _run toml set --toml-path pyproject.toml project.version "$py_next_version"
+      py_commit_pyproject "$py_cur_version" "$py_next_version"
     else
       # Setuptools / bump-my-version
       # shellcheck disable=SC2086
@@ -909,9 +996,14 @@ variables:
   }
 
   function py_publish() {
+    if [[ "$PYTHON_BUILD_SYSTEM" =~ ^hatch ]] && [[ "$PYTHON_PACKAGE_ENABLED" != "true" ]]
+    then
+      py_package
+    fi
+
     if [[ "$PYTHON_BUILD_SYSTEM" =~ ^poetry ]]
     then
-      maybe_install_poetry
+      maybe_install_build_system
 
       if [[ "$PYTHON_PACKAGE_ENABLED" != "true" ]]
       then
@@ -924,7 +1016,7 @@ variables:
       poetry publish ${TRACE+--verbose} --username "$PYTHON_REPOSITORY_USERNAME" --password "$PYTHON_REPOSITORY_PASSWORD" --repository user_defined
     elif [[ "$PYTHON_BUILD_SYSTEM" =~ ^uv ]]
     then
-      maybe_install_uv
+      maybe_install_build_system
   
       if [[ "$PYTHON_PACKAGE_ENABLED" != "true" ]]
       then
@@ -934,6 +1026,11 @@ variables:
 
       log_info "--- publish packages (uv) to $PYTHON_REPOSITORY_URL with user $PYTHON_REPOSITORY_USERNAME..."
       uv publish ${TRACE+--verbose} --username "$PYTHON_REPOSITORY_USERNAME" --password "$PYTHON_REPOSITORY_PASSWORD" --publish-url "$PYTHON_REPOSITORY_URL"
+    elif [[ "$PYTHON_BUILD_SYSTEM" =~ ^hatch ]]
+    then
+      maybe_install_build_system
+      log_info "--- publish packages (hatch) to $PYTHON_REPOSITORY_URL with user $PYTHON_REPOSITORY_USERNAME..."
+      hatch publish ${TRACE+--verbose} --no-prompt --yes --user "$PYTHON_REPOSITORY_USERNAME" --auth "$PYTHON_REPOSITORY_PASSWORD" --repo "$PYTHON_REPOSITORY_URL"
     else
       # shellcheck disable=SC2086
       pip install ${PIP_OPTS} build twine
@@ -997,6 +1094,8 @@ stages:
     POETRY_CACHE_DIR: "$CI_PROJECT_DIR/.cache/poetry"
     PIPENV_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pipenv"
     UV_CACHE_DIR: "$CI_PROJECT_DIR/.cache/uv"
+    HATCH_CACHE_DIR: "$CI_PROJECT_DIR/.cache/hatch/.cache/"
+    HATCH_DATA_DIR: "$CI_PROJECT_DIR/.cache/hatch/.local"
     POETRY_VIRTUALENVS_IN_PROJECT: "false"
   cache:
     key: "$CI_COMMIT_REF_SLUG-python"
@@ -1297,9 +1396,14 @@ py-trivy:
           ;;
         uv*)
           log_info "$PYTHON_BUILD_SYSTEM build system used (\\e[32mmust generate pinned requirements.txt from uv.lock\\e[0m)"
-          maybe_install_uv
+          maybe_install_build_system
           uv export > ./reports/requirements.txt
           ;;
+        hatch*)
+          log_info "$PYTHON_BUILD_SYSTEM build system used (\\e[32mmust generate pinned requirements.txt\\e[0m)"
+          maybe_install_build_system
+          hatch run python -m pip freeze > reports/requirements.txt # hatch dep show requirements not working and complete
+          ;;
         *)
           log_info "$PYTHON_BUILD_SYSTEM build system used (\\e[32mmust generate pinned requirements.txt\\e[0m)"
           install_requirements
@@ -1356,9 +1460,14 @@ py-sbom:
           ;;
         uv*)
           log_info "$PYTHON_BUILD_SYSTEM build system used (\\e[32mmust generate pinned requirements.txt from uv.lock\\e[0m)"
-          maybe_install_uv
+          maybe_install_build_system
           uv export > ./reports/requirements.txt
           ;;
+        hatch*)
+          log_info "$PYTHON_BUILD_SYSTEM build system used (\\e[32mmust generate pinned requirements.txt from uv.lock\\e[0m)"
+          maybe_install_build_system
+          hatch run python -m pip freeze > reports/requirements.txt # hatch dep show requirements not working and complete
+          ;;
         *)
           log_info "$PYTHON_BUILD_SYSTEM build system used (\\e[32mmust generate pinned requirements.txt\\e[0m)"
           install_requirements