Skip to content
Snippets Groups Projects
Commit d22ffbac authored by euri10's avatar euri10 Committed by Gaëtan Montury
Browse files

feat(uv): add uv support as a new build system

parent fd87fcf1
No related branches found
No related tags found
No related merge requests found
...@@ -65,10 +65,11 @@ and/or `setup.py` and/or `requirements.txt`), but the build system might also be ...@@ -65,10 +65,11 @@ and/or `setup.py` and/or `requirements.txt`), but the build system might also be
`$PYTHON_BUILD_SYSTEM` variable: `$PYTHON_BUILD_SYSTEM` variable:
| Value | Build System (scope) | | Value | Build System (scope) |
| ---------------- | ---------------------------------------------------------- | | ---------------- |--------------------------------------------------------------------------------------------------------|
| _none_ (default) or `auto` | The template tries to **auto-detect** the actual build system | | _none_ (default) or `auto` | The template tries to **auto-detect** the actual build system |
| `setuptools` | [Setuptools](https://setuptools.pypa.io/) (dependencies, build & packaging) | | `setuptools` | [Setuptools](https://setuptools.pypa.io/) (dependencies, build & packaging) |
| `poetry` | [Poetry](https://python-poetry.org/) (dependencies, build, test & packaging) | | `poetry` | [Poetry](https://python-poetry.org/) (dependencies, build, test & packaging) |
| `uv` | [uv](https://docs.astral.sh/uv/) (dependencies, build, test & packaging)|
| `pipenv` | [Pipenv](https://pipenv.pypa.io/) (dependencies only) | | `pipenv` | [Pipenv](https://pipenv.pypa.io/) (dependencies only) |
| `reqfile` | [Requirements Files](https://pip.pypa.io/en/stable/user_guide/#requirements-files) (dependencies only) | | `reqfile` | [Requirements Files](https://pip.pypa.io/en/stable/user_guide/#requirements-files) (dependencies only) |
...@@ -372,9 +373,10 @@ This job is **disabled by default** and allows to perform a complete release of ...@@ -372,9 +373,10 @@ This job is **disabled by default** and allows to perform a complete release of
3. build the [Python packages](https://packaging.python.org/), 3. build the [Python packages](https://packaging.python.org/),
4. publish the built packages to a PyPI compatible repository ([GitLab packages](https://docs.gitlab.com/ee/user/packages/pypi_repository/) by default). 4. publish the built packages to a PyPI compatible repository ([GitLab packages](https://docs.gitlab.com/ee/user/packages/pypi_repository/) by default).
The Python template supports two packaging systems: The Python template supports three packaging systems:
* [Poetry](https://python-poetry.org/): uses Poetry-specific [version](https://python-poetry.org/docs/cli/#version), [build](https://python-poetry.org/docs/cli/#build) and [publish](https://python-poetry.org/docs/cli/#publish) commands. * [Poetry](https://python-poetry.org/): uses Poetry-specific [version](https://python-poetry.org/docs/cli/#version), [build](https://python-poetry.org/docs/cli/#build) and [publish](https://python-poetry.org/docs/cli/#publish) commands.
* [uv](https://docs.astral.sh/uv/): uses [uv](https://docs.astral.sh/uv/) as version management, [build](https://docs.astral.sh/uv/guides/publish/#building-your-package) as package builder and [publish](https://docs.astral.sh/uv/guides/publish/) to publish.
* [Setuptools](https://setuptools.pypa.io/): uses [bump-my-version](https://github.com/callowayproject/bump-my-version) as version management, [build](https://pypa-build.readthedocs.io/) as package builder and [Twine](https://twine.readthedocs.io/) to publish. * [Setuptools](https://setuptools.pypa.io/): uses [bump-my-version](https://github.com/callowayproject/bump-my-version) as version management, [build](https://pypa-build.readthedocs.io/) as package builder and [Twine](https://twine.readthedocs.io/) to publish.
The release job is bound to the `publish` stage, appears only on production and integration branches and uses the following variables: The release job is bound to the `publish` stage, appears only on production and integration branches and uses the following variables:
......
...@@ -21,7 +21,7 @@ ...@@ -21,7 +21,7 @@
"name": "PYTHON_BUILD_SYSTEM", "name": "PYTHON_BUILD_SYSTEM",
"description": "Python build-system to use to install dependencies, build and package the project", "description": "Python build-system to use to install dependencies, build and package the project",
"type": "enum", "type": "enum",
"values": ["auto", "setuptools", "poetry", "pipenv", "reqfile"], "values": ["auto", "setuptools", "poetry", "pipenv", "reqfile", "uv"],
"default": "auto", "default": "auto",
"advanced": true "advanced": true
}, },
......
...@@ -29,6 +29,7 @@ spec: ...@@ -29,6 +29,7 @@ spec:
- poetry - poetry
- pipenv - pipenv
- reqfile - reqfile
- uv
default: auto default: auto
reqs-file: reqs-file:
description: |- description: |-
...@@ -569,7 +570,7 @@ variables: ...@@ -569,7 +570,7 @@ variables:
case "${PYTHON_BUILD_SYSTEM:-auto}" in case "${PYTHON_BUILD_SYSTEM:-auto}" in
auto) auto)
;; ;;
poetry*|setuptools*|pipenv*) poetry*|setuptools*|pipenv*|uv*)
log_info "--- Build system explicitly declared: ${PYTHON_BUILD_SYSTEM}" log_info "--- Build system explicitly declared: ${PYTHON_BUILD_SYSTEM}"
return return
;; ;;
...@@ -589,6 +590,17 @@ variables: ...@@ -589,6 +590,17 @@ variables:
return return
fi fi
if [[ -f "uv.lock" ]]
then
if [[ -f "pyproject.toml" ]]
then
log_info "--- Build system auto-detected: uv (uv.lock and pyproject.toml)"
export PYTHON_BUILD_SYSTEM="uv"
return
fi
log_error "--- Build system auto-detected: uv (uv.lock) but no pyproject.toml found: please read template doc"
fi
if [[ -f "pyproject.toml" ]] if [[ -f "pyproject.toml" ]]
then then
# that might be PEP 517 if a build-backend is specified # that might be PEP 517 if a build-backend is specified
...@@ -636,6 +648,13 @@ variables: ...@@ -636,6 +648,13 @@ variables:
pip install ${PIP_OPTS} "$PYTHON_BUILD_SYSTEM" pip install ${PIP_OPTS} "$PYTHON_BUILD_SYSTEM"
fi fi
} }
function maybe_install_uv() {
if [[ "$PYTHON_BUILD_SYSTEM" =~ ^uv ]] && ! command -v uv > /dev/null
then
# shellcheck disable=SC2086
pip install ${PIP_OPTS} "$PYTHON_BUILD_SYSTEM"
fi
}
# install requirements # install requirements
function install_requirements() { function install_requirements() {
...@@ -680,6 +699,13 @@ variables: ...@@ -680,6 +699,13 @@ variables:
log_warn "--- requirements build system defined, but no ${PYTHON_REQS_FILE} file found" log_warn "--- requirements build system defined, but no ${PYTHON_REQS_FILE} file found"
fi fi
;; ;;
uv*)
if [[ ! -f "uv.lock" ]]; then
log_warn "Using uv but \\e[33;1muv.lock\\e[0m file not found: you shall commit it with your project files"
fi
maybe_install_uv
uv sync --frozen ${PYTHON_EXTRA_DEPS:+--extras "$PYTHON_EXTRA_DEPS"}
;;
esac esac
} }
...@@ -688,6 +714,10 @@ variables: ...@@ -688,6 +714,10 @@ variables:
then then
maybe_install_poetry maybe_install_poetry
poetry run "$@" poetry run "$@"
elif [[ "$PYTHON_BUILD_SYSTEM" =~ ^uv ]]
then
maybe_install_uv
uv run "$@"
else else
"$@" "$@"
fi fi
...@@ -698,8 +728,15 @@ variables: ...@@ -698,8 +728,15 @@ variables:
} }
function _pip() { function _pip() {
if [[ "$PYTHON_BUILD_SYSTEM" =~ ^uv ]]
then
maybe_install_uv
# shellcheck disable=SC2086
uv pip ${PIP_OPTS} "$@"
else
# shellcheck disable=SC2086 # shellcheck disable=SC2086
_run pip ${PIP_OPTS} "$@" _run pip ${PIP_OPTS} "$@"
fi
} }
function py_package() { function py_package() {
...@@ -707,6 +744,10 @@ variables: ...@@ -707,6 +744,10 @@ variables:
then then
maybe_install_poetry maybe_install_poetry
poetry build poetry build
elif [[ "$PYTHON_BUILD_SYSTEM" =~ ^uv ]]
then
maybe_install_uv
uv build
else else
# shellcheck disable=SC2086 # shellcheck disable=SC2086
pip install ${PIP_OPTS} build pip install ${PIP_OPTS} build
...@@ -772,6 +813,33 @@ variables: ...@@ -772,6 +813,33 @@ variables:
py_commit_message=$(python -c "print('$PYTHON_RELEASE_COMMIT_MESSAGE'.format(current_version='$py_cur_version', new_version='$py_next_version'))") py_commit_message=$(python -c "print('$PYTHON_RELEASE_COMMIT_MESSAGE'.format(current_version='$py_cur_version', new_version='$py_next_version'))")
git commit -m "$py_commit_message" git commit -m "$py_commit_message"
git tag "$py_next_version" git tag "$py_next_version"
elif [[ "$PYTHON_BUILD_SYSTEM" =~ ^uv ]]
then
maybe_install_uv
if [[ -z "$py_next_version" ]]
then
# quick version waiting for uv to manage bump
# related uv MR https://github.com/astral-sh/uv/pull/7248#issuecomment-2395465334
mkdir -p -m 777 tbc_tmp
uvx --from toml-cli toml get --toml-path pyproject.toml project.version > tbc_tmp/version.txt
py_cur_version=$(cat tbc_tmp/version.txt)
py_release_part="$PYTHON_RELEASE_NEXT"
log_info "[bump-my-version] increase \\e[1;94m${py_release_part}\\e[0m (from current \\e[1;94m${py_cur_version}\\e[0m)"
uvx bump-my-version bump ${TRACE+--verbose} --current-version "$py_cur_version" "$py_release_part" tbc_tmp/version.txt
py_next_version=$(cat tbc_tmp/version.txt)
rm -fr tbc_tmp/version.txt
fi
log_info "[uv] change version \\e[1;94m${py_cur_version}\\e[0m → \\e[1;94m${py_next_version}\\e[0m"
uvx --from toml-cli toml set --toml-path pyproject.toml project.version "$py_next_version"
# Git commit and tag
git add pyproject.toml
# emulate bump-my-version to generate commit message
py_commit_message=$(python -c "print('$PYTHON_RELEASE_COMMIT_MESSAGE'.format(current_version='$py_cur_version', new_version='$py_next_version'))")
git commit -m "$py_commit_message"
git tag --force "$py_next_version"
else else
# Setuptools / bump-my-version # Setuptools / bump-my-version
# shellcheck disable=SC2086 # shellcheck disable=SC2086
...@@ -825,6 +893,18 @@ variables: ...@@ -825,6 +893,18 @@ variables:
log_info "--- publish packages (poetry) to $PYTHON_REPOSITORY_URL with user $PYTHON_REPOSITORY_USERNAME..." log_info "--- publish packages (poetry) to $PYTHON_REPOSITORY_URL with user $PYTHON_REPOSITORY_USERNAME..."
poetry config repositories.user_defined "$PYTHON_REPOSITORY_URL" poetry config repositories.user_defined "$PYTHON_REPOSITORY_URL"
poetry publish ${TRACE+--verbose} --username "$PYTHON_REPOSITORY_USERNAME" --password "$PYTHON_REPOSITORY_PASSWORD" --repository user_defined poetry publish ${TRACE+--verbose} --username "$PYTHON_REPOSITORY_USERNAME" --password "$PYTHON_REPOSITORY_PASSWORD" --repository user_defined
elif [[ "$PYTHON_BUILD_SYSTEM" =~ ^uv ]]
then
maybe_install_uv
if [[ "$PYTHON_PACKAGE_ENABLED" != "true" ]]
then
log_info "--- build packages (uv)..."
uv build ${TRACE+--verbose}
fi
log_info "--- publish packages (uv) to $PYTHON_REPOSITORY_URL with user $PYTHON_REPOSITORY_USERNAME..."
uv publish ${TRACE+--verbose} --username "$PYTHON_REPOSITORY_USERNAME" --password "$PYTHON_REPOSITORY_PASSWORD" --publish-url "$PYTHON_REPOSITORY_URL"
else else
# shellcheck disable=SC2086 # shellcheck disable=SC2086
pip install ${PIP_OPTS} build twine pip install ${PIP_OPTS} build twine
...@@ -887,6 +967,7 @@ stages: ...@@ -887,6 +967,7 @@ stages:
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip" PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
POETRY_CACHE_DIR: "$CI_PROJECT_DIR/.cache/poetry" POETRY_CACHE_DIR: "$CI_PROJECT_DIR/.cache/poetry"
PIPENV_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pipenv" PIPENV_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pipenv"
UV_CACHE_DIR: "$CI_PROJECT_DIR/.cache/uv"
POETRY_VIRTUALENVS_IN_PROJECT: "false" POETRY_VIRTUALENVS_IN_PROJECT: "false"
cache: cache:
key: "$CI_COMMIT_REF_SLUG-python" key: "$CI_COMMIT_REF_SLUG-python"
...@@ -956,7 +1037,7 @@ py-black: ...@@ -956,7 +1037,7 @@ py-black:
script: script:
- install_requirements - install_requirements
- _pip install black - _pip install black
- _run black . --check - _run black . --check --extend-exclude '(\/\.cache\/|\/\.venv\/)'
rules: rules:
# exclude if $PYTHON_BLACK_ENABLED not set # exclude if $PYTHON_BLACK_ENABLED not set
- if: '$PYTHON_BLACK_ENABLED != "true"' - if: '$PYTHON_BLACK_ENABLED != "true"'
...@@ -969,7 +1050,7 @@ py-isort: ...@@ -969,7 +1050,7 @@ py-isort:
script: script:
- install_requirements - install_requirements
- _pip install isort - _pip install isort
- _run isort . --check-only --extend-skip .cache - _run isort . --check-only --extend-skip .cache --extend-skip .venv
rules: rules:
# exclude if $PYTHON_ISORT_ENABLED not set # exclude if $PYTHON_ISORT_ENABLED not set
- if: '$PYTHON_ISORT_ENABLED != "true"' - if: '$PYTHON_ISORT_ENABLED != "true"'
...@@ -1018,7 +1099,7 @@ py-mypy: ...@@ -1018,7 +1099,7 @@ py-mypy:
- mkdir -p -m 777 reports - mkdir -p -m 777 reports
- install_requirements - install_requirements
- _pip install mypy mypy-to-codeclimate - _pip install mypy mypy-to-codeclimate
- _run mypy ${MYPY_ARGS} ${MYPY_FILES:-$(find -type f -name "*.py" -not -path "./.cache/*")} | tee reports/py-mypy.console.txt || true - _run mypy ${MYPY_ARGS} ${MYPY_FILES:-$(find -type f -name "*.py" -not -path "./.cache/*" -not -path "./.venv/*")} | tee reports/py-mypy.console.txt || true
# mypy-to-codeclimate will fail if any error was found # mypy-to-codeclimate will fail if any error was found
- _run mypy-to-codeclimate reports/py-mypy.console.txt reports/py-mypy.codeclimate.json - _run mypy-to-codeclimate reports/py-mypy.console.txt reports/py-mypy.codeclimate.json
artifacts: artifacts:
...@@ -1140,15 +1221,15 @@ py-bandit: ...@@ -1140,15 +1221,15 @@ py-bandit:
- | - |
if [[ "$SONAR_HOST_URL" ]] if [[ "$SONAR_HOST_URL" ]]
then then
_run bandit ${TRACE+--verbose} --exit-zero --exclude ./.cache --format csv --output reports/py-bandit.bandit.csv ${BANDIT_ARGS} _run bandit ${TRACE+--verbose} --exit-zero --exclude ./.cache --exclude ./.venv --format csv --output reports/py-bandit.bandit.csv ${BANDIT_ARGS}
fi fi
# JSON (for DefectDojo) # JSON (for DefectDojo)
- | - |
if [[ "$DEFECTDOJO_BANDIT_REPORTS" ]] if [[ "$DEFECTDOJO_BANDIT_REPORTS" ]]
then then
_run bandit ${TRACE+--verbose} --exit-zero --exclude ./.cache --format json --output reports/py-bandit.bandit.json ${BANDIT_ARGS} _run bandit ${TRACE+--verbose} --exit-zero --exclude ./.cache --exclude ./.venv --format json --output reports/py-bandit.bandit.json ${BANDIT_ARGS}
fi fi
- _run bandit ${TRACE+--verbose} --exclude ./.cache ${BANDIT_ARGS} - _run bandit ${TRACE+--verbose} --exclude ./.cache --exclude ./.venv ${BANDIT_ARGS}
artifacts: artifacts:
when: always when: always
name: "$CI_JOB_NAME artifacts from $CI_PROJECT_NAME on $CI_COMMIT_REF_SLUG" name: "$CI_JOB_NAME artifacts from $CI_PROJECT_NAME on $CI_COMMIT_REF_SLUG"
...@@ -1194,6 +1275,11 @@ py-trivy: ...@@ -1194,6 +1275,11 @@ py-trivy:
log_info "$PYTHON_BUILD_SYSTEM build system (\\e[32muse lock file\\e[0m)" log_info "$PYTHON_BUILD_SYSTEM build system (\\e[32muse lock file\\e[0m)"
cp poetry.lock Pipfile.lock ./reports 2>/dev/null || true cp poetry.lock Pipfile.lock ./reports 2>/dev/null || true
;; ;;
uv*)
log_info "$PYTHON_BUILD_SYSTEM build system used (\\e[32mmust generate pinned requirements.txt from uv.lock\\e[0m)"
maybe_install_uv
uv export > ./reports/requirements.txt
;;
*) *)
log_info "$PYTHON_BUILD_SYSTEM build system used (\\e[32mmust generate pinned requirements.txt\\e[0m)" log_info "$PYTHON_BUILD_SYSTEM build system used (\\e[32mmust generate pinned requirements.txt\\e[0m)"
install_requirements install_requirements
...@@ -1244,6 +1330,11 @@ py-sbom: ...@@ -1244,6 +1330,11 @@ py-sbom:
poetry*|pipenv*) poetry*|pipenv*)
log_info "$PYTHON_BUILD_SYSTEM build system (\\e[32muse lock file\\e[0m)" log_info "$PYTHON_BUILD_SYSTEM build system (\\e[32muse lock file\\e[0m)"
;; ;;
uv*)
log_info "$PYTHON_BUILD_SYSTEM build system used (\\e[32mmust generate pinned requirements.txt from uv.lock\\e[0m)"
maybe_install_uv
uv export > ./reports/requirements.txt
;;
*) *)
log_info "$PYTHON_BUILD_SYSTEM build system used (\\e[32mmust generate pinned requirements.txt\\e[0m)" log_info "$PYTHON_BUILD_SYSTEM build system used (\\e[32mmust generate pinned requirements.txt\\e[0m)"
install_requirements install_requirements
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment