diff --git a/README.adoc b/README.adoc index c5993788b5f4810d41c9c2dfc62a6aef90e1b9f6..eb4ff72ac250cb16e94fc65a5f2de93ee3c565cc 100644 --- a/README.adoc +++ b/README.adoc @@ -1,55 +1,10 @@ = GitLab Templates +:toc: This repository contains miscellaneous resources that need to be accessible without authentication. -== GitLab CI templates -Two templates with default CI pipeline configurations are provided, for any repo to make use of: - -- `.gitlab-industrial-ci-no-rosinstall.yml` -- `.gitlab-industrial-ci-with-rosinstall.yml` - -=== ROS distribution selection -These pipelines will try to detect the ROS distro from the branch name. For example, if the branch starts with `noetic-` it will build for noetic. If the branch doesn't start with any ROS distro, it will build the default distro, which can be specified with the variable `DEFAULT_DISTRO`. For convenience and backwards compatibility, some templates are provided that already set this variable: - -- `.gitlab-industrial-ci-kinetic-no-rosinstall.yml` -- `.gitlab-industrial-ci-melodic-no-rosinstall.yml` -- `.gitlab-industrial-ci-noetic-no-rosinstall.yml` -- And the same with the `with-rosinstall` files. - -=== Multiple ROS distributions - -If a repo wants to compile using multiple distributions, one can specify them by setting the `BUILD` variables: - -```yml -include: 'https://git.code.tecnalia.com/tecnalia_robotics-public/gitlab_templates/raw/master/.gitlab-industrial-ci-with-rosinstall.yml' - -variables: - BUILD_KINETIC: 1 - BUILD_MELODIC: 1 - BUILD_NOETIC: 1 -``` - -=== Pipeline - -All files follow the same pipeline, consisting of the following stages and jobs: - -- Stage: `build` - * Check python syntax - * Run `industrial_ci` - ** Uses the ros distro defined above - ** Uses `flexbotics-base-devel` as a starting image - ** Runs link:scripts/ci_run_entry_points.sh[] in the `BEFORE_SCRIPT` stage, which in turn finds and runs every `ci_entry_script.bash` script found in the source tree (target repo + repos from `.rosinstall`) - * Only when tags are pushed: - ** Run some cleanup in the `industrial_ci` context (e.g. delete sources) - ** Commit image used by `industrial_ci` - ** Build deployment image on top of it and push it to our registry -- Stage: `.post` (jobs in this stage are allowed to fail) - * Check `bash`/`sh` script syntax - * Check Markdown syntax - * Check files have no trailing whitespace and are UTF-8 encoded - -It is important to note that both images set the variable `GIT_LFS_SKIP_SMUDGE`, which means that when the repositories are cloned/fetched, the files hosted using Git LFS will not be downloaded. An example on how to explicitely fetch the files from LFS can be found in the https://git.code.tecnalia.com/tecnalia_robotics/common_config/blob/e93dbe6379d94e6a895a33c59bf17ccab26eacf9/.gitlab-ci.yml#L132[common_config repository]. +include::ci-templates/README.adoc[leveloffset=+1] == Custom `rosdep` definitions @@ -60,5 +15,5 @@ Custom `rosdep` definitions for the packages hosted in our internal debian repos Resources required for building the deployment docker images are also hosted in this repository. This basically boils down to: -- link:scripts/pre-deploy-cleanup.bash[] is run in the `industrial_ci` container before commiting to an image +- link:scripts/pre-deploy-cleanup.bash[] is run in the `.industrial_ci_deploy` template before commiting to an image - link:deploy/Dockerfile[] is used to perform some additional cleanup and install link:deploy/ros_entrypoint.bash[] as entrypoint script on top of the `industrial_ci` image diff --git a/ci-templates/README.adoc b/ci-templates/README.adoc new file mode 100644 index 0000000000000000000000000000000000000000..d4f1a0dc2ad0e28e0dcdb1ef45ca9819866f7c7f --- /dev/null +++ b/ci-templates/README.adoc @@ -0,0 +1,54 @@ + += GitLab CI templates + +Templates for GitLab Continuous Integration are provided in the folder link:ci-templates[]. These can be included from any repo by using `include:project`: + +```yml +include::examples/simple_include.yml[] +``` + +== Core Pipeline +The Core pipeline, defined in link:ci-templates/core.yml[], has the following structure: + +- Stage: `build` + * Check python syntax + * `.industrial_ci` template (see https://github.com/ros-industrial/industrial_ci[the industrial_ci repo]), which can be used to define jobs that build `ROS` packages + ** Needs the `ROS_DISTRO` variable to be defined + ** If the package has `.rosinstall` dependencies, they can be installed using the variable `UPSTREAM_WORKSPACE` + ** Uses `flexbotics-base-devel:${ROS_DISTRO}` as a starting image + ** Runs link:scripts/ci_run_entry_points.sh[] in the `AFTER_SETUP_TARGET_WORKSPACE` stage, which in turn finds and runs every `ci_entry_script.bash` script found in the target repo. + * `.industrial_ci_deploy` template, which: + ** Runs `industrial_ci` by extending `.industrial_ci` + ** Runs some cleanup in the `industrial_ci` context (e.g. delete sources) + ** Commits image used by `industrial_ci` + ** Builds deployment image on top of it and push it to our registry +- Stage: `.post` + * Check `bash`/`sh` script syntax + * Check Markdown and AsciiDoc syntax + * Check files have no trailing whitespace and are `UTF-8` encoded + * Check that `C++` files follow this link:clang-format[] + +It is important to note that both templates set the variable `GIT_LFS_SKIP_SMUDGE`, which means that when the repositories are cloned/fetched, the files hosted using Git LFS will not be downloaded. An example on how to explicitely fetch the files from LFS can be found in the https://git.code.tecnalia.com/tecnalia_robotics/common_config/blob/e93dbe6379d94e6a895a33c59bf17ccab26eacf9/.gitlab-ci.yml#L132[common_config repository]. + +== Manual Jobs + +Using the templates defined by the core pipeline, one can easily define jobs to build the packages in the repo. + +```yml +include::examples/manual_job.yml[] +``` + +More examples can be found in link:ci-templates/examples[]. + +== Auto Rules +In the link:ci-templates/auto-rules[] folder, automatic pipelines are defined that will try to detect the ROS distro from the branch name and create jobs. +For example, if the branch starts with `noetic-` it will build for noetic. +If the branch doesn't start with any ROS distro, it will build the default distro, which can be specified with the variable `DEFAULT_DISTRO`. +For convenience, some templates are provided that already set this variable. +If your repo needs to uses `.rosinstall`, you can use the templates in the link:ci-templates/auto-rules/with-rosinstall[] folder. + +The auto rules also allow for using variables to explicitely activate each job: + +```yml +include::examples/build_vars.yml[] +``` diff --git a/ci-templates/examples/build_vars.yml b/ci-templates/examples/build_vars.yml new file mode 100644 index 0000000000000000000000000000000000000000..276246b6e078206fcba7bfe2e9ebc87924b63d85 --- /dev/null +++ b/ci-templates/examples/build_vars.yml @@ -0,0 +1,8 @@ +include: + - project: tecnalia_robotics-public/gitlab_templates + file: ci-templates/auto-rules/no-default.yml + +variables: + BUILD_KINETIC: 1 + BUILD_MELODIC: 1 + BUILD_NOETIC: 1 diff --git a/ci-templates/examples/manual_job.yml b/ci-templates/examples/manual_job.yml new file mode 100644 index 0000000000000000000000000000000000000000..8cf4e1d377becdd725bf22ed91cbf05638fc0a74 --- /dev/null +++ b/ci-templates/examples/manual_job.yml @@ -0,0 +1,9 @@ +include: + - project: tecnalia_robotics-public/gitlab_templates + file: ci-templates/core.yml + +industrial_ci_kinetic: + extends: .industrial-ci + variables: + ROS_DISTRO: kinetic + UPSTREAM_WORKSPACE: .rosinstall \ No newline at end of file diff --git a/ci-templates/examples/simple_include.yml b/ci-templates/examples/simple_include.yml new file mode 100644 index 0000000000000000000000000000000000000000..3b1a85dd85aa9387ef7b436e45e221b8928f6c25 --- /dev/null +++ b/ci-templates/examples/simple_include.yml @@ -0,0 +1,3 @@ +include: + - project: tecnalia_robotics-public/gitlab_templates + file: ci-templates/core.yml \ No newline at end of file