Skip to content
Snippets Groups Projects
Commit 717af381 authored by Iñigo Moreno i Caireta's avatar Iñigo Moreno i Caireta
Browse files

Resolve "Update deprecated deploy job"

parent a60c7c08
Branches
No related tags found
1 merge request!70Resolve "Update deprecated deploy job"
shellcheck: shellcheck:
image: tecnalia-docker-dev.artifact.tecnalia.com/koalaman/shellcheck-alpine image: tecnalia-docker-dev.artifact.tecnalia.com/koalaman/shellcheck-alpine
script: shellcheck deploy/base/ros_entrypoint.bash scripts/*sh script: shellcheck scripts/*sh
tags: tags:
- docker - docker
# The ignored rules in hadolint checks are
# DL3006 Always tag the version of an image explicitly
# One of the Dockerfiles uses a temporary image which isn't important to tag
# and the other uses a tagged image through an ARG, which apparently is not
# properly recognized.
# DL3008 Pin versions in apt-get install
# A bit overkill
dockerfiles:
image: tecnalia-docker-dev.artifact.tecnalia.com/hadolint/hadolint:latest-debian
script:
- RET=0
- hadolint --ignore DL3006 --ignore DL3008 --ignore DL3059 deploy/base/Dockerfile || RET=1
- hadolint --ignore DL3006 --ignore DL3008 --ignore DL3059 deploy/dev/Dockerfile || RET=1
- exit $RET
tags:
- docker
yamllint: yamllint:
image: tecnalia-docker-dev.artifact.tecnalia.com/sdesbure/yamllint image: tecnalia-docker-dev.artifact.tecnalia.com/sdesbure/yamllint
......
...@@ -9,11 +9,3 @@ include::ci-templates/README.adoc[leveloffset=+1] ...@@ -9,11 +9,3 @@ include::ci-templates/README.adoc[leveloffset=+1]
== Custom `rosdep` definitions == Custom `rosdep` definitions
Custom `rosdep` definitions for the packages hosted in our internal debian repository are declared in link:rosdistro/rosdep_tecnalia.yaml[]. Custom `rosdep` definitions for the packages hosted in our internal debian repository are declared in link:rosdistro/rosdep_tecnalia.yaml[].
== Deployment image build resources
Resources required for building the deployment docker images are also hosted in this repository.
This basically boils down to:
- link:scripts/pre-deploy-cleanup.bash[] is run in the `.industrial_ci_deploy` template before commiting to an image
- link:deploy/Dockerfile[] is used to perform some additional cleanup and install link:deploy/ros_entrypoint.bash[] as entrypoint script on top of the `industrial_ci` image
...@@ -17,11 +17,14 @@ The Core pipeline, defined in link:ci-templates/core.yml[], has the following st ...@@ -17,11 +17,14 @@ The Core pipeline, defined in link:ci-templates/core.yml[], has the following st
** If the package has `.rosinstall` dependencies, they can be installed using the variable `UPSTREAM_WORKSPACE` ** If the package has `.rosinstall` dependencies, they can be installed using the variable `UPSTREAM_WORKSPACE`
** Uses `flexbotics-base-devel:${ROS_DISTRO}` as a starting image ** Uses `flexbotics-base-devel:${ROS_DISTRO}` as a starting image
** Runs link:scripts/ci_run_entry_points.sh[] in the `AFTER_SETUP_TARGET_WORKSPACE` stage, which in turn finds and runs every `ci_entry_script.bash` script found in the target repo. ** Runs link:scripts/ci_run_entry_points.sh[] in the `AFTER_SETUP_TARGET_WORKSPACE` stage, which in turn finds and runs every `ci_entry_script.bash` script found in the target repo.
* `.industrial_ci_deploy` template, which: - Stage: `deploy`
** Runs `industrial_ci` by extending `.industrial_ci` * `.ddeploy` template (see https://git.code.tecnalia.com/tecnalia_robotics/flexbotics/flexbotics_utils/ddeploy[the ddeploy repo]), which:
** Runs some cleanup in the `industrial_ci` context (e.g. delete sources) ** Is run automatically for tags, but can also be run manually on normal commits
** Commits image used by `industrial_ci` ** Calls `ddeploy` to generate the docker image
** Builds deployment image on top of it and push it to our registry ** Renames the image created by ddeploy with different tags and pushes them to the registry.
*** Tag `${COMMIT_SHA}` (hash of commit)
*** Tag `${CI_COMMIT_REF_SLUG}` (branch or tag name)
*** Tag `latest` only on the default branch
- Stage: `.post` - Stage: `.post`
* Check `bash`/`sh` script syntax * Check `bash`/`sh` script syntax
* Check Markdown and AsciiDoc syntax * Check Markdown and AsciiDoc syntax
...@@ -52,6 +55,8 @@ The auto rules also allow for using variables to explicitely activate each job: ...@@ -52,6 +55,8 @@ The auto rules also allow for using variables to explicitely activate each job:
include::examples/build_vars.yml[] include::examples/build_vars.yml[]
``` ```
If a `ddeploy.yaml` file exists, the auto-rules will add a job extending the `.ddeploy` template defined
== LFS == LFS
By default, lfs files are not downloaded. If you need to add LFS, include the link:ci-templates/lfs-pull.yml[] along with anything else: By default, lfs files are not downloaded. If you need to add LFS, include the link:ci-templates/lfs-pull.yml[] along with anything else:
```yml ```yml
......
...@@ -2,40 +2,23 @@ include: ci-templates/core.yml ...@@ -2,40 +2,23 @@ include: ci-templates/core.yml
workflow: workflow:
rules: rules:
- if: $CI_COMMIT_BRANCH && $CI_COMMIT_REF_NAME =~ /^kinetic-.*/ - if: $CI_COMMIT_REF_NAME =~ /^kinetic-.*/
variables: variables:
DEFAULT_DISTRO: "kinetic" DEFAULT_DISTRO: "kinetic"
- if: $CI_COMMIT_BRANCH && $CI_COMMIT_REF_NAME =~ /^melodic-.*/ - if: $CI_COMMIT_REF_NAME =~ /^melodic-.*/
variables: variables:
DEFAULT_DISTRO: "melodic" DEFAULT_DISTRO: "melodic"
- if: $CI_COMMIT_BRANCH && $CI_COMMIT_REF_NAME =~ /^noetic-.*/ - if: $CI_COMMIT_REF_NAME =~ /^noetic-.*/
variables: variables:
DEFAULT_DISTRO: "noetic" DEFAULT_DISTRO: "noetic"
- if: $CI_COMMIT_TAG && $CI_COMMIT_REF_NAME =~ /^kinetic-.*/
variables:
DEFAULT_DISTRO: "kinetic"
DEPLOY: "true"
- if: $CI_COMMIT_TAG && $CI_COMMIT_REF_NAME =~ /^melodic-.*/
variables:
DEFAULT_DISTRO: "melodic"
DEPLOY: "true"
- if: $CI_COMMIT_TAG && $CI_COMMIT_REF_NAME =~ /^noetic-.*/
variables:
DEFAULT_DISTRO: "noetic"
DEPLOY: "true"
- if: $CI_COMMIT_TAG - if: $CI_COMMIT_TAG
variables:
DEPLOY: "true"
- if: $CI_COMMIT_BRANCH - if: $CI_COMMIT_BRANCH
industrial_ci_kinetic: industrial_ci_kinetic:
extends: .industrial_ci extends: .industrial_ci
variables: variables:
ROS_DISTRO: kinetic ROS_DISTRO: kinetic
rules: rules:
- if: $DEPLOY
when: never
- if: $DEFAULT_DISTRO == "kinetic" - if: $DEFAULT_DISTRO == "kinetic"
- if: $BUILD_KINETIC - if: $BUILD_KINETIC
...@@ -44,8 +27,6 @@ industrial_ci_melodic: ...@@ -44,8 +27,6 @@ industrial_ci_melodic:
variables: variables:
ROS_DISTRO: melodic ROS_DISTRO: melodic
rules: rules:
- if: $DEPLOY
when: never
- if: $DEFAULT_DISTRO == "melodic" - if: $DEFAULT_DISTRO == "melodic"
- if: $BUILD_MELODIC - if: $BUILD_MELODIC
...@@ -54,32 +35,17 @@ industrial_ci_noetic: ...@@ -54,32 +35,17 @@ industrial_ci_noetic:
variables: variables:
ROS_DISTRO: noetic ROS_DISTRO: noetic
rules: rules:
- if: $DEPLOY
when: never
- if: $DEFAULT_DISTRO == "noetic" - if: $DEFAULT_DISTRO == "noetic"
- if: $BUILD_NOETIC - if: $BUILD_NOETIC
ddeploy:
industrial_kinetic_deploy: extends: .ddeploy
extends:
- industrial_ci_kinetic
- .industrial_ci_deploy
rules:
- if: $DEPLOY && $DEFAULT_DISTRO == "kinetic"
- if: $DEPLOY && $BUILD_KINETIC
industrial_melodic_deploy:
extends:
- industrial_ci_melodic
- .industrial_ci_deploy
rules: rules:
- if: $DEPLOY && $DEFAULT_DISTRO == "melodic" - if: $CI_COMMIT_TAG
- if: $DEPLOY && $BUILD_MELODIC exists:
- ddeploy.yaml
industrial_noetic_deploy: when: always
extends: - exists:
- industrial_ci_noetic - ddeploy.yaml
- .industrial_ci_deploy when: manual
rules: allow_failure: true
- if: $DEPLOY && $DEFAULT_DISTRO == "noetic"
- if: $DEPLOY && $BUILD_NOETIC
...@@ -16,6 +16,7 @@ services: ...@@ -16,6 +16,7 @@ services:
stages: stages:
- build - build
- deploy
before_script: before_script:
- apk add --update bash coreutils tar wget - apk add --update bash coreutils tar wget
......
...@@ -14,24 +14,51 @@ ...@@ -14,24 +14,51 @@
CMAKE_ARGS: -DCMAKE_CXX_FLAGS=-Wno-ignored-attributes -Wno-int-in-bool-context CMAKE_ARGS: -DCMAKE_CXX_FLAGS=-Wno-ignored-attributes -Wno-int-in-bool-context
.industrial_ci_deploy: .ddeploy:
extends: .industrial_ci stage: deploy
variables: variables:
AFTER_SCRIPT: "wget -q -O - https://git.code.tecnalia.com/tecnalia_robotics-public/gitlab_templates/raw/master/scripts/pre-deploy-cleanup.bash | bash" DOCKER_PUSH_REGISTRY: ${ARTIFACT_DOCKER_URL}
DOCKER_COMMIT: industrial_ci_image DOCKER_PUSH_NAME: ${CI_PROJECT_PATH}
DOCKER_PUSH_TAG: ${CI_COMMIT_REF_SLUG}
DDEPLOY_YAML: ddeploy.yaml
before_script:
- apk add --update python3 git py3-pip git-lfs
# forward the SSH authentication into the Docker executor
- "which ssh-agent || ( apk update && apk add openssh-client )"
- eval $(ssh-agent -s)
- echo "$SSH_PRIVATE_KEY" | tr -d '\r' | ssh-add - > /dev/null
- mkdir -p ~/.ssh
- echo "$SSH_SERVER_HOSTKEYS" > ~/.ssh/known_hosts
- chmod 700 ~/.ssh
# setup LFS authentication
- git lfs install
- git config --global credential.helper store
- echo https://$ARTIFACT_CI_USER:$ARTIFACT_CI_TOKEN@artifact.tecnalia.com > ~/.git-credentials
# install ddeploy
- pip install git+ssh://git@git.code.tecnalia.com/tecnalia_robotics/flexbotics/flexbotics_utils/ddeploy.git@v0.1.0
- docker login ${ARTIFACT_DOCKER_URL} -u ${ARTIFACT_CI_USER} -p ${ARTIFACT_CI_TOKEN}
script: script:
- .ci_config/gitlab.sh # Run ddeploy
- cd $(mktemp -d) - ddeploy --yaml ${DDEPLOY_YAML}
- wget https://git.code.tecnalia.com/tecnalia_robotics-public/gitlab_templates/-/archive/master/gitlab_templates-master.zip # Get ID of image created by ddeploy
- unzip gitlab_templates-master.zip && cd gitlab_templates-master/deploy - 'DOCKER_ID=$(docker images --format="{{.ID}}" | head -1)'
- cd base # Tag and push with the branch or tag name.
- chmod a+x ros_entrypoint.bash - echo "Pushing to ${DOCKER_PUSH_REGISTRY}/${DOCKER_PUSH_NAME}:${DOCKER_PUSH_TAG}"
- docker build -t ${ARTIFACT_DOCKER_URL}/${CI_PROJECT_NAME}:${CI_COMMIT_REF_NAME} . - docker tag ${DOCKER_ID} ${DOCKER_PUSH_REGISTRY}/${DOCKER_PUSH_NAME}:${DOCKER_PUSH_TAG}
- docker push ${ARTIFACT_DOCKER_URL}/${CI_PROJECT_NAME}:${CI_COMMIT_REF_NAME} - docker push ${DOCKER_PUSH_REGISTRY}/${DOCKER_PUSH_NAME}:${DOCKER_PUSH_TAG}
- cd .. # Tag and push with commit sha
- cd dev - echo "Pushing to ${DOCKER_PUSH_REGISTRY}/${DOCKER_PUSH_NAME}:${CI_COMMIT_SHA}"
- docker build --build-arg APPLICATION_IMAGE=${ARTIFACT_DOCKER_URL}/${CI_PROJECT_NAME}:${CI_COMMIT_REF_NAME} -t ${ARTIFACT_DOCKER_URL}/${CI_PROJECT_NAME}:${CI_COMMIT_REF_NAME}-dev . - docker tag ${DOCKER_ID} ${DOCKER_PUSH_REGISTRY}/${DOCKER_PUSH_NAME}:${CI_COMMIT_SHA}
- docker push ${ARTIFACT_DOCKER_URL}/${CI_PROJECT_NAME}:${CI_COMMIT_REF_NAME}-dev - docker push ${DOCKER_PUSH_REGISTRY}/${DOCKER_PUSH_NAME}:${CI_COMMIT_SHA}
- cd .. # Tag and push with latest only if it's on the default branch
- >
if [ "$CI_COMMIT_BRANCH" == "$CI_DEFAULT_BRANCH" ]; then
echo "Pushing to ${DOCKER_PUSH_REGISTRY}/${DOCKER_PUSH_NAME}:latest"
docker tag ${DOCKER_ID} ${DOCKER_PUSH_REGISTRY}/${DOCKER_PUSH_NAME}:latest &&
docker push ${DOCKER_PUSH_REGISTRY}/${DOCKER_PUSH_NAME}:latest;
fi
rules: rules:
- if: $DEPLOY - if: $CI_COMMIT_TAG
when: always
- when: manual
allow_failure: true
FROM industrial_ci_image
RUN rm -rf /root/ici /root/src
RUN apt-get update -qq && apt-get install -y -qq --no-install-recommends \
ros-kinetic-rosbash \
&& rm -rf /var/lib/apt/lists/*
COPY ./ros_entrypoint.bash /
ENTRYPOINT ["/ros_entrypoint.bash"]
CMD ["bash"]
#!/bin/bash
set -e
# setup application environment
# shellcheck disable=SC1091
source "/root/target_ws/install/setup.bash"
exec "$@"
ARG APPLICATION_IMAGE
FROM ${APPLICATION_IMAGE}
RUN apt-get update -qq && apt-get install -y -qq --no-install-recommends \
apt-transport-https \
curl \
libasound2 \
ca-certificates \
&& rm -rf /var/lib/apt/lists/*
SHELL ["/bin/bash", "-o", "pipefail", "-c"]
RUN curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > microsoft.gpg \
&& mv microsoft.gpg /etc/apt/trusted.gpg.d/microsoft.gpg \
&& sh -c 'echo "deb [arch=amd64] https://packages.microsoft.com/repos/vscode stable main" > /etc/apt/sources.list.d/vscode.list'
RUN apt-get update -qq && apt-get install -y -qq --no-install-recommends \
code \
&& rm -rf /var/lib/apt/lists/*
RUN echo "alias code='code --user-data-dir /root'" >> /root/.bashrc
RUN code --user-data-dir /root \
--install-extension ms-iot.vscode-ros \
--install-extension joaompinto.asciidoctor-vscode \
--install-extension ms-python.python \
--install-extension ms-vscode.cpptools \
--install-extension ms-azuretools.vscode-docker \
--install-extension twxs.cmake
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment