Merge branch 'feat/official-vars' into 'master'

Multiple improvements Closes #16 and #15 See merge request to-be-continuous/sonar!24

Merge branch 'feat/official-vars' into 'master'
6cd3ce94 · Pierre Smeyers · ca86d2d7 · 1c4c9ccf · 6cd3ce94 · 6cd3ce94
Commit 6cd3ce94 authored 2 years ago by Pierre Smeyers
--- a/README.md
+++ b/README.md
@@ -25,43 +25,32 @@ It is bound to the `test` stage, and uses the following variables:
 | Name                     | description                     | default value |
 | ------------------------ | ------------------------------- | ----------------------------- |
 | `SONAR_SCANNER_IMAGE`    | The Docker image used to run [sonar-scanner](https://docs.sonarqube.org/latest/analysis/scan/sonarscanner/) | `sonarsource/sonar-scanner-cli:latest` |
-| `SONAR_URL`              | SonarQube server url            | _none_ (disabled) |
+| `SONAR_HOST_URL`         | SonarQube server url            | _none_ (disabled) |
-| :lock: `SONAR_AUTH_TOKEN`| SonarQube authentication [token](https://docs.sonarqube.org/latest/user-guide/user-token/) (depends on your authentication method) | _none_ |
+| `SONAR_PROJECT_KEY`      | SonarQube Project Key (might also be set in the `sonar-project.properties` file) | fallbacks to `$CI_PROJECT_PATH_SLUG` (see below) |
+| `SONAR_PROJECT_NAME`     | SonarQube Project Name (might also be set in the `sonar-project.properties` file) | fallbacks to `$CI_PROJECT_PATH` (see below) |
+| :lock: `SONAR_TOKEN`     | SonarQube authentication [token](https://docs.sonarqube.org/latest/user-guide/user-token/) (depends on your authentication method) | _none_ |
 | :lock: `SONAR_LOGIN`     | SonarQube login (depends on your authentication method)                | _none_ |
 | :lock: `SONAR_PASSWORD`  | SonarQube password (depends on your authentication method)             | _none_ |
-| `SONAR_BASE_ARGS`        | SonarQube [analysis arguments](https://docs.sonarqube.org/latest/analysis/analysis-parameters/) | `-Dsonar.host.url=${SONAR_URL} -Dsonar.projectKey=${CI_PROJECT_PATH_SLUG} -Dsonar.projectName=${CI_PROJECT_PATH} -Dsonar.projectBaseDir=. -Dsonar.links.homepage=${CI_PROJECT_URL} -Dsonar.links.ci=${CI_PROJECT_URL}/-/pipelines -Dsonar.links.issue=${CI_PROJECT_URL}/-/issues` |
+| `SONAR_BASE_ARGS`        | SonarQube [analysis arguments](https://docs.sonarqube.org/latest/analysis/analysis-parameters/) | `-Dsonar.links.homepage=${CI_PROJECT_URL} -Dsonar.links.ci=${CI_PROJECT_URL}/-/pipelines -Dsonar.links.issue=${CI_PROJECT_URL}/-/issues` |
-| :lock: `SONAR_GITLAB_TOKEN` | GitLab [access token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html) with `api` scope. When set, activates the [Sonar GitLab plugin](https://github.com/gabrie-allaigre/sonar-gitlab-plugin/#plugins-properties) integration. | _none_ |
+| `SONAR_QUALITY_GATE_ENABLED` | Set to `true` to enable SonarQube [Quality Gate](https://docs.sonarqube.org/latest/user-guide/quality-gates/) verification.<br/>_Uses `sonar.qualitygate.wait` parameter ([see doc](https://docs.sonarqube.org/latest/analysis/ci-integration-overview/#header-1))._ | _none_ (disabled) |
-| `SONAR_BRANCH_ANALYSIS_DISABLED` | Set to `true` to disable automatic [Pull Request Analysis](https://docs.sonarqube.org/latest/analysis/pull-request/) and [Branch Analysis](https://docs.sonarqube.org/latest/branches/overview/)  | _none_ (enabled) |
-| `SONAR_GITLAB_ARGS`      | Extra arguments to use with [Sonar GitLab plugin](https://github.com/gabrie-allaigre/sonar-gitlab-plugin/#plugins-properties) | `-Dsonar.gitlab.url=${CI_SERVER_URL} -Dsonar.gitlab.user_token=${SONAR_GITLAB_TOKEN} -Dsonar.gitlab.project_id=${CI_PROJECT_ID} -Dsonar.gitlab.commit_sha=${CI_COMMIT_SHA} -Dsonar.gitlab.ref_name=${CI_COMMIT_REF_NAME}` |
-#### Automatic Branch Analysis & Pull Request Analysis
+### Automatic Branch Analysis & Merge Request Analysis
-By default, this template tries to auto-detect and use [Pull Request Analysis](https://docs.sonarqube.org/latest/analysis/pull-request/) or [Branch Analysis](https://docs.sonarqube.org/latest/branches/overview/) (depending on the context).
+This template relies on SonarScanner's [GitLab integration](https://docs.sonarqube.org/latest/analysis/gitlab-integration), that is able to auto-detect whether to launch Branch Analysis or Merge Request Analysis
+from GitLab's environment variables.
-Those is a great SonarQube features but it assumes one of the following conditions:
+:warning: This feature also depends on your SonarQube server version and license.
+If using Community Edition, you'll have to install the [sonarqube-community-branch-plugin](https://github.com/mc1arke/sonarqube-community-branch-plugin) to enable automatic Branch & Merge Request analysis (only works from SonarQube version 8).
-* you are using a [Developer Edition](https://www.sonarqube.org/developer-edition/) version,
+:warning: Merge Request Analysis only works if you're running [Merge Request pipeline](https://docs.gitlab.com/ee/ci/yaml/workflow.html#switch-between-branch-pipelines-and-merge-request-pipelines) strategy (default).
-* or you are using Community Edition with an opensource plugin emulating those features, such as [sonarqube-community-branch-plugin](https://github.com/mc1arke/sonarqube-community-branch-plugin).
-If you're not in one of those cases, then you shall disable this feature by setting `SONAR_BRANCH_ANALYSIS_DISABLED`.
+### Configuring SonarQube project key, project name and other parameters
-If you leave the feature enabled, if `SONAR_GITLAB_TOKEN` is provided, the template will try to autodetect (using GitLab APIs) an opened merge request matching the current branch:
+You shall define your SonarQube project key and project name in a `sonar-project.properties` file located at the root of your repository (as respectively `sonar.projectKey` and `sonar.projectName` entries), although they might alternately be set as `$SONAR_PROJECT_KEY` and `$SONAR_PROJECT_NAME` variables.
-* If one is found, a SonarQube [Pull Request Analysis](https://docs.sonarqube.org/latest/analysis/pull-request/) will be made.
+Note that when not explictly set, the template will use `$CI_PROJECT_PATH_SLUG` and `$CI_PROJECT_PATH` as fallback project key and project name.
-* Otherwise, a simple [Branch Analysis](https://docs.sonarqube.org/latest/branches/overview/) is performed on the current branch.
-### About Sonar GitLab plugin
+The `sonar-project.properties` file is also the recommended way to configure other SonarQube [analysis parameters](https://docs.sonarqube.org/latest/analysis/analysis-parameters/)
+as well as [language specific parameters](https://docs.sonarqube.org/latest/analysis/languages/overview/).
-The [Sonar GitLab plugin](https://github.com/gabrie-allaigre/sonar-gitlab-plugin) uses the GitLab APIs to inline comments
+Each _to-be-continuous_ build template shall briefly document the supported language-specific SonarQube parameters.
-into your commits directly in GitLab for each new anomaly.
\ No newline at end of file
-As explained above, this template automatically enables the Sonar GitLab plugin if `SONAR_GITLAB_TOKEN` is set.
-It will then simply append the `SONAR_GITLAB_ARGS` (overridable) to the SonarQube analysis arguments.
-Comments added to GitLab will appear as owned by the user associated to the GitLab [access token](https://docs.gitlab.com/ee/user/profile/personal_access_tokens.html).
-### How should I configure other SonarQube arguments ?
-The simplest and recommended way to configure other SonarQube [analysis arguments](https://docs.sonarqube.org/latest/analysis/analysis-parameters/)
-(and even [language specific args](https://docs.sonarqube.org/latest/analysis/languages/overview/)) is to use a
-`sonar-project.properties` file at the root of your repository.
--- a/kicker.json
+++ b/kicker.json
@@ -10,13 +10,23 @@
      "default": "sonarsource/sonar-scanner-cli:latest"
    },
    {
-      "name": "SONAR_URL",
+      "name": "SONAR_HOST_URL",
      "type": "url",
      "description": "SonarQube server url",
      "mandatory": true
    },
    {
-      "name": "SONAR_AUTH_TOKEN",
+      "name": "SONAR_PROJECT_KEY",
+      "description": "SonarQube Project Key (might also be set in the `sonar-project.properties` file)",
+      "advanced": true
+    },
+    {
+      "name": "SONAR_PROJECT_NAME",
+      "description": "SonarQube Project Name (might also be set in the `sonar-project.properties` file)",
+      "advanced": true
+    },    
+    {
+      "name": "SONAR_TOKEN",
      "description": "SonarQube authentication [token](https://docs.sonarqube.org/latest/user-guide/user-token/) (depends on your authentication method)",
      "secret": true
    },
@@ -33,24 +43,13 @@
    {
      "name": "SONAR_BASE_ARGS",
      "description": "SonarQube [analysis arguments](https://docs.sonarqube.org/latest/analysis/analysis-parameters/)",
-      "default": "-Dsonar.host.url=${SONAR_URL} -Dsonar.projectKey=${CI_PROJECT_PATH_SLUG} -Dsonar.projectName=${CI_PROJECT_PATH} -Dsonar.projectBaseDir=. -Dsonar.links.homepage=${CI_PROJECT_URL} -Dsonar.links.ci=${CI_PROJECT_URL}/-/pipelines -Dsonar.links.issue=${CI_PROJECT_URL}/-/issues",
+      "default": "-Dsonar.links.homepage=${CI_PROJECT_URL} -Dsonar.links.ci=${CI_PROJECT_URL}/-/pipelines -Dsonar.links.issue=${CI_PROJECT_URL}/-/issues",
      "advanced": true
    },
    {
-      "name": "SONAR_GITLAB_TOKEN",
+      "name": "SONAR_QUALITY_GATE_ENABLED",
-      "description": "GitLab API access token. When set, activates the [Sonar GitLab plugin](https://github.com/gabrie-allaigre/sonar-gitlab-plugin/#plugins-properties) integration, and enables SonarQube [Pull Request Analysis](https://docs.sonarqube.org/latest/analysis/pull-request/)",
+      "description": "Enables SonarQube [Quality Gate](https://docs.sonarqube.org/latest/user-guide/quality-gates/) verification.\n\n_Uses `sonar.qualitygate.wait` parameter ([see doc](https://docs.sonarqube.org/latest/analysis/ci-integration-overview/#header-1))._",
-      "secret": true
-    },
-    {
-      "name": "SONAR_BRANCH_ANALYSIS_DISABLED",
-      "description": "Set to disable automatic [Pull Request Analysis](https://docs.sonarqube.org/latest/analysis/pull-request/) and [Branch Analysis](https://docs.sonarqube.org/latest/branches/overview/)",
      "type": "boolean"
-    },
-    {
-      "name": "SONAR_GITLAB_ARGS",
-      "description": "Extra arguments to use with [Sonar GitLab plugin](https://github.com/gabrie-allaigre/sonar-gitlab-plugin/#plugins-properties)",
-      "default": "-Dsonar.gitlab.url=${CI_SERVER_URL} -Dsonar.gitlab.user_token=${SONAR_GITLAB_TOKEN} -Dsonar.gitlab.project_id=${CI_PROJECT_ID} -Dsonar.gitlab.commit_sha=${CI_COMMIT_SHA} -Dsonar.gitlab.ref_name=${CI_COMMIT_REF_NAME}",
-      "advanced": true
    }
  ]
 }
--- a/templates/gitlab-ci-sonar.yml
+++ b/templates/gitlab-ci-sonar.yml
@@ -51,24 +51,10 @@ variables:
  # see: https://docs.sonarqube.org/latest/analysis/analysis-parameters/
  # default uses branch analysis: https://docs.sonarqube.org/latest/branches/overview/
  SONAR_BASE_ARGS: >-
-    -Dsonar.host.url=${SONAR_URL}
-    -Dsonar.projectKey=${CI_PROJECT_PATH_SLUG}
-    -Dsonar.projectName=${CI_PROJECT_PATH}
-    -Dsonar.projectBaseDir=.
    -Dsonar.links.homepage=${CI_PROJECT_URL}
    -Dsonar.links.ci=${CI_PROJECT_URL}/-/pipelines
    -Dsonar.links.issue=${CI_PROJECT_URL}/-/issues
-  # Sonar GitLab plugin args
-  # see: https://github.com/gabrie-allaigre/sonar-gitlab-plugin/#plugins-properties
-  # -> enabled if $SONAR_GITLAB_TOKEN is set
-  SONAR_GITLAB_ARGS: >-
-    -Dsonar.gitlab.url=${CI_SERVER_URL}
-    -Dsonar.gitlab.user_token=${SONAR_GITLAB_TOKEN}
-    -Dsonar.gitlab.project_id=${CI_PROJECT_ID}
-    -Dsonar.gitlab.commit_sha=${CI_COMMIT_SHA}
-    -Dsonar.gitlab.ref_name=${CI_COMMIT_REF_NAME}
  # default production ref name (pattern)
  PROD_REF: '/^(master|main)$/'
  # default integration ref name (pattern)
@@ -148,39 +134,6 @@ stages:
    fi
  }
-  function sonar_autodetect_mr() {
-    if [[ "$SONAR_BRANCH_ANALYSIS_DISABLED" == "true" ]]
-    then
-      log_info "Branch Analysis and Merge Request Analysis are disabled"
-      return
-    fi
-    if [[ "$CI_MERGE_REQUEST_ID" ]]
-    then
-      # we are in an MR pipeline: no need to pass arguments as SonarScanner will
-      log_info "Merge Request pipeline detected: let SonarScanner handle..."
-      return
-    fi
-    if [[ -n "$SONAR_GITLAB_TOKEN" ]]
-    then 
-      wget -q "$CI_API_V4_URL/projects/${CI_PROJECT_ID}/merge_requests?state=opened&source_branch=${CI_COMMIT_REF_NAME}&private_token=$SONAR_GITLAB_TOKEN" -O mr.json || log_warn "Failed requesting GitLab API: check \$SONAR_GITLAB_TOKEN"
-      if [[ -f mr.json ]] && [[ "$(cat mr.json)" != "[]" ]]
-      then
-        mr_title=$(sed -E 's/\[\{[^{]*"title":"([^"]*)".*/\1/g' < mr.json)
-        mr_target=$(sed -E 's/\[\{[^{]*"target_branch":"([^"]*)".*/\1/g' < mr.json)
-        mr_id=$(sed -E 's/\[\{[^{]*"iid":([0-9]+).*/\1/g' < mr.json)
-        log_info "Merge Request \\e[33;1m$mr_title\\e[0m detected associated to this branch: trigger MR analysis..."
-        export sonar_mr_args="-Dsonar.pullrequest.key=$mr_id -Dsonar.pullrequest.branch=${CI_COMMIT_REF_NAME} -Dsonar.pullrequest.base=$mr_target"
-      else 
-        log_info "No Merge Request associated to this branch: trigger branch analysis..."
-        export sonar_mr_args="-Dsonar.branch.name=${CI_COMMIT_REF_NAME}"
-      fi
-    else 
-      log_info "\$SONAR_GITLAB_TOKEN unset: trigger branch analysis..."
-      export sonar_mr_args="-Dsonar.branch.name=${CI_COMMIT_REF_NAME}"
-    fi
-  }
  function unscope_variables() {
    _scoped_vars=$(env | awk -F '=' "/^scoped__[a-zA-Z0-9_]+=/ {print \$1}" | sort)
    if [[ -z "$_scoped_vars" ]]; then return; fi
@@ -290,6 +243,13 @@ stages:
    fi
  }
+  # determines whether the given SonarQube param is defined 
+  # either in the sonar-project.properties or in the $SONAR_BASE_ARGS variable
+  function has_sonar_param() {
+    sonar_param="$1"
+    grep -e "^sonar\.${sonar_param}[ \t]*[:=]" sonar-project.properties > /dev/null || echo "$SONAR_BASE_ARGS" | grep -e "-Dsonar\.${sonar_param}=" > /dev/null
+  }
  unscope_variables
  # ENDSCRIPT
@@ -304,20 +264,52 @@ sonar:
    - name: "$TBC_TRACKING_IMAGE"
      command: ["--service", "sonar", "3.0.0" ]
  variables:
-    # Tells git to fetch all the branches of the project, required by the analysis task
+    # see: https://docs.sonarqube.org/latest/analysis/gitlab-integration/#header-4
-    # see: https://docs.sonarqube.org/8.5/analysis/gitlab-cicd/#header-2
+    SONAR_USER_HOME: "${CI_PROJECT_DIR}/.sonar" # Defines the location of the analysis task cache
-    GIT_DEPTH: 0
+    GIT_DEPTH: 0 # Tells git to fetch all the branches of the project, required by the analysis task
+  cache:
+    key: "sonar"
+    paths:
+      - .sonar/cache
  before_script:
    - *sonar-scripts
    - install_ca_certs "${CUSTOM_CA_CERTS:-$DEFAULT_CA_CERTS}"
    - eval_java_proxy_args
  script:
-    - sonar_autodetect_mr
+    - |
-    - if [[ "$SONAR_GITLAB_TOKEN" ]]; then sonar_extra_args=$SONAR_GITLAB_ARGS; fi
+      if [[ -z "$SONAR_PROJECT_KEY" ]] && ! has_sonar_param projectKey
-    - sonar-scanner ${TRACE+-Dsonar.verbose=true} $java_proxy_args ${SONAR_AUTH_TOKEN+-Dsonar.login=$SONAR_AUTH_TOKEN} ${SONAR_LOGIN+-Dsonar.login=$SONAR_LOGIN} ${SONAR_PASSWORD+-Dsonar.password=$SONAR_PASSWORD} $SONAR_BASE_ARGS $sonar_extra_args $sonar_mr_args
+      then
+        log_info "No Sonar Project Key explicitly set: use default \\e[33;1m$CI_PROJECT_PATH_SLUG\\e[0m"
+        log_info "If you want another value, either set the 'sonar.projectKey' entry in your sonar-project.properties, or set it as \$SONAR_PROJECT_KEY"
+        export SONAR_PROJECT_KEY="$CI_PROJECT_PATH_SLUG"
+      fi
+    - |
+      if [[ -z "$SONAR_PROJECT_NAME" ]] && ! has_sonar_param projectName
+      then
+        log_info "No Sonar Project Name explicitly set: use default \\e[33;1m$CI_PROJECT_PATH\\e[0m"
+        log_info "If you want another value, either set the 'sonar.projectName' entry in your sonar-project.properties, or set it as \$SONAR_PROJECT_NAME"
+        export SONAR_PROJECT_NAME="$CI_PROJECT_PATH"
+      fi
+    - |
+      if [[ "$SONAR_URL" ]] && [[ -z "$SONAR_HOST_URL" ]]
+      then 
+        log_warn '$SONAR_URL variable detected: use $SONAR_HOST_URL instead (see doc)'
+        export SONAR_HOST_URL="$SONAR_URL"
+      fi
+    - |
+      if [[ "$SONAR_AUTH_TOKEN" ]] && [[ -z "$SONAR_TOKEN" ]]
+      then 
+        log_warn '$SONAR_AUTH_TOKEN variable detected: use $SONAR_TOKEN instead (see doc)'
+        export SONAR_TOKEN="$SONAR_AUTH_TOKEN"
+      fi
+    - >-
+      sonar-scanner ${TRACE+-Dsonar.verbose=true} $java_proxy_args 
+      ${SONAR_LOGIN+-Dsonar.login=$SONAR_LOGIN} 
+      ${SONAR_PASSWORD+-Dsonar.password=$SONAR_PASSWORD} 
+      ${SONAR_PROJECT_KEY+-Dsonar.projectKey=$SONAR_PROJECT_KEY} 
+      ${SONAR_PROJECT_NAME+-Dsonar.projectName=$SONAR_PROJECT_NAME} 
+      ${SONAR_QUALITY_GATE_ENABLED+-Dsonar.qualitygate.wait=$SONAR_QUALITY_GATE_ENABLED}
+      $SONAR_BASE_ARGS
  rules:
-    # exclude if $SONAR_URL not set
-    - if: '$SONAR_URL == null || $SONAR_URL == ""'
-      when: never
    - !reference [.test-policy, rules]