From 2f6b7c9ff528fb6b78ad3175573e5bb75a184f4b Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?I=C3=B1aki=20Etxaniz?= <Inaki.Etxaniz@Tecnalia.com> Date: Thu, 21 Nov 2024 16:36:24 +0100 Subject: [PATCH] Updated readme file, component_integration readme --- README.md | 22 ++++----- component_integration.md | 98 +++++++++++++++++++--------------------- 2 files changed, 57 insertions(+), 63 deletions(-) diff --git a/README.md b/README.md index a041199..30f0d2e 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ -# Emerald Caas Framework Contribution Guidelines +# Emerald CaaS Framework Contribution Guidelines -This repository contains Contribution Guidelines for Caas Framework in emerald project. +This repository contains Contribution Guidelines for CaaS Framework in emerald project. ## Table of contents - [Table of contents](#table-of-contents) @@ -14,21 +14,21 @@ This repository contains Contribution Guidelines for Caas Framework in emerald p - [Project status](#project-status) ## Description -This repository contains diferent utility guidelines on how to contribute to the Emerald Caas Framework. The guidelines are divided into different sections, each one with a specific purpose. +This repository contains diferent utility guidelines on how to contribute to the Emerald CaaS Framework. The guidelines are divided into different sections, each one with a specific purpose. - [Enroll](https://git.code.tecnalia.com/emerald/public/enroll/-/blob/master/README.md) how to join to the repository for contributions. -- [Component Development](./component_development.md): Contains the guidelines for developing a new component for the Caas Framework. -- [Component Integration](./component_integration.md): Contains the guidelines for integrating a new component into the Caas Framework. -- [Component Validation](./component_validation.md): Contains the guidelines for validating a new component for the Caas Framework. -- [Keycloak Integration](./keycloak_integration.md): Contains the guidelines for integrating Keycloak into the components of the Caas Framework. -- [Component Update](./component_update.md): Contains the guidelines for updating a component of the Caas Framework. -- [Component Troubleshooting](./component_troubleshooting.md): Contains the guidelines for troubleshooting a component of the Caas Framework. -- [Semantic Versioning with Gitlab CI/CD](./semantic_versioning_ci_cd.md): Contains the guidelines for versioning the components of the Caas Framework. +- [Component Development](./component_development.md): Contains the guidelines for developing a new component for the CaaS Framework. +- [Component Integration](./component_integration.md): Contains the guidelines for integrating a new component into the CaaS Framework. +- [Component Validation](./component_validation.md): Contains the guidelines for validating a new component for the CaaS Framework. +- [Keycloak Integration](./keycloak_integration.md): Contains the guidelines for integrating Keycloak into the components of the CaaS Framework. +- [Component Update](./component_update.md): Contains the guidelines for updating a component of the CaaS Framework. +- [Component Troubleshooting](./component_troubleshooting.md): Contains the guidelines for troubleshooting a component of the CaaS Framework. +- [Semantic Versioning with Gitlab CI/CD](./semantic_versioning_ci_cd.md): Contains the guidelines for versioning the components of the CaaS Framework. - [Component Publish with Gitlab CI/CD](./component_publish_ci_cd.md): Contains the guidelines for setting up the CI/CD pipeline for a component image publishing in gitlab. Version: 1.0.0 ## Support -If you require support for improving the template add an [issue](../../../-/issues) to the template and contact the [Owners/Maintainers](../-/project_members?sort=access_level_desc) of the repo. +If you require support for improving the template, add an [issue](../../../-/issues) to the template and contact the [Owners/Maintainers](../-/project_members?sort=access_level_desc) of the repo. Note: remember to change this section to match the support options for your project. diff --git a/component_integration.md b/component_integration.md index 231db3f..1cb3367 100644 --- a/component_integration.md +++ b/component_integration.md @@ -1,14 +1,14 @@ # Component Integration -Once the components are developed and their images are published in some accesible registry, the next step is to integrate them into the Caas Framework. The integration of a new component into the Caas Framework is done by adding the component manifests files to the repository and extending the Kustomization file to include those manifests files in the deployment. +Once the components are developed and their images published in some accesible registry, the next step is to integrate them into the CaaS Framework. The integration of a new component consist in adding the component manifests files to the repository and extending the Kustomization file to include those manifests files in the deployment. -This document contains the guidelines for integrating a new component into the Caas Framework. The final target in the integration of a new component into the Caas Framework is to submit a merge request. The merge request will be reviewed by the project maintainers and if it is accepted it will be merged into the main branch. +This document contains the guidelines for integrating a new component into the CaaS Framework. The final purpose of the integration is to submit a merge request. The merge request will be reviewed by the project maintainers and, if accepted, merged into the main branch. The merge request should contain the following information: - A folder with the component manifests files. -- The required sections in the Kustomization file to include those manifests in the Caas Framework deployment. +- The required sections in the Kustomization file to include those manifests in the CaaS Framework deployment. -In the following sections we will explain how to create the merge request with the required information. +In the following sections we explain how to create the merge request with the required information. ## Table of contents @@ -33,33 +33,31 @@ In the following sections we will explain how to create the merge request with t ## Merge request initialisation -There are many ways to create a merge request in Gitlab. In this section we will explain some of them. -- From an issue, which is the recommended way. +There are many ways to create a merge request in GitLab. In this section we will explain two of them. +- From an issue (recommended way). - From a branch. ### From an issue -This is the recommended way to create a merge request. The issue should be created before starting the integration of a new component into the Caas Framework. +This is the recommended way to create a merge request. The issue should be created before starting the integration of a new component into the CaaS Framework. - [Create a new issue](../../../-/issues/new) You can additionally link the issue with the [requirement](https://git.code.tecnalia.com/emerald/private/requirements/-/issues) that this issue contributes to fulfil. -Once the issue is created, you can create a merge request from the issue. The merge request will be linked to the issue and the issue will be automatically closed when the merge request is accepted. - -The creation of the merge request from the issue is done by clicking on the **Create merge request** button in the issue page. That will also include the creation of a new branch in the repository to hold the changes necessary to integrate the component. +Once the issue is created, you can create a linked merge request by clicking on the <kbd>**Create merge request**</kbd> button in the issue page. That will also create a new branch in the repository that hold the changes. When the merge request is accepted, the linked issue will be automatically closed. The merge request will be annotated as **draft** so that you can work without the intervention of the project maintainers. Once the component is ready for review, you can change the status of the merge request to **ready for review**. -### From a branch before adding the component +### From a branch *before* adding the component -This is an alternative way to create a merge request. You can create a new branch in the repository and then create a merge request from that branch. In this case, it is recommended to create the merge request as a **draft** so that you can work without the intervention of the project maintainers. Once the component is ready for review, you can change the status of the merge request to **ready for review**. +This is an alternative way to create a merge request. You can create a new branch in the repository and then create a merge request from that branch. when the merge request is accepted. It is recommended to create the merge request as a **draft**, so that you can work without the intervention of the project maintainers. Once the component is ready for review, you can change the status of the merge request to **ready for review**. -### From a branch after adding the component +### From a branch *after* adding the component -Another way to work is to clone the repository, add the component to the repository, and then create a merge request from the branch where you added the component. +Another way to work is to clone the repository, add the component in a new branch, push it to the repository, and then create a merge request from that branch. ```bash -git clone git@git.code.tecnalia.com:emerald/private/devops/caas-framework.git -cd caas-framework +git clone git@git.code.tecnalia.com:emerald/private/devops/CaaS-framework.git +cd CaaS-framework echo "Add the component files to the repository" git checkout -b feature/component_integration git add . @@ -69,24 +67,20 @@ git push origin feature/component_integration ## (Optional) Download the branch and add the component -This is applicable if you create the branch from the Gitlab interface before adding the component. You can download the branch as follows: +This is applicable if you create the branch from the GitLab interface before adding the component. You can download the branch as follows: ```bash -git clone git@git.code.tecnalia.com:emerald/private/devops/caas-framework.git -cd caas-framework +git clone git@git.code.tecnalia.com:emerald/private/devops/CaaS-framework.git +cd CaaS-framework git checkout feature/component_integration ``` ## Create the component folder -The first step to integrate a new component into the Caas Framework is to create a folder in the repository to hold the component manifests files. The folder should be created in the root of the repository and should have the name of the component. The folder will contain the manifests files of the component. - -i.e. for the amoe component, the folder should be named `amoe` and should contain the manifests files of the component. +The first step to integrate a new component into the CaaS Framework is to create a folder in the repository to hold the component manifests files. The folder should be created in the root of the repository and should have the name of the component. -```bash -mkdir amoe -``` +E.g., for the amoe component, the folder should be named `amoe` and should contain the manifests files of the component. -The type of the manifests files can be different depending on the component. The most common types are: +The manifests files can be different depending on the component. The most common types are: - jobs - services - deployments @@ -94,7 +88,7 @@ The type of the manifests files can be different depending on the component. The ### Add the component manifests files -The next step is to add the component manifests files to the folder created in the previous step. We recommend to prepend the name of the file with a number to indicate the order in which the files should be applied. There are example of names into the `rcm` component. +The next step is to add the component manifests files to the created folder. We recommend to prepend the name of the file with a number to indicate the order in which the files should be applied. There are examples of names into the `rcm` component.[link?]() ### ConfigMaps @@ -102,23 +96,23 @@ It is recommended to create Configmaps using the `kustomize` tool. The contents ### Secrets -The Secrets will be created using the `kustomize` tool. The files to be used for the secrets generation should be **NOT** be stored in the repository. The secrets should be added in the component folder in a folder named `.secrets`. The secrets will be excluded from the repository because `.gitignore` file contains the `.secrets/` folder. +The Secrets will be created using the `kustomize` tool. The files to be used for the secrets generation SHOULD NOT BE STORED in the repository. The secrets should be added in the component folder in a folder named `.secrets`, that will be excluded from the repository adding the `.secrets/` folder to the `.gitignore` file.  ## Extend the Kustomization file -The next step is to extend the Kustomization file to include the manifests files of the component. The Kustomization file is located in the root of the repository and is named `kustomization.yaml`. The Kustomization file is a YAML file that contains the list of the manifests files that should be included in the deployment. +The next step is to extend the Kustomization file. The Kustomization file (`kustomization.yaml`) is a YAML file, located in the root of the repository, that contains the list of the manifests files that should be included in the deployment. -To extend the Kustomization file it is recommended to use the `kustomize` tool. The `kustomize` tool is a CLI tool that allows you to manage the Kustomization file. To install the `kustomize` tool you can follow the instructions in the [official documentation](https://kubectl.docs.kubernetes.io/installation/kustomize/). +To extend the Kustomization file it is recommended to use the `kustomize` CLI tool. To install the `kustomize` tool you can follow the instructions in the [official documentation](https://kubectl.docs.kubernetes.io/installation/kustomize/). The relevant sections in the Kustomization file are: - - `resources`: This section contains the list of the manifests files that should be included in the deployment. The manifests files should be relative to the root of the repository. - - `configMapGenerator`: This section contains the list of the ConfigMaps that should be included in the deployment. The ConfigMaps should be relative to the root of the repository. - - `secretGenerator`: This section contains the list of the Secrets that should be included in the deployment. The Secrets should be relative to the root of the repository. + - `resources` + - `configMapGenerator` + - `secretGenerator` ### Kustomize resources -This is the most common section in the Kustomization file. The `resources` section contains the list of the manifests files that should be included in the deployment. The manifests files should be relative to the root of the repository. +The `resources` section contains the list of the manifests files that should be included in the deployment. The manifests files should be relative to the root of the repository. ### Kustomize configMapGenerator This section contains the list of the ConfigMaps that should be included in the deployment. If we follow the previous example, the ConfigMaps should use the `configmaps` folder in the component folder. @@ -126,18 +120,18 @@ This section contains the list of the ConfigMaps that should be included in the ### Kustomize secretGenerator This section contains the list of the Secrets that should be included in the deployment. The secrets should be in the root of the repository and they should be excluded from the repository in the `.gitignore` file. -The secrets will be added during the CI/CD pipeline execution. The secrets that are not in the repository should be added to the Gitlab CI/CD variables as files. The files should be named with the name of the secret and should contain the secret value. +The secrets that are not in the repository should be added to the GitLab CI/CD variables as files. The files should be named with the name of the secret and should contain the secret value. -Latter, during the CI/CD pipeline execution, the secrets will be added to the root of the repository by the k8s-pre-apply.sh. This script takes the secrets from the temporary folder and adds them to the root of the repository. +Latter, during the CI/CD pipeline execution, the secrets will be added to the root of the repository by the `k8s-pre-apply.sh`. This script takes the secrets from the temporary folder and adds them to the root of the repository. -In summary the steps to add a secret are: -- Create the secret file in the component folder, to verify the kustomize configuration. -- Add configuration in the kustomization.yaml file to include the secret file. -- Check the kustomize configuration. -- Add the secret CI/CD setup at the end of the k8s-pre-apply.sh file. -- Request a gitlab project [owner or maintainer](https://git.code.tecnalia.com/emerald/private/devops/caas-framework/-/project_members) to create the secret in the Gitlab CI/CD variables. +In summary, the steps to add a secret are: +1. Create the secret file in the component folder, to verify the kustomize configuration. +2. Add configuration in the kustomization.yaml file to include the secret file. +3. Check the kustomize configuration. +4. Add the secret CI/CD setup at the end of the `k8s-pre-apply.sh` file. +5. Request to some GitLab project [owner or maintainer](https://git.code.tecnalia.com/emerald/private/devops/CaaS-framework/-/project_members) to create the secret in the GitLab CI/CD variables. -example of the secret file creation and kustomize configuration: +Example of the secret file creation and kustomize configuration: ```bash cat <<EOF > amoe_redis_secrets @@ -157,11 +151,11 @@ kustomize build . echo 'copy_from_cicd_variables "amoe_redis_secrets" "amoe/.secrets/amoe_redis"' >> k8s-pre-apply.sh ``` -Once the secret is in the Gitlab CI/CD variables, the k8s-pre-apply.sh script will add the secret to the root of the repository. +Once the secret is placed in the GitLab CI/CD variables, the `k8s-pre-apply.sh` script will add the secret to the root of the repository. ## Kustomize check configuration -The Kustomize tool allows you to check the configuration of the Kustomization file. The `kustomize` tool has a command to check the configuration of the Kustomization file. The command is: +The `Kustomize` tool allows you to check the configuration of the Kustomization file. The `kustomize` tool has a command to check the configuration of the Kustomization file. The command is: ```bash kustomize build . @@ -178,10 +172,10 @@ kustomize build . | kubectl apply -f - ### Getting the kubeconfig -To get the kubeconfig login into kubernetes https://k8so.emerald.digital.tecnalia.dev/ and download the kubeconfig. **Be aware that the kubeconfig is valid for one month**. +To get the kubeconfig, login into kubernetes https://k8so.emerald.digital.tecnalia.dev/ and download the kubeconfig. **Be aware that the kubeconfig is valid for one month**. - - +{width=400} +{width=400} The kubeconfig should be stored in the `~/.kube/config` file. @@ -190,7 +184,7 @@ cat << EOF > ~/.kube/config <content of the kubeconfig file> EOF ``` - +{width=50%} ```bash kubectl config get-contexts @@ -203,12 +197,12 @@ The above is valid for one kubernetes cluster. If you have more than one kuberne The kubernetes cluster includes a rancher server that allows you to access the kubernetes cluster. The rancher server is available at https://k8so.emerald.digital.tecnalia.dev/. - - +{width=400} +{width=400} There you can access the kubernetes cluster and check the deployment of the component. - +{width=400} ## Submit the merge request -- GitLab