@@ -12,31 +12,35 @@ Emerald project architecture is based in containers and containers orchestration
-[Login](#login)
-[Publish](#publish)
## Basics of component development
## Basics of component publishing
Independent of the component nature: open source, or proprietary. It must have a repository in the Tecnalia GitLab. The repository must have a README.md and a LICENSE file.
Independently of the component nature -open source or proprietary- it must have a repository in the Tecnalia GitLab. The repository must have a README.md and a LICENSE file.
In the case of open source components, the README.md will describe the usage of the component and the LICENSE file will contain the license of the component. By default, the license in Emerald project is Apache 2.0.
In the case of **open source** components, the README.md will describe the usage of the component, and the LICENSE file will contain the license of the component. By default, the license in Emerald project is Apache 2.0.
In the case of proprietary components, the README.md will describe the objective of the component and the means to access the component. The LICENSE file will contain the license of the component.
In the case of **proprietary** components, the README.md will describe the objective of the component, and how to access the component. The LICENSE file will contain the license of the component.
**Note: during the public release of the component, for proprietary components, only the README.md and the LICENSE file will be copied to the public repository.**
**Note: for the public release of the proprietary components, only the README and the LICENSE files will be copied to the public repository.**
## Component placement in the Tecnalia GitLab
The components must be placed in the Tecnalia GitLab. The components must be placed in the `emerald/private/components` group. Inside we have prepared individual groups for each component, i.e`emerald/private/components/amoe`, `emerald/private/components/rcm`, etc.
The components must be placed in the Tecnalia GitLab. The components must be placed in the `emerald/private/components` group. Inside we have prepared individual groups for each component, e.g.,`emerald/private/components/amoe`, `emerald/private/components/rcm`, etc.
**Note: if your component is not in the list, please contact the CaaS Framework administrator to create the group for your component.**
Inside, the component group, you can create repos for each of the images that your component require. For example, the `rcm` component has the following repositories: `frontend`, `backend` and `converter`.
Inside the component group, you can create a repo for each of the images that your component require. For example, the `rcm` component has the following repositories: `frontend`, `backend` and `converter`.
Adittionally, if you require more repositories for your component, you can create them. For example, the `amoe` component has an internal group `emerald/private/components/amoe/python-clients` that contains the code generated with the client libraries. For example, `rcm` has a repository `docker-compose` that contains a compostion that we use to develop rcm, and it also has a `initial-data` component that contain the initial configuration data that we use in the docker-compose project.
Adittionally, if you require more repositories for your component, you can create them. For example,
- The `amoe` component has an internal group `emerald/private/components/amoe/python-clients` that contains the code generated with the client libraries.
- The `rcm` has a repository `docker-compose` that contains a composition that we use to build rcm; and it also has an `initial-data` repository that contain the initial configuration data that we use in the docker-compose project.
## Manual build and publish
The manual build and publish is the simplest way to build and publish a component. It is useful for testing purposes.
The simplest way to build and publish a component is manually. It is useful for testing purposes.
To build and publish a component manually you will need a Dockerfile.
To build and publish a component manually, you will need a Dockerfile.
### Dockerfile
...
...
@@ -46,19 +50,14 @@ The following is an example of a Dockerfile:
```Dockerfile
FROM python:3.8-slim
WORKDIR /app
COPY requirements.txt requirements.txt
RUN pip install-r requirements.txt
COPY src src
CMD ["python", "src/main.py"]
```
The Dockerfile is a simple example of a python application. It installs the requirements, copies the source code and runs the main.py file.
The above file is a simple example of a python application. It installs the requirements, copies the source code and runs the main.py file.
### Build
...
...
@@ -68,19 +67,17 @@ To build the docker image, you can use the following command:
docker build -t my-component:latest .
```
This command will build the docker image with the tag `my-component:latest`.
that will build the docker image with the tag `my-component:latest`.
### Login
### Publish
To publish the docker image, you need to login to the docker image registry. You can use the following command:
To publish the docker image in some binary repository, you need first to login into the docker image registry. You can use the following command:
When developing the dockerfile you should have in mind that depending of how you write the dockerfile, the docker image will be more or less efficient, stable and big. Here are some recommendations:
When developing the dockerfile you should have in mind that depending of how you write the dockerfile, the resulting docker image will be more (or less) efficient, stable or large. Here are some recommendations:
-**SIZE**. Check the size of the resulting image. The size of the image is important because it affects the time it takes to download the image, the time it takes to start the container, and the amount of disk space used. You can use the `docker image ls` command to check the size of the image. If it is too big, you can try to reduce the size by removing unnecessary files, using a smaller base image, or using multi-stage builds.
-**SET-UP FILES**. Avoid including large setup files inside the image. If you need to include them, you can use a volume to mount the files at runtime. This will reduce the size of the image.
-**BASE IMAGES**. Use versioned base images. For example, if you are developing a python application, you can use the `python:3.8` image instead of the `python:latest` image. Using the `latest` tag can lead to unexpected behavior when the base image is updated.
-**IMAGE SIZE**. Use the smallest base image that you can. For example, if you are developing a python application, you can use the `python:3.8-slim` image instead of the `python:3.8` image.
-**DEPENDENCIES**. Install dependecies at the beginning of the Dockerfile. This will allow you to take advantage of the docker cache, so that if you change the source code, the dependencies will not be installed again, saving lots of time. For example:
- Check the size of the resulting image. The size of the image is important because it affects the time it takes to download the image, the time it takes to start the container, and the amount of disk space that is used. You can use the `docker image ls` command to check the size of the image. If it is too big, you can try to reduce the size by removing unnecessary files, using a smaller base image, or using multi-stage builds.
- Avoid including large setup files inside the image. If you need to include large setup files, you can use a volume to mount the files at runtime. This will reduce the size of the image.
- Use versioned base images. For example, if you are developing a python application, you can use the `python:3.8-slim` image instead of the `python:latest` image. Using the `latest` tag can lead to unexpected behavior when the base image is updated.
- Use the smallest base image that you can. For example, if you are developing a python application, you can use the `python:3.8-slim` image instead of the `python:3.8` image.
- Install dependecies at the beginning of the Dockerfile. This will allow you to take advantage of the docker cache. If you change the source code, the dependencies will not be installed again. Saving lots of time. i.e.:
```Dockerfile
FROM python:3.8-slim
WORKDIR /app
COPY requirements.txt requirements.txt
RUN pip install-r requirements.txt
COPY src src
CMD ["python", "src/main.py"]
```
- Use multi-stage builds. If your application requires build stage with packages that are not needed in the final image, you can use multi-stage builds. i.e.:
-**MULTI-STAGE**. If your application requires a build stage with packages that are not needed in the final image, you can use multi-stage builds. For example:
```Dockerfile
FROMmaven:3.6.3-jdk-11ASbuild
WORKDIR /app
COPY pom.xml .
RUN mvn dependency:go-offline
COPY src src
RUN mvn package
FROM openjdk:11-jre-slim
WORKDIR /app
COPY --from=build /app/target/my-app.jar .
CMD ["java", "-jar", "my-app.jar"]
```
- Use COPY instead of ADD. The ADD instruction is more powerful than the COPY instruction. However, the ADD instruction can be confusing and lead to unexpected behavior. For example, the ADD instruction can download files from the internet. This can be a security risk.
- If you need to ensure permissions after COPY use `--chown` option. For example, if you copy a file and you need to ensure that the file has the correct permissions, you can use the `--chown` option. i.e.:
-**COPY vs ADD**. Use COPY instead of ADD. The ADD instruction is more powerful than the COPY instruction. However, the ADD instruction can be confusing and lead to unexpected behavior. For example, the ADD instruction can download files from the internet. This can be a security risk. If you need to ensure permissions after COPY use `--chown` option. For example, if you copy a file and you need to ensure that the file has the correct permissions, you can use the `--chown` option. i.e.:
```Dockerfile
FROM python:3.8-slim
WORKDIR /app
COPY --chown=1000:1000 src src
CMD ["python", "src/main.py"]
```
- Combine instructions. If you can combine instructions, you can reduce the number of layers in the docker image. Do not forget to clean up the cache in the same instruction. Use Heredoc syntax to combine instructions. i.e.:
```Dockerfile
-**COMBINE INSTRUCTIONS**. Combining instructions, you can reduce the number of layers in the docker image. Do not forget to clean up the cache in the same instruction. Use *Heredoc* syntax to combine instructions. For example:
```Dockerfile
FROM python:3.8-slim
WORKDIR /app
RUN <<EOF
set -e
apt-get update
...
...
@@ -161,72 +142,47 @@ RUN <<EOF
apt-get clean
rm -rf /var/lib/apt/lists/*
EOF
COPY requirements.txt requirements.txt
RUN pip install-r requirements.txt
COPY src src
CMD ["python", "src/main.py"]
```
- Define the EXPOSE instruction. The EXPOSE instruction informs not only the user but also other utility containers (i.e. traefik) that the container listens on the specified network ports at runtime. i.e.:
-**USE EXPOSE**. The EXPOSE instruction informs not only the user, but also other utility containers (i.e. traefik) that the container listens on the specified network ports at runtime. For example:
```Dockerfile
FROM python:3.8-slim
WORKDIR /app
COPY requirements.txt requirements.txt
RUN pip install-r requirements.txt
COPY src src
EXPOSE 8080
CMD ["python", "src/main.py"]
```
- (Optional) Use the HEALTHCHECK instruction. The HEALTHCHECK instruction tells Docker how to test a container to check that it is still working. This can be used to detect when a container is unhealthy. i.e.:
-**(Optional) USE HEALTHCHECK**. The HEALTHCHECK instruction tells Docker how to test a container to check that it is working. This can be used to detect when a container is unhealthy. For example:
- (Optional) Define VOLUME instruction. The VOLUME instruction creates a mount point with the specified name and marks it as holding externally mounted volumes from native host or other containers. i.e.:
-**(Optional) DEFINE VOLUMES**. The VOLUME instruction creates a mount point with the specified name and marks it as holding externally mounted volumes from native host or other containers. For example:
