From 70ff95dbb7e5442c07da0afab83011f3170521a8 Mon Sep 17 00:00:00 2001 From: "Osaba Icedo, Eneko" <eneko.osaba@tecnalia.com> Date: Wed, 19 Jul 2023 10:19:42 +0200 Subject: [PATCH] Update README.md --- README.md | 313 ++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 266 insertions(+), 47 deletions(-) diff --git a/README.md b/README.md index c8e1652..a8fcdc3 100644 --- a/README.md +++ b/README.md @@ -105,59 +105,278 @@ And to those generic described at [development-services](git/deploy/development- * https://auth.192.168.56.1.nip.io:8443/ to access keycloak * ... -# Using docker directly +************ +Optimization Module +************ -The iop-optimizer can also be used directly with docker, i.e. +The optimization problem formulated in PIACERE and solved by the IOP consists on having +a service to be deployed, with the principal challenge of finding an optimized deployment +configuration of the IaC on the appropriate infrastructural elements that best meet the +predefined constraints. In this context,it is the IOP component which is the responsible +for finding the best possible infrastructure given the input data received. -```bash -docker login optima-piacere-docker-dev.artifact.tecnalia.com -u user@domain.net -p repositoryToken -docker pull optima-piacere-docker-dev.artifact.tecnalia.com/wp5/iop-optimizer:y1 -docker run -d -p 8081:8081 --name iop optima-piacere-docker-dev.artifact.tecnalia.com/wp5/iop-optimizer:y1 -curl -X 'POST' http://localhost:8081/api/optimize -H 'accept: */*' -H 'Content-Type: application/json' -d 'doml nginx_openstack application app { software_component nginx { properties { // site source_code=\"/usr/share/nginx/html/index.html\"; } }}infrastructure infra { vm vm1 { ifaces { iface i1 { address \"16.0.0.1\" } } } vm_image v_img { generates vm1 } net net1 { address \"16.0.0.0/24\" protocol \"tcp/ip\" } autoscale_group ag { vm1 }} deployment config { nginx -> vm1} active deployment config concretizations { concrete_infrastructure con_infra { provider openstack { vm concrete_vm { properties { vm_name = \"nginx-host\"; vm_flavor = \"small\"; vm_security_groups = \"default\"; vm_key_name = \"user1\"; ssh_user = \"ubuntu\"; ssh_key_file = \"/home/user1/.ssh/openstack.key\"; } maps vm1 } vm_image concrete_vm_image { properties { name = \"ubuntu-20.04.3\"; } maps v_img } net concrete_net { properties { name = \"ostack2\"; } maps net1 } } } active con_infra}optimization opt { objectives { \"cost\" => min } nonfunctional_requirements { req1 \"Cost <= 200\" max 200 => \"cost\"; req2 \"Availability >= 98%\" min 98 => \"availability\"; req3 \"Region\" values \"00EU\" => \"region\"; }}' +=========================== +How to run the IOP? +=========================== -``` +In order to run the IOP, an **String** should be introduced as input. This input should be +formated in DOML language with all the details about the optimization introduced (this is +explained later). -If the service is called correctly, it will return a String in JSON format, following this structure: +=========================== +How the optimization information is introduced in the input DOML +=========================== -``` -{ - "Solutions":[ - { - "Objectives":[ - 200.0, - 99.76666666666665, - 8.0 - ], - "Solution":"[Storage4_Europe, db.dynamo.3, C8_Germany]" - }, - { - "Objectives":[ - 45.53, - 99.03333333333335, - 11.0 - ], - "Solution":"[Storage1_Spain, db.dynamo.3, t2.nano]" - } - } - ] +This input data is provided in DOML format and will include the optimization objectives +(such as the cost, performance, or availability), and optimization requirements. Then, +the IOP performs the matchmaking for the infrastructure via the execution of optimization +intelligent techniques by using the information taken as input against the available +infrastructure and historical data, available from the IEC. + +Having said that, in order to correctly run the optimization module, the following structure +should be added to the input DOML + +------------------------------------------------------------------------------------------------------------------------ +optimization opt { + objectives { + + } + nonfunctional_requirements { + + } } -``` -Basically, for each solution we show the objectives (cost: 200, availability: 99.7 and performance: 8), and the solution itself (that is, the elements of the IEC that have been chosen). +------------------------------------------------------------------------------------------------------------------------ -If the IOP is called using an String in an incorrect format, it will return an incoherent solution, composed by the whole IEC with the "objectives" clause empty: +At this moment, three different objectives can be contemplated in the IOP: minimize the **cost**, +maximize the **availability** and maximize the **performance**. Any combination of these objectives +can be introduced. Here is an example of the full three objectives and how they should be introduced: -``` -{ - "Solutions":[ - { - "Objectives":[], - "Solution":"[Storage4_Europe]" - }, - { - "Objectives":[], - "Solution":"[t2.nano]" - } - } - ] +------------------------------------------------------------------------------------------------------------------------ + objectives { + "cost" => min + "availability" => max + "performance" => max + } +------------------------------------------------------------------------------------------------------------------------ + +Regarding the non funcional requirements, different kind of requirements are contemplated in +the current version of the IOP, such as assign a maximum cost for the overall configuration, assign a minimum +availability for the overall configuration, assign a minimum performance for the overall configuration +restrict the region of the selected elements and restrict the providers of the elements. Here is an +example of the five requirements, and how they should be introduced: + +------------------------------------------------------------------------------------------------------------------------ + nonfunctional_requirements { + req1 "Cost <= 200" max 200.0 => "cost"; + req2 "Availability >= 94.0%" min 94.0 => "availability"; + req3 "Performance >= 5.0%" min 5.0 => "performance"; + req4 "Region" values "Europe" => "region"; + req5 "Provider" values "aws" => "provider"; + } +------------------------------------------------------------------------------------------------------------------------ + +Thus, this could be a possible input for the IOP, represented in the input DOML. + +------------------------------------------------------------------------------------------------------------------------ +optimization opt { + objectives { + "cost" => min + "availability" => max + } + nonfunctional_requirements { + req1 "Cost <= 400" max 400.0 => "cost"; + req2 "Performance >= 60%" min 60.0 => "performance"; + req3 "Region" values "Europe" => "region"; + } } -``` +------------------------------------------------------------------------------------------------------------------------ + +A special attention should be made here to the multi-element optimization. In order to contemplate the multi-element +optimization, the user can introduce the following structure as a new requirement: + +------------------------------------------------------------------------------------------------------------------------ + +nonfunctional_requirements { + req1 "elements" => ""; +} + +------------------------------------------------------------------------------------------------------------------------ + +Using this structure, the use is able to introduce the combination of elements he/she is searching for. For example, if +the user needs to deploy a service with 3 different virtual machines and 1 database, the element part should be: + +------------------------------------------------------------------------------------------------------------------------ + +nonfunctional_requirements { + req1 "elements" => "VM VM VM Database"; +} + +------------------------------------------------------------------------------------------------------------------------ + +That is, the elements requirement should be introduced as a list of Strings, being VM the acronym of Virtual Machine. In +order to facilitate the usage of this multi-optimization feature, several aspects should be taken into account: + + +• The IOP is flexible to the order of the elements part. That is, if the user needs to deploy 2 virtual machines and 2 +storages, he/she can define the elements part without adapting to any established order. In other words, the following +three examples represent the same situation above mentioned: + +------------------------------------------------------------------------------------------------------------------------ + +req1 "elements" => "VM VM storage storage"; + +req1 "elements" => " storage storage VM VM"; + +req1 "elements" => "VM storage VM storage"; + +------------------------------------------------------------------------------------------------------------------------ + +• The number of units per element can be equal to 0. In these cases, the elements parts should not contain the elements +that are not desired. For example, if the user needs the best configuration for three different virtual machines, not +needing any storage nor database, so the elements part should be defined as follows: + +------------------------------------------------------------------------------------------------------------------------ + +req1 "elements" => "VM VM VM"; + +------------------------------------------------------------------------------------------------------------------------ + +• The elements part of the nonfunctional_requirements is optional. In case the user does not introduce this information, +the IOP consider the basic problem. That is, the finding of the best configuration combining one single unit per element +of the IEC. + +=========================== +How the IOP returns the results +=========================== + +Once the IOP conduct its optimization, it returns a complete DOML. This DOML is the one introduced +as input, with the optimization results introduced. First, the IOP complements the "optimization opt" +part of the DOML adding the found solutions. This is an example: + +------------------------------------------------------------------------------------------------------------------------ +optimization opt { + objectives { + "cost" => min + "performance" => max + "availability" => max + } + nonfunctional_requirements { + req1 "Cost <= 400" max 400.0 => "cost"; + req2 "Performance >= 60%" min 60.0 => "performance"; + req3 "Region" values "00EU" => "region"; + } + solution sol2 { + objectives { + cost 192.0 euro + performance 65.83333333333333 metric + availability 105.0 % + } + decisions ["Storage1_Spain VM_1_CS"] + } + solution sol3 { + objectives { + cost 201.0 euro + performance 65.93333333333334 metric + availability 103.0 % + } + decisions ["Storage4_Europe VM_1_CS"] + } +} +------------------------------------------------------------------------------------------------------------------------ +In this example, the IOP returns two different solutions, composed each one of a Virtual Machine and a +Storage element. For each solutions, the value of each objective and the elements chosen are returned. + +Furthermore, the IOP introduces the features of each found solution in the *concretizations* part of +the DOML. If *concretizations* part is not present in the input DOML, the IOP creates it. This is an +example of returning concretizations. + +------------------------------------------------------------------------------------------------------------------------ +concrete_infrastructure con_infra { + provider openstack { + vm concrete_vm { + properties { + vm_name = "nginx-host"; + vm_key_name = "user1"; + vm_flavor = "t2.nano"; + vm_Availability = 98; + vm_Response_time_Virtual_Machine_Performance = 3; + vm_Memory = 0.5; + vm_Zone = "IEEU"; + vm_Frequency_per_Core = 1500; + vm_Virtual_CPU_Cores = 1; + vm_provider_OU = "AMAZ"; + vm_public_IP_type = "IPV4"; + vm_Cost_Currency = 4.53; + vm_Instance_Storage = 40; + vm_Optimized_for = "GEPU"; + vm_Region = "00EU"; + } + maps vm1 + } + + vm_image concrete_vm_image { + properties { + name = "ubuntu-20.04.3"; + } + maps v_img + } + + net concrete_net { + properties { + name = "ostack2"; + } + maps net1 + } + } + } +------------------------------------------------------------------------------------------------------------------------ + +Finally, the last aspect to highlight in this documentation is related to how the IOP ranks the solutions provided as +output. As mentioned, if the problem to solve has two or three different objectives to optimize, the problem converts into +a multi-objective one. This situation implies that more than one optimal solution exists. Indeed, the number of optimal +solutions are all the ones that meet the users requirements and that are placer in the Pareto front of the problem. + +For this reason, the IOP provides the different deployment configurations found using one of the optimization objectives +as reference. In this sense, and in order to do it flexible enough, the IOP adapts to the needs of the user, who can +define which is the objective considered as reference. For doing that, the input DOML is used. More concretely, the +objectives part of the optimization opt section of the DOML is used, deeming the first value as reference objective. + +Thus, if the user inserts the following objectives as input: + +------------------------------------------------------------------------------------------------------------------------ + +objectives { + "cost" => min + "availability" => max + "performance" => max +} + +------------------------------------------------------------------------------------------------------------------------ +All the deployments returned by the IOP will be ranked using the cost as reference, choosing the solution with the less +cost the first one to appear. On the contrary, if the user introduces the following objectives: + +------------------------------------------------------------------------------------------------------------------------ + +objectives { + "performance" => max + "availability" => max +} + +------------------------------------------------------------------------------------------------------------------------ + +The performance will be the reference to rank the returned solutions, choosing these solutions with the best performance +as the best ones. + +=========================== +How to install the IOP +=========================== + +The whole IOP is built in a compressed folder, which can be imported by any JAVA development framework such as NetBeans of +Eclipse. Furthermore, the project can also be imported as a Maven project, being this option even more comfortable than just +importing the whole project folder. Related also with this last aspect, jMetal framework is imported to the project and used +by the IOP using Maven functionality. + +The entry point of the IOP if the user wants to run it via JAVA code is the class named as OptimizerServiceIT.java, which is +part of the package com.piacere.iop.optimizer.service, within the test branch of the code. The method that runs the AOT is the +one coined as optimizedService.run(call). This is the method that it is called by the PIACERE components to make the IOP run +and to find the most optimized deployment configurations. -- GitLab