Skip to content
Snippets Groups Projects
Commit 833966d9 authored by Zitnik, Anze's avatar Zitnik, Anze
Browse files

Merge branch 'delivery-ansible-scripts' into 'master'

Delivery ansible scripts

See merge request medina/security-monitoring!6
parents d5637b66 2eeb5547
Branches
No related tags found
No related merge requests found
......@@ -4,6 +4,9 @@ ENV_DIR = $(DEPLOY_DIR)/environments/$(ENVIRONMENT)
ANSIBLE_DIR = $(DEPLOY_DIR)/ansible
include $(ENV_DIR)/$(ENVIRONMENT).mk
CUSTOM_PROVISIONING_DIR = $(DEPLOY_DIR)/custom-provision
include $(CUSTOM_PROVISIONING_DIR)/custom-provision.mk
ANSIBLE_ARGS = -i $(ENV_DIR)/inventory.txt \
-e ansible_dir=$(ANSIBLE_DIR) \
-e environment_dir=$(ENV_DIR)
......
......@@ -4,9 +4,15 @@ This project is meant for quickly setting up a demo of evidence collection with
Project is deployed using Ansible scripts on top of infrastructure provisioned with Vagrant.
It creates 4 VMs: a Wazuh server (manager), two machines acting as Wazuh agents, the Evidence Collector, and Clouditor.
It creates 5 CentOS virtual machines (if ran in `full-setup` mode):
- Wazuh server (manager),
- 2x machines acting as Wazuh agents,
- Evidence Collector,
- Clouditor.
In addition to Wazuh, ClamAV is also installed on agent machines (for testing purposes).
In addition to Wazuh, ClamAV is also installed on agent machines.
---
## Requirements
......@@ -14,45 +20,54 @@ In addition to Wazuh, ClamAV is also installed on agent machines (for testing pu
* Ansible 2.9.16
* (optional / integrations) `npm` / `npx` in order to run the simple HTTP server for the integrations
---
## Setting up the demo
First, checkout Wazuh's tag `v4.1.5` into the current directory:
1. Checkout Wazuh's tag `v4.1.5` into the current directory:
```
$ make clone-wazuh
```
or
```
$ git clone https://github.com/wazuh/wazuh-ansible.git
$ git checkout tags/v4.1.5
```
Then select your `ENVIRONMENT` in `Makefile`. Set it to `full-setup` or `no-collector`.
2. Select your `ENVIRONMENT` in `Makefile`. Set it to `full-setup` or `no-collector` (for development purposes, when `evidence-collector` runs on local machine).
3. Create and provision VMs:
If the case of `full-setup`, installation of additional Ansible roles is required:
```
$ ansible-galaxy install --roles-path ansible/ gantsign.golang
$ make create provision
```
1. Provision Wazuh server and Wazuh agents:
---
## Using demo components
### Alert forwarding
To test Wazuh's alert forwarding, run HTTP Simple server using `npx`:
```
$ make create provision
$ PORT=8088 npx http-echo-server
```
2. Check the running instances:
### Clouditor
Navigate browser to: `https://192.168.33.10:5601`, login with default credentials `admin:changeme`. Navigate to `wazuh` section on the left hand-side.
Clouditor starts automatically when Clouditor VM is provisioned.
You should see 2 agents registered and running with Wazuh.
To see Clouditor's output, `ssh` to its machine and examine the log file:
3. Run HTTP Simple server using `npx`
```
$ make ssh-clouditor
$ tail /var/log/clouditor.log
```
To manually (re)start Clouditor (normally not needed), you can use the following command on the Clouditor VM (inside `/home/vagrant/clouditor`):
```
$ PORT=8088 npx http-echo-server
$ make run
```
## Using Evidence Collector
### Evidence Collector
To see Evidence Collector's output, `ssh` to its machine and open Docker logs:
......@@ -61,29 +76,130 @@ $ make ssh-evidence-collector
$ docker logs -ft evidence-collector
```
## Testing with Clouditor
### Wazuh
Clouditor starts automatically when clouditor VM is provisioned.
To check running instances (via Wazuh web interface):
To see Clouditor's output, `ssh` to its machine and examine the log file:
1. Navigate browser to: `https://192.168.33.10:5601`.
2. Login with default credentials `admin:changeme`.
3. Navigate to `Wazuh` section on the left hand-side.
You should see 2 agents registered and running with Wazuh.
---
## Vagrant & Ansible environment configuration
Vagrant boxes (and variables later used by Ansible) are defined inside `/environments/` folder. Each environment contains 3 main files:
- `inventory.txt`:
contains environment variables/configs that will be used by Ansible when provisioning.
- `Makefile`:
named the same as the environment (for easier referencing in the main `Makefile` in root directory), adds additional commands that are environment specific.
- `Vagrantfile`:
contains Vagrant configuration. IPs, hostnames etc. of machines have to match those defined in the corresponding `inventory.txt`.
> Note: `full-setup` environment contains additional `.env` file containing environment variables required by `evidence-collector`.
To deploy to some other existing machines (assuming they run same/similar Linux distro etc.), use `custom-provision` functionality.
---
## Provision existing machines
Ansible playbooks allow for easy installation and set-up of Wazuh (both manager and agents) and Evidence collector.
As part of the Wazuh agent deploy, machines will also have ClamAV installed.
Wazuh manager and Evidence collector should be installed on the same, clean machine, while Wazuh agents can be (and should be) installed onto existing machines with other software running.
> Note: this functionality was developed primarily for CentOS based machines (as it uses YUM package manager).
> Possible problems: CentOS 7 versions with RHEL 7.9 could have problems starting Docker containers due to `libseccomp-devel` package deprecation.
1. Generate SSH key-pair on the remote server(s) as well as on your local machine (if you haven't yet done so or want to use separate credentials):
```
$ make ssh-clouditor
$ tail /var/log/clouditor.log
$ ssh-keygen -t rsa
```
To manually start clouditor (normally not needed), you can use the following command on the clouditor VM (in `/home/vagrant/`):
2. Copy your SSH public key to remote server's `authorized_keys` file:
```
$ make run
$ ssh-copy-id root@192.168.0.13
```
> Note: this will copy your default SSH pub-key from `~/.ssh/id_rsa.pub`.
3. Add machine info to `/custom-provision/custom-inventory.txt` file (see `/environments/.../inventory.txt` file for example).
Make sure to set correct variables:
| Variable | Description |
| -------- | -------- |
| `public_ip` | Machine's IP address. |
| `ansible_sudo_pass` | Machine's root password. |
| `ansible_ssh_user` | Username used to SSH (and later used by Ansible). |
| `ansible_ssh_pass` | SSH password (corresponding to `ansible_ssh_user`). |
| `ansible_ssh_private_key_file` | Location of your private key (corresponding to public key set in previous step). |
Example (user: `root`, password: `admin`, @ `192.168.0.13`):
```
192.168.0.13 public_ip=192.168.0.13 ansible_sudo_pass=admin ansible_ssh_pass=admin ansible_ssh_user=root ansible_ssh_private_key_file=~/.ssh/id_rsa
```
4. Set `evidence-collector` environment variables in `/custom-provision/.env`. See [Evidence collector's documentation](https://git.code.tecnalia.com/medina/wp3/task_3.2/evidence-collector) for more information.
If you're installing both Evidence collector as well as Wazuh manager on the same machine (as intended), you have to set only `clouditor_host`, `elastic_host` & `wazuh_host` variables (where `elastic_host` & `wazuh_host` are the same).
> Note: empty line in `.env` file can cause `Invalid line in environment file` Docker error. This happens only on certain Docker builds - distro dependant.
5. Set variables in `/ansible/globals/globals.yml`:
| Variable | Description |
| ---- | ---- |
| `elasticsearch_host_ip` | IP of the machine running Elasticsearch (same as Wazuh manager). |
| `wazuh_manager_ip` | IP of the machine running Wazuh manager. |
6. Provision:
```
$ make -B custom-provision
```
### Wazuh troubleshooting
Depending on your machine and network configuration, Wazuh could have problem connecting agents to manager. Check Wazuh's web interface to see if agents work corrrectly.
If not, check the logs in `/var/ossec/logs/ossec.log` and consult [official troubleshooting manual](https://documentation.wazuh.com/current/user-manual/agent-enrollment/troubleshooting.html).
Two of the most common problems are [missing open ports](https://documentation.wazuh.com/current/user-manual/agent-enrollment/troubleshooting.html#testing-communication-with-wazuh-manager) and [invalid agent names](https://documentation.wazuh.com/current/user-manual/agent-enrollment/troubleshooting.html#invalid-agent-name-for-enrollment) (if agent machine's hostname matches hostname of any already existing Wazuh machine).
### Minimum hardware requirements
| Component | Wazuh manager + <br>Evidence collector machine | Wazuh agent<br>machine |
| ---- | ---- | ---- |
| Memory | 2 GB | 1 GB |
| CPU | 2 | 1 |
| Storage | 10 GB | 10 GB |
---
## Potential issues
### ClamAV (re)start failed/timed out
ClamAV restart can time-out due to slow disk read/write speeds (if using HDD) and lack of memory. To resolve this, provide the machine with more RAM. Current implementation has it set to `1024` MB (which should suffice for the majoirty of host machine configurations). If you're using SSD, you can lower it to `512` MB.
ClamAV restart can time-out due to slow disk read/write speeds (if using HDD) and lack of memory. To resolve this, provide the machine with more RAM. Current implementation has it set to `1024 MB` (which should suffice for the majoirty of host machine configurations). If you're using SSD, you can lower it to `512 MB`.
### Vagrant issue:
### Vagrant issue
```
The following SSH command responded with a non-zero exit status.
......@@ -94,14 +210,15 @@ Stdout from the command:
Stderr from the command:
umount: /mnt: not mounted.
```
Solved:
```
$ vagrant plugin uninstall vagrant-vbguest
```
### Ansible failing due to ssh issues.
### Ansible failing due to ssh issues
This is important for `manager` and `agents` - VMs need to be running already.
......
---
- name: add Docker CE repository
- name: Remove container tools
yum:
name:
- buildah
- podman
- runc
state: absent
- name: Install (updated) runc
yum:
name:
- runc
state: present
- name: Add Docker CE repository
yum_repository:
name: docker-ce-stable
file: docker
......@@ -9,17 +23,25 @@
gpgcheck: yes
gpgkey: https://download.docker.com/linux/centos/gpg
- name: create a docker group
- name: Create a docker group
group: name=docker
- name: install Docker CE from repository
yum: name=docker-ce state=installed
- name: Install Docker CE from repository
yum:
name: docker-ce
state: installed
- name: add curent '{{ ansible_user }}' to docker groups
user: name={{ ansible_user }} groups=docker append=yes
- name: Add curent "{{ ansible_user }}" to docker groups
user:
name: "{{ ansible_user }}"
groups: docker
append: yes
- name: enable docker service
service: name=docker enabled=yes state=started
- name: Enable docker service
service:
name: docker
enabled: yes
state: started
- name: reset connection to apply group permissions
- name: Reset connection to apply group permissions
meta: reset_connection
\ No newline at end of file
......@@ -15,11 +15,28 @@
- name: Install docker-py
pip:
name: docker-py
executable: pip3
vars:
ansible_python_interpreter: /usr/bin/python3.6
- name: Create 'evidence-collector' directory
file:
path: /home/security-monitoring
state: directory
owner: "{{ ansible_ssh_user }}"
group: "{{ ansible_ssh_user }}"
- name: Copy .env file
copy:
src: "{{ environment_dir }}/.env"
dest: /home/security-monitoring/.env
owner: "{{ ansible_ssh_user }}"
group: "{{ ansible_ssh_user }}"
- name: Log in to Docker image registry
docker_login:
registry: "{{ docker_registry }}"
username: "{{ docker_username }}"
password: "{{ docker_token }}"
vars:
ansible_python_interpreter: /usr/bin/python3.6
- name: Run Docker container
docker_container:
name: evidence-collector
......@@ -27,3 +44,6 @@
state: started
pull: yes
restart_policy: always
env_file: /home/security-monitoring/.env
vars:
ansible_python_interpreter: /usr/bin/python3.6
\ No newline at end of file
demo_mode=false
wazuh_host=192.168.0.12
wazuh_port=55000
wazuh_username=wazuh-wui
wazuh_password=wazuh-wui
elastic_host=192.168.0.12
elastic_port=9200
elastic_username=admin
elastic_password=changeme
redis_host=localhost
redis_port=6379
redis_queue=low
clouditor_host=192.168.0.13
clouditor_port=9090
\ No newline at end of file
#
# Ansible hosts
#
[wazuh_managers]
192.168.0.12 public_ip=192.168.0.12 ansible_sudo_pass=admin ansible_ssh_pass=admin ansible_ssh_user=root ansible_ssh_private_key_file=~/.ssh/is_rsa
[wazuh_agents]
192.168.0.13 public_ip=192.168.0.13 ansible_sudo_pass=admin ansible_ssh_pass=admin ansible_ssh_user=root ansible_ssh_private_key_file=~/.ssh/is_rsa
[evidence_collector]
192.168.0.12 public_ip=192.168.0.12 ansible_sudo_pass=admin ansible_ssh_pass=admin ansible_ssh_user=root ansible_ssh_private_key_file=~/.ssh/is_rsa
#
# Ansible hosts' variables
#
[wazuh_managers:vars]
ansible_ssh_common_args='-o StrictHostKeyChecking=no'
[wazuh_agents:vars]
ansible_ssh_common_args='-o StrictHostKeyChecking=no'
[evidence_collector:vars]
ansible_ssh_common_args='-o StrictHostKeyChecking=no'
\ No newline at end of file
CUSTOM_ARGS = -i $(CUSTOM_PROVISIONING_DIR)/custom-inventory.txt \
-e ansible_dir=$(ANSIBLE_DIR) \
-e environment_dir=$(CUSTOM_PROVISIONING_DIR)
custom-provision:
@ANSIBLE_HOST_KEY_CHECKING=False ansible-playbook $(CUSTOM_ARGS) $(ANSIBLE_DIR)/provision.yml
\ No newline at end of file
demo_mode=false
wazuh_host=192.168.33.10
wazuh_port=55000
wazuh_username=wazuh-wui
wazuh_password=wazuh-wui
elastic_host=192.168.33.10
elastic_port=9200
elastic_username=admin
elastic_password=changeme
redis_host=localhost
redis_port=6379
redis_queue=low
clouditor_host=192.168.33.14
clouditor_port=9090
\ No newline at end of file
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment