Open Source SIEM Solution Using Wazuh and TheHive

11 min readMar 5, 2023

By: Samson Idowu

Introducing Wazuh

Wazuh uses a multi-tier architecture that includes these major components; agent(s), indexer(s), dashboard and central server(s).

The agents are installed on each system to be monitored, and they collect events data, logs, and other system information. Agents can be installed on both virtual (cloud inclusive) and physical machines.

The wazuh central server is responsible for managing the agents, configuring and updating them remotely. The data collected by the agents are sent to the server, which is responsible for processing and analyzing the information to look for compromise based on preconfigured rules and it’s threat intelligence engine.

Data is then sent from the server to the indexer which is responsible for storing and indexing logs and alerts. The indexer is also a scalable text search and analysis engine.

Every data in the wazuh system can be viewed from the dashboard which is also a great way for managing the wazuh components, creating rules, and analysing the data.

The Wazuh SIEM also includes an API that enables integration with other security tools and services, allowing for a more comprehensive security solution. It can be implemented as a single-node cluster, or a distributed solution. However in the course of this implementation, I will work with wazuh as a single-node cluster.

Figure 1.0: Overview of Wazuh Architecture.

Open Source Solutions have some advantages that might suite your need:

They offer more flexibility for customization, as users have access to the source code, can contribute to the development and modify the software to suit their specific use-cases.
They are mostly free to use and distribute; this then makes them a cost-effective option for users.
They are often backed by a community of users (developers) who contribute to the software, provide support, and share knowledge.

Setting-up Wazuh Infrastructure

1. Configuring Wazuh Server

I installed the Wazuh server using the installation assistant. Detailed steps on how to do this are shown here. Run the following command:

curl -sO https://packages.wazuh.com/4.3/wazuh-install.sh && sudo bash ./wazuh-install.sh -a

The server is now installed successfully and I accessed it using the https://<server's ip> and logged in using the initial admnin credentials provided after installation.

Figure 2.1: Showing initial login to Wazuh Dashboard.

2. Configuring Wazuh Agents

Device 1 (Windows Endpoint):

I installed the wazuh agent on my windows endpoint by folowing these steps.

Figure 2.2: Installing and configuring Wazuh agent on Windows.

I refreshed the page and my windows agent is registered and detected as an agent on the dashboard as shown in the below image.

Figure 2.3: Showing Windows Machine on Agent Dashboard.

Device 2 (Linux Endpoint):

I repeated the above steps for my ubuntu machine by following these steps.

Figure 2.4: Installing and configuring Wazuh agent on Linux (Ubuntu).

I refreshed the page and my linux agent is registered and detected as an agent on the dashboard as shown in the below image.

Figure 2.5: Showing Linux Machine on Agent Dashboard.

Device 3 (Linux Endpoint):

I on-boarded the device 3 which is an ubuntu machine by following same steps used for device 2.

Figure 2.6: Showing Device 3 on Agent Dashboard.

I could now view logs from all 3 monitored endpoints as shown below.

Figure 2.7: Validating that SIEM receives logs.

What do the logs mean?

Log 1: This is a windows logon success event that was logged from the windows server endpoint. This event can serve as an indicator of a defense evasion, privilege escalation and successful bruteforce attack if the preceeding events were indicators of bruteforce, such as multiple failed login attempts.

On the windows machine, the logon success is registered as security event log. Wazuh agent then ships these logs top the wazuh core fabric as explained earlier. As highlighted in figure 2.8, you can identify the IP address of the machine and other relevant details to aid analysis of the log data.

It is left for an analyst to determine if this is an indicator of successful compromise having looked at preceeding logs.

Figure 2.8: Showing Logon Success Event.

Log 2: This is a logon failure to one of my linux endpoints. This can also serve as an indicator of attempted compromise if there is repeated failures over a short period of time. The linux machine registers the event as a log and saves it into /var/log/auth.log, from which it is collected by the wazuh agent and shipped to the wazuh central fabric for analysis and presentation.

It can also be a false-positive (a legitimate user can forget login details).

Figure 2.9: Showing Logon Failure Event.Figure 2.9: Showing Logon Failure Event.

Why Can I View the Logs?
I can view these logs because the wazuh agent installed on the endpoints collects the logs from the machines and sends them to the wazuh central server (manager) which then analyzes the logs and runs it against configured rulesets , including threat intelligence. The logs are then sent to the indexer where it is stored in indexes for ease of query. I can then query the logs from the indexer and analyze them using the wazuh dashboard as shown earlier in figure 1.8 above.

Blocking Malicious IP Addresses

Setting up Wazuh Agent:

To begin blocking malicious IP addresses, I installed Nginx on device 3 using the below command.

sudo apt install nginx

Figure 3.0: Confirming Nginx Installation.

I tested that I can reach the server from my “malicious machine” with IP address 10.1.1.42 using curl.

Figure 3.1: Accessing the Web Server from Malicious Host.

To enable wazuh agent monitor Nginx logs, I added the following configuration to the /var/ossec/etc/ossec.conf file and restarted the wazuh agent using sudo systemctl restart wazuh-agent.

<localfile>
  <log_format>syslog</log_format>
  <location>/var/log/nginx/access.log</location>
</localfile>

2. Setting up Wazuh Server:

I downloaded Alienvault’s IP reputation database and outputed it to a file /var/ossec/etc/lists/alienvault_reputation.ipset using the below command.

sudo wget https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/alienvault_reputation.ipset -O /var/ossec/etc/lists/alienvault_reputation.ipset

I then entered the attacking machine’s IP into the Alienvault’s IP reputation database using the below command.

sudo echo "10.1.1.42" >> /var/ossec/etc/lists/alienvault_reputation.ipset

Figure 3.2: Appending Malicious IP in Reputation Database.

I downloaded a script to convert .ipset format to .cdb from the wazuh using the following command.

sudo wget https://wazuh.com/resources/iplist-to-cdblist.py -O /tmp/iplist-to-cdblist.py

I then converted the alienvault_reputation.ipset file created earlier into .cdb format using the previously downloaded script. Below command does the conversion.

sudo /var/ossec/framework/python/bin/python3 /tmp/iplist-to-cdblist.py /var/ossec/etc/lists/alienvault_reputation.ipset /var/ossec/etc/lists/blacklist-alienvault

Since I no longer needed the alienvault_reputation.ipset file and the iplist-to-cdblist.py script, I discarded them using the following commands.

sudo rm -rf /var/ossec/etc/lists/alienvault_reputation.ipset
sudo rm -rf /tmp/iplist-to-cdblist.py

I gave wazuh adequate permissions to the new & converted /var/ossec/etc/lists/blacklist-alienvault file using the command below.

sudo chown wazuh:wazuh /var/ossec/etc/lists/blacklist-alienvault

After completing the above procedures successfully, I proceeded to configure Wazuh’s active response module to block the “malicious IP”
I created a custom rule in the /var/ossec/etc/rules/local_rules.xml file to trigger the active response script with the rule severity to level 12. The custom rulesets are as follows:

<group name="attack,">
  <rule id="100100" level="12">
    <if_group>web|attack|attacks</if_group>
    <list field="srcip" lookup="address_match_key">etc/lists/blacklist-alienvault</list>
    <description>IP address found in AlienVault reputation database.</description>
  </rule>
</group>

Figure 3.3: Adding Custom Ruleset to Local Rules

I edited the /var/ossec/etc/ossec.conf file in the wazuh server and created a etc/lists/blacklist-alienvault list under ruleset section.

Figure 3.4: Adding the Blacklist file to the Ruleset Lists.

I edited the active response section of the /var/ossec/etc/ossec.conf file to make sure traffic from the attacker machine is dropped for 300 seconds as shown in figure 3.5 below.

Figure 3.5: Showing Active Response Config.

A quick test on the attacker’s machine shows that the first curl test is successfuly, but subsequent curl operations within the following 5 minutes are unsuccessful.

Figure 3.6: Demonstrating that Malicious IP is Blocked.

I was also able to detect the event on my wazuh dashboard as shown in the following images.

Figure 3.7: Showing Number of Alerts in Level 12 Severity Level.

Figure 3.8: Showing Security Alerts Above Level 12.

Figure 3.9: Showing More Details About the Alert.

Simulating Brute Force Attack Detection

I simulated a brute force attack on all 3 endpoints by entering wrong credentials over ssh and rdp sessions within a short time frame. I used an invalid credential with username attacker for this simulation.

Below images show the detection of the attack on my SIEM dashboard.

I identified an increment in the amount of authentication failures on the security events dashboard

Figure 4.0: Showing Security Events Dashboard.

As an analyst, this should raise a dust, not only because of the severity level, but because of the frequency over a short period of time. This can be an idicator of a bruteforce attack from automated tools like Hydra, etc.

I drilled down to identify the root cause of the spike in authentication failures by exploring the security alerts. I identified that there had been multiple consistent logon failures over a short period of time on hosts within my infrastructure.

Figure 4.1: Showing Brute Force Attack Indicators in Security Events.

I went further to analyze one of the logs that indicated the attack and I could detect that the login session came from a remote IP 10.1.1.100 with a non-existent user attacker. The consistency in these attempts is a clear indicator of a brute-force attempt and can be remediated temporarily by blocking the attacker’s IP using a network protection device or the wazuh agent on that node, which can be achieved by following this guide.

A more lasting fix can be reviewing IAM policies to further harden the infrastructure.

Figure 4.2: Showing More Details on an Event Log.

Configuring email or slack channel alerts can also be used to detect such alerts by setting alert for the rule’s severity level or adjusting the rule’s severity level to match the minimum threshold for pre-configured email alerting.

Integrating theHive with Wazuh for SOC

1. Setting up Hive Server using Docker:

To begin this implementation, I decided to work with the container version of theHive. I ran the following docker command to run the latest version of theHive.

sudo docker run -d --rm -p 9000:9000 strangebee/thehive:latest

I logged in to hive and created an organization as shown in figure 5.0 below.

I added a new admin user to the organization with full admin permissions as shown below.

I also created more users like boss and analyst and added them to the organization in order to have a better simulation.

I generated an API key for the user account created earlier and copied it.

2. Configuring Wazuh Server to Integrate with theHive:

I proceeded to configure the wazuh manager by first installing theHive python module using the below command.

sudo /var/ossec/framework/python/bin/pip3 install thehive4py==1.8.1

I pasted the following python script in the /var/ossec/integrations/custom-w2thive.py file in order to create a custom integration script.

#!/var/ossec/framework/python/bin/python3
import json
import sys
import os
import re
import logging
import uuid
from thehive4py.api import TheHiveApi
from thehive4py.models import Alert, AlertArtifact

# start user config
# Global vars

#threshold for wazuh rules level
lvl_threshold=0
#threshold for suricata rules level
suricata_lvl_threshold=3
debug_enabled = False
#info about created alert
info_enabled = True
#end user config
# Set paths
pwd = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
log_file = '{0}/logs/integrations.log'.format(pwd)
logger = logging.getLogger(__name__)
#set logging level
logger.setLevel(logging.WARNING)
if info_enabled:
    logger.setLevel(logging.INFO)
if debug_enabled:
    logger.setLevel(logging.DEBUG)
# create the logging file handler
fh = logging.FileHandler(log_file)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
fh.setFormatter(formatter)
logger.addHandler(fh)

def main(args):
    logger.debug('#start main')
    logger.debug('#get alert file location')
    alert_file_location = args[1]
    logger.debug('#get TheHive url')
    thive = args[3]
    logger.debug('#get TheHive api key')
    thive_api_key = args[2]
    thive_api = TheHiveApi(thive, thive_api_key )
    logger.debug('#open alert file')
    w_alert = json.load(open(alert_file_location))
    logger.debug('#alert data')
    logger.debug(str(w_alert))
    logger.debug('#gen json to dot-key-text')
    alt = pr(w_alert,'',[])
    logger.debug('#formatting description')
    format_alt = md_format(alt)
    logger.debug('#search artifacts')
    artifacts_dict = artifact_detect(format_alt)
    alert = generate_alert(format_alt, artifacts_dict, w_alert)
    logger.debug('#threshold filtering')
    if w_alert['rule']['groups']==['ids','suricata']:
        #checking the existence of the data.alert.severity field
        if 'data' in w_alert.keys():
            if 'alert' in w_alert['data']:
                #checking the level of the source event
                if int(w_alert['data']['alert']['severity'])<=suricata_lvl_threshold:
                    send_alert(alert, thive_api)
    elif int(w_alert['rule']['level'])>=lvl_threshold:
        #if the event is different from suricata AND suricata-event-type: alert check lvl_threshold
        send_alert(alert, thive_api)

def pr(data,prefix, alt):
    for key,value in data.items():
        if hasattr(value,'keys'):
            pr(value,prefix+'.'+str(key),alt=alt)
        else:
            alt.append((prefix+'.'+str(key)+'|||'+str(value)))
    return alt

def md_format(alt,format_alt=''):
    md_title_dict = {}
    #sorted with first key
    for now in alt:
        now = now[1:]
        #fix first key last symbol
        dot = now.split('|||')[0].find('.')
        if dot==-1:
            md_title_dict[now.split('|||')[0]] =[now]
        else:
            if now[0:dot] in md_title_dict.keys():
                (md_title_dict[now[0:dot]]).append(now)
            else:
                md_title_dict[now[0:dot]]=[now]
    for now in md_title_dict.keys():
        format_alt+='### '+now.capitalize()+'\n'+'| key | val |\n| ------ | ------ |\n'
        for let in md_title_dict[now]:
            key,val = let.split('|||')[0],let.split('|||')[1]
            format_alt+='| **' + key + '** | ' + val + ' |\n'
    return format_alt

def artifact_detect(format_alt):
    artifacts_dict = {}
    artifacts_dict['ip'] = re.findall(r'\d+\.\d+\.\d+\.\d+',format_alt)
    artifacts_dict['url'] =  re.findall(r'http[s]?://(?:[a-zA-Z]|[0-9]|[$-_@.&+]|[!*\(\),]|(?:%[0-9a-fA-F][0-9a-fA-F]))+',format_alt)
    artifacts_dict['domain'] = []
    for now in artifacts_dict['url']: artifacts_dict['domain'].append(now.split('//')[1].split('/')[0])
    return artifacts_dict

def generate_alert(format_alt, artifacts_dict,w_alert):
    #generate alert sourceRef
    sourceRef = str(uuid.uuid4())[0:6]
    artifacts = []
    if 'agent' in w_alert.keys():
        if 'ip' not in w_alert['agent'].keys():
            w_alert['agent']['ip']='no agent ip'
    else:
        w_alert['agent'] = {'id':'no agent id', 'name':'no agent name'}
    for key,value in artifacts_dict.items():
        for val in value:
            artifacts.append(AlertArtifact(dataType=key, data=val))
    alert = Alert(title=w_alert['rule']['description'],
              tlp=2,
              tags=['wazuh', 
              'rule='+w_alert['rule']['id'], 
              'agent_name='+w_alert['agent']['name'],
              'agent_id='+w_alert['agent']['id'],
              'agent_ip='+w_alert['agent']['ip'],],
              description=format_alt ,
              type='wazuh_alert',
              source='wazuh',
              sourceRef=sourceRef,
              artifacts=artifacts,)
    return alert

def send_alert(alert, thive_api):
    response = thive_api.create_alert(alert)
    if response.status_code == 201:
        logger.info('Create TheHive alert: '+ str(response.json()['id']))
    else:
        logger.error('Error create TheHive alert: {}/{}'.format(response.status_code, response.text))

if __name__ == "__main__":
    try:
       logger.debug('debug mode') # if debug enabled       
       # Main function
       main(sys.argv)
    except Exception:
       logger.exception('EGOR')ssec/integrations/custom-w2thive

I created a bash script to execute the python script created earlier in this file /var/ossec/integrations/custom-w2thive.

#!/bin/sh
# Copyright (C) 2015-2020, Wazuh Inc.
# Created by Wazuh, Inc. <info@wazuh.com>.
# This program is free software; you can redistribute it and/or modify it under the terms of GP>

WPYTHON_BIN="framework/python/bin/python3"
SCRIPT_PATH_NAME="$0"
DIR_NAME="$(cd $(dirname ${SCRIPT_PATH_NAME}); pwd -P)"
SCRIPT_NAME="$(basename ${SCRIPT_PATH_NAME})"
case ${DIR_NAME} in
    */active-response/bin | */wodles*)
        if [ -z "${WAZUH_PATH}" ]; then
            WAZUH_PATH="$(cd ${DIR_NAME}/../..; pwd)"
        fi
    PYTHON_SCRIPT="${DIR_NAME}/${SCRIPT_NAME}.py"
    ;;
    */bin)
    if [ -z "${WAZUH_PATH}" ]; then
        WAZUH_PATH="$(cd ${DIR_NAME}/..; pwd)"
    fi
    PYTHON_SCRIPT="${WAZUH_PATH}/framework/scripts/${SCRIPT_NAME}.py"
    ;;
     */integrations)
        if [ -z "${WAZUH_PATH}" ]; then
            WAZUH_PATH="$(cd ${DIR_NAME}/..; pwd)"
        fi
    PYTHON_SCRIPT="${DIR_NAME}/${SCRIPT_NAME}.py"
    ;;
esac

${WAZUH_PATH}/${WPYTHON_BIN} ${PYTHON_SCRIPT} $@

For wazuh to successfully run the integration script, I added the following lines to the wazuh manager’s confi file /var/ossec/etc/ossec.conf. I also entered the IP address, port and previously generated API key.

<ossec_config>
…
  <integration>
    <name>custom-w2thive</name>
    <hook_url>http://10.1.1.34:9000</hook_url>    
    <api_key>bs+UW9HBLkDbw4tHeiAwQw413El3qO4W</api_key>
    <alert_format>json</alert_format>
  </integration>
…
</ossec_config>

I restarted the wazuh-manager service using sudo systemctl restart wazuh-manager.
To test if theHive is receiving logs, I logged in as one of the users and I could see all cases created and drill down on them, as well as assign to users within the organization.