Splunk Enterprise Integration with Zero Trust Alarms

• Overview
• Support Statement
• Limitations
• Solution Overview
• What is Splunk Enterprise?
• Integration Architecture
• Solution Configuration in Splunk Enterprise and Defender Zero Trust
• Prerequisites
• Configuration in Splunk Enterprise
• Configuration Steps on Eyeglass Virtual Machine
• High Level steps
• Configuration Step by Step
• Configure the Service start and python integration files
• Configure the python packages and customize the splunk integration python code
• Configure Defender Zero Trust Webhooks
• How to test the Integration with Splunk Enterprise
• Splunk Enterprise SecOps administrators Integration Experience
• How to Configure Data Security Essentials
• Overview
• Requirements
• Features
• Steps to Configure Event Sync to Splunk Host Containment Integration Download the integration code from here.

Overview

Customers using Splunk Enterprise can leverage this integration to send real time zero trust alerts using webhooks and maintain full payload parsing using json defined fields.   The Splunk HTTP Event Connector capability allows inbound webhook json payload to be parsed and tagged with a source type to allow for dashboards and triggers to be leveraged in workflows.

Support Statement

1. NOTE:  This documentation is provided "as is" without support for 3rd party software.  The level of support for this integration guide is best effort without any SLA on response time.  No 3rd party product support can be provided by Superna directly.  3rd party components require support contracts

Limitations

1. HTTPS with un-signed certificates is the default configuration and certificate checking has been disabled in this sample code.

Solution Overview

Superna Defender Zero Trust API is the cornerstone technology used to integrate with SIEM, SOAR and XDR platforms.   Automation begins with data that summarizes the threat and places that information into a security tools to be acted on by Secops and run playbooks to protect corporate IT assets from vulnerabilities and insider or external attackers.   In order to allow a Splunk to act on the data it must be parsed.   The Splunk platform can dynamically extract fields that are encoded in the alert payload and exposed inside the index.  The example code shows how to take a subset of the fields available in the Superna Zero Trust alarms and have them exposed to the index as key value pairs.

What is Splunk Enterprise?

Enables you to search, analyze, and visualize the data gathered from the components of your IT infrastructure or business.  Create triggers and workflows and dashboards to monitoring critical infrastructure.

Integration Architecture

Solution Configuration in Splunk Enterprise and Defender Zero Trust

Prerequisites

1. Installed Ransomware Defender and or Easy Auditor or Defender for AWS
2. Eyeglass OS appliance version 15.5
   1. cat /etc/os-release
3. License key for the Zero Trust API 

Configuration in Splunk Enterprise

1. Login to the console and select Settings --> Data Input --> HTTP Event collector.  
   1. Fill in the Fields as per the below image.
   2.  
   3. Enable all indexes and set the default to Main
   4. 
   5. Click submit
   6. Go back to Data Inputs --> HTTP Event Collector Screen and click Global Settings
      1. Set the default Source Type to _json
      2. Click Enable All Tokens
      3. Click Save
      4.   
2. On the list of HTTP Event Tokens record the token to be used with the python code.  Click the token field to copy the value.
   1.  

Configuration Steps on Eyeglass Virtual Machine

High Level steps

1. Create python location to run the application on the Eyeglass vm
2. Create python main application script
3. Create linux systemd service and set to auto start
4. Create Zero Trust configuration in Defender
5. Update the main script to customize with Splunk Enterprise  python code
6. Test the script is running as a service
7. Create a test event in Defender to validate the alerts appear as indexed parsed events in Splunk Enterprise

Configuration Step by Step

Configure the Service start and python integration files

1. Login to eyeglass vm using ssh as the admin user to create zero trust application
   1. sudo -s 
   2. mkdir -p /opt/superna/cgi-bin
   3. touch /opt/superna/cgi-bin/splunk.py
   4. touch /opt/superna/cgi-bin/splunk.sh 
   5. sca:users /opt/superna/cgi-bin/splunk.*
   6. chmod +x  /opt/superna/cgi-bin/splunk.py 
   7. chmod +x /opt/superna/cgi-bin/splunk.sh 
2. Create systemd configuration
   1. nano  /etc/systemd/system/splunk.service
   2. Copy the values below and Paste these contents into the file opened in nano editor in the step above
   3. Save the file 
      1. press control+x
      2. Answer yes to save and exit
3. Restart systemd
   1. systemctl daemon-reload
4. Set to enabled
   1. systemctl enable splunk
5. Create splunk.sh service script
   1. Copy the values below
   2. nano /opt/superna/cgi-bin/splunk.sh
   3. paste the script values below into the nano editor
   4. Save the file
      1. press control+x
      2. Answer yes to save and exit the editor
6. Once the script is created below,  do not start the service at this step.
7. Done

Configure the python packages and customize the splunk integration python code

1. Now install required python packages for the SCA users that will run the service.  NOTE this is the same user that runs the main eyeglass application code. NOTE: This installs flask library for webhook support in python, boto3 for API support with AWS and gunicorn a multi threaded web server to run and execute the python application.
   3. su - sca
   4. pip3 install flask
   5. pip3 install boto3
   6. pip3 install requests
   7. exit 
   8. NOTE: you must type exit to ensure you are the root user for the remaining steps.  Type whoami to make sure you are the root user.
2. Customize the application code by downloading the python code from this link to download
   3. Open the python template file in a text editor. NOTE: make sure to only replace the values and do not delete any of the commas
   4. Locate this section in the file
      3. SPLUNK_HEC_TOKEN = 'xxxxxxxxxxxxxxxx'
      4. SPLUNK_HEC_URL = 'https://yyyyyyyyyyyyyy:8088/services/collector/event'
      5. Replace the xxxxx with the HEC token recorded from the steps above and replace yyyy with the IP address of the Splunk server running the HTTP Event Collector
   5. Paste the updated script into the production file
      3. nano /opt/superna/cgi-bin/splunk.py
      4. Open the file locally in Windows OS notepad and use control-A or select all the text.
      5. Paste the clipboard into the ssh terminal session with the open nano editor file
      6. save the file
         3. press control+x
         4. Answer yes to save and exit the nano editor 
3. Start the service and verify it is running
   1. systemctl start splunk
   2. systemctl status -l splunk 
   3. Verify the service is started successfully and returns "active and running".
   4. 
      
4. If the service does not start do not proceed and double check the steps above are completed. 

Configure Defender Zero Trust Webhooks

1. The next step creates an Zero Trust Webhook URL.    
   
   1. Configure Zero Trust endpoint in Ransomware Defender Zero Trust tab.
      1. Recommended Configuration: Only Critical and Major events and only the webhooks that set lockout or delayed lockout.   Customers can customize based on specific requirements. The goal is to send findings versus a list of alarms that do not pinpoint a security incident.
      2.   
      3. The endpoint url above will use localhost and will send Webhooks to the application service listening on port 5000. URL to use in the configuration
         1. http://localhost:5000/splunk
         2. Add the Content-Type header with value of application/json as shown above to complete the webhook configuration.
         3. Click save to commit the configuration.
         4. Click save on the main Webhook configuration page
2. Test the configuration is working following the next section

How to test the Integration with Splunk Enterprise

1. To test the integration follow these steps
   1. download this curl command template and open with a text editor
   2. Copy all the text
   3. ssh to the eyeglass vm as the admin user
   4. paste the entire cli command to the ssh prompt to send sample data to the running Zero Trust application.  This will send test data directly to the application to be processed and sent to Splunk Enterprise.
   5. The output of a successfully processed webhook test will return this text in the ssh terminal
      1. done sending event to Splunk and check for http 200 and success count in response
   6. How to review the process logs from the web application
      1. sudo -s 
      2. journalctl -f -u splunk 
      3. This allows you to view the logs generated by the application.
      4. To log to a file and review with nano showing only the most recent 250 lines.
      5. journalctl -f -n 250 -u splunk  > /tmp/ztwebhook.log
      6. nano /tmp/splunk.log
      7. In the log below the response code from the Splunk Enterprise api call should show http 200 status code and successCount 1 to indicate the Finding was successfully created.
      8.   You can also view splunkd.log for specific error messages.
   7. Login to the Splunk Enterprise Main Dashboard to run a search for all supernaZT source messages.  enter a search with sourcetype="supernaZT" to locate events sent from testing.
      1.    
      2.  
2. Done

Splunk Enterprise SecOps administrators Integration Experience

1. Once the integration has been tested, it is possible to build triggers from indexed parsed zero trust fields.

How to Configure Data Security Essentials

Overview

Data Security Essentials saves alerts to the Windows Event log to a custom application log.   This integration will extract the log entries and the json payload contained within them and format as webhook data to send to Splunk to issue the host containment to the endpoint provided by Splunk HEC endpoint.

Requirements

1. Follow the Splunk integration steps above to configure the Scripted REST API endpoint integration.
2. Install python for Windows on the Data Security Essentials host and allow the installation to add python to the path variable. https://www.python.org/downloads/windows/
   1. NOTE: Ensure it's installed for all users so service accounts have access, by enabling for all users during installation.
3. The integration code matches event log entries from Custom trigger BOT policies and File management policies
   1. The event log source names are below 
      1. Superna Data Security Essentials BOT Service
      2. Superna Data Security Policy Engine 

Features

1. Logs for each execution are stored in the path below along with the python code. 
   1.  
2. Each execution will store the event log record ID and date time and event number of the last processed event in a file called last_processed_record.txt.  This file is read on each execution to continue processing from last record.  The record number is not visible in Event Viewer, the date time and event number can be used to understand where in the log the last processing occurred.
3. The schedule sync runs every minute and only processes events since the last record and logs all processing as well as event matches in the log file for debugging.
4. Only events that match the severity filter will be processed for client ip address.   The sample code defaults to Major and Critical severity events all other severities will be ignored. 

Steps to Configure Event Sync to Splunk Host Containment IntegrationDownload the integration code from here.

1. Modify the python and update the Splunk integration settings highlighted in yellow to match your environment.
   1. #Splunk HEC authentication
      SPLUNK_HEC_TOKEN = 'xxxxxxx'
      SPLUNK_HEC_URL = 'https://yyyyyy:8088/services/collector/event
      
      
      # Configuration
      server = 'localhost'
      log_name = 'Data Security Essentials'
      source_names = ['Superna Data Security Essentials BOT Service', 'Superna Data Security Policy Engine']
      trigger_severities = ['MAJOR', 'CRITICAL','WARNING'] # Configure severities that should trigger event forwarding
2. Copy the Python code into C:\Program Files\Superna\cgi-bin on the Data Security Essentials host and name the file splunk-dse.py
3. Install python dependancies
   1. MANDATORY Step: Open a command prompt as the service account (hold down shift and right click the command prompt icon) run as  option to login as the service account dse-service.  This is required to install the python packages under the correct user account.
   2. Install the dependencies with this command
      1. pip install pywin32 requests
   3. This command installs the windows API python libraries needed to read the event log.
4. Test the script can correctly reach your Splunk instance from a dos prompt
   1. cd C:\Program Files\Superna\cgi-bin
   2. python  splunk-dse.py
      
   3. The output should display event log entries that match  and  will ignore severities that do not match script settings .  It will display the event payload details in json format and will show the data that was sent to the Splunk endpoint.   
   4. Example output
      1.   
   5. You can also review this file last_processed_record.txt to see the last date and time within the event log and event id that was last processed.  This file can be deleted to have the script start reading from the oldest windows event log entry.  This can be done for testing puposes.
   6. The above examples shows the output when a client ip address is successfully located and the host containment api returns success.
   7. The example below is logged when event data is processed and inspected for severity and forwarded with key fields parsed and extracted.
      1. Right Click here to download and save and open. 
      2. 
         
5. Steps to Schedule sync alerts into Splunk Security Incidents 
   1. Open secpol.msc, Local policies
   2. Give the Data security essentials AD service account the  "Logon as a batch job"  OS permission.
   3.   
   4. Create a file 
6. Right click this link to save to a file. The task scheduler XML file to import into Task Scheduler
   1. You can rename the xml from Servicenow alert sync.xml to Splunk integration.xml. 
7. Open Task Scheduler
   1. Run taskschd.msc, right click task schedule library 
   2.   
   3. Import the xml file
   4. You will be prompted with the task definition, you will need to change the service account to match your service account name and domain and re-enter the password to save the task scheduler.  All other settings are configured and match the path and settings used above.  Adjust any parameters that you changed if you did not use all the steps above. 
      1. Edit the name "Splunk integration"
      2. Click the Actions tab and browse to select the file C:\Program Files\Superna\cgi-bin\splunk-dse.py
      3. Click ok to save 
      4.   
   5. done.