Eyeglass Solutions Publication

Rapid7 InsightIDR Superna Zero Trust Alerts

Home


Overview

Customers using Rapid7 InsightIDR can leverage this integration to send real time zero trust alerts using webhooks and maintain full payload parsing using json defined fields.   The InsightIDR Network port syslog data collector will be used.   In order to leverage real time webhooks, a python integration application will be used process webhooks and format the data in syslog format to send to Rapid7.


Support Statement

  1. NOTE:  This documentation is provided "as is" without support for 3rd party software.  The level of support for this integration guide is best effort without any SLA on response time.  No 3rd party product support can be provided by Superna directly.  3rd party components require support contracts

Limitations

  1. HTTPS with un-signed certificates is the default configuration and certificate checking has been disabled in this sample code.

Solution Overview

Superna Defender Zero Trust API is the cornerstone technology used to integrate with SIEM, SOAR and XDR platforms.   Automation begins with data that summarizes the threat and places that information into a security tools to be acted on by Secops and run playbooks to protect corporate IT assets from vulnerabilities and insider or external attackers.   In order to allow a InsightIDR to act on the data it must be parsed.   The InsightIDR platform can dynamically extract fields that are encoded in the alert payload.  The example code sends all fields in the webhook to Rapid7,  the application code could be customized to extract fields and only send a subset of the available data.


What is Rapid7 InsightIDR?

A cloud-based SIEM built for security teams in need of a solution that can quickly detect and respond to threats in today's ever-evolving hybrid and multi-cloud IT environments.


Integration Architecture



Solution Configuration in Rapid7 InsightIDR and Defender Zero Trust

Prerequisites

  1. Installed Superna Security Edition
  2. Eyeglass OS appliance version 15.5
    1. cat /etc/os-release
  3. License key for the Zero Trust API  or Subscription license type


Configuration in Rapid7 InsightIDR

  1. Login to the console and install a Data collector,  this example will use the Windows OS collector
    1. Select Add Raw Logs Custom Event Type, select the collector that will be used.
    2. Use an unused network port on the collector host
    3. Select Listen on Network port option and use 6675 or change to another port if this port is already used.
    4. Leave the protocol set to UDP
    5.  Check the box for RFC 3164  
    6.  Done


Configuration Steps on Eyeglass Virtual Machine

High Level steps

  1. Create python location to run the application on the Eyeglass vm
  2. Create python main application script
  3. Create linux systemd service and set to auto start
  4. Create Zero Trust configuration in Defender
  5. Update the main script to customize with Rapid7 InsightIDR  python code
  6. Test the script is running as a service
  7. Create a test event in Defender to validate the alerts appear as indexed parsed events in Rapid7 InsightIDR

Configuration Step by Step


Configure the Service start and python integration files
  1. Login to eyeglass vm using ssh as the admin user to create zero trust application
    1. sudo -s 
    2. mkdir -p /opt/superna/cgi-bin
    3. touch /opt/superna/cgi-bin/insightidr.py
    4. touch /opt/superna/cgi-bin/insightidr.sh 
    5. sca:users /opt/superna/cgi-bin/insightidr.*
    6. chmod +x  /opt/superna/cgi-bin/insightidr.py 
    7. chmod +x /opt/superna/cgi-bin/insightidr.sh 
  2. Create systemd configuration
    1. nano  /etc/systemd/system/insightidr.service
    2. Copy the values below and Paste these contents into the file opened in nano editor in the step above
    3. Save the file 
      1. press control+x
      2. Answer yes to save and exit
  3. Restart systemd
    1. systemctl daemon-reload
  4. Set to enabled
    1. systemctl enable insightidr
  5. Create insightidr.sh service script
    1. Copy the values below
    2. nano /opt/superna/cgi-bin/insightidr.sh
    3. paste the script values below into the nano editor
    4. Save the file
      1. press control+x
      2. Answer yes to save and exit the editor
  6. Once the script is created below,  do not start the service at this step.
  7. Done

### Service definition text to copy into /etc/systemd/system/InsightIDR.service. ###
[Unit]
Description=Webhook listener for Zero Trust api translations and integrations
After=network.target

[Service]
User=sca
Group=users
WorkingDirectory=/opt/superna/cgi-bin
Environment="PATH=/opt/.pyenv/shims:/usr/bin:/usr/local/bin"

ExecStart=/bin/bash /opt/superna/cgi-bin/insightidr.sh

[Install]
WantedBy=multi-user.target



copy for bash script


### service launch script /opt/superna/cgi-bin/insightidr.sh ### 
#!/bin/bash
export PATH="/opt/.pyenv/bin:$PATH"
/opt/.pyenv/shims/python3 insightidr.py


Configure the python packages and customize the InsightIDR integration python code
  1. Now install required python packages for the SCA users that will run the service.  NOTE this is the same user that runs the main eyeglass application code. NOTE: This installs flask library for webhook support in python, boto3 for API support with AWS and gunicorn a multi threaded web server to run and execute the python application.
    1. su - sca
    2. pip3 install flask
    3. pip3 install boto3
    4. pip3 install requests
    5. exit 
    6. NOTE: you must type exit to ensure you are the root user for the remaining steps.  Type whoami to make sure you are the root user.
  2. Customize the application code by downloading the python code from this link to download
    1. Open the python template file in a text editor. NOTE: make sure to only replace the values and do not delete any of the commas
    2. Locate this section in the file and update the yellow highlighted values.  Optionally change the facility and severity defaults used in the syslog headers. 
      1. SYSLOG_SERVER = 'syslog.server.address' # Replace with your Data Collector server address, keep the single quotes
        SYSLOG_PORT = 6675 # Replace with your Data Collector UDP port number configured for the Event Source
        FACILITY = 13 # 1 - User-level messages, 13 - Security audit log
        SEVERITY = 2 # 1 - Alert, 2 - critical, 3 - Error, 4 - Warning, 5- Notice, 6 - Informational message
    3. Paste the updated script into the production file
      1. nano /opt/superna/cgi-bin/insightidr.py
      2. Open the file locally in Windows OS notepad and use control-A or select all the text.
      3. Paste the clipboard into the ssh terminal session with the open nano editor file
      4. save the file
        1. press control+x
        2. Answer yes to save and exit the nano editor 
  3. Start the service and verify it is running
    1. systemctl start insightidr
    2. systemctl status -l insightidr 
    3. Verify the service is started successfully and returns "active and running".
  4. If the service does not start do not proceed and double check the steps above are completed. 

Configure Defender Zero Trust Webhooks

  1. The next step creates an Zero Trust Webhook URL.    
    1. Configure Zero Trust endpoint in Ransomware Defender Zero Trust tab.
      1. Recommended Configuration: Only Critical and Major events and only the webhooks that set lockout or delayed lockout.   Customers can customize based on specific requirements. The goal is to send findings versus a list of alarms that do not pinpoint a security incident.
      2.   
      3. The endpoint url above will use localhost and will send Webhooks to the application service listening on port 5000. URL to use in the configuration
        1. http://localhost:5000/webhook
        2. Add the Content-Type header with value of application/json as shown above to complete the webhook configuration.
        3. Click save to commit the configuration.
        4. Click save on the main Webhook configuration page
  2. Test the configuration is working following the next section

How to test the Integration with Rapid7 InsightIDR

  1. To test the integration follow these steps
    1. download this curl command template and open with a text editor
    2. Copy all the text
    3. ssh to the eyeglass vm as the admin user
    4. paste the entire cli command to the ssh prompt to send sample data to the running Zero Trust application.  This will send test data directly to the application to be processed and sent to Rapid7 InsightIDR Data Collector.
      1. It can take upto 5 minutes for the data collector to forward events to the portal.  Once webhooks are processed. The log search will show the records and using json format the key value pairs are displayed.
        1.  

    5. The output of a successfully processed webhook test will return this text in the ssh terminal
      1. done sending event to InsightIDR and check for http 200 and success count in response
    6. How to review the process logs from the web application
      1. sudo -s 
      2. journalctl -f -u insightidr
      3. This allows you to view the logs generated by the application.
      4. To log to a file and review with nano showing only the most recent 250 lines.
      5. journalctl -f -n 250 -u insightidr  > /tmp/ztwebhook.log
      6. nano /tmp/insightidr.log
      7. In the log below the response code from the Rapid7 InsightIDR api call should show http 200 status code.
      8.   You can also view InsightIDRd.log for specific error messages.
    7. Login to the Rapid7 InsightIDR view the event source 
      1.  
      2. Select the collector used for the integration and click view raw log to verify data has been recieved by the Rapid7 portal from the on premise collector.
      3.  
  2. Done


How to Create Detection Rules in Rapid7 InsightIDR

Overview

This Log Pattern Detection rule configuration will create critical incidents since the Zero Trust alert is not a log entry it represents a root cause detection, the log entries can be set up a trigger to create critical incident in InsightIDR. 


How to Configure Detection Rule in InisightIDR

  1. NOTE: Ensure log entries have been tested and visible in the log search before starting this configuration.
  2. Run a search using the severity query filter and select the Superna Zero Trust log collector.
    1. Use this query syntax where("severity" = "CRITICAL") 
    2. Select the 3 dots to create a Log Pattern Detection Rule
    3.  
  3. To Create Log Pattern Detection Rule triggers follow these steps
    1.  
    2.  
    3. Click Create and then view the detection rules page to verify it is enabled as per below image.
    4.  
  4. Procedure Completed.   When new log entries arrive they should trigger the creation of Critical incidents.



Example Incident in InsightIDR


  1.  
  2. Evidence log
© Superna Inc