Eyeglass Alarm forwarding Syslog Guide

Overview:

Alarms can be forwarded over syslog.     This guide explains how to filter and then forward alarms matching certain criteria.    This guide contains the most common examples of how to filter by application or by severity of the alarms.

Limitations

1. Syslog are limited in what information can be sent to these protocols.   Email alerts will contain more information not available over Syslog
2. The intent of SNMP forwarding is to make basic alarm type and severity, and detailed alarm data available in the GUI or via email that supports more text and attachments.  This alarm solution provides notification of an alarm, the application that generated it, and the severity.
3. Only the documented forwarding solutions below are supported.   

Deprecation Notice

1. SNMP forwarding has been deprecated as of 2.5.8.1
2. NOTE: /var/log/messages will be deprecated and no longer supported in an up coming release as a log to use for forwarding alarms. In a release 2.5.7 syslog will not be used and dedicated alarms log with syslog-ng.
   

Supported Alarms

1. Eyeglass alarms listed here

Requirements:

1. Eyeglass OVF version 2.5.8 or greater. Open suse 15.1 or later.  Upgrade to the latest OVF if required with the guide here.

Configuration of SYSLOG Forwarding 

1. The new alarm architecture will use a dedicated log that will roll over and provide alarm history external from the database and alarm history available in the GUI.  Release 2.5.7 or later is required.
2. Review all the filter examples to match your requirement and replace the filter with one of the following scenarios.
   1. How to Filter by Severity, Application, or Alarm codes (recommended)
      
3. Log location
   1. /opt/superna/sca/logs/igls_alarms.log
4. Configure Syslog Forwarding
   1. Ssh to the appliance as admin user
   2. Sudo -s
   3. Enter admin password
   4. nano /etc/syslog-ng/conf.d/superna.conf
   5. The example below is going to forward specific ransomware defender events.
      1. Paste the text below into the file and change the text as follows:
      2. replace x.x.x.x with the ip address of the syslog server ip address you want to forward messages
         1. NOTE Run this command  syslog-ng --version and use the version number for Config Version field
            1. 
               
         2. NOTE: Add the version number returned in the first line @version: x.xx

                           @version: 3.29 

                               filter  f_superna {

                                 };

source igls_src { file("/opt/superna/sca/logs/igls_alarms.log"); };

destination logserver { udp("x.x.x.x" port(514)); };

log {

    source(igls_src);

    filter(f_superna);

    destination(logserver);

};

How to Filter and Forward alarms

This section provides examples of how to filter for alarms to forward to syslog.

How to forward by alarm Severity

To combine multiple Alarm severities or combine message strings see example below:

.

filter f_superna {

    message("Severity:CRITICAL") or message("Severity:MAJOR") ;

};

How to Forward by Alarm code (Recommended and Supported Method)

Use this filter example to the best option to forward exactly the alarms you need using the alarm code guide.  All possible alarms are listed and provides the best option to simplify forwarding exactly the alarms you need to external systems.  Get the Alarm codes and use them in the filters.

filter f_superna {

    message("RSW0002") or message("RSW0011") ;

};

How to Filter by Application

Each Eyeglass application has an alarm code to easily forward alarms based on the prefix.

1. Ransomware Defender prefix - RSW
2. Easy Auditor Prefix - EAU
3. DR - SCA

How to Forward Ransomware Defender User Lockout and Restored Alarms Except for Security Guard Alarms

In the example below 2 commonly used Ransomware Defender alarms are needed.

1. User locked out is RSW0002
2. User access restored is RSW0011
3. NOTE: Replace the yellow highlight with the security guard service account that you have configured.
   

filter f_superna {

    (message("RSW0002") or message("RSW0011"))  and not message("igls-sg") ;

};

This example forwards all Ransomware Defender and Easy Auditor alarms

filter f_superna {

    message("RSW") or message("EAU") ;

};

This example forwards all Ransomware Defender alarms Except for Security Guard alarms

NOTE: Replace the yellow highlight with the security guard service account that you have configured.

filter f_superna {

           (message("RSW")) and not message("igls-sg");

How to Integrate Ransomware Defender Events with a SIEM

1. The syslog message alarms generated by Ransomware Defender when a user is detect with Ransomware an alarm includes details with user ID, ip address and a subset of some of the files that were detected. The ip address can be used in a SIEM trigger to find the Ethernet port of the IP address and disable the port.  See the example message format below.
2. Use the yellow highlighted sections below to build your parsing and trigger to capture the user name and PC IP address.  Using this information build a trigger in your SIEM to take action on the Ethernet port the PC is connected.
3. [DEBUG] IGLS_ALARMS:168 - Eyeglass, , Event: 2021-02-26 19:28:23.916, AID:AD02\sgdemo, Port:Nil, Type:null, EntityType:, Extra Data:{"clientIps":"172.31.1.65","info":"Successfully locked out user AD02\\sgdemo"}, Description:Locked user access.172.31.1.65, NSA, Severity:CRITICAL, Impact:false, Category:RSW0002 

Syslog format examples to be used for Parsing with a Syslog server

How to search the Eyeglass appliance logs for examples of syslog alarm formatting

1. Login to eyeglass vm over ssh as admin
2. cat /opt/superna/sca/logs/igls_alarms.log

Example alarm formats

[DEBUG] IGLS_ALARMS:168 - Eyeglass, , Event: 2021-02-26 20:15:23.983, AID:\ifs\data\dfsdata\dlp\, Port:Nil, Type:null, EntityType:, Extra Data:{"reason":"There is no smart quota for /ifs/data/dfsdata/dlp/ limited by a Data Loss Prevention threat detector. no limit is enforced."}, Description:There is no smart quota for a path limited by a Data Loss Prevention threat detector , NSA, Severity:MAJOR, Impact:false, Category:EAU0005 

[DEBUG] IGLS_ALARMS:168 - Eyeglass, , Event: 2021-02-26 19:28:14.496, AID:AD02\sgdemo, Port:Nil, Type:null, EntityType:, Extra Data:{"clientIps":"172.31.1.65","event severity":"CRITICAL","user name":"AD02\\sgdemo","affected files":"\\\\prod8\\System\\ifs\\igls-securityguard\\igls-securityguard-test-file-1614385692201.iglsrswtest","affected Isilon clusters":["prod8"],"detectors":"THREAT_DETECTOR_06","number of affected files":"1","info":"Lockout required."}, Description:Ransomware event received. Event severity: CRITICAL, user: AD02\sgdemo172.31.1.65, NSA, Severity:CRITICAL, Impact:false, Category:RSW0001

[DEBUG] IGLS_ALARMS:168 - Eyeglass, , Event: 2021-02-26 19:28:23.916, AID:AD02\sgdemo, Port:Nil, Type:null, EntityType:, Extra Data:{"clientIps":"172.31.1.65","info":"Successfully locked out user AD02\\sgdemo"}, Description:Locked user access.172.31.1.65, NSA, Severity:CRITICAL, Impact:false, Category:RSW0002

How to Trouble shoot SYSLOG Forwarding

1. enable verbose logging
2. ssh to eyeglass as admin
3. sudo -s (enter admin password)
4. syslog-ng-ctl verbose --set=on
5. Check the statistics of the forwarding to the logserver label (this is the name assigned to the destination in all the examples)
6. syslog-ng-ctl stats | grep logserver
   1. If the counters are not incrementing or show zeros it means nothing has matched your filter and nothing was forwarded to your destination 
7. Reset the stats to zero to test forwarding again to help trouble sheet the processed counter incrementing
   1. syslog-ng-ctl stats --reset 

How to use packet capture to see syslog messages sent to your target syslog server

1. Use this command to monitor any udp syslog messages sent based on matching alarms
2. Login as admin
3. sudo -s (enter admin password)
4. Replace x.x.x.x in the command below with the ip address of your syslog server configured in the above settings file /etc/syslog-ng/config.d/f_superna.conf.   This command will NOT display any data until a packet is sent to your syslog server based on the matching logic configured in your filter.  Leave the command prompt running and continue to the next step.
   1. tcpdump -nnAs0 -i eth0 udp port 514 -v | grep -A 2 "x.x.x.x"  
5. Open new ssh session as the admin user leaving the other session running
6. run the random test alarm command, this command will create a random alarm (NOTE: The random alarm may not match your filter logic, adjust your filter logic to match on severity using the example above following all steps to edit the file and restart syslog-ng)
7. Run this command below several times until you see a packet appear in the first ssh session that is packet capturing all packets sent to your syslog server.  This will help troubleshoot your filter and allow monitoring in realtime for any packets that are sent.   
   
   1. igls test AlarmTest 
   2. You may also run this command to verify any matches processed by Syslog-ng filter logic
   3. syslog-ng-ctl stats | grep logserver
8. Repeat the test command and stats command to verify your forwarding is working.  Check your syslog server to verify the messages are appearing after you have verified the stats and packet capture show successful packets are sent.
9. Done. 

Syslog Format Documentation

Syslog messages sent by Eyeglass are based on the messages recorded in Eyeglass log file /opt/superna/sca/logs/igls_alarms.log. When they are  forwarded to the Syslog management server,  they are received and displayed as messages with several sections separated by comma.

Example:

This Eyeglass forwarded syslog message has the following format (each section is separated by “comma”)

Example: Alarm for Ransomware Event

Example: DR Failover

Example: Easy Auditor