Scripted API Searches with Search & Recover GraphQL API
- Overview
- Scheduled Content Search with Email Alerts Example
- Description:
- Script:
- Scheduled File Count Monitoring in a Folder with email alerts Example
- Description:
- Script:
- Monitor a Directory for files being created in the last hour
- Description:
- Script:
- Setup a script to run on a Schedule with Cron
- How to Search for last modified date files and save the results to a file (sample code provided)
Overview
The API can be used to automate searches and apply logic to the results. Several examples are available below.
- Search for credit card or SSN on newly added files and send an alert email on any hits.
- Monitor a folder for a threshold number of files and send an alert email if the threshold is crossed.
Scheduled Content Search with Email Alerts Example
Description:
To search file with specific path that contain specific Social Security Number (SSN) (i.e: filePath:"/ifs/data/search3/folder1",content:"333-44-5555")
If found that file, send e-mail by utilizing mailx function on Eyeglass Search node.
Script:
Login to any Eyeglass Search node as ecaadmin user.
Create a script (Example: /home/ecadmin/contentsearchtrigger.sh).
Bash Script:
======================
#!/bin/bash
## declare mail variables
##email subject
subject="Eyeglass Search Found file with requested SSN"
##sending mail as
from="eyeglassSR@exampledomain.com"
## sending mail to
to="admin1@exampledomain.com"
## send carbon copy to
also_to="admin2@exampledomain.com"
## token
token=$(curl -s -G -k https://<eyelgassSR-node1-IPaddress>/graphql --data-urlencode 'query={
login(id:"searchuser@exampledomain.com", pass:"NotReal!") {
token
}
}'| awk -F'"' '{print $8}')
## Search
found=$(curl -s -Gk -H Authorization:"Bearer $token" https://<eyelgassSR-node1-IPaddress>/graphql --data-urlencode 'query={
fileInfo(filePath:"/ifs/data/search3/folder1",content:"333-44-5555") {
pageInfo {
totalNum
}
}
}'| tr -dc '0-9')
## check if found
if [[ "$found" > 0 ]]; then
## send email if file with requested SSN found
echo -e "Found\n\nthe requested Social Security Number" | mailx -s "$subject" -r "$from" -c "$to" "$also_to"
fi
exit 0
Change mode to executable:
chmod +x contentsearchtrigger.sh
Run the script:
./contentsearchtrigger.sh
Check e-mail.
Then follow setup on a schedule instructions.
Scheduled File Count Monitoring in a Folder with email alerts Example
Description:
This example will count the number of files in a folder (Example: Path:"/ifs/data/search3/folder1"). If the number of files is greater than threshold, send e-mail by utilizing mailx function on Eyeglass Search node.
Script:
- Login to any Eyeglass Search node as ecaadmin user
- Create a script (Example: /home/ecadmin/foldermonitor.sh)
- Requires modifications for emails and password to authenticate to the index
- Can use eccaadmin user
- The depth property is how many folder levels the search should use. This example /ifs/data/path1/path2 is a depth 5 to limit the search to files in path2 folder, level 5 depth means the folders equal a depth and the file name is also included in the depth value. Adjust the script to use a depth value equal to the path used in the search. See bolded yellow highlight in the script to set the depth value of the search. Note: This is not a recursive search and will not count files below the path in the search.
- Change mode to executable
- chmod +x foldermonitor.sh
- Run the script
- ./foldermonitor.sh
- Check e-mail
- Then follow setup on a schedule instructions.
Monitor a Directory for files being created in the last hour
Description:
- To count the number of files created for the last 1 hour in a folder (i.e.: Path:"/ifs/data/search3/folder1") .
- If the number of files is zero, send e-mail by utilizing mailx function on Eyeglass Search node.
- Time format: Epoch .
Script:
- Login to any Eyeglass Search node as ecaadmin user .
- Create a script (Example: /home/ecadmin/monitornewfileslasthour.sh).
- Change mode to executable.
- chmod +x monitornewfileslasthour.sh
- Run the script:
- ./monitornewfileslasthour.sh
- Check e-mail.
- Then follow setup on a schedule instructions.
Setup a script to run on a Schedule with Cron
- Login to node 1 as ecaadmin use examples below for hourly, daily or use a different cron tab string. Find examples here.
- Type crontab -e .
- Press letter i (to insert).
- Copy and paste the job information into the editor.
- Then press esc key.
- Then press :
- Followed by wq (write and quit).
Example cron entries:
- Runs hourly:
- 0 * * * * $HOME/myscript.sh (note $HOME = /home/ecaadmin path)
- Runs Daily at midnight:
- 0 0 * * * $HOME/myscript.sh
How to Search for last modified date files and save the results to a file (sample code provided)
- Use this sample code in python to search for modified files between 2 dates and save the results to a file.
- Download the sample here.
- If using python3
- pip3 install requests
- or python2
- pip install requests
- How to customize the script for your environment
- the script uses ecaadmin and default password change values based on your installation
- The url must be changed to use node 1 IP address on the Search & Recover appliance
- Search for url in the file and edit the ip address to match your installation
- Search for modStart and modEnd and set the date and time using epoch time
- Use an epoch conversion tool to get the numeric value of the dates. tool here.
- change the file name by searching for data_file.csv and change the file name
- How to execute the script (python3)
- python3 <file name>.py