ECA Cluster Monitoring Tools

NOTE: All tools listed here are for support only and not for normal administration, only use if directed by support.

The ECA is built on docker containers with each container providing a different function to process messages. If both Ransomware And Auditor are licensed on an ECA cluster the following additional UI’s exist for monitoring HDFS write performance, spark job analysis jobs.

Note x.x.x.x is any node in the cluster, the master function only runs active on one node and can move from one node to another based on voting on startup.

HBASE Database Monitoring UI

http://x.x.x.x/hbase-master (master controls all region servers and provides overall cluster configuration. One node is elected the master but all ECA nodes can become HBASE Master that oversees each Region server.
http://x.x.x.x/hbase-rs (Region server controls a portion of the audit database and controls writes to this portion of the database)

Spark Monitoring UI

http://x.x.x.x/spark-master - Spark master shows all running queries across the cluster, each node runs
http://x.x.x.x:spark-worker - Spark worker shows running queries on each node in the cluster and the number of tasks to complete along with job logs and errors
http://x.x.x.x/spark-history - Spark History server provides interface to see all searches done across the cluster and all logs related to these searches for debugging. The searches are encoded by ID visible in Eyeglass reports.

Spark History Server Report logs Monitoring UI

http://x.x.x.x/spark-history - Spark queries have logs stored on the PowerScale for each query for diagnostics and reviewing post query execution. This UI can be used to download query logs based on the Auditor report driver ID number. This will also be available for download in the Eyeglass report table as well.

Kafka Monitoring UI

http://x.x.x.x:kafkahq/ - Kafka monitoring of message event processing, queues depths, event rates. On ECA node one only open ui click add cluster enter zookeeper:2181 for the cluster to have auto detection find the correct Kafka broker to add. Note only 1 UI is required to monitor all brokers. This process handles events for fast analysis and writing to the Analytics DB