Search

Ad Hoc Searching: Global Pages

Table of contents

1. Search
   1. Ad Hoc Searching: Global Pages
   2. Events and Flows
   3. Message Flow

Events and Flows

There are two main databases that messages (logs) are stored in:

• Event Database: Contains the incoming message and how it got there (its metadata)
• Flow Database: Contains merged messages that share a common tuple along with enhanced data.

Both databases are useful and serve different purposes. Where in the processing of data the database is populated changes what is in the databases.

Message Flow

Theoverview->charts->sankeydiagram shows an active sankey diagram of information flowing through the system.

On the left side of the diagram is information entering the system.

Metaflow Engine data is the Fluency protocol analyzer adding flow data to fill in the gaps that are missed by the security tools.

1. The boxes below that are data sources provide messages. The information is forwarded to the stream INPUT.
2. The message collector (stream INPUT) examines the message and sends it to the correct parser.
3. The parsers
   1. normalize the data into key-value fields producing a JSON document
   2. define fields that are sensitive and have the value replaced with a Pseudonym.
4. The record and how the data is collected is
   1. placed in the Event Database
   2. sent to the Fusion engine.
5. The fusion engine
   1. enhances the record with table and third party lookups
   2. merges the record with other records that share the same network tuple.
6. Before the record is stored, the RiskScore processor provides a confidence score and saved that data in the Flow Database.

---

Table of contents

• Defining the Query
• Flow Search
• Event Search