Parsers
Table of contents
For each device integration/feed, Fluency will treat it as a logical “event stream”. Each event stream should be associated with a correct parser. Since Event processing is done on the server side, this configuration only needs to be done once, on the server, per each different event type.
Refer to the User Manual for a full list of currently supported devices. Fluency’s event parser utilizes open-source Grok patterns. If your device is not supported, a new parser could be added easily on request.
Event Parser Configuration:
Event processing configuration is in the “Event Streams” section under the “Admin” dropdown tab of the Main Menu Bar.
“Admin” => “Event Streams”
There are three event streams on the Event Stream Configuration page: “INPUT”, “SAVE” and “DISCARD.” In the above Figure, a default Event Pipe for embedded Suricata IDS events, named “suricata_alert_process” is already attached to the “INPUT” stream.
Another Event Pipe named “save_all_to_es” is attached to the “SAVE” stream. This filter will save the incoming event into the Elasticsearch database.
Event Stream Lambdas are an advanced feature; more information can be found in the User Manual.
Click the “+ NEW RULE” button in any stream to add a new parser to that stream.
Example:
Let’s start by adding a new Event Pipe for the Cisco ASA firewalls located at IP addresses “10.1.0.50” and “10.1.0.51”.
Click the “+ NEW RULE” button within the “INPUT” stream block.
First we fill the “Name” and “Description” field. Choose “save” for the Sink to save the imported events to the database. By default, a new pipe will be added to the head of the filter chain. For greater efficiency, we recommend putting a high bandwidth event type at the top of the filter chain.
Then we set the “Event Selector” section. Here we can define a list of “matches” on “@sender”, “@source” and “@tags” fields. An “Exclude” checkbox is available to “invert” the match result. Incoming event will be “selected” if all matches evaluated to be “true”. In case we want to match all events, “MatchAll” could be checked to create a “match-all” selector. Here we create a rule to match the “@sender” field with an IP address (10.1.0.50).
Multiple items can be added to emulate the “OR” logic. Press “Enter” to confirm addition. We now add another IP address for matching the @sender” field (10.1.0.51).
An additional item added. Press the “+ ADD” button to finish the rule, and add the changes.
The final step is the “Event filters” section; here we can add one, or multiple, predefined filters to be applied to the selected events.
For the Cisco ASA firewall, we choose filter “parser\string_asa” from the drop-down menu.
Click “+ ADD” to confirm the changes.
After clicking “+ SAVE”, a new Event Pipe is added to stream “INPUT” (See red arrow below). This new event pipe will be applied in real-time, and will begin working immediately.
Another example of a Fluency device with multiple parsers configured.
