Fluency has introduced a new, robust method of building queries for both the Event Search and the Event Watch pages. Previously, the Event Search queries were built using either Lucene queries or Regular Expressions. While these were sufficient, they were also limited in some cases. The new mechanism allows more complex queries to be used.

 

Query Tree Structure

The new query tree structure is built using an ordering similar to reverse polish notation. As such, queries are built downward will Booleans coming before individual operations.

Here is a somewhat complex example of an LVDB Query taken from a SIGMA rule that was imported for detecting mimikatz:

First, notice that the query begins with a root element at the top. All other elements branch off of this root.

After the root, typically, there are one or more Boolean operations. These Boolean operations are essential to building complex queries as they ties the leaf elements together. While technically a query can consist of just a single leaf element, that is typically not the case. Cases where a single leaf can be used may suffice just as well being a Lucene or Regular Expression query.

Each Boolean operates on all of its child nodes. If there will be multiple top-level conditions (for complex queries), then there will most likely be nested Booleans. See below for groups of Booleans and their children (colored circles).

Each color represents a nested level from the root to the innermost query. In this manner complex queries can combine booleans AND/OR.

Build the tree in Event Search

To build a query, it is recommended that this be done initially in Event Search in order to verify the results being sought prior to finalizing in Event Watch.

On the Event Search page, look for the LVDB Query checkbox near the top right of the page.

Once this box has been checked, the LVDB Query Builder will be shown on the page. The three buttons on the top left of the query builder are Delete, Copy, and Paste. These are useful for starting over, pasting in an already created query (such as from Event Watch) or copying the query to be saved/used elsewhere (such as creating an Event Watch behavior).

The root element will also be shown with no children.

To add a child element, press the Plus (+) button to the right of the root node.

This will provide a dialog for adding a query filter.

In most cases, at the root, this will typically be a Boolean so that more query filters can be added.

Once the Boolean is added, the tree will be updated with it as the new child of root.

At this point, it becomes a matter of continuing to build the query out either by adding more Booleans to adding the combination of leaf filters needed to provide the results being sought. Once this has been completed, set the time frame and click search. If the desired results are shown then the query may be complete. Otherwise, make changes as needed and re-test until the desired results are shown.

Copy the Query to a new Behavior (Event Watch)

Once the query has been built and tested, use the Copy button from the Query Builder to make a copy of the completed query.

Switch to the Event Watch page (if the intent is to build a behavior from this query). Create a new behavior as normal (clicking the Plus button on the top right). The only difference now is the need to paste the LVDB query into the behavior as it is built. Look for the LVDB Query checkbox under Selection Criteria, and check it. The same three buttons should be shown as were shown on the Event Search page. In this case, click the Paste from Clipboard button to open the past dialog box. Past the text (currently in the clipboard) into the dialog box and click Save.

Now, after verifying that the LVDB Query tree is in the new behavior dialog and correct, complete the rest of the behavior as needed.

Summary

While this post explains the basics of the LVDB Query feature that was added, it is just meant as the starting point for users to test different creations and complex queries. It may seem daunting at first, but with practice, it will become almost second nature for most analysts in creating complex queries for their searches and custom behaviors.

Appendix – Query Filter Options

• bool – this can be the boolean AND or OR an operates upon leaf elements below this node, including other booleans. This is the only non-leaf element in the tree aside from the root
• content – the entirety of the field specified must match the criteria
• contains – the field must contain the criteria within its data
• startswith – the field starts with the criteria
• endswith – the field ends with the criteria
• range – specifies a range for the field
• regexp – a regular expression criteria for the specified field
• entityinfo – the contents of the field must be contained in the specified entityinfo list
• wildcard – matches anything in the field that is not null
• isnull – the field is null

MODIFIER – EXCLUDE – on any of the above filter options, the Exclude checkbox may be checked in order to provide the opposite effect (such as does not contain or is not null).

CASE INSENSITIVE – All filter items are case insensitive.