Filters

How filters work

A filter is a saved view of your events. It sets two things: which sources to include, and which events from those sources to keep. Once you save a filter, you can open it in the Log Explorer in one click, and an alert can watch it.

To create a filter, open Filters in the sidebar and choose Add filter. Only Name is required, and each name can be used once per account. Everything else is optional.

Users with Read Only access can see filters but cannot add, edit or delete them.

Scope: which sources

The scope is set on the Details card with three fields. You can use any combination of them:

• Sources - specific sources, picked by name. The filter always includes these.
• Source pattern - a pattern matched against the whole source name, where * stands for any run of characters. web-*.example.com covers every source named that way, including sources that appear later. * is the only wildcard, and the match ignores upper and lower case.
• Environments - every source tagged with one of the chosen environments. A source tagged later joins the filter at that point.

A source that matches any one of the three fields is in scope. If you leave all three empty, the filter covers every source in the account.

A source marked Pending approval is left out of Source pattern and Environments matches, and out of a filter that covers every source. It joins those filters once it is approved. Picking it by name under Sources includes it straight away.

Conditions: which events

Conditions keep only some of the events from the sources in scope - for example, messages containing some text, a set of levels, one program, or a test on a structured field. Conditions explains each kind and how they combine.

Conditions are optional. A filter with no conditions keeps every event from its sources.

Using a filter

In the Log Explorer, filters are listed in the source picker, and Open in Log Explorer on a filter's page opens it directly. Choosing a filter applies its scope and conditions. The other controls still work on top of it, so you can search, pick a level or add a field condition within the filter.

In alerts, an alert watches one filter. Subscribe on a filter's page starts a new alert for it. The alert always uses the filter's current definition, so when you edit the filter you do not need to change the alert.

On a source's page, the Filters entry lists every filter that covers the source, whichever part of the scope matched, with a link to each. To change which filters include a source, edit those filters - the full scope is shown on the filter's form.

Editing and deleting

When you edit a filter, every alert watching it uses the new definition from its next check.

Delete filter is refused while an alert still watches the filter. Delete that alert, or change it to watch another filter, and then delete the filter.

Conditions

Conditions decide which events a filter keeps. They are arranged in groups:

• An event must match every group.
• Within a group, matching one condition is enough.

This is the only combining rule, and it covers most views. For example, "errors or criticals, from the payments program, mentioning a timeout" is three groups: a level group, a program group and a message group.

The condition kinds

Each condition row starts with a field picker. The field decides which operators the row offers.

Every operator has an opposite ("does not contain", "is none of", "is not"), which matches exactly the events the positive operator does not.

To keep unwanted events out of a filter, add a group holding one negated condition, such as Program is not health-check.

Redaction with is present matches every line any detector marked, including lines marked by detectors that are switched on after the filter is saved. Use it for a filter or alert on "any personal data" rather than listing every detector. Detectors mark lines in both the Detect only and Redact modes.

Pattern with is none of keeps a noisy message pattern out of the filter. Is any of keeps only the lines of the patterns listed, such as one cluster from Log Analysis.

Field conditions

An Attribute condition tests one structured attribute, named by its dotted path such as context.order.id, status or request.method. The form suggests paths already seen in your account, and you can type any other path.

The operators are is, is not, contains, does not contain, is present and is absent. They match exactly as the same operators do in a query expression, where each one is described. Two points matter most when building a filter:

• is compares the value's text exactly, including upper and lower case, and numbers compare as text, so status is 500 matches a numeric 500 and a string "500". An object or a list at the path never matches is or contains.
• is not and does not contain also match events where the path is missing. To match only events that carry the attribute, add a second group with is present.

Building them

On a new filter, the Conditions card starts empty. Choose Add conditions to add the first group.

• Or condition adds an alternative to a group. The event then needs to match only one of the group's conditions.
• And group adds a new group. The event must match this group as well as the others.
• Remove condition (the x beside a row) removes that condition. A group left with no conditions is removed.

A filter can hold up to five groups, and each group up to five conditions.

Every row needs a value before you can save, except is present and is absent. A row left blank is shown as an error rather than dropped, because dropping an alternative would let more events through.

The Log Explorer uses the same structure. Its search, level set, program and each field condition are one group each, so an expression copied from its advanced search has the same meaning in a filter.

Editing as text

Edit as text on the Conditions card shows the conditions as one expression. It uses the same grammar as the Log Explorer's advanced search, described in full under Query expressions. Edit as builder turns the text back into rows.

To save a Log Explorer search as a filter, copy the expression from advanced search and paste it here.

For example, this expression becomes three groups in the builder - the two programs as alternatives, the level set, and the excluded route:

(program is php-fpm or program is laravel) and level is error+ and route does not contain "/health"

Things to know:

• Saving from the text view saves what the text says.
• If the expression does not parse, the error appears under the box with its reason and character position, and nothing is saved. The text view stays open until the expression parses.
• The text view has the same limit as the builder: five groups of five conditions.
• Rows left blank in the builder are left out when you switch to the text view.