Query expressions

What an expression is

An expression is a way to write event conditions as text. You can type one in three places, and it means the same in each:

• The Log Explorer's advanced search.
• A filter's Conditions card, after choosing Edit as text.
• The API's q parameter.

An expression is made of groups of conditions. An event must match every group, and within a group, matching one condition is enough. Filters, alerts and the Log Explorer's pickers use the same structure, so an expression can move between them without changing meaning.

program is nginx and (level is error or message contains "timeout") and request_id is present

This reads as: the program is nginx, and the event is an error or mentions a timeout, and the event has a request_id attribute.

Writing an expression

Shape

• Join conditions with and when all of them must match.
• Put alternatives in parentheses and join them with or. Matching one of them is enough.
• An expression that uses only or needs no parentheses: program is nginx or program is php-fpm.
• Keywords, and the field names message, level, program, pattern and redaction, can be written in any case: Program IS nginx AND Level IS error reads the same as the lower-case form.
• Spaces and line breaks between words make no difference.

Things that are refused, with the reason and the character position in the error:

• and and or side by side without parentheses. program is a or level is error and message contains "x" is refused, and the error says to wrap the or conditions in parentheses.
• and inside parentheses. Parentheses hold or alternatives only.
• Parentheses inside parentheses. A group holds conditions, not other groups.
• Two conditions with nothing between them. Every condition after the first needs and or or before it.

Fields and operators

A condition is a field, an operator and, for most operators, a value.

Field	Operators	Example
message	contains, does not contain	message contains "timeout"
program	is, is not	program is not cron
level	is, is not, is any of, is none of, and a level with +	level is warning+
pattern	is, is not, is any of, is none of	pattern is "Job ? failed"
redaction	is, is not, is any of, is none of, is present, is absent	redaction is present
an attribute path	is, is not, contains, does not contain, is present, is absent	context.order.id is 889

Every operator has an opposite, which matches exactly the events the positive operator does not. On level, pattern and redaction, is means is any of and is not means is none of.

You can also put not in front of a single condition to write its opposite: not program is cron is the same as program is not cron. not cannot go in front of a group in parentheses.

message

• message contains matches the text anywhere in the message. Upper and lower case are treated as the same for the letters A to Z. Other letters, such as é or Ä, must match their case exactly.
• A quoted string on its own is short for message contains: "payment failed" means message contains "payment failed", and not "payment failed" means message does not contain "payment failed".
• message takes only contains and does not contain. message is "x" is refused.

program

• program is matches the whole program name exactly, including upper and lower case. program is php does not match php-fpm.

pattern

• pattern is matches the event's message pattern exactly, against one pattern or a list: pattern is any of "Job ? failed", "Deploy ? finished".
• A message pattern is the message with numbers, IP addresses, UUIDs and long hexadecimal ids replaced by ?, and quoted text replaced by "?". Log Analysis lists patterns, and opening a cluster from there writes these conditions for you.
• Quote each pattern, because most patterns contain ? or spaces. A quote inside a pattern is written \": pattern is "User \"?\" logged in".
• pattern is none of in a filter keeps a noisy pattern out of everything the filter shows.

redaction

• redaction is matches events in which a redaction detector found something when the line arrived. Name detectors by their key: email, card, iban, phone, au_tfn, au_medicare, us_ssn, uk_nino, private_key, bearer, jwt, aws_key, vendor_key or secret_value.
• redaction is any of email, card matches every line that contained either.
• redaction is present matches every line any detector marked, including detectors switched on later. Use this form in a filter or alert for "any personal data". redaction is absent matches every other line.
• Detectors mark lines in the Detect only mode, where the line is stored as it arrived, and in the Redact mode, where the match was replaced.
• The counts on the Redaction page open the Log Explorer with these conditions.

attributes

• is compares the stored value's text exactly, including upper and lower case. Numbers and booleans compare as their text, so status is 500 matches a numeric 500 and a string "500".
• contains matches the text anywhere in the value, with the same case rule as message contains.
• is present matches when the path holds a value: a string (even an empty one), a number, a boolean or a list. is absent matches when the path is missing or holds null.
• An object at the path counts as absent, and neither an object nor a list matches is or contains. Attributes are addressed by their leaves, so test request.method, not request.

Negated operators and missing values

is not, does not contain and is none of also match events where the field has no value, because "not this value" includes "no value". Add is present as a separate condition when you want only events that carry the field. Excluding noise shows examples.

Levels

Levels run from least to most severe: debug, info, notice, warning, error, critical, alert, emergency. Level names can be written in any case.

• level is error - exactly that level.
• level is any of error, critical - any level in the list. level is error, critical means the same.
• level is none of debug, info - every level except those. level is not debug means level is none of debug.
• level is warning+ - that level and every more severe level. The + also works inside a list: level is any of notice, error+.

Attribute paths

A path names a structured attribute by its keys, with a dot between nested keys: status, request_id, context.order.id, request-id. Each key can hold letters, digits, underscores and dashes. A path can have up to 16 keys and 200 characters.

The quickest way to find a path's exact spelling is the Log Explorer's field dialog, which lists the paths seen in the events you are viewing.

Paths are matched with their exact upper and lower case.

If an attribute's name is also a field name (message, level, program, pattern or redaction) or a keyword (such as and, not, is or contains), in any case, put @ in front of it: @message is "x" tests an attribute named message, not the event's message, and @and is "x" tests an attribute named and.

Quoting

A value can be written without quotes when it is a single word made of the letters A to Z, digits and the characters _ - . : / @ +. For example: nginx, 500, /api/orders, req-1, user@example.com.

Put any other value in double quotes, including:

• A value with spaces or commas: program is "php fpm".
• A value with any other character, including letters outside A to Z: message contains "Ärger".
• A keyword used as a value: program is "and".

Inside quotes, write a double quote as \" and a backslash as \\: message contains "he said \"no\"".

The words present and absent after is are operators. state is present tests whether state has a value, and state is "present" tests for the word "present".

What an expression cannot say

• Time. An expression has no dates or time ranges. Use Go to time in the Log Explorer, or the API's time parameters.
• Comparisons. There is no greater-than or less-than, so duration > 2 cannot be written.
• Wildcards and regular expressions. contains is a plain text search, and * and ? in a value are matched as those characters. The one exception is ? in a message pattern, which stands for the parts Log Analysis replaced.

How much fits

• Filters and the API - up to five groups joined by and, each holding up to five conditions joined by or.
• The Log Explorer's advanced search - up to twelve groups, to leave room for the conditions from its pickers, and five conditions per group.

An expression over the limit is refused with the reason.

Examples

One condition

Expression	Finds
level is error	Errors only.
level is warning+	Warnings and every more severe level.
program is nginx	Everything nginx wrote.
"payment failed"	Messages containing "payment failed", in any case.
status is 500	Events whose status attribute is 500.
request_id is present	Events that have a request_id attribute.

Narrowing with and

Expression	Finds
program is nginx and level is warning+	nginx warnings and more severe levels.
program is laravel and "Exception"	Laravel lines that mention an exception.
route contains "/api/" and status is not 200	API requests that did not return 200, including requests with no status recorded.
level is error and request_id is present	Errors that can be traced to a request.

Alternatives with or

Expression	Finds
program is php-fpm or program is laravel	Lines from either program.
(level is error or message contains "exception")	Events at the error level, and events at any level that mention an exception.
(status is 500 or status is 502 or status is 504)	Events with any of the three server-error statuses.
program is nginx and (level is error or message contains "upstream")	nginx errors, and nginx lines at any level that mention "upstream".

Excluding noise

Expression	Finds
program is not health-check	Everything except the health-check program.
message does not contain "GET /up"	Everything except lines containing "GET /up", such as uptime checks.
level is none of debug, info	Notice and every more severe level.
route does not contain "/health"	Events whose route does not contain "/health", including events with no route.
status is not 200 and status is present	Events that have a status other than 200.

The last two rows show how negation treats missing values. is not and does not contain also match events where the field is missing, because "not this value" includes "no value". Add is present when you want only events that carry the field.

Following one request

Expression	Finds
request_id is "4f2c-9a"	Every event from one request, across all the sources you are viewing.
user.id is 7 and level is warning+	One user's warnings and more severe levels.
context.order.id is 889	Events whose context.order.id attribute is 889. The path must name the exact nesting.
trace_id is absent and level is error	Errors that have no trace id.

Combining everything

(program is php-fpm or program is laravel)
and (level is error or message contains "Exception")
and request_id is present
and route does not contain "/health"

This finds events from php-fpm or laravel that are errors or mention an exception, that belong to a request, and whose route is not a health check.

Where expressions work

Log Explorer. Advanced search (the code-bracket button at the right end of the bar) replaces the pickers with an expression box, which starts with whatever the pickers held. The expression narrows the selected source, environment or filter, and never replaces it. Some expressions cannot be shown by the pickers and stay in the box - Advanced search lists them.

Filters. Edit as text on a filter's Conditions card shows the conditions as an expression and turns an edited expression back into conditions. To keep a Log Explorer search as a filter, paste its expression here. An alert can then watch the filter.

API. The q parameter is read as an expression when it contains one of the words is, contains or does, or starts with a quote or an opening parenthesis. Anything else is a plain message search, so q=not found searches for the words "not found". The response shows which way q was read, and an expression that does not parse returns an error naming the position. See Query expressions in the API reference.

Asking in plain English

In the Log Explorer's advanced search, a request typed in plain language, such as "all production errors" or "warnings and worse except cron", is turned into an expression by an assistant. The suggestion replaces the text in the box, and nothing runs until you press Enter. The assistant receives the grammar, the text and the names used in the account (programs, attribute paths, sources, environments and filters), never log events.

Asking in plain English in the Log Explorer section has examples, tips for requests that give good results, and the cases where no suggestion is given.