Expression syntax

Expressions provide flexibility in manipulating, filtering, and routing data. The following sections provide information about expression syntax in Data Prepper.

Key terms

The following key terms are used in the context of expressions.

Operators

The following table lists the supported operators. Operators are listed in order of precedence (top to bottom, left to right).

Relational operators

Relational operators compare numeric values or JSON pointers that resolve to numeric values. The operators are used to test the relationship between two operands, determining if one is greater than, less than, or equal to the other. The syntax for using relational operators is as follows:

<Number | JSON Pointer> < <Number | JSON Pointer>
<Number | JSON Pointer> <= <Number | JSON Pointer>
<Number | JSON Pointer> > <Number | JSON Pointer>
<Number | JSON Pointer> >= <Number | JSON Pointer>

For example, to check if the value of the status_code field in an event is within the range of successful HTTP responses (200–299), you can use the following expression:

/status_code >= 200 and /status_code < 300

Equality operators

Equality operators are used to test whether two values are equivalent. These operators compare values of any type, including JSON pointers, literals, and expressions. The syntax for using equality operators is as follows:

<Any> == <Any>
<Any> != <Any>

The following are some example equality operators:

• /is_cool == true: Checks if the value referenced by the JSON pointer is equal to the Boolean value.
• 3.14 != /status_code: Checks if the numeric value is not equal to the value referenced by the JSON pointer.
• {1, 2} == /event/set_property: Checks if the array is equal to the value referenced by the JSON pointer.

Conditional expressions

Conditional expressions allow you to combine multiple expressions or values using logical operators to create more complex evaluation criteria. The available conditional operators are and, or, and not. The syntax for using these conditional operators is as follows:

<Any> and <Any>
<Any> or <Any>
not <Any>

The following are some example conditional expressions:

/status_code == 200 and /message == "Hello world"
/status_code == 200 or /status_code == 202
not /status_code in {200, 202}
/response == null
/response != null

Reserved symbols

Reserved symbols are symbols that are not currently used in the expression syntax but are reserved for possible future functionality or extensions. Reserved symbols include ^, *, /, %, +, -, xor, =, +=, -=, *=, /=, %=, ++, --, and ${<text>}.

Syntax components

Syntax components are the building blocks of expressions in Data Prepper. They allow you to define sets, specify evaluation order, reference values within events, use literal values, and follow specific white space rules. Understanding these components is crucial for creating and working with expressions effectively in Data Prepper pipelines.

Priority expressions

Priority expressions specify the evaluation order of expressions. They are enclosed in parentheses (). Priority expressions must contain an expression or value (empty parentheses are not supported). The following is an example priority expression:

/is_cool == (/name == "Steven")

JSON pointers

JSON pointers are used to reference values within an event. They start with a leading forward slash / followed by alphanumeric characters or underscores that are separated by additional forward slashes /.

JSON pointers can use an extended character set by wrapping the entire pointer in double quotation marks "" and escaping characters using a backslash \. Note that the ~ and / characters are considered to be part of the pointer path and do not need to be escaped. The following are some examples of valid JSON pointers: ~0 to represent the literal character ~ or ~1 to represent the literal character /.

Shorthand syntax

The shorthand syntax for a JSON pointer can be expressed using the following regular expression pattern, where \w represents any word character (A–Z, a-z, 0–9, or underscore):

/\w+(/\w+)*`

The following is an example of this shorthand syntax:

/Hello/World/0

Escaped syntax

The escaped syntax for a JSON pointer can be expressed as follows:

"/<Valid String Characters | Escaped Character>(/<Valid String Characters | Escaped Character>)*"

The following is an example of an escaped JSON pointer:

# Path
# { "Hello - 'world/" : [{ "\"JsonPointer\"": true }] }
"/Hello - 'world\//0/\"JsonPointer\""

Literals

Literals are fundamental values that have no children. Data Prepper supports the following literal types:

• Float: Supports values from 3.40282347 x 10^38 to 1.40239846 x 10^-45.
• Integer: Supports values from -2,147,483,648 to 2,147,483,647.
• Boolean: Supports true or false.
• JSON pointer: See JSON pointers for more information.
• String: Supports valid Java strings.
• Null: Supports null to check if a JSON pointer exists.

White space rules

White space is optional around relational operators, regex equality operators, equality operators, and commas. White space is required around set initializers, priority expressions, set operators, and conditional expressions.

Related articles

• Functions