Expression syntax

Expressions provide flexibility in manipulating, filtering, and routing data. The following sections provide information about expression syntax in Data Prepper.

Key terms

The following key terms are used in the context of expressions.

Operators

The following table lists the supported operators. Operators are listed in order of precedence (top to bottom, left to right).

Relational operators

Relational operators compare numeric values or JSON pointers that resolve to numeric values. The operators are used to test the relationship between two operands, determining if one is greater than, less than, or equal to the other. The syntax for using relational operators is as follows:

<Number | JSON Pointer> < <Number | JSON Pointer>
<Number | JSON Pointer> <= <Number | JSON Pointer>
<Number | JSON Pointer> > <Number | JSON Pointer>
<Number | JSON Pointer> >= <Number | JSON Pointer>

For example, to check if the value of the status_code field in an event is within the range of successful HTTP responses (200–299), you can use the following expression:

/status_code >= 200 and /status_code < 300

Equality operators

Equality operators are used to test whether two values are equivalent. These operators compare values of any type, including JSON pointers, literals, and expressions. The syntax for using equality operators is as follows:

<Any> == <Any>
<Any> != <Any>

The following are some example equality operators:

• /is_cool == true: Checks if the value referenced by the JSON pointer is equal to the Boolean value.
• 3.14 != /status_code: Checks if the numeric value is not equal to the value referenced by the JSON pointer.
• {1, 2} == /event/set_property: Checks if the array is equal to the value referenced by the JSON pointer.

Conditional expressions

Conditional expressions allow you to combine multiple expressions or values using logical operators to create more complex evaluation criteria. The available conditional operators are and, or, and not. The syntax for using these conditional operators is as follows:

<Any> and <Any>
<Any> or <Any>
not <Any>

The following are some example conditional expressions:

/status_code == 200 and /message == "Hello world"
/status_code == 200 or /status_code == 202
not /status_code in {200, 202}
/response == null
/response != null

Arithmetic expressions

Arithmetic expressions enable basic mathematical operations like addition, subtraction, multiplication, and division. These expressions can be combined with conditional expressions to create more complex conditional statements. The available arithmetic operators are +, -, *, and /. The syntax for using the arithmetic operators is as follows:

<Any> + <Any>
<Any> - <Any>
<Any> * <Any>
<Any> / <Any>

The following are example arithmetic expressions:

/value + length(/message)
/bytes / 1024
/value1 - /value2
/TimeInSeconds * 1000

The following are some example arithmetic expressions used in conditional expressions :

/value + length(/message) > 200
/bytes / 1024 < 10
/value1 - /value2 != /value3 + /value4

String concatenation expressions

String concatenation expressions enable you to combine strings to create new strings. These concatenated strings can also be used within conditional expressions. The syntax for using string concatenation is as follows:

<String Variable or String Literal> + <String Variable or String Literal>

The following are example string concatenation expressions:

/name + "suffix"
"prefix" + /name
"time of " + /timeInMs + " ms"

The following are example string concatenation expressions that can be used in conditional expressions:

/service + ".com" == /url
"www." + /service != /url

Reserved symbols

Certain symbols, such as ^, %, xor, =, +=, -=, *=, /=, %=, ++, –, and ${}, are reserved for future functionality or extensions. Reserved symbols include `^`, `%`, `xor`, `=`, `+=`, `-=`, `*=`, `/=`, `%=`, `++`, `--`, and `${}`.

Syntax components

Syntax components are the building blocks of expressions in Data Prepper. They allow you to define sets, specify evaluation order, reference values within events, use literal values, and follow specific white space rules. Understanding these components is crucial for creating and working with expressions effectively in Data Prepper pipelines.

Priority expressions

Priority expressions specify the evaluation order of expressions. They are enclosed in parentheses (). Priority expressions must contain an expression or value (empty parentheses are not supported). The following is an example priority expression:

/is_cool == (/name == "Steven")

JSON pointers

JSON pointers are used to reference values within an event. They start with a leading forward slash / followed by alphanumeric characters or underscores that are separated by additional forward slashes /.

JSON pointers can use an extended character set by wrapping the entire pointer in double quotation marks "" and escaping characters using a backslash \. Note that the ~ and / characters are considered to be part of the pointer path and do not need to be escaped. The following are some examples of valid JSON pointers: ~0 to represent the literal character ~ or ~1 to represent the literal character /.

Shorthand syntax

The shorthand syntax for a JSON pointer can be expressed using the following regular expression pattern, where \w represents any word character (A–Z, a-z, 0–9, or underscore):

/\w+(/\w+)*`

The following is an example of this shorthand syntax:

/Hello/World/0

Escaped syntax

The escaped syntax for a JSON pointer can be expressed as follows:

"/<Valid String Characters | Escaped Character>(/<Valid String Characters | Escaped Character>)*"

The following is an example of an escaped JSON pointer:

# Path
# { "Hello - 'world/" : [{ "\"JsonPointer\"": true }] }
"/Hello - 'world\//0/\"JsonPointer\""

Literals

Literals are fundamental values that have no children. Data Prepper supports the following literal types:

• Float: Supports values from 3.40282347 x 10^38 to 1.40239846 x 10^-45.
• Integer: Supports values from -2,147,483,648 to 2,147,483,647.
• Boolean: Supports true or false.
• JSON pointer: See JSON pointers for more information.
• String: Supports valid Java strings.
• Null: Supports null to check if a JSON pointer exists.

White space rules

White space is optional around relational operators, regex equality operators, equality operators, and commas. White space is required around set initializers, priority expressions, set operators, and conditional expressions.

Related articles

• Functions