Fields available in simple search
The search queries with field=value, word(s), phrases or logical combination of such without any use of "|" commands are defined as simple search. Each result row in the result of a simple search query can be taken as a set of field=value pairs and the actual logs. All the field=value pairs that are generated during normalization can be accessed. In addition, the following fields are added by default; log_ts, col_ts, device_name,device_ip,col_type.
example: Users failing to login
label=User label=Login label=Fail
Fields available in aggregation query results
There may be use-cases where a simple search doesn't suffice and there‘s a need to make aggregations based on one or more fields.
Example:
label=User label=Login label=Fail | chart count() as FailCount by target_user
The result of this query is a set of target_user values and the corresponding count, represented as FailCount. Only these two variables are available at this stage. If you want to make a search of results with fails more than hundred then you can do the following.
label=User label=Login label=Fail | chart count() as FailCount by target_user | search FailCount>100
Let's take a non-working example.
label=Allowed label=Connection | chart count() as CNT by source_address,destination_address | search destination_port <1024
The result of the query "label=Allowed label=Connection" will be a set of logs and field-value pairs and destination_port will also be contained on each result row. However, when "| chart count() as CNT by source_address,destination_address" is done we are basically reducing the result set in such a way that each row consists of three field-value pairs viz source_address,destination_address,CNT. This can also be seen as a table with 3 columns. If you now need to search for all connections that went to ports below 1024 you can't do "| search destination_port <1024" because the field destination_port is not available at this stage.
If you need to make search on the destination_port then you can do the following.
label=Allowed label=Connection | chart count() as CNT by source_address,destination_address,destination_port | search destination_port <1024 | chart sum(CNT) by source_address,destination_address
To make the destination_port available for the search segment "| search destination_port < 1024" we explicitly carried that in the first aggregation. We then filtered out all rows corresponding to destination_port < 1024 with the query "| search destination_port <1024". Finally we used another aggregation to get the sum of counts for each source_address,destination_address pair.
Alternatively (and optimally), this query could have been written as given below. The above was taken for sake of clarifying the usage.
label=Allow label=Connection destination_port<1024 | chart count() as CNT by source_address,destination_address
Fields available in join queries
The result of join queries are again rows of fields=value pairs. The raw logs are not included. If you are joining two result sets s1 and s2 then all the fields that are part of the result set can be referenced appending with s1. or s2. as applicable.
For example
psource_address=*] as s1 join jdevice_ip=*] as s2 on s1.source_address=s2.device_ip
In this case, all the fields that are part of the result set source_address=* can be accessed as s1.field_name. For example if the result set have a field called destination_port you will be able to access that as s1.destination_port. Likewise, all the fields in the result set of the second can be accessed as s2.field_name. You can rename the fields and do further aggregations as necessary.
Fields available in stream queries
/5 label=User label=Login label=Fail having same target_user within 5 minutes]
In this case, only the log_ts and the target_user is available for further aggregations and analysis.
Fields available in alert results
An alert is fired when a set of logs matches some pre-specified condition. When an alert is fired, the following variables are exposed to the notification interface.
alert_name, description,risk_level,detection_timestamp,search_link,rows_count,rows.
The rows_count contains the count of the the events particiapting in firing the alert. The variable rows is the set of results of the search query firing the alert. For notification each item in the result set can be accessed by iterating on this list using jinja templating.
For example if the query is as below
device_name=some_device| chart count() as CNT by source_address
You can use the following jinja template to access the values in the returned list.
{% for row in rows %}
{{row.source_address}}:{{row.CNT}}
{% endfor %}
The fields available in the generated alert for stream queries are slightly different. For example take the below query.
5 label=User label=Login label=Fail having same target_user within 5 minutes]
The jinja templating can also be used to access the fields in the participating events. For details please refer