-
Notifications
You must be signed in to change notification settings - Fork 474
[checkpoint] Improve firewall session log handling #12167
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[checkpoint] Improve firewall session log handling #12167
Conversation
Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices) |
BTW, I mentioned it in the changelog but not in the PR commit message but this change will probably result in a mapping change for the three non-ECS fields ( |
@andrewkroh, what are you thoughts on making this a breaking change? When the types of existing fields are changed ( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When the types of existing fields are changed (keyword -> integer in this case), how does that affect existing, indexed documents?
It will not change any existing indexed data. Only the the new index that is created after the package upgrade, which triggers and data stream index rollover, will have the new mapping.
This means that queries that span older indices can have problems when they involve these fields since numeric comparisons cannot execute on the older data.
packages/checkpoint/data_stream/firewall/_dev/test/pipeline/test-r81x.log-expected.json
Outdated
Show resolved
Hide resolved
packages/checkpoint/data_stream/firewall/elasticsearch/ingest_pipeline/default.yml
Outdated
Show resolved
Hide resolved
packages/checkpoint/data_stream/firewall/elasticsearch/ingest_pipeline/default.yml
Outdated
Show resolved
Hide resolved
packages/checkpoint/data_stream/firewall/elasticsearch/ingest_pipeline/default.yml
Show resolved
Hide resolved
/test |
💚 Build Succeeded
|
|
Package checkpoint - 1.35.0 containing this change is available at https://epr.elastic.co/package/checkpoint/1.35.0/ |
Add handling of Check Point firewall session logs in accordance with the ECS structure. Session logs aggregate multiple connection logs from the same network activity into a single event. The aggregation creates the following fields: - creation_time: UNIX timestamp of the first connection in the session. - last_hit_time: UNIX timestamp of the last recorded connection in the session. - duration: Duration (in seconds) of the session. - aggregated_log_count: Number of connection logs aggregated into the session. - connection_count: Number of connections recorded in the session. - update_count: Number of times the session was updated. This commit will: 1. Interpret creation_time and last_hit_time as dates, storing them in the ECS fields event.start and event.end, respectively. 2. Convert duration to nanoseconds, as per the ECS event.duration specification, and store it in the event.duration field. 3. Ensure checkpoint.aggregated_log_count, checkpoint.connection_count, and checkpoint.update_count are mapped to numeric types. Note that `checkpoint.aggregated_log_count`, `checkpoint.connection_count`, and `checkpoint.update_count` which were previously mapped dynamically as keyword data types are now statically mapped as integer data types. Closes elastic#11894
Add handling of Check Point firewall session logs in accordance with the ECS structure. Session logs aggregate multiple connection logs from the same network activity into a single event. The aggregation creates the following fields: - creation_time: UNIX timestamp of the first connection in the session. - last_hit_time: UNIX timestamp of the last recorded connection in the session. - duration: Duration (in seconds) of the session. - aggregated_log_count: Number of connection logs aggregated into the session. - connection_count: Number of connections recorded in the session. - update_count: Number of times the session was updated. This commit will: 1. Interpret creation_time and last_hit_time as dates, storing them in the ECS fields event.start and event.end, respectively. 2. Convert duration to nanoseconds, as per the ECS event.duration specification, and store it in the event.duration field. 3. Ensure checkpoint.aggregated_log_count, checkpoint.connection_count, and checkpoint.update_count are mapped to numeric types. Note that `checkpoint.aggregated_log_count`, `checkpoint.connection_count`, and `checkpoint.update_count` which were previously mapped dynamically as keyword data types are now statically mapped as integer data types. Closes elastic#11894
Proposed commit message
Checklist
changelog.yml
file.Author's Checklist
How to test this PR locally
Related issues