Skip to content

[checkpoint] Improve firewall session log handling #12167

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 0 commits into from
Jan 31, 2025
Merged

[checkpoint] Improve firewall session log handling #12167

merged 0 commits into from
Jan 31, 2025

Conversation

srilumpa
Copy link
Contributor

@srilumpa srilumpa commented Dec 19, 2024

Proposed commit message

Add handling of Check Point firewall session logs in accordance with the ECS
structure.

Session logs aggregate multiple connection logs from the same network
activity into a single event. The aggregation creates the following fields:

- creation_time: UNIX timestamp of the first connection in the session.
- last_hit_time: UNIX timestamp of the last recorded connection in the session.
- duration: Duration (in seconds) of the session.
- aggregated_log_count: Number of connection logs aggregated into the session.
- connection_count: Number of connections recorded in the session.
- update_count: Number of times the session was updated.

This commit will:

1. Interpret creation_time and last_hit_time as dates, storing them in the ECS
fields event.start and event.end, respectively.
2. Convert duration to nanoseconds, as per the ECS event.duration specification,
and store it in the event.duration field.
3. Ensure checkpoint.aggregated_log_count, checkpoint.connection_count, and
checkpoint.update_count are mapped to numeric types.

Note that `checkpoint.aggregated_log_count`, `checkpoint.connection_count`, and
`checkpoint.update_count` which were previously mapped dynamically as keyword
data types are now statically mapped as integer data types.

Closes #11894

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

How to test this PR locally

Related issues

@srilumpa srilumpa requested a review from a team as a code owner December 19, 2024 15:09
@andrewkroh andrewkroh added enhancement New feature or request Integration:checkpoint Check Point Team:Security-Deployment and Devices Deployment and Devices Security team [elastic/sec-deployment-and-devices] labels Dec 19, 2024
@elasticmachine
Copy link

Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices)

@srilumpa
Copy link
Contributor Author

BTW, I mentioned it in the changelog but not in the PR commit message but this change will probably result in a mapping change for the three non-ECS fields (checkpoint.aggregated_log_count, checkpoint.connection_count and checkpoint.update_count) as they were dynamically mapped to a keyword but are now mapped to an integer.

@taylor-swanson
Copy link
Contributor

@andrewkroh, what are you thoughts on making this a breaking change? When the types of existing fields are changed (keyword -> integer in this case), how does that affect existing, indexed documents?

Copy link
Member

@andrewkroh andrewkroh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the types of existing fields are changed (keyword -> integer in this case), how does that affect existing, indexed documents?

It will not change any existing indexed data. Only the the new index that is created after the package upgrade, which triggers and data stream index rollover, will have the new mapping.

This means that queries that span older indices can have problems when they involve these fields since numeric comparisons cannot execute on the older data.

@andrewkroh andrewkroh changed the title Fix/checkpoint session logs [checkpoint] Improve firewall session log handling Jan 31, 2025
@andrewkroh
Copy link
Member

/test

@andrewkroh andrewkroh enabled auto-merge (squash) January 31, 2025 22:40
@elasticmachine
Copy link

💚 Build Succeeded

@andrewkroh andrewkroh merged this pull request into elastic:main Jan 31, 2025
3 checks passed
Copy link

@elastic-vault-github-plugin-prod

Package checkpoint - 1.35.0 containing this change is available at https://epr.elastic.co/package/checkpoint/1.35.0/

harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 4, 2025
Add handling of Check Point firewall session logs in accordance with the ECS
structure.

Session logs aggregate multiple connection logs from the same network
activity into a single event. The aggregation creates the following fields:

- creation_time: UNIX timestamp of the first connection in the session.
- last_hit_time: UNIX timestamp of the last recorded connection in the session.
- duration: Duration (in seconds) of the session.
- aggregated_log_count: Number of connection logs aggregated into the session.
- connection_count: Number of connections recorded in the session.
- update_count: Number of times the session was updated.

This commit will:

1. Interpret creation_time and last_hit_time as dates, storing them in the ECS
fields event.start and event.end, respectively.
2. Convert duration to nanoseconds, as per the ECS event.duration specification,
and store it in the event.duration field.
3. Ensure checkpoint.aggregated_log_count, checkpoint.connection_count, and
checkpoint.update_count are mapped to numeric types.

Note that `checkpoint.aggregated_log_count`, `checkpoint.connection_count`, and
`checkpoint.update_count` which were previously mapped dynamically as keyword
data types are now statically mapped as integer data types.

Closes elastic#11894
harnish-crest-data pushed a commit to chavdaharnish/integrations that referenced this pull request Feb 5, 2025
Add handling of Check Point firewall session logs in accordance with the ECS
structure.

Session logs aggregate multiple connection logs from the same network
activity into a single event. The aggregation creates the following fields:

- creation_time: UNIX timestamp of the first connection in the session.
- last_hit_time: UNIX timestamp of the last recorded connection in the session.
- duration: Duration (in seconds) of the session.
- aggregated_log_count: Number of connection logs aggregated into the session.
- connection_count: Number of connections recorded in the session.
- update_count: Number of times the session was updated.

This commit will:

1. Interpret creation_time and last_hit_time as dates, storing them in the ECS
fields event.start and event.end, respectively.
2. Convert duration to nanoseconds, as per the ECS event.duration specification,
and store it in the event.duration field.
3. Ensure checkpoint.aggregated_log_count, checkpoint.connection_count, and
checkpoint.update_count are mapped to numeric types.

Note that `checkpoint.aggregated_log_count`, `checkpoint.connection_count`, and
`checkpoint.update_count` which were previously mapped dynamically as keyword
data types are now statically mapped as integer data types.

Closes elastic#11894
@srilumpa srilumpa deleted the fix/checkpoint_session_logs branch February 7, 2025 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Integration:checkpoint Check Point Team:Security-Deployment and Devices Deployment and Devices Security team [elastic/sec-deployment-and-devices]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Checkpoint]: Processing and mapping of Checkpoint Firewall session logs
4 participants