Skip to content

#11767 - Remove deprecated httpjson from packages zeek, apache, system, windows, aws.cloudtrail and nginx #13246

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 12 commits into from
May 6, 2025
Merged

#11767 - Remove deprecated httpjson from packages zeek, apache, system, windows, aws.cloudtrail and nginx #13246

merged 12 commits into from
May 6, 2025

Conversation

qcorporation
Copy link
Contributor

@qcorporation qcorporation commented Mar 21, 2025 •
edited
Loading

Parent Ticket

#11767

Proposed commit message

Remove httpjson from packages zeek, apache, system, windows, aws.cloudtrail and nginx

  • removed docker service if applicable
  • removed system tests if they use the httpjson input that's been deprecated
  • removed httpjson input from the manifest file
  • update each integration to a new major version as this can potentially be a breaking change

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

Ran system and pipeline test and validated no failures

Exceptions

There is still the windows integration where all the system tests are based upon the httpjson input
Speaking with the teams with code ownership to get recommendations

Screenshots

Screenshot 2025-03-21 at 3 24 56 PM
Screenshot 2025-03-21 at 3 03 27 PM
Screenshot 2025-03-21 at 2 49 48 PM
Screenshot 2025-03-21 at 2 11 15 PM

@qcorporation
Remove httpjson from packages zeek, apache, system and nginx
9a1ea02 
- removed docker service if applicable
- removed pipeline tests if they use the httpjson input that's been deprecated
- update each integration to a new major version as this can potentially be a breaking change
@qcorporation qcorporation added Integration:system System Integration:apache Apache HTTP Server Integration:zeek Zeek Integration:nginx Nginx Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] Team:Security-Deployment and Devices Deployment and Devices Security team [elastic/sec-deployment-and-devices] Team:Security-Linux Platform Linux Platform Security team [elastic/sec-linux-platform] Team:Security-Windows Platform Security Windows Platform team [elastic/sec-windows-platform] labels Mar 21, 2025
@qcorporation qcorporation self-assigned this Mar 21, 2025
@qcorporation qcorporation changed the title #11767 - Remove httpjson from packages zeek, apache, system and nginx #11767 - Remove deprecated httpjson from packages zeek, apache, system and nginx Mar 21, 2025
@qcorporation qcorporation added Team:Service-Integrations Label for the Observability Service Integrations team Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] and removed Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] Team:Service-Integrations Label for the Observability Service Integrations team labels Mar 21, 2025
@elastic-vault-github-plugin-prod
Copy link

elastic-vault-github-plugin-prod bot commented Mar 21, 2025 •
edited
Loading

🚀 Benchmarks report

Package apache 👍(0) 💚(0) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
access 4366.81 2577.32 -1789.49 (-40.98%) 💔
error 6060.61 4784.69 -1275.92 (-21.05%) 💔

Package aws 👍(12) 💚(6) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
guardduty 1001 773.4 -227.6 (-22.74%) 💔
apigateway_logs 8403.36 5882.35 -2521.01 (-30%) 💔

Package nginx 👍(0) 💚(0) 💔(2)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
access 3322.26 2392.34 -929.92 (-27.99%) 💔
error 18181.82 10204.08 -7977.74 (-43.88%) 💔

Package system 👍(1) 💚(1) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
auth 11111.11 9259.26 -1851.85 (-16.67%) 💔

Package windows 👍(3) 💚(1) 💔(5)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
applocker_exe_and_dll 7692.31 4739.34 -2952.97 (-38.39%) 💔
applocker_msi_and_script 7812.5 5464.48 -2348.02 (-30.05%) 💔
forwarded 1116.07 884.96 -231.11 (-20.71%) 💔
powershell 1904.76 1607.72 -297.04 (-15.59%) 💔
windows_defender 10101.01 8130.08 -1970.93 (-19.51%) 💔

Package zeek 👍(23) 💚(15) 💔(5)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
ntlm 20000 16129.03 -3870.97 (-19.35%) 💔
pe 22727.27 8064.52 -14662.75 (-64.52%) 💔
dns 22727.27 18867.92 -3859.35 (-16.98%) 💔
dpd 19230.77 15625 -3605.77 (-18.75%) 💔
files 23255.81 14084.51 -9171.3 (-39.44%) 💔

To see the full report comment with /test benchmark fullreport

@qcorporation
removing splunk httpjson support for windows
868cdea 
- removed docker deployment
- removed manifest configuration
- remove system test configuration
@qcorporation qcorporation marked this pull request as ready for review March 25, 2025 19:52
@qcorporation qcorporation requested review from a team as code owners March 25, 2025 19:52
@elasticmachine
Copy link

Pinging @elastic/sec-deployment-and-devices (Team:Security-Deployment and Devices)

@elasticmachine
Copy link

Pinging @elastic/sec-linux-platform (Team:Security-Linux Platform)

@elasticmachine
Copy link

Pinging @elastic/sec-windows-platform (Team:Security-Windows Platform)

leehinman
leehinman requested changes Mar 25, 2025
View reviewed changes
Copy link
Contributor

@leehinman leehinman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. I just think there is one more file that can be removed.

marc-gr
marc-gr reviewed Mar 28, 2025
View reviewed changes
marc-gr
marc-gr approved these changes Mar 28, 2025
View reviewed changes
Copy link
Contributor

@marc-gr marc-gr left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@botelastic
Copy link

botelastic bot commented Apr 30, 2025

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

@botelastic botelastic bot added the Stalled label Apr 30, 2025
@botelastic botelastic bot removed the Stalled label Apr 30, 2025
@botelastic
Copy link

botelastic bot commented May 1, 2025

Hi! We just realized that we haven't looked into this PR in a while. We're sorry! We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1. Thank you for your contribution!

@botelastic botelastic bot added the Stalled label May 1, 2025
ishleenk17
ishleenk17 reviewed May 2, 2025
View reviewed changes
@botelastic botelastic bot removed the Stalled label May 2, 2025
@elasticmachine
Copy link

💚 Build Succeeded

History

cc @qcorporation

@elastic-sonarqube Elastic SonarQube
Copy link

Quality Gate passed Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
55.6% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

@qcorporation qcorporation requested a review from ishleenk17 May 5, 2025 18:57
@qcorporation qcorporation merged commit 1becadf into main May 6, 2025
7 checks passed
@qcorporation qcorporation deleted the 11767-remove-deprecation branch May 6, 2025 16:11
@elastic-vault-github-plugin-prod
Copy link

Package apache - 2.0.0 containing this change is available at https://epr.elastic.co/package/apache/2.0.0/

@elastic-vault-github-plugin-prod
Copy link

Package aws - 3.3.0 containing this change is available at https://epr.elastic.co/package/aws/3.3.0/

@elastic-vault-github-plugin-prod
Copy link

Package nginx - 2.0.0 containing this change is available at https://epr.elastic.co/package/nginx/2.0.0/

@elastic-vault-github-plugin-prod
Copy link

Package system - 2.0.0 containing this change is available at https://epr.elastic.co/package/system/2.0.0/

@elastic-vault-github-plugin-prod
Copy link

Package windows - 3.0.0 containing this change is available at https://epr.elastic.co/package/windows/3.0.0/

@elastic-vault-github-plugin-prod
Copy link

Package zeek - 3.0.0 containing this change is available at https://epr.elastic.co/package/zeek/3.0.0/

@efd6 efd6 mentioned this pull request Jun 11, 2025
Open
5 tasks
@efd6 efd6 mentioned this pull request Jul 2, 2025
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@P1llus P1llus P1llus left review comments

@marc-gr marc-gr marc-gr approved these changes

@nicholasberlin nicholasberlin nicholasberlin approved these changes

@leehinman leehinman leehinman approved these changes

@efd6 efd6 efd6 approved these changes

@taylor-swanson taylor-swanson taylor-swanson approved these changes

@muthu-mps muthu-mps muthu-mps approved these changes

@khushijain21 khushijain21 Awaiting requested review from khushijain21 khushijain21 is a code owner automatically assigned from elastic/elastic-agent-data-plane

@ishleenk17 ishleenk17 Awaiting requested review from ishleenk17

Assignees

@qcorporation qcorporation

Labels
Integration:apache Apache HTTP Server Integration:aws AWS Integration:nginx Nginx Integration:system System Integration:windows Windows Integration:zeek Zeek Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] Team:Security-Deployment and Devices Deployment and Devices Security team [elastic/sec-deployment-and-devices] Team:Security-Linux Platform Linux Platform Security team [elastic/sec-linux-platform] Team:Security-Windows Platform Security Windows Platform team [elastic/sec-windows-platform]
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.