Skip to content

[AWS] Add WAF datastream #1886

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Oct 13, 2021
Merged

[AWS] Add WAF datastream #1886

merged 4 commits into from
Oct 13, 2021

Conversation

legoguy1000
Copy link
Contributor

What does this PR do?

Adds a new datastream to parse WAF logs

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • If I'm introducing a new feature, I have modified the Kibana version constraint in my package's manifest.yml file to point to the latest Elastic stack release (e.g. ^7.13.0).

Author's Checklist

  • [ ]

How to test this PR locally

cd integrations/packages/aws
elastic-package build && elastic-package stack down && elastic-package stack up -d -v && eval "$(elastic-package stack shellinit)" && elastic-package test  -v --data-streams waf

Related issues

Screenshots

@elasticmachine
Copy link

elasticmachine commented Oct 9, 2021

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-10-13T09:53:04.890+0000

  • Duration: 18 min 40 sec

  • Commit: a900db8

Test stats 🧪

Test Results
Failed 0
Passed 260
Skipped 0
Total 260

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine
Copy link

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@jamiehynds jamiehynds added New Integration Issue or pull request for creating a new integration package. Integration:aws AWS labels Oct 11, 2021
@P1llus P1llus requested a review from kaiyan-sheng October 11, 2021 14:20
@P1llus
Copy link
Member

P1llus commented Oct 11, 2021

This is LGTM from my side in terms of how the data is parsed, would want an approval from @kaiyan-sheng especially on the addition to the AWS package, as I know these are made as subpackages etc?

@legoguy1000
Copy link
Contributor Author

The only thing i don't know about the WAF logs is if they are nested like the other JSON AWS logs

{
"Records": [
{},
{},....
]
}

@P1llus
Copy link
Member

P1llus commented Oct 11, 2021

The only thing i don't know about the WAF logs is if they are nested like the other JSON AWS logs

{
"Records": [
{},
{},....
]
}

From what I can see @legoguy1000, that is more specific to Cloudtrail logs. When WAF logs are written directly to S3 (for example with Kinesis) it does not come in that nested format.

@kaiyan-sheng
Copy link
Contributor

/test

@P1llus
Copy link
Member

P1llus commented Oct 13, 2021

/test

@P1llus
Copy link
Member

P1llus commented Oct 13, 2021

Anything else needed for this @kaiyan-sheng ?

@kaiyan-sheng
Copy link
Contributor

@P1llus Looks good to me 👍 Thanks!!

@P1llus P1llus merged commit a216567 into elastic:master Oct 13, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Integration:aws AWS New Integration Issue or pull request for creating a new integration package.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants