[AWS] add support to include linked accounts when using log name prefix to select log groups #11484

Kavindu-Dodan · 2024-10-21T21:23:04Z

Proposed commit message

Adds support to include linked accounts when using Log Group Name Prefix to select the desired log groups to monitor.

The boolean option Include Linked Accounts with prefix is disabled by default and internally maps to filebeat input include_linked_accounts_for_prefix_mode.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.
I have verified that any added dashboard complies with Kibana's Dashboard good practices

Related issues

Closes [AWS] Support linked accounts when using log_group_name_prefix to select log groups #11457

Screenshots

elastic-vault-github-plugin-prod · 2024-10-21T22:58:50Z

🚀 Benchmarks report

Package `aws` 👍(9) 💚(8) 💔(2)

Expand to view

Data stream	Previous EPS	New EPS	Diff (%)	Result
`route53_public_logs`	19607.84	14084.51	-5523.33 (-28.17%)	💔
`waf`	7194.24	5813.95	-1380.29 (-19.19%)	💔

To see the full report comment with /test benchmark fullreport

elasticmachine · 2024-11-07T16:46:06Z

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

packages/aws/changelog.yml

packages/aws/manifest.yml

packages/aws/data_stream/cloudtrail/agent/stream/aws-cloudwatch.yml.hbs

agithomas · 2024-11-25T04:29:38Z

Could we consider adding details of this flag in the README, such as at here, the dependency of log_group_name_prefix on include_linked_accounts_with_prefix ?

Kavindu-Dodan · 2024-11-25T19:13:47Z

Could we consider adding details of this flag in the README, such as at here, the dependency of log_group_name_prefix on include_linked_accounts_with_prefix ?

Good point. See this commit - d0b2f1a . I added a small section on logs :) Hope this is sufficient, but else let me know

agithomas

LGTM!

elasticmachine · 2024-11-27T16:52:32Z

💚 Build Succeeded

Buildkite Build
Commit: 6e233c79776b132440046c31d9024928e90becfd

History

💚 Build #18681 succeeded 59565636a2e5982bea821d9cb4f4f9add53cff06
💔 Build #18680 failed d0b2f1a00a2d7d3b5c68b84f94ced65719bf3a0d
💚 Build #18616 succeeded 7c72aed02b500b8e9c55c5ef3f1d25a3a0bdd46c
💚 Build #18519 succeeded 60c99efc14168c43cc4808e3476bebf30d355717
💚 Build #18348 succeeded 0f2102c26b7bb28b72ee3988ded5e3f0ccf9f819
💚 Build #18028 succeeded f3be5ec2f3c8f38eb7805d90da74c56e40a47f6e

elastic-sonarqube · 2024-11-27T16:52:38Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
85.4% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

elastic-vault-github-plugin-prod · 2024-11-27T17:10:02Z

Package aws - 2.33.0 containing this change is available at https://epr.elastic.co/package/aws/2.33.0/

elastic-vault-github-plugin-prod · 2024-11-27T17:10:03Z

Package aws_bedrock - 0.14.0 containing this change is available at https://epr.elastic.co/package/aws_bedrock/0.14.0/

elastic-vault-github-plugin-prod · 2024-11-27T17:10:04Z

Package aws_logs - 1.5.0 containing this change is available at https://epr.elastic.co/package/aws_logs/1.5.0/

…ix to select log groups (elastic#11484) * add support to include linked accounts when using log name prefix Signed-off-by: Kavindu Dodanduwa <kavindu.dodanduwa@elastic.co> add changelog link Signed-off-by: Kavindu Dodanduwa <kavindu.dodanduwa@elastic.co> trigger build # Conflicts: # packages/aws/changelog.yml # Conflicts: # packages/aws/changelog.yml # packages/aws_logs/changelog.yml # packages/aws_logs/manifest.yml * Update packages/aws/changelog.yml Co-authored-by: kaiyan-sheng <kaiyan.sheng@elastic.co> * Update packages/aws/manifest.yml Co-authored-by: kaiyan-sheng <kaiyan.sheng@elastic.co> --------- Co-authored-by: kaiyan-sheng <kaiyan.sheng@elastic.co>

Kavindu-Dodan added enhancement New feature or request Team:obs-ds-hosted-services Observability Hosted Services team [elastic/obs-ds-hosted-services] labels Oct 21, 2024

andrewkroh added Integration:aws AWS Integration:aws_bedrock Amazon Bedrock Integration:aws_logs Custom AWS Logs labels Oct 21, 2024

Kavindu-Dodan requested a review from kaiyan-sheng October 21, 2024 21:39

Kavindu-Dodan force-pushed the feat/aws-linked-accounts-for-prefix-mode branch 3 times, most recently from dd5efc3 to 689e5fc Compare October 24, 2024 17:03

Kavindu-Dodan force-pushed the feat/aws-linked-accounts-for-prefix-mode branch 4 times, most recently from 03a1d8b to 6d8b3f8 Compare November 4, 2024 20:43

Kavindu-Dodan force-pushed the feat/aws-linked-accounts-for-prefix-mode branch from 6d8b3f8 to f3be5ec Compare November 7, 2024 16:39

Kavindu-Dodan marked this pull request as ready for review November 7, 2024 16:39

Kavindu-Dodan requested review from a team as code owners November 7, 2024 16:39

Kavindu-Dodan changed the title ~~[Draft] [AWS] add support to include linked accounts when using log name prefix to select log groups~~ [AWS] add support to include linked accounts when using log name prefix to select log groups Nov 7, 2024

andrewkroh added Team:Obs-InfraObs Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Nov 7, 2024

Kavindu-Dodan marked this pull request as draft November 7, 2024 17:15

Kavindu-Dodan force-pushed the feat/aws-linked-accounts-for-prefix-mode branch from f3be5ec to 2511dba Compare November 14, 2024 18:00

Kavindu-Dodan marked this pull request as ready for review November 14, 2024 18:01

Kavindu-Dodan force-pushed the feat/aws-linked-accounts-for-prefix-mode branch from 2511dba to 0f2102c Compare November 14, 2024 18:02

kaiyan-sheng reviewed Nov 20, 2024

View reviewed changes

packages/aws/changelog.yml Outdated Show resolved Hide resolved

kaiyan-sheng reviewed Nov 20, 2024

View reviewed changes

packages/aws/manifest.yml Outdated Show resolved Hide resolved

kaiyan-sheng reviewed Nov 20, 2024

View reviewed changes

packages/aws/data_stream/cloudtrail/agent/stream/aws-cloudwatch.yml.hbs Show resolved Hide resolved

Kavindu-Dodan requested a review from kaiyan-sheng November 20, 2024 19:56

kaiyan-sheng approved these changes Nov 20, 2024

View reviewed changes

Kavindu-Dodan enabled auto-merge (squash) November 20, 2024 20:37

Kavindu-Dodan requested review from efd6 and agithomas November 20, 2024 21:48

efd6 approved these changes Nov 20, 2024

View reviewed changes

Kavindu-Dodan force-pushed the feat/aws-linked-accounts-for-prefix-mode branch from 60c99ef to 7c72aed Compare November 22, 2024 16:19

Kavindu-Dodan force-pushed the feat/aws-linked-accounts-for-prefix-mode branch from 7c72aed to d0b2f1a Compare November 25, 2024 19:12

Kavindu-Dodan force-pushed the feat/aws-linked-accounts-for-prefix-mode branch from d0b2f1a to 5956563 Compare November 25, 2024 19:29

agithomas approved these changes Nov 27, 2024

View reviewed changes

Kavindu-Dodan force-pushed the feat/aws-linked-accounts-for-prefix-mode branch from 5956563 to 6e233c7 Compare November 27, 2024 15:15

Kavindu-Dodan merged commit 57a1300 into elastic:main Nov 27, 2024
3 checks passed

Kavindu-Dodan deleted the feat/aws-linked-accounts-for-prefix-mode branch November 27, 2024 18:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[AWS] add support to include linked accounts when using log name prefix to select log groups #11484

[AWS] add support to include linked accounts when using log name prefix to select log groups #11484

Uh oh!

Kavindu-Dodan commented Oct 21, 2024 •

edited

Loading

Uh oh!

elastic-vault-github-plugin-prod bot commented Oct 21, 2024 •

edited

Loading

Uh oh!

elasticmachine commented Nov 7, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

agithomas commented Nov 25, 2024 •

edited

Loading

Uh oh!

Kavindu-Dodan commented Nov 25, 2024

Uh oh!

agithomas left a comment

Uh oh!

elasticmachine commented Nov 27, 2024

Uh oh!

Uh oh!

elastic-sonarqube bot commented Nov 27, 2024

Uh oh!

elastic-vault-github-plugin-prod bot commented Nov 27, 2024

Uh oh!

elastic-vault-github-plugin-prod bot commented Nov 27, 2024

Uh oh!

elastic-vault-github-plugin-prod bot commented Nov 27, 2024

Uh oh!

Uh oh!

[AWS] add support to include linked accounts when using log name prefix to select log groups #11484

[AWS] add support to include linked accounts when using log name prefix to select log groups #11484

Uh oh!

Conversation

Kavindu-Dodan commented Oct 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed commit message

Checklist

Related issues

Screenshots

Uh oh!

elastic-vault-github-plugin-prod bot commented Oct 21, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🚀 Benchmarks report

Package aws 👍(9) 💚(8) 💔(2)

Uh oh!

elasticmachine commented Nov 7, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

agithomas commented Nov 25, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Kavindu-Dodan commented Nov 25, 2024

Uh oh!

agithomas left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Nov 27, 2024

💚 Build Succeeded

History

Uh oh!

Uh oh!

elastic-sonarqube bot commented Nov 27, 2024

Quality Gate passed

Uh oh!

elastic-vault-github-plugin-prod bot commented Nov 27, 2024

Uh oh!

elastic-vault-github-plugin-prod bot commented Nov 27, 2024

Uh oh!

elastic-vault-github-plugin-prod bot commented Nov 27, 2024

Uh oh!

Uh oh!

Kavindu-Dodan commented Oct 21, 2024 •

edited

Loading

elastic-vault-github-plugin-prod bot commented Oct 21, 2024 •

edited

Loading

Package `aws` 👍(9) 💚(8) 💔(2)

agithomas commented Nov 25, 2024 •

edited

Loading