Enhance AWS S3 integration dashboard

[kaiyan-sheng]

What does this PR do?

This PR is to improve AWS S3 overview dashboard. Some metrics are added to be collected in order to show in the visualizations as well.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Screenshots

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-12-01T22:53:27.991+0000

• Duration: 37 min 55 sec

Test stats 🧪

Test	Results
Failed	0
Passed	177
Skipped	3
Total	180

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[ravikesarwani]

Wow! this is great. Exceptional work and thank you for helping it drive forward.

Just a few minor comments:
What is this graph showing? Is it # of objects? Would be good to make it clear and add the appropriate text.

On the Downloads and uploads row, what does the “download bytes per request” really mean?

[kaiyan-sheng]

Just a few minor comments: What is this graph showing? Is it # of objects? Would be good to make it clear and add the appropriate text.

Ah good catch! I missed the title there! Will add it. This visualization is for number of objects per bucket.

On the Downloads and uploads row, what does the “download bytes per request” really mean?

So this one is to show the BytesDownloaded metric from S3. BytesDownloaded metric shows the number of bytes downloaded for requests made to an Amazon S3 bucket, where the response includes a body. With the average statistic method, this metric represents the average number of bytes downloaded per request.

[elasticmachine]

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (14/14)	💚
Files	93.333% (14/15)	👎 -4.401
Classes	93.333% (14/15)	👎 -4.401
Methods	85.156% (218/256)	👎 -6.135
Lines	96.018% (5908/6153)	👍 4.005
Conditionals	100.0% (0/0)	💚

[drewdaemon]

Looking great!

I noticed a couple of things that might point to overuse of horizontal space:

1. On my screen (still quite large), the row labels get clipped

2. I see that we are putting the legends inside the charts, which overlaps with the data.

I am wondering if it would make sense to limit the rows to max two visualizations each. This would allow you more space to place the legends outside the charts and increase the width of the row labels

In this spirit

• the "Average 4xx errors" and "Average 5xx errors" visualizations could be replaced with one stacked bar chart.
• do we need separate visualizations for the GET and PUT requests? After all, the user can always add a dashboard filter if they only want to see one request type

These would result in more horizontal space and actually better dashboard performance when the data gets large. WDYT?

[kaiyan-sheng]

Looking great!

I noticed a couple of things that might point to overuse of horizontal space:

1. On my screen (still quite large), the row labels get clipped

Fixed! I changed it to font size 12 and they all should be visible in smaller screens now.

2. I see that we are putting the legends inside the charts, which overlaps with the data.
   I am wondering if it would make sense to limit the rows to max two visualizations each. This would allow you more space to place the legends outside the charts and increase the width of the row labels

I adjusted the visualizations to only have 2 in one row instead of 3. This way we have more space to show the legends and data points. Thanks for the suggestion!

In this spirit

• the "Average 4xx errors" and "Average 5xx errors" visualizations could be replaced with one stacked bar chart.
• do we need separate visualizations for the GET and PUT requests? After all, the user can always add a dashboard filter if they only want to see one request type

These would result in more horizontal space and actually better dashboard performance when the data gets large. WDYT?

I'd like to still keep the 4xx and 5xx separate in two visualizations, same with GET and PUT just because they are basic metrics to show for S3 requests and depends on the specific bucket or use case, user might want to see these charts separately 🤔

[kaiyan-sheng]

Note: I'm keeping the screenshot as it is (instead of showing 2 visualizations per row) so all the visualizations can fit in one page of screenshot 😂

[girodav]

LGTM! Great work :)

[elasticmachine]

Package aws - 1.28.0 containing this change is available at https://epr.elastic.co/search?package=aws