[Apache Tomcat] Add integration package with catalina data stream #5926

niraj-elastic · 2023-04-19T16:28:17Z

Enhancement

What does this PR do?

Added 1 data stream (Catalina logs).
Added data collection logic for the data streams.
Added the ingest pipeline for the data streams.
Mapped fields according to the ECS schema and added Fields metadata in the appropriate yaml files.
Added dashboards and visualizations.
Added system test cases for the data stream.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

Clone integrations repo.
Install elastic-package locally.
Start elastic stack using elastic-package.
Move to integrations/packages/tomcat) directory.
Run the following command to run tests. elastic-package test

Screenshots

Meta issue

[Meta] Create Package for Apache Tomcat Integration #5200

Related PRs

elasticmachine · 2023-04-19T16:47:41Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2023-05-05T16:55:12.417+0000
Duration: 15 min 54 sec

Test stats 🧪

Test	Results
Failed	0
Passed	8
Skipped	0
Total	8

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

elasticmachine · 2023-04-19T16:47:51Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	100.0% (`1/1`)	💚
Files	100.0% (`1/1`)	💚
Classes	100.0% (`1/1`)	💚
Methods	90.909% (`10/11`)	👎 -5.245
Lines	88.889% (`80/90`)	👎 -11.111
Conditionals	100.0% (`0/0`)	💚

harnish-crest-data

LGTM!

ishleenk17 · 2023-04-28T09:42:35Z

packages/tomcat/_dev/deploy/docker/setup/ROOT/RELEASE-NOTES.txt

@@ -0,0 +1,173 @@
+================================================================================
+  Licensed to the Apache Software Foundation (ASF) under one or more
+  contributor license agreements.  See the NOTICE file distributed with


This release note looks to be quite detailed. Is there a need for this ?

I think we can skip it.

Or we can shorten it to keep just the important information.

We don't think we require it. let us know your thoughts over it.

Lets remove this then

niraj-elastic · 2023-05-04T05:33:52Z

@ishleenk17 currently we are facing one issue with timestamp in which we see logs in incorrect time window in kibana discover page. We are looking in to it and fix it soon.

ishleenk17 · 2023-05-04T08:15:31Z

@niraj-elastic : The sample logs files under deploy/docker are same for all the 3 log datastreams.
These should be as per the actual access, localhost, and catalina logs.

niraj-elastic · 2023-05-04T08:43:16Z

@niraj-elastic : The sample logs files under deploy/docker are same for all the 3 log datastreams. These should be as per the actual access, localhost, and catalina logs.

@ishleenk17 We are able to generate log files of access and catalina in docker itself, but we are not able to generate log file of localhost. so we only have to add localhost file in sample logs.

ishleenk17 · 2023-05-05T05:10:48Z

@ishleenk17 currently we are facing one issue with timestamp in which we see logs in incorrect time window in kibana discover page. We are looking in to it and fix it soon.

Please mention this issue and its details in the Meta Issue for tracking

ishleenk17 · 2023-05-08T05:23:19Z

packages/apache_tomcat/_dev/build/build.yml

+dependencies:
+  ecs:
+    reference: git@v8.7.0
+    import_mappings: true


We are using import_mappings and having ecs.yml file as well.

@ishleenk17 Could you please refer to this comment? Let me know if you have still any doubts regarding this!

For log datastreams, there is no TSDB to be enabled. So that won't hold true here?
Is the purpose to keep then in README?

User can see these fields from the sample event

@ishleenk17 Yes, if we dont mention these fields in ecs.yml than these fields wont be reflected in readme. So for better user experience we are keeping this fields in ecs.yml.

ishleenk17

Looks good!

elasticmachine · 2023-05-08T12:52:18Z

Package apache_tomcat - 0.1.0 containing this change is available at https://epr.elastic.co/search?package=apache_tomcat

add Apache Tomcat package with catalina data stream

add7f6d

niraj-elastic requested a review from a team as a code owner April 19, 2023 16:28

update changelog entry in changelog.yml

b00bed8

kush-elastic mentioned this pull request Apr 20, 2023

[Meta] Create Package for Apache Tomcat Integration #5200

Closed

17 tasks

kush-elastic requested a review from ishleenk17 April 20, 2023 05:17

kush-elastic assigned niraj-elastic Apr 20, 2023

kush-elastic added enhancement New feature or request Integration:apache_tomcat Apache Tomcat v8.7.0 Team:Service-Integrations Label for the Observability Service Integrations team labels Apr 20, 2023

kush-elastic self-requested a review April 20, 2023 05:19

update readme

135324a

harnish-crest-data approved these changes Apr 24, 2023

View reviewed changes

ishleenk17 reviewed Apr 28, 2023

View reviewed changes

ishleenk17 requested a review from a team May 1, 2023 07:32

niraj-elastic added 2 commits May 3, 2023 21:53

undo changes in tomcat package

747b304

create apache_tomcat package with catalina data stream

142492f

address review comments

1ba48b2

niraj-elastic added 2 commits May 5, 2023 22:21

update pipeline

05d9a2d

update dashboard image

d3a009c

ishleenk17 reviewed May 8, 2023

View reviewed changes

ishleenk17 approved these changes May 8, 2023

View reviewed changes

kush-elastic merged commit 707ba7c into elastic:main May 8, 2023

harnish-crest-data mentioned this pull request Jun 9, 2023

[Apache Tomcat] Add integration package with memory data stream #6527

Merged

7 tasks

milan-elastic mentioned this pull request Jun 14, 2023

[Apache Tomcat] Add integration package with session data stream #6573

Merged

7 tasks

kush-elastic mentioned this pull request Jun 16, 2023

[Apache Tomcat] Add integration package with connection pool data stream #6596

Merged

7 tasks

harnish-crest-data mentioned this pull request Jun 19, 2023

[O11y][Apache Tomcat] Add integration package with thread pool data stream #6609

Merged

7 tasks

harnish-crest-data mentioned this pull request Jun 29, 2023

[O11y][Apache Tomcat] Create Overview dashboard #6750

Closed

harnish-crest-data mentioned this pull request Jul 17, 2023

[O11y][Apache Tomcat] Add overview dashboards for logs and metrics data streams #6982

Merged

4 tasks

andrewkroh added the New Integration Issue or pull request for creating a new integration package. label Aug 13, 2024

[Apache Tomcat] Add integration package with catalina data stream #5926

[Apache Tomcat] Add integration package with catalina data stream #5926

Uh oh!

Conversation

niraj-elastic commented Apr 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Checklist

How to test this PR locally

Screenshots

Meta issue

Related PRs

Uh oh!

elasticmachine commented Apr 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

Uh oh!

elasticmachine commented Apr 19, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🌐 Coverage report

Uh oh!

harnish-crest-data left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

niraj-elastic commented May 4, 2023

Uh oh!

ishleenk17 commented May 4, 2023

Uh oh!

niraj-elastic commented May 4, 2023

Uh oh!

ishleenk17 commented May 5, 2023

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ishleenk17 left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented May 8, 2023

Uh oh!

Uh oh!

niraj-elastic commented Apr 19, 2023 •

edited

Loading

elasticmachine commented Apr 19, 2023 •

edited

Loading

elasticmachine commented Apr 19, 2023 •

edited

Loading