Skip to content

[O11y][Apache Spark] Add filters in the visualizations #8117

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Oct 17, 2023
Merged

[O11y][Apache Spark] Add filters in the visualizations #8117

merged 4 commits into from
Oct 17, 2023

Conversation

harnish-crest-data
Copy link
Contributor

  • Bugfix

What does this PR do

  • Fix the missing filter for the following Apache Spark visualizations by adding the filter ("field_name": *)

    • Number of Cores Used
    • Memory Used (MB)
    • Number of Threadpool tasks over time [Metrics Apache Spark]
    • Number of Tasks over time [Metrics Apache Spark]

  • Since the changes are not affecting the dashboard screenshot, no need to update the screenshot.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Related issues

@harnish-crest-data harnish-crest-data self-assigned this Oct 6, 2023
@harnish-crest-data harnish-crest-data marked this pull request as ready for review October 6, 2023 12:36
@harnish-crest-data harnish-crest-data requested a review from a team as a code owner October 6, 2023 12:36
@elasticmachine
Copy link

elasticmachine commented Oct 6, 2023 •
edited
Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-10-17T06:19:00.211+0000

  • Duration: 18 min 9 sec

Test stats 🧪

Test Results
Failed 0
Passed 15
Skipped 0
Total 15

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

@elasticmachine
Copy link

elasticmachine commented Oct 6, 2023 •
edited
Loading

🌐 Coverage report

Name Metrics % (covered/total) Diff
Packages 100.0% (0/0) 💚
Files 100.0% (0/0) 💚 4.463
Classes 100.0% (0/0) 💚 4.463
Methods 75.0% (12/16) 👎 -16.614
Lines 100.0% (0/0) 💚 12.285
Conditionals 100.0% (0/0) 💚

ali786XI
ali786XI approved these changes Oct 9, 2023
View reviewed changes
Copy link
Contributor

@ali786XI ali786XI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@muthu-mps
Copy link
Contributor

@harnish-elastic,

  • When looking into the changes. I was able to understand that we apply explicit filters when we have single metrics to show in the LENS and when there are multiple metrics displayed in single chart then advanced filter is applied which is not visible to the user similar to the explicit ones.

  • Here is a catch where the LENS has redundant filter applied for a single metric. Please see the issue below.

Memory usage over time [Metrics Apache Spark] Memory
Screenshot 2023-10-10 at 3 18 16 PM

muthu-mps
muthu-mps approved these changes Oct 16, 2023
View reviewed changes
Copy link
Contributor

@muthu-mps muthu-mps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@harnish-crest-data
Merge branch 'main' of https://github.com/harnish-elastic/integrations …
91fe76a 
…into apache_spark-add-filter

Conflicts:
	packages/apache_spark/changelog.yml
	packages/apache_spark/manifest.yml
@harnish-crest-data harnish-crest-data merged commit bb52565 into elastic:main Oct 17, 2023
@elasticmachine
Copy link

Package apache_spark - 0.7.9 containing this change is available at https://epr.elastic.co/search?package=apache_spark

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@muthu-mps muthu-mps muthu-mps approved these changes

@ali786XI ali786XI ali786XI approved these changes

Assignees

@harnish-crest-data harnish-crest-data

Labels
Integration:apache_spark Apache Spark
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[O11y][Apache Spark] Add filters in dashboard visualizations panels
4 participants
@harnish-crest-data @elasticmachine @muthu-mps @ali786XI