[Prometheus] Update default dashboard, update sample_event files #4275

tetianakravchenko · 2022-09-22T12:57:40Z

Signed-off-by: Tetiana Kravchenko tetiana.kravchenko@elastic.co

What does this PR do?

Prepare to the prometheus GA

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

[ ]

How to test this PR locally

Related issues

Closes [Prometheus integration] Refactor OOTB dashboard #3668

Screenshots

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

elasticmachine · 2022-09-22T13:08:09Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-09-26T13:42:35.251+0000
Duration: 14 min 7 sec

Test stats 🧪

Test	Results
Failed	0
Passed	7
Skipped	0
Total	7

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

elasticmachine · 2022-09-22T13:14:34Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	100.0% (`0/0`)	💚
Files	100.0% (`0/0`)	💚 2.586
Classes	100.0% (`0/0`)	💚 2.586
Methods	66.667% (`6/9`)	👎 -23.401
Lines	100.0% (`0/0`)	💚 8.631
Conditionals	100.0% (`0/0`)	💚

MichaelKatsoulis · 2022-09-22T13:24:51Z

@tetianakravchenko I understand that this dashboard is only for the prometheus server. Could we have visualisations showing uptime, memory used, operations, scrapes (successful and unsuccessful), active alerts? Or it is not in the scope of this PR to add more visualisations?

gizas · 2022-09-23T06:36:49Z

packages/prometheus/kibana/visualization/prometheus-19886730-b2e7-11e9-9a23-67ee28886a4b.json

                "filter": {
                    "language": "kuery",
                    "query": ""
                },
                "id": "b2579fe0-b2e6-11e9-96a9-535735f478e7",
                "index_pattern": "metrics-*",
-                "interval": "auto",
+                "interval": "1m",


Was this done on purpose? Does this affect sampling period? I see it in all visualisations

with auto on remote_write as metrics are coming with interval 1m:

gizas · 2022-09-23T06:38:42Z

packages/prometheus/kibana/visualization/prometheus-19886730-b2e7-11e9-9a23-67ee28886a4b.json

    "id": "prometheus-19886730-b2e7-11e9-9a23-67ee28886a4b",
    "migrationVersion": {
-        "visualization": "7.8.0"
+        "visualization": "8.3.0"


Should not this be 8.4.0? Probably minor not affecting functionality though

I got those visualisations/dashboard as result of running:

elastic-package-0.63.0 export dashboards -d 1153ffb0-3a6c-11ed-afd9-719992d5bc04

my local stack is on version - 8.4.0

gizas · 2022-09-23T06:41:49Z

packages/prometheus/kibana/visualization/prometheus-19886730-b2e7-11e9-9a23-67ee28886a4b.json

@@ -58,21 +53,27 @@
                        "stacked": "none",
                        "terms_field": "prometheus.labels.handler",
                        "terms_size": "5",
-                        "value_template": "{{value}}"
+                        "time_range_mode": "entire_time_range",
+                        "value_template": "{{value}}/s"


Have you tested this visualisation with period less than 15min? This is the one having problems

gizas · 2022-09-23T06:42:48Z

packages/prometheus/_dev/build/docs/README.md

-This integration periodically fetches metrics from [Prometheus](https://prometheus.io/) servers.
-This integration can collect metrics from Prometheus Exporters, receive metrics from Prometheus using Remote Write
+This integration periodically fetches metrics from [Prometheus](https://prometheus.io/) metrics endpoints.
+This integration can collect metrics from Prometheus Exporters, receive metrics from Prometheus server using Remote Write
 and execute specific Prometheus queries against Promethes Query API.


Suggested change

and execute specific Prometheus queries against Promethes Query API.

or execute specific Prometheus queries against Promethes Query API.

gizas · 2022-09-23T06:43:47Z

packages/prometheus/_dev/build/docs/README.md

+
+Prometheus integration is shipped including default overview dashboard.
+Default dashboard works only for `remote_write` datastream and `collector` darastream, if metrics are scraped from the Prometheus server metrics endpoint.


Suggested change

Default dashboard works only for `remote_write` datastream and `collector` darastream, if metrics are scraped from the Prometheus server metrics endpoint.

Default dashboard tested with `remote_write` datastream and `collector` darastream, if metrics are scraped from the Prometheus server metrics endpoint.

Dont know if we want to write something like:
Provided dashboards can not guarantee coverage in case of Prometheus Query configuration scenarios

This dashboard will not work with query, as metrics will be stored under prometheus.query.*, that is why I wrote Default dashboard works only for ...

gizas · 2022-09-23T06:55:37Z

@tetianakravchenko I understand that this dashboard is only for the prometheus server. Could we have visualisations showing uptime, memory used, operations, scrapes (successful and unsuccessful), active alerts? Or it is not in the scope of this PR to add more visualisations?

@MichaelKatsoulis in general in this PR we wanted to have as generic dashboards as we can to support both remote_write and collector scenarios OOTB. The existing dashboards had problems with remote_write. So now we have the minimum I would say needed for someone to evaluate that prometheus is working and not having problems.

That said (alerts are in inside) I would say that if we had to add something from your list it would be memory perhaps to help troubleshoot. All other might not be so necessary. I would let @tetianakravchenko to argue in favour or not and have final saying on this :)

MichaelKatsoulis · 2022-09-23T07:08:31Z

in general in this PR we wanted to have as generic dashboards as we can to support both remote_write and collector scenarios OOTB.

@gizas the name of the dashboard (Prometheus Server Overview) does not reveal the generic approach.

tetianakravchenko · 2022-09-23T09:11:14Z

@gizas the name of the dashboard (Prometheus Server Overview) does not reveal the generic approach.

@MichaelKatsoulis motivation for this rename: metrics that are coming from remote_write metricset are anyway specific to the prometheus server (the list of metrics defined by the prometheus scrape_configs). to make the same dashboard work for both datastream - remote_write and collector (latest for the case of scraping prometheus_server:9090/metrics) - should be used only metrics that are coming from the prometheus server /metrics endpoint.
All the metrics used initially in this dashboard are coming from prometheus server /metrics endpoint.

@tetianakravchenko I understand that this dashboard is only for the prometheus server. Could we have visualisations showing uptime, memory used, operations, scrapes (successful and unsuccessful), active alerts? Or it is not in the scope of this PR to add more visualisations?

uptime - not provided by /metrics endpoint metrics.
memory used - is not part of /metrics endpoint metrics.
operations - could you explain?
scrapes - I will add it
active alerts - we already have it on the dashboard

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

tetianakravchenko · 2022-09-23T14:22:13Z

@MichaelKatsoulis I've added TSDB Series, Appended Samples rate, Rejected Scrapes, Target sync duration in milliseconds, Target scrape sync total charts.

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

MichaelKatsoulis · 2022-09-26T07:42:14Z

packages/prometheus/kibana/dashboard/prometheus-1153ffb0-3a6c-11ed-afd9-719992d5bc04.json

+    "migrationVersion": {
+        "dashboard": "8.4.0"
+    },
+    "references": [


According to https://docs.google.com/document/d/1uyyFGx6xA5Kvl8c-ZdvXdvBGrHTylxU9F69TGqfzdmw/edit#heading=h.o6it0jm3zlmv we should migrate existing dashboards “By value”. This means that in the dashboard you should unlink the various visualisations from the library. That way they will be just saved to the dashboard itself and not referenced. So then we can deliver only the dashboard json and not the ones under visualization folder (can be deleted).

does it mean that dashboard should be migrated to lens?

I found what unlink the visualiation means, from the document somehow I thought that it required migration to lens

done in a836f4d

MichaelKatsoulis · 2022-09-26T07:42:46Z

packages/prometheus/kibana/dashboard/prometheus-1153ffb0-3a6c-11ed-afd9-719992d5bc04.json

+                                    },
+                                    "formatter": "s,ms,2",
+                                    "id": "2f5a9030-b2e5-11e9-b248-0162f01eb4ee",
+                                    "label": "Targen sync duration",


Target maybe here.

fixed it in the same commit - a836f4d#diff-5a3dc417971082efc7b6d31d90ad8ad9c8a3d8b59d91e9cc13e5a2a62ed0caf6R466

MichaelKatsoulis · 2022-09-26T07:46:22Z

@MichaelKatsoulis I've added TSDB Series, Appended Samples rate, Rejected Scrapes, Target sync duration in milliseconds, Target scrape sync total charts.

Great! That is very good. I left some comments regarding the by value visualisations. Also about the name of the various visualizations, in others we add the [Metrics Integrations_name] in the end of the name. Like in Kubernetes dashboards. I believe this help in better grouping.

gizas · 2022-09-26T07:59:18Z

Also about the name of the various visualizations, in others we add the [Metrics Integrations_name] in the end of the name.

Regarding that can we use tags from now on to group specific dashboards? I am not sure if tags need to preexist before the assets in order to work and appear

…s Prometheus] in the end for visualisations names Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

tetianakravchenko · 2022-09-26T12:48:57Z

Regarding that can we use tags from now on to group specific dashboards? I am not sure if tags need to preexist before the assets in order to work and appear

it is not possible to add tags for the visualisations (at least to the unlinked vizualisation)

It seems you talk about adding tags on the dashboard itself, not on the visualisation, right? In case of Prometheus there is just 1 dashboard

gizas · 2022-09-26T12:55:03Z

Correct was referring to Dashboards, although comment about [Metrics Integrations_name] was for visualisations.
Sorry for that, so dont see the point of adding such extra info to visualisations.

If it was dashboards to assist searching ok. But minor

tetianakravchenko · 2022-09-26T14:23:07Z

Sorry for that, so dont see the point of adding such extra info to visualisations.

@gizas as I see this naming is used for lots of dashboards, example - AWS all dashboards have [Metrics AWS] as well, example - https://github.com/elastic/integrations/blob/main/packages/aws/kibana/dashboard/aws-07d67a60-d872-11eb-8220-c9141cc1b15c.json.
I am not sure if there is any convention on that, but for now I would keep [Metrics Integrations_name] for visualisation for consistency

update default dashboard; update sample_events

7db86e3

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

tetianakravchenko requested a review from a team as a code owner September 22, 2022 12:57

tetianakravchenko added 2 commits September 22, 2022 14:58

update pr number

5484257

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

run elastic-package format; fix screenshot path

4a78ea0

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

gizas reviewed Sep 23, 2022

View reviewed changes

tetianakravchenko added 2 commits September 23, 2022 16:17

update dashboard and screenshot

0f2e5fc

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

update chart title

5b87cb3

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

readme adjustments

f22d52e

Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

MichaelKatsoulis reviewed Sep 26, 2022

View reviewed changes

unlink visualisations from the library; rename dashboard; add [Metric…

a836f4d

…s Prometheus] in the end for visualisations names Signed-off-by: Tetiana Kravchenko <tetiana.kravchenko@elastic.co>

MichaelKatsoulis approved these changes Sep 26, 2022

View reviewed changes

update the dashboard image

5e6c8c4

tetianakravchenko merged commit 7987b9d into elastic:main Sep 26, 2022

	and execute specific Prometheus queries against Promethes Query API.
	or execute specific Prometheus queries against Promethes Query API.


		Prometheus integration is shipped including default overview dashboard.
		Default dashboard works only for `remote_write` datastream and `collector` darastream, if metrics are scraped from the Prometheus server metrics endpoint.

	Default dashboard works only for `remote_write` datastream and `collector` darastream, if metrics are scraped from the Prometheus server metrics endpoint.
	Default dashboard tested with `remote_write` datastream and `collector` darastream, if metrics are scraped from the Prometheus server metrics endpoint.

[Prometheus] Update default dashboard, update sample_event files #4275

[Prometheus] Update default dashboard, update sample_event files #4275

Uh oh!

Conversation

tetianakravchenko commented Sep 22, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Checklist

Author's Checklist

How to test this PR locally

Related issues

Screenshots

Uh oh!

elasticmachine commented Sep 22, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

Uh oh!

elasticmachine commented Sep 22, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🌐 Coverage report

Uh oh!

MichaelKatsoulis commented Sep 22, 2022

Uh oh!

gizas Sep 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

gizas commented Sep 23, 2022

Uh oh!

MichaelKatsoulis commented Sep 23, 2022

Uh oh!

tetianakravchenko commented Sep 23, 2022

Uh oh!

tetianakravchenko commented Sep 23, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

MichaelKatsoulis commented Sep 26, 2022

Uh oh!

gizas commented Sep 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tetianakravchenko commented Sep 26, 2022

Uh oh!

gizas commented Sep 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tetianakravchenko commented Sep 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

tetianakravchenko commented Sep 22, 2022 •

edited

Loading

elasticmachine commented Sep 22, 2022 •

edited

Loading

elasticmachine commented Sep 22, 2022 •

edited

Loading

gizas Sep 23, 2022 •

edited

Loading

gizas commented Sep 26, 2022 •

edited

Loading

gizas commented Sep 26, 2022 •

edited

Loading

tetianakravchenko commented Sep 26, 2022 •

edited

Loading