GCP Vertex AI LLM Integration #11225

ishleenk17 · 2024-09-24T06:17:35Z

GCP Vertex AI LLM Integration

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

Related issues

Relates https://github.com/elastic/obs-integration-team/issues/64

Screenshots

muthu-mps

LGTM!

agithomas · 2024-09-30T08:42:14Z

Based on the screenshot attached to the issue, here below are some of the suggestions for improvements

You may consider removing the decimals from Total invocations
Consider including the account selection / region control dropdown at the top of the dashboard
Very surprised to see the model invocation latency dashboard is empty. For every LLM request , chance of not having an LLM invocation latency is nil. Please re-look if the dashboard is correctly configured
Are there any available attributes that identify which of the CPU's are utilised? Please check the unit mapping to select percentage (if that is the case so). Same with memory as well. Are there any available attributes to identify the resource's whose memory is referred in the visualisation
Network utilisation graph has label "Received Bytes". But, it is indented for Sent & received bytes. Suggest to change the Y-axis label to network utilisation.
Observed [GCP Billing] prefix towards the end of the screenshot. Consider removing the prefix.

agithomas · 2024-09-30T08:46:10Z

Based on the screenshot attached to the issue, here below are some of the suggestions for improvements

You may consider removing the decimals from Total invocations

Consider including the account selection / region control dropdown at the top of the dashboard

Very surprised to see the model invocation latency dashboard is empty. For every LLM request , chance of not having an LLM invocation latency is nil. Please re-look if the dashboard is correctly configured

Are there any available attributes that identify which of the CPU's are utilised? Please check the unit mapping to select percentage (if that is the case so). Same with memory as well. Are there any available attributes to identify the resource's whose memory is referred in the visualisation

Network utilisation graph has label "Received Bytes". But, it is indented for Sent & received bytes. Suggest to change the Y-axis label to network utilisation.

Observed [GCP Billing] prefix towards the end of the screenshot. Consider removing the prefix.

I see that the screenshot attached to the PR is not updated. So, some of the comments mentioned above may be obsolete and may not need any further action.

agithomas · 2024-09-30T08:49:12Z

Please do consider having received & sent as a single panel. Having them separate may take away the advantage of in-line visualisation comparison.
Any reason why 50 percentile latency is considered for determining LLM invocation latency?

packages/gcp_vertexai/_dev/build/docs/README.md

packages/gcp_vertexai/changelog.yml

agithomas · 2024-09-30T09:05:20Z

packages/gcp_vertexai/data_stream/metrics/elasticsearch/ingest_pipeline/default.yml

@@ -0,0 +1,74 @@
+---
+description: Pipeline for parsing GCP Vertex AI metrics.


We have used available semconv metric names for AWS bedrock field names.

Is the below field name pattern the preferred pattern for GCP?

cc @lalit-satapathy

agithomas · 2024-09-30T09:09:14Z

Is there any reason why the TSDS enablement is not present?

ishleenk17 · 2024-09-30T10:43:30Z

Is there any reason why the TSDS enablement is not present?

We are doing this for Tech Preview and to start getting the initial feedbacks.
Will be enabled when we move it to Beta.

elasticmachine · 2024-10-03T05:42:56Z

💚 Build Succeeded

Buildkite Build
Commit: 7fea35a

History

💚 Build #16640 succeeded 62180d2
💚 Build #16598 succeeded 8806565
💚 Build #16596 succeeded cd29c4c
💚 Build #16595 succeeded 50e49ea
💔 Build #16594 failed 6c5ccbd
💔 Build #16582 failed 02d3d4a

cc @ishleenk17

elastic-sonarqube · 2024-10-03T05:43:00Z

Quality Gate passed

Issues
0 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
89.8% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

elastic-vault-github-plugin-prod · 2024-10-08T04:30:51Z

Package gcp_vertexai - 0.0.1 containing this change is available at https://epr.elastic.co/search?package=gcp_vertexai

* Update null checks anf if checks in the rename * Update changelog.yml * Inital PR for GCP Vertex AI Integration * update dashboard * Update dashboard * Add metrics and update dashboard * add logo image * update PR id * Add fields, update dashboard * update dashboard * update title * Update dashboard * draft documentation * update dashboard screenshot * update metric type for fields * documentation update * update README * update unit type * update dashboard * update dashboard image * merge bytes transferred in a single chart --------- Co-authored-by: muthu-mps <muthukumar.paramasivam@elastic.co>

ishleenk17 and others added 20 commits September 22, 2023 12:00

Update null checks anf if checks in the rename

23cf3a5

Update changelog.yml

9bcd85d

Merge branch 'elastic:main' into main

1462704

Merge branch 'elastic:main' into main

3bfa4f3

Merge branch 'elastic:main' into main

7df11eb

Merge branch 'elastic:main' into main

9f4a04a

Merge branch 'elastic:main' into main

58330e6

Merge branch 'elastic:main' into main

5c5a7eb

Merge branch 'elastic:main' into main

8e5e367

Merge branch 'elastic:main' into main

56e3d83

Merge branch 'elastic:main' into main

3be9458

Merge branch 'elastic:main' into main

e97f93c

Merge branch 'elastic:main' into main

c6afaab

Merge branch 'elastic:main' into main

feac62a

Merge branch 'elastic:main' into main

e901508

Merge branch 'elastic:main' into main

fcc7361

Merge branch 'elastic:main' into main

3ed8f2d

Merge branch 'elastic:main' into main

1f3d86b

Merge branch 'elastic:main' into main

7c8c614

Inital PR for GCP Vertex AI Integration

7b923f8

ishleenk17 self-assigned this Sep 24, 2024

andrewkroh added the New Integration Issue or pull request for creating a new integration package. label Sep 24, 2024

ishleenk17 and others added 8 commits September 24, 2024 14:54

Merge branch 'elastic:main' into gcp_vertexai

1ddf23f

update dashboard

fa0b3fd

Update dashboard

f33e7ba

Add metrics and update dashboard

ac45c78

add logo image

33de8a4

update PR id

be6e3b7

Add fields, update dashboard

3a682f5

update dashboard

0d8e44d

ishleenk17 requested a review from a team September 27, 2024 07:25

ishleenk17 and others added 2 commits September 27, 2024 13:01

Update dashboard

2d992dd

draft documentation

485c8b3

muthu-mps marked this pull request as ready for review September 27, 2024 09:59

update dashboard screenshot

f0b9581

muthu-mps approved these changes Sep 27, 2024

View reviewed changes

muthu-mps and others added 2 commits September 30, 2024 12:12

update metric type for fields

02d3d4a

documentation update

6c5ccbd

update README

50e49ea

agithomas reviewed Sep 30, 2024

View reviewed changes

packages/gcp_vertexai/_dev/build/docs/README.md Show resolved Hide resolved

agithomas reviewed Sep 30, 2024

View reviewed changes

packages/gcp_vertexai/changelog.yml Show resolved Hide resolved

agithomas reviewed Sep 30, 2024

View reviewed changes

update unit type

cd29c4c

muthu-mps added 2 commits September 30, 2024 15:46

update dashboard

bc57a5d

update dashboard image

8806565

muthu-mps and others added 2 commits October 1, 2024 12:01

merge bytes transferred in a single chart

62180d2

Merge branch 'elastic:main' into gcp_vertexai

7fea35a

ishleenk17 merged commit a20055b into elastic:main Oct 8, 2024
5 checks passed

andrewkroh added the Integration:gcp_vertexai GCP Vertex AI label Oct 9, 2024

ishleenk17 deleted the gcp_vertexai branch February 7, 2025 03:43

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

GCP Vertex AI LLM Integration #11225

GCP Vertex AI LLM Integration #11225

Uh oh!

ishleenk17 commented Sep 24, 2024 •

edited

Loading

Uh oh!

muthu-mps left a comment

Uh oh!

agithomas commented Sep 30, 2024

Uh oh!

agithomas commented Sep 30, 2024

Uh oh!

agithomas commented Sep 30, 2024

Uh oh!

Uh oh!

Uh oh!

agithomas Sep 30, 2024

Uh oh!

agithomas commented Sep 30, 2024

Uh oh!

ishleenk17 commented Sep 30, 2024

Uh oh!

elasticmachine commented Oct 3, 2024

Uh oh!

elastic-sonarqube bot commented Oct 3, 2024

Uh oh!

Uh oh!

elastic-vault-github-plugin-prod bot commented Oct 8, 2024

Uh oh!

Uh oh!

		@@ -0,0 +1,74 @@
		---
		description: Pipeline for parsing GCP Vertex AI metrics.

GCP Vertex AI LLM Integration #11225

GCP Vertex AI LLM Integration #11225

Uh oh!

Conversation

ishleenk17 commented Sep 24, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Checklist

Author's Checklist

Related issues

Screenshots

Uh oh!

muthu-mps left a comment

Choose a reason for hiding this comment

Uh oh!

agithomas commented Sep 30, 2024

Uh oh!

agithomas commented Sep 30, 2024

Uh oh!

agithomas commented Sep 30, 2024

Uh oh!

Uh oh!

Uh oh!

agithomas Sep 30, 2024

Choose a reason for hiding this comment

Uh oh!

agithomas commented Sep 30, 2024

Uh oh!

ishleenk17 commented Sep 30, 2024

Uh oh!

elasticmachine commented Oct 3, 2024

💚 Build Succeeded

History

Uh oh!

elastic-sonarqube bot commented Oct 3, 2024

Quality Gate passed

Uh oh!

Uh oh!

elastic-vault-github-plugin-prod bot commented Oct 8, 2024

Uh oh!

Uh oh!

ishleenk17 commented Sep 24, 2024 •

edited

Loading