Skip to content

[ES|QL] Introduce AggregateMetricDoubleBlock #127299

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 8 commits into from
May 1, 2025
Merged

[ES|QL] Introduce AggregateMetricDoubleBlock #127299

merged 8 commits into from
May 1, 2025
+487 −130

Conversation

limotova
Copy link
Contributor

@limotova limotova commented Apr 24, 2025 •
edited
Loading

This commit adds a new Block type named AggregateMetricDoubleBlock and
refactors AggregateMetricDouble to use it instead of CompositeBlock.

The reasoning behind this is that there were many areas where we lost
the distinction between a generic CompositeBlock vs using CompositeBlock
specifically for AggregateMetricDoubles, and we felt it was best to
split them into their own types.

@limotova limotova force-pushed the agg-metric-block branch 8 times, most recently from 066a38d to daba3fc Compare April 28, 2025 05:12
@limotova limotova mentioned this pull request Apr 28, 2025
Open
15 tasks
@limotova limotova changed the title agg metric block [ES|QL] Introduce AggregateMetricDoubleBlock Apr 28, 2025
@limotova limotova requested a review from dnhatn April 28, 2025 17:33
limotova
limotova commented Apr 28, 2025
View reviewed changes
...in/esql/compute/src/main/java/org/elasticsearch/compute/data/AggregateMetricDoubleBlock.java
}

@Override
public int getTotalValueCount() {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the same as CompositeBlock but I'm not sure if this is what we want for AggregateMetricDouble?

dnhatn
dnhatn reviewed Apr 28, 2025
View reviewed changes
Copy link
Member

@dnhatn dnhatn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've left several suggestions, but the PR looks great. Thanks Larisa!

@limotova limotova requested a review from dnhatn April 29, 2025 07:48
dnhatn
dnhatn approved these changes Apr 29, 2025
View reviewed changes
Copy link
Member

@dnhatn dnhatn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, Larisa!

@limotova limotova marked this pull request as ready for review April 29, 2025 22:28
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Apr 29, 2025
@elasticsearchmachine
Copy link
Collaborator

Hi @limotova, I've created a changelog YAML for you.

@limotova limotova added the :StorageEngine/TSDB You know, for Metrics label Apr 29, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

@limotova limotova requested review from nik9000 and kkrik-es April 30, 2025 03:26
kkrik-es
kkrik-es reviewed Apr 30, 2025
View reviewed changes
...in/esql/compute/src/main/java/org/elasticsearch/compute/data/AggregateMetricDoubleBlock.java

@Override
public boolean mayHaveNulls() {
return Stream.of(minBlock, maxBlock, sumBlock, countBlock).anyMatch(Block::mayHaveNulls);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I thought there was a push to skip streams as they're unnecessarily expensive?

@limotova limotova merged commit 4241842 into elastic:main May 1, 2025
17 checks passed
@limotova limotova deleted the agg-metric-block branch May 1, 2025 05:23
@limotova limotova mentioned this pull request May 2, 2025
Merged
limotova added a commit to limotova/elasticsearch that referenced this pull request May 2, 2025
@limotova
[8.19] [ES|QL] Introduce AggregateMetricDoubleBlock (elastic#127299)
c86702d 
Backports the following commits to 8.19:
- [ES|QL] Introduce AggregateMetricDoubleBlock (elastic#127299)
limotova added a commit to limotova/elasticsearch that referenced this pull request May 2, 2025
@limotova
[8.19] [ES|QL] Introduce AggregateMetricDoubleBlock (elastic#127299)
a142e1d 
Backports the following commits to 8.19:
- [ES|QL] Introduce AggregateMetricDoubleBlock (elastic#127299)
limotova added a commit to limotova/elasticsearch that referenced this pull request May 2, 2025
@limotova
[8.19] [ES|QL] Introduce AggregateMetricDoubleBlock (elastic#127299)
4f1f5a7 
Backports the following commits to 8.19:
- [ES|QL] Introduce AggregateMetricDoubleBlock (elastic#127299)
@limotova limotova mentioned this pull request May 2, 2025
Closed
elasticsearchmachine pushed a commit that referenced this pull request May 2, 2025
@limotova
[8.19] [ES|QL] Introduce AggregateMetricDoubleBlock (#127299) (#127623)
8c18619 
* [8.19] [ES|QL] Introduce AggregateMetricDoubleBlock (#127299)

Backports the following commits to 8.19:
- [ES|QL] Introduce AggregateMetricDoubleBlock (#127299)

* borrow test skip from b563145
nik9000 added a commit to nik9000/elasticsearch that referenced this pull request May 2, 2025
@nik9000
ESQL: Fix transport versions
2c93edc 
In elastic#127623 we backported elastic#127299 and added a backport transport version
for it - `ESQL_AGGREGATE_METRIC_DOUBLE_BLOCK_8_19` aka `8_841_0_24`.
This brings that version forwards to `main` and adds support for parsing
streams with that version.

In elastic#127639 we backported elastic#126401 and added a backport transport version
for it - `PINNED_RETRIEVER_8_19` aka `8_841_0_23`. This brings that
version forwards to `main` and adds support for parsing streams with
that versions.

In elastic#127633 we a claimed a backport transport version to backport elastic#127314 -
`INTRODUCE_FAILURES_LIFECYCLE_BACKPORT_8_19` aka `8_841_0_23`. That's
the same versions as `PINNED_RETRIEVER_8_19`. It's just that this one is
in `main` and `PINNED_RETRIEVER_8_19` is in `8.19`. To allow me to bring
`PINNED_RETRIEVER_8_19` for wards I've had to revert elastic#127633.

Closes elastic#127667
@nik9000 nik9000 mentioned this pull request May 2, 2025
Merged
elasticsearchmachine pushed a commit that referenced this pull request May 3, 2025
@nik9000
ESQL: Fix transport versions (#127668)
2acd563 
In #127623 we backported #127299 and added a backport transport version
for it - `ESQL_AGGREGATE_METRIC_DOUBLE_BLOCK_8_19` aka `8_841_0_24`.
This brings that version forwards to `main` and adds support for parsing
streams with that version.

In #127639 we backported #126401 and added a backport transport version
for it - `PINNED_RETRIEVER_8_19` aka `8_841_0_23`. This brings that
version forwards to `main` and adds support for parsing streams with
that versions.

In #127633 we a claimed a backport transport version to backport #127314
- `INTRODUCE_FAILURES_LIFECYCLE_BACKPORT_8_19` aka `8_841_0_23`. That's
the same versions as `PINNED_RETRIEVER_8_19`. It's just that this one is
in `main` and `PINNED_RETRIEVER_8_19` is in `8.19`. To allow me to bring
`PINNED_RETRIEVER_8_19` for wards I've had to revert #127633.

Closes #127667
jfreden pushed a commit to jfreden/elasticsearch that referenced this pull request May 12, 2025
@nik9000 @jfreden
ESQL: Fix transport versions (elastic#127668)
788d9ce 
In elastic#127623 we backported elastic#127299 and added a backport transport version
for it - `ESQL_AGGREGATE_METRIC_DOUBLE_BLOCK_8_19` aka `8_841_0_24`.
This brings that version forwards to `main` and adds support for parsing
streams with that version.

In elastic#127639 we backported elastic#126401 and added a backport transport version
for it - `PINNED_RETRIEVER_8_19` aka `8_841_0_23`. This brings that
version forwards to `main` and adds support for parsing streams with
that versions.

In elastic#127633 we a claimed a backport transport version to backport elastic#127314
- `INTRODUCE_FAILURES_LIFECYCLE_BACKPORT_8_19` aka `8_841_0_23`. That's
the same versions as `PINNED_RETRIEVER_8_19`. It's just that this one is
in `main` and `PINNED_RETRIEVER_8_19` is in `8.19`. To allow me to bring
`PINNED_RETRIEVER_8_19` for wards I've had to revert elastic#127633.

Closes elastic#127667
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dnhatn dnhatn dnhatn approved these changes

@kkrik-es kkrik-es kkrik-es approved these changes

@nik9000 nik9000 Awaiting requested review from nik9000

Assignees
No one assigned
Labels
:Analytics/ES|QL AKA ESQL >enhancement :StorageEngine/TSDB You know, for Metrics Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) Team:StorageEngine v9.1.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@limotova @elasticsearchmachine @dnhatn @kkrik-es