Skip to content

Using a temp IndexService for template validation #129507

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Jun 17, 2025
Merged

Using a temp IndexService for template validation #129507

merged 5 commits into from
Jun 17, 2025

Conversation

masseyke
Copy link
Member

@masseyke masseyke commented Jun 16, 2025 •
edited
Loading

We currently create and then quickly delete a real IndexService for template validation in MetadataIndexTemplateService::validateCompositeTemplate. If a cluster state update happens to come in while the IndexService exists, then we can see an assertion failure like this:

java.lang.AssertionError: index [VnipvFiyQtGvaLjGyo4xiQ/osjlH3fjQeuz66QRpH04vg] does not exist in the cluster state, it should either have been deleted or the cluster must be new
	at org.elasticsearch.server@9.1.0-SNAPSHOT/org.elasticsearch.indices.cluster.IndicesClusterStateService.removeIndicesAndShards(IndicesClusterStateService.java:491)
	at org.elasticsearch.server@9.1.0-SNAPSHOT/org.elasticsearch.indices.cluster.IndicesClusterStateService.doApplyClusterState(IndicesClusterStateService.java:318)
	at org.elasticsearch.server@9.1.0-SNAPSHOT/org.elasticsearch.indices.cluster.IndicesClusterStateService.applyClusterState(IndicesClusterStateService.java:276)
	at org.elasticsearch.server@9.1.0-SNAPSHOT/org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:572)
	at org.elasticsearch.server@9.1.0-SNAPSHOT/org.elasticsearch.cluster.service.ClusterApplierService.callClusterStateAppliers(ClusterApplierService.java:558)
	at org.elasticsearch.server@9.1.0-SNAPSHOT/org.elasticsearch.cluster.service.ClusterApplierService.applyChanges(ClusterApplierService.java:531)
	at org.elasticsearch.server@9.1.0-SNAPSHOT/org.elasticsearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:460)
	at org.elasticsearch.server@9.1.0-SNAPSHOT/org.elasticsearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:159)
	at org.elasticsearch.server@9.1.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:977)
	at org.elasticsearch.server@9.1.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:218)
	at org.elasticsearch.server@9.1.0-SNAPSHOT/org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:184)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1095)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:619)
	at java.base/java.lang.Thread.run(Thread.java:1447)

This PR uses a temp IndexService instead to avoid this problem.
Closes #129473

@masseyke masseyke added >bug :Data Management/Indices APIs APIs to create and manage indices and templates v8.19.0 auto-backport Automatically create backport pull requests when merged labels Jun 16, 2025
@elasticsearchmachine
Copy link
Collaborator

Hi @masseyke, I've created a changelog YAML for you.

@masseyke masseyke marked this pull request as ready for review June 17, 2025 13:15
@masseyke masseyke requested review from dakrone and nielsbauman June 17, 2025 13:16
@elasticsearchmachine elasticsearchmachine added the Team:Data Management Meta label for data/management team label Jun 17, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

nielsbauman
nielsbauman approved these changes Jun 17, 2025
View reviewed changes
Copy link
Contributor

@nielsbauman nielsbauman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks for the quick fix, Keith!

@masseyke masseyke enabled auto-merge (squash) June 17, 2025 19:03
@masseyke masseyke disabled auto-merge June 17, 2025 19:26
@masseyke masseyke merged commit e969ec2 into elastic:main Jun 17, 2025
26 of 27 checks passed
@masseyke masseyke deleted the using-temp-indexservice-for-template-validation branch June 17, 2025 19:28
@masseyke masseyke mentioned this pull request Jun 17, 2025
Merged
@elasticsearchmachine
Copy link
Collaborator

💚 Backport successful

Status Branch Result
✅ 8.19

masseyke added a commit to masseyke/elasticsearch that referenced this pull request Jun 17, 2025
@masseyke
Using a temp IndexService for template validation (elastic#129507)
f3f47ae
elasticsearchmachine pushed a commit that referenced this pull request Jun 17, 2025
@masseyke
Using a temp IndexService for template validation (#129507) (#129588)
d73d771
kderusso pushed a commit to kderusso/elasticsearch that referenced this pull request Jun 23, 2025
@masseyke @kderusso
Using a temp IndexService for template validation (elastic#129507)
c5afbd4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nielsbauman nielsbauman nielsbauman approved these changes

@dakrone dakrone Awaiting requested review from dakrone

Assignees
No one assigned
Labels
auto-backport Automatically create backport pull requests when merged >bug :Data Management/Indices APIs APIs to create and manage indices and templates Team:Data Management Meta label for data/management team v8.19.0 v9.1.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[CI] DataStreamsClientYamlTestSuiteIT class failing
3 participants
@masseyke @elasticsearchmachine @nielsbauman