Skip to content

Disable hashing on effective_visibilities #144927

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

blyxyas
Copy link
Member

@blyxyas blyxyas commented Aug 4, 2025

Make effective_visibilities no_hash. We were wasting too much time on hashing the FxIndexMap (15 million instructions on tokio just for the hashing) so it was not worth it.

@rustbot
Copy link
Collaborator

rustbot commented Aug 4, 2025

r? @nnethercote

rustbot has assigned @nnethercote.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Aug 4, 2025
@jieyouxu
Copy link
Member

jieyouxu commented Aug 4, 2025

@bors2 try @rust-timer queue

@rust-timer

This comment has been minimized.

rust-bors bot added a commit that referenced this pull request Aug 4, 2025
Disable hashing on `effective_visibilities`
@rust-bors
Copy link

rust-bors bot commented Aug 4, 2025

⌛ Trying commit 64189bf with merge afc29b9

To cancel the try build, run the command @bors try cancel.

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Aug 4, 2025
@nnethercote
Copy link
Contributor

Hmm, I am not familiar with the query modifiers like no_hash and eval_always and the documentation isn't very clear, so I'm not comfortable reviewing this. @blyxyas, can you explain how/why this is a valid and good change? Thanks!

@cjgillot cjgillot self-assigned this Aug 4, 2025
@rust-bors
Copy link

rust-bors bot commented Aug 5, 2025

☀️ Try build successful (CI)
Build commit: afc29b9 (afc29b9bbebfe04fc6635a69253e150adb83e511, parent: 0060d5a2a8a86a31f6299311fe64b1d755a91c4f)

@rust-timer

This comment has been minimized.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (afc29b9): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request means it may be perf-sensitive – we'll automatically label it not fit for rolling up. You can override this, but we strongly advise not to, due to possible changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please do so in sufficient writing along with @rustbot label: +perf-regression-triaged. If not, please fix the regressions and do another perf run. If its results are neutral or positive, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

Our most reliable metric. Used to determine the overall result above. However, even this metric can be noisy.

mean range count
Regressions ❌
(primary)
3.0% [0.2%, 11.0%] 83
Regressions ❌
(secondary)
0.9% [0.2%, 3.7%] 18
Improvements ✅
(primary)
-2.2% [-11.0%, -0.3%] 45
Improvements ✅
(secondary)
-1.0% [-2.5%, -0.1%] 36
All ❌✅ (primary) 1.2% [-11.0%, 11.0%] 128

Max RSS (memory usage)

Results (primary 1.9%, secondary -0.1%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
1.9% [0.7%, 4.2%] 18
Regressions ❌
(secondary)
2.1% [1.9%, 2.4%] 2
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-2.3% [-2.4%, -2.3%] 2
All ❌✅ (primary) 1.9% [0.7%, 4.2%] 18

Cycles

Results (primary 3.4%, secondary -2.0%)

A less reliable metric. May be of interest, but not used to determine the overall result above.

mean range count
Regressions ❌
(primary)
6.0% [2.6%, 10.2%] 40
Regressions ❌
(secondary)
3.5% [2.7%, 4.0%] 3
Improvements ✅
(primary)
-4.5% [-7.9%, -2.1%] 13
Improvements ✅
(secondary)
-4.4% [-6.2%, -2.0%] 7
All ❌✅ (primary) 3.4% [-7.9%, 10.2%] 53

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 466.337s -> 468.665s (0.50%)
Artifact size: 376.97 MiB -> 376.94 MiB (-0.01%)

@rustbot rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Aug 5, 2025
@nnethercote
Copy link
Contributor

Those perf results are not so good, unfortunately :(

@nnethercote nnethercote added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 5, 2025
@blyxyas
Copy link
Member Author

blyxyas commented Aug 5, 2025

:/ Curious results, but not what I'm looking for.

can you explain how/why this is a valid and good change? Thanks!

The principle for this change was that if the effective_visibilities query was very prone to change, and produced a very complex value (an FxIndexMap), so hashing it didn't make much sense.

Also, while inspecting some Valgrind output, we were wasting a significant amount of time on hashing EffectiveVisibiltiies types. Kinda weird that in my cachegrind output it looked like an improvement.

NOTE: Thinking about it, I just remembered that I have the bad habit of only benchmarking the Full scenario and I should change that.

@blyxyas blyxyas closed this Aug 5, 2025
@rustbot rustbot removed the S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. label Aug 5, 2025
@cjgillot
Copy link
Contributor

cjgillot commented Aug 6, 2025

Also, while inspecting some Valgrind output, we were wasting a significant amount of time on hashing EffectiveVisibiltiies types. Kinda weird that in my cachegrind output it looked like an improvement.

This appears in the perf results. The -11% improvements show this.

My understanding of the regressions:
Marking the query no_hash means it only has information about dependencies to know whether the result has changed. And effective_visibilities tends to look at the whole crate's contents, so will very probably be marked red, even if the results are not changed. So dependent queries need to be recomputed, only to discover their results are unchanged.

A possible way forward would be to hide the large effective_visibilities array behind a targeted query effective_visibility(LocalDefId) -> EffectiveVisibility. This would have a query overhead, but may avoid recomputing of some dependents.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
perf-regression Performance regression. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants