[SPARK-53098][SQL] DeduplicateRelations shouldn't remap expressions if old ExprId still exists in output

[mihailotim-db]

What changes were proposed in this pull request?

DeduplicateRelations shouldn't remap expressions from oldExprId -> newExprId if oldExprId still exists in output of one of the children.

Why are the changes needed?

This is needed in order to implement ExprId comparison framework for single-pass analyzer: https://issues.apache.org/jira/browse/SPARK-53099.

For example, if we have a dataframe self join plan like:

Join(a#1 = a#1)
+-Project(a#1, b#2)
+-Project(a#1, b#2)

Current implementation of DeduplicateRelations would regenerate the right hand side to a#3, b#4 and would then also remap the join condition to a#3 = a#3. However, this behavior can't be matched in single-pass because there is no actual need to remap the join condition. This PR proposes a change to only remap the condition if old ExprId is no longer present in any of the child outputs.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Added a new test to DataframeSelfJoinSuite

Was this patch authored or co-authored using generative AI tooling?

No

[violetnspct]

@mihailotim-db Should you be adding tests to cover the following scenarios?

1. Mixed scenario with some ExprIds needing remapping and others preserved
2. Edge case:
   • Multiple children with overlapping ExprIds. Important because it tests the AttributeSet merging logic. Should add test case with 3+ way joins having overlapping ExprIds.