26 Pull requests opened by 15 people

• [SPARK-53069][SS] Fix incorrect state store metrics with virtual column families

  #51790 opened Aug 2, 2025

• [SPARK-53084][CORE] Supplement default GC options in SparkContext initialization

  #51796 opened Aug 3, 2025

• Fix invalid exit codes and enhance CLI validation tools

  #51797 opened Aug 3, 2025

• Just test

  #51798 opened Aug 3, 2025

• [SPARK-53094][SQL] Fix cube-related data quality problem

  #51810 opened Aug 4, 2025

• [SPARK-53097][CONNECT] Make WriteOperationV2 in SparkConnectPlanner side effect free

  #51813 opened Aug 4, 2025

• [SPARK-53103][SS] Throw an error if state directory is not empty on batch 0

  #51817 opened Aug 4, 2025

• [SPARK-53074][SQL] Avoid partial clustering in SPJ to meet a child's required distribution

  #51818 opened Aug 4, 2025

• [SPARK-53094][SQL] Fix CUBE with aggregate containing HAVING clauses

  #51820 opened Aug 4, 2025

• [SPARK-53104][PS] Introduce ansi_mode_context to avoid multiple config checks per API call

  #51821 opened Aug 4, 2025

• [SPARK-53106] Add schema evolution tests for TWS Scala spark connect suites

  #51822 opened Aug 4, 2025

• [SPARK-53107][SQL] Implement the time_trunc function in Scala

  #51823 opened Aug 4, 2025

• [SPARK-53113][SQL] Support the time type by try_make_timestamp()

  #51824 opened Aug 4, 2025

• [SPARK-53110][SQL][PYTHON][CONNECT] Implement the time_trunc function in PySpark

  #51825 opened Aug 4, 2025

• [SPARK-53108][SQL] Implement the time_diff function in Scala

  #51826 opened Aug 4, 2025

• [SPARK-51462][SPARK-51564][SQL][FOLLOWUP][TESTS] Add E2E SQL tests for TIME literals in 24hr and 12hr formats

  #51827 opened Aug 4, 2025

• [SPARK-53109][SQL] Support TIME in the make_timestamp_ntz and try_make_timestamp_ntz functions in Scala

  #51828 opened Aug 4, 2025

• [SPARK-53111][SQL][PYTHON][CONNECT] Implement the time_diff function in PySpark

  #51829 opened Aug 4, 2025

• [SQL] Run scalafmt on DateTimeUtils and DateTimeUtilsSuite

  #51830 opened Aug 4, 2025

• [SPARK-53112][SQL][PYTHON][CONNECT] Support TIME in the make_timestamp_ntz and try_make_timestamp_ntz functions in PySpark

  #51831 opened Aug 4, 2025

• [SPARK-53105][Structured Streaming] Fix tests for checkpoint v2 in RocksDBSuite

  #51834 opened Aug 4, 2025

• [SPARK-53116][CONNECT] Fix conf setter when client is built with jar containing SQLConf but not its engine counterparts

  #51835 opened Aug 4, 2025

• [SPARK-53124][SQL] Prune unnecessary fields from JsonTuple

  #51843 opened Aug 5, 2025

• [SPARK-53126][SQL][CONNECT] Refactor `SparkErrorUtils#stackTraceToString` to handle `null` input and use it to replace `ExceptionUtils#getStackTrace`

  #51844 opened Aug 5, 2025

• [SPARK-53077][CORE] reduce insertion count in SparkBloomFilterSuite

  #51845 opened Aug 5, 2025

• [SPARK-53125][TEST] RemoteSparkSession prints whole `spark-submit` command

  #51846 opened Aug 5, 2025

20 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• [SPARK-52989][SS] Add explicit close() API to State Store iterators

  #51701 commented on Aug 5, 2025 • 27 new comments

• [SPARK-52998][Core] Multiple variables inside declare

  #51705 commented on Aug 4, 2025 • 13 new comments

• [SPARK-52394][PS] Fix autocorr divide-by-zero error under ANSI mode

  #51192 commented on Aug 5, 2025 • 5 new comments

• [SPARK-51585][SQL] Oracle dialect supports pushdown datetime functions

  #50353 commented on Aug 5, 2025 • 4 new comments

• [SPARK-52777][SQL] Enable shuffle cleanup mode configuration in Spark SQL

  #51458 commented on Aug 5, 2025 • 4 new comments

• [SPARK-53066][SQL] Improve EXPLAIN output for DSv2 Join pushdown

  #51781 commented on Aug 5, 2025 • 2 new comments

• [SPARK-53064][CORE] Rewrite MDC LogKey in Java

  #51775 commented on Aug 5, 2025 • 2 new comments

• [SPARK-52593][PS] Avoid CAST_INVALID_INPUT of `MultiIndex.to_series`, `Series.dot` and `DataFrame.dot` in ANSI mode

  #51310 commented on Aug 5, 2025 • 2 new comments

• [SPARK-52976][PYTHON] Fix Python UDF not accepting collated string as input param/return type

  #51688 commented on Aug 5, 2025 • 2 new comments

• [SPARK-52582][SQL] Improve the memory usage of XML parser

  #51287 commented on Aug 4, 2025 • 1 new comment

• [SPARK-52971] [PYTHON] Limit idle Python worker queue size

  #51684 commented on Aug 4, 2025 • 1 new comment

• [SPARK-53030][PYTHON] Support Arrow writer for streaming Python data sources

  #51757 commented on Aug 4, 2025 • 0 new comments

• [SPARK-53015][BUILD] Upgrade log4j to 2.25.1

  #51719 commented on Aug 4, 2025 • 0 new comments

• [SPARK-52931][Core] Restrict declare variable naming

  #51669 commented on Aug 5, 2025 • 0 new comments

• [SPARK-52943][PYTHON] Enable arrow_cast for all pandas UDF eval types

  #51635 commented on Aug 5, 2025 • 0 new comments

• [SPARK-52729][SQL] Add MetadataOnlyTable in DS v2 API

  #51419 commented on Aug 4, 2025 • 0 new comments

• [MINOR][DOCS] Updated the docstring of DataStreamWriter.foreach() method

  #51316 commented on Aug 5, 2025 • 0 new comments

• [SPARK-52407][SQL] Add support for Theta Sketch

  #51298 commented on Aug 4, 2025 • 0 new comments

• [SPARK-52388][SQL] Handle named and positional parameters under `PlanWithUnresolvedIdentifier`

  #51073 commented on Aug 5, 2025 • 0 new comments

• [WIP][SPARK-51348][BUILD][SQL] Upgrade Hive to 4.0

  #50213 commented on Aug 4, 2025 • 0 new comments