Spark 3.5.5 版本发布

Spark 3.5.5 是第四个维护版本，包含安全性和正确性修复。 此版本基于 Spark 的 branch-3.5 维护分支。 我们强烈建议所有 3.5 用户升级到此稳定版本。

主要变更

  • [SPARK-45013]: Flaky Test with NPE: track allocated resources by taskId
  • [SPARK-48899]: Fix ENV key value format in K8s Dockerfiles
  • [SPARK-50091]: Query fails when aggregate expression is in left-hand operand of IN-subquery
  • [SPARK-50624]: Add TimestampNTZType to ColumnarRow/MutableColumnarRow
  • [SPARK-50769]: ClassCastException in HistogramNumeric
  • [SPARK-50784]: Fix lint-scala not to ignore scalastyle errors
  • [SPARK-50797]: Move HiveCharVarcharTestSuite from o/a/s/sql to o/a/s/sql/hive
  • [SPARK-50853]: Close temp shuffle file writable channel
  • [SPARK-50870]: DataSourceV2 relations may contain unresolved cast expressions after materializing current_timestamp()
  • [SPARK-50890]: Skip test_take in Spark Connect only build
  • [SPARK-50946]: When using Java version 17.0.14, the SPARK-35907: createDirectory test in UtilsSuite will fail.
  • [SPARK-50984]: Make ExpressionImplUtilsSuite robust by matching JDK msgs via regex
  • [SPARK-50987]: Make spark-connect-overview.md’s version strings up-to-date
  • [SPARK-50998]: Fix spark.kubernetes.configMap.maxSize default value in docs
  • [SPARK-51027]: Prevent HiveClient.runSqlHive invocation in non-testing environment
  • [SPARK-51033]: Fix CryptoStreamUtilsSuite to use sameElements for array comparision
  • [SPARK-51039]: Fix hive-llap-common dependency to use hive.llap.scope in root pom.xml
  • [SPARK-51042]: Read and write CalendarIntervals using one call to get/putLong consistently
  • [SPARK-51061]: Hide Jetty info in REST Submission API
  • [SPARK-51071]: Upgrade upload-artifact to v4 to recover branch-3.5 CI
  • [SPARK-51093]: Fix minor endianness issues in tests
  • [SPARK-51104]: Use self-host javascript and CSS in Spark website
  • [SPARK-51128]: Self host docsearch.min.css.map
  • [SPARK-51129]: Fix code tab switching in Spark Website
  • [SPARK-51157]: Add missing @varargs Scala annotation for scala functon APIs
  • [SPARK-51164]: Fix CallerContext test by enabling hadoop.caller.context.enabled
  • [SPARK-51173]: Add configName Scalastyle rule
  • [SPARK-51185]: Revert simplifications to PartitionedFileUtil API due to increased risk of OOM
  • [SPARK-51187]: Implement the graceful deprecation of incorrect config introduced in SPARK-49699
  • [SPARK-51219]: Catalog.listTables fails on temp table when non-V2SessionCatalog is in use
  • [SPARK-51244]: Upgrade left Github Action image from ubuntu-20.04 to ubuntu-22.04 and solved the TPCDSQueryBenchmark compatibility issue

依赖变更

虽然是一个维护版本，但我们仍然在此版本中升级了一些依赖项，它们是

您可以查阅 JIRA 以获取详细更改。

我们要感谢所有社区成员为本次发布贡献了补丁。


Spark 新闻存档

最新消息

存档