Comet 0.17.0 Release

[andygrove]

What is the problem the feature request solves?

I would like to start planning the 0.17.0 release.

Feature PRs that I would to see included:

• feat: Add experimental support for accelerated PyArrow UDFs #4234
• feat: Add 100% Spark-compatible regex support via codegen dispatcher #4239
• feat: 100% Spark-compatible JSON support via codegen dispatcher #4305
• feat: re-enable COUNT for mixed Spark partial / Comet final aggregates #4374

Bugs that I would like to see fixed:

• Correctness issues

Describe the potential solution

No response

Additional context

No response

[mbutrovich]

I would like #4507 and the followup that uses ArrowStreamReader, closes #3770, and makes all FFI safe is embedded in #4393 and will be peeled off after #4507 merges.

[mbutrovich]

I would love to get DataFusion 54 in the next release (#3978) since the last sync we did Comet CI was green and it brings in a ton of stuff we want, but that timeline is not concrete right now (apache/datafusion#21080). Also, considering we'd have to get the iceberg-rust bump in before Comet, I don't think we can make 54 a blocker for 0.17.0.

[mbutrovich]

I am creating branch-0.17 now.

[mbutrovich]

Current status:

main:

• chore: start 0.18.0 development #4664

branch-0.17:

• chore: [branch-0.17] change version from 0.17.0-SNAPSHOT to 0.17.0 #4661
• docs: [branch-0.17] generate release docs, update script #4663
• docs: [branch-0.17] 0.17.0 changelog #4667

I also opened a handful of PRs to update release_process.md based on discrepancies found.

[mbutrovich]

I believe @andygrove will build and propose an RC tomorrow.

[mbutrovich]

0.17.0 is done. I just opened a 0.17.1 tracking issue.

#4811