Co-authored-by: BharatDeva <278575558+BharatDeva@users.noreply.github.com>

…Exception (apache#1720)

* Report physical partitioning, and stop two panics escaping as panics

Groundwork for a multi-library distributed-execution example. Each item here
is something that example needs and cannot get today.

`ExecutionPlan.output_partitioning` is new. `partition_count` already existed
but discards everything except the count, so a driver deciding how to split
work across workers could not tell hash-distributed output from merely counted
output, nor read the hash keys. It returns a `PhysicalPartitioning`, named to
keep it distinct from `datafusion.expr.Partitioning` — that one is the logical
partitioning `repartition_by_hash` takes as a request, this one is what a built
plan does. Physical expressions have no Python representation, so the hash keys
are returned in their displayed form.

`SessionContext.execute` now bounds-checks the partition index. The plan's
leaves index their partition vector directly, so an out-of-range index reached
`MemorySourceConfig` and panicked; the panic was caught as a tokio `JoinError`
and arrived as `index out of bounds: the len is 2 but the index is 5`, naming
neither the plan nor the index the caller passed.

`SessionConfig.set` no longer routes through `SessionConfig::set_str`, which
unwraps. An unknown namespace — `datafusion.runtime.*`, or a config extension
not yet installed — aborted with a `PanicException`, which derives from
`BaseException` and so escapes `except Exception`. `information_schema.
df_settings` lists keys in both categories, so replaying settings onto a worker
hit this first.

Two docstrings on `ExecutionPlan` claimed that a table registered from record
batches cannot be serialized. That is true of `LogicalPlan`, whose
`try_encode_table_provider` has no arm for one, and false of the physical
layer, which inlines the batches: verified by decoding on a context sharing
nothing with the encoder and executing. A test pins it, since it is what lets a
worker run a plan the driver encoded.

Also documents, rather than fixes, the `ForeignExecutionPlan` arm in the
example provider's physical codec. It claims every other library's nodes, which
the extension guide tells authors not to do — but it is load-bearing:
`EnsureCooperative` runs during a foreign planner's `create_physical_plan` and
hands the library back a `ForeignExecutionPlan` wrapping the host's
`CooperativeExec`, which has no reachable `try_to_proto`. Narrowing the arm
makes 31 of the 51 tests in the query-planner example fail, all on that node.
The comment now says so, and says a planner that controls its own physical
optimizer rules needs no such arm.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Name the execute partition index for what it is, trim the upgrade guide

`SessionContext.execute` took its second argument as `partitions`, which reads
as a count when it is a single partition index. Rename it to `partition` and
give the method a real docstring with a doctest covering both a full sweep over
`partition_count` and the out-of-range `ValueError`. Every call site in the
repo, docs, and examples passes it positionally, so add a short note to the
upgrade guide for anyone passing it by keyword.

Drop two changelog-shaped sections from the upgrade guide. `output_partitioning`
is additive and `SessionContext.execute` / `SessionConfig.set` only trade a
panic for a raise, so neither asks the reader to change anything. The
`with_extensions` recommendation goes for the same reason; it is advice, and the
extension guide already carries it under `extension_bundles`.

Also remove the comment above the partition range check. It explained the check
by way of a `MemorySourceConfig` panic, which reads as a complaint about
DataFusion's leaves. The index arrives straight from Python and no planner has
seen it, so validating it needs no more justification than any other argument
check at the boundary.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Link the upstream FFI issue, stop teaching execute by counter-example

The `ForeignExecutionPlan` arm's comment explained the symptom but left the
reader no way to find out whether the workaround is still needed. Name the
upstream umbrella issue, apache/datafusion#25152, and the cascade behind it:
`FFI_PlanProperties` carries no `scheduling_type`, so `EnsureCooperative`
reads every foreign leaf as non-cooperative and wraps it, and the resulting
`ForeignExecutionPlan` then cannot serialize itself. Fixing either half
retires the arm.

Drop the out-of-range call from `SessionContext.execute`'s doctest. A
docstring example shows a reader how to use the method, and this one put a
wrong call in front of them; the message it asserted is already pinned by
`test_execute_rejects_an_out_of_range_partition`. The `Raises:` section is
the right home for that behaviour, so complete it: a negative or oversized
index raises `OverflowError` from the `usize` conversion, not the
`ValueError` the bounds check produces.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Document what SessionConfig.set actually does

The docstring was wrong in three ways at once, and the panic fix earlier in
this branch is what made it worth opening: it promised "a new SessionConfig
object" when the method mutates in place and returns self, it mis-indented
the `Args` entries so Sphinx rendered them as body text rather than a field
list, and it had no `Raises` at all -- so the one behaviour this branch
changed, an unknown key raising instead of aborting the interpreter, was
documented only in the Rust source that no user reads.

Give it a truthful `Returns`, a `Raises` covering both an unknown key and an
unparsable value, and a doctest that reads the option back out through
`information_schema.df_settings`.

The `datafusion.runtime.*` trap goes to `configuration.md`, which is where a
reader is when they need it: those keys appear in `df_settings` but are not
settable, so replaying that table verbatim onto a worker fails on the first
such row. The docstring states it in one sentence and points at the guide.
That page also now records that the `with_*` methods modify in place, which
is what makes the chained style it already demonstrates work.

Note the same false `Returns` line appears on every other `with_*` method on
this class; correcting those is a separate sweep, not this branch's business.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Raise ValueError from SessionConfig.set, not a bare Exception

Trading a `PanicException` for an untyped `Exception` only half-fixed the
problem: a caller replaying `information_schema.df_settings` still could not
catch the failure without swallowing every other error this crate raises,
and was left matching on message text. A rejected key or an unparsable value
is an argument error, so it should raise `ValueError` -- the same thing an
out-of-range partition index gets from `execute` two methods away.

Map through `from_datafusion_error`, which already produces `PyValueError`
and which `SessionContext.sql` already uses in this file, rather than
propagating a `PyDataFusionError` and taking its blanket conversion. That
conversion is deliberately untouched: reclassifying every error out of this
crate is a much wider change with its own compatibility story.

The test can now assert something. It previously ended in
`assert isinstance(excinfo.value, Exception)` under a
`pytest.raises(Exception, ...)` that had already proven exactly that, so the
line could never fail. Assert the type instead, and add a case for a known
key with a value of the wrong type, which reaches the same path by a route a
settings-replay loop is just as likely to take.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Point output_partitioning at a page that discusses partitioning

The docstring referred the reader to `distributed_query_engines`, which is
wrong twice over: that page's premise is that you do *not* partition by hand
because the engine does it for you, and it documents work that is not yet
usable from datafusion-python. A reader following the link to find out what
a scheme means landed on a status page for a different road.

Nothing under docs/source/ discussed plan partitioning from the caller's
side, so give the claim a home on the page that already tells readers to
call `repartition` and `repartition_by_hash` to keep their cores busy --
and that, until now, gave them no way to check whether it worked.

The new section is worth more than a pointer. `repartition_by_hash(col("a"),
num=8)` followed by an aggregation reports `Hash([a@0], 16)`: the optimizer
discarded the requested repartition and inserted its own at
`target_partitions`, so neither the scheme nor the count is what was asked
for. Drop the aggregation and the repartition vanishes entirely, leaving
`UnknownPartitioning` over the source's partition count. Both were verified
against a Parquet source, and both are invisible without this accessor,
which is the concrete form of the have-versus-ask distinction the class
docstring asserts.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Drop PhysicalPartitioning's unused inbound conversion

`From<PyPhysicalPartitioning> for Partitioning` had no callers. The type is
a read-only report of what a built plan does, nothing accepts a partitioning
as an argument, and a Rust caller wanting the `Partitioning` reads it off the
plan, so there is no inbound direction to support.

Removing the impl alone left a deprecation warning: pyo3 auto-derives
`FromPyObject` for a `#[pyclass]` that implements `Clone` and now wants the
choice made explicitly. Omission is not the way to decline it, so say
`skip_from_py_object`, which is what the example crate's providers already
use. Clippy is clean again, and `--all-targets` keeps it that way.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Cover RoundRobinBatch, stop claiming a plan can report Range

The `scheme` docstring listed four values as though a plan could report any
of them. Two of the four needed opposite corrections.

`RoundRobinBatch` is reachable, and now tested. It is easy to miss because
it never survives at the root: the optimizer inserts one only above a source
with fewer partitions than `target_partitions` and CPU work above it to
parallelize, so a single-file Parquet scan under a filter and a grouped
aggregate produces `RepartitionExec: partitioning=RoundRobinBatch(8)` three
levels down. The test walks `children` and asserts all three schemes in that
tree, which also pins that the accessor reads each node's own partitioning
rather than the root's.

`Range` is the other way: it is in the upstream enum, but nothing constructs
one in a physical plan. `RangePartitioning` says optimizer and execution
support is deliberately unimplemented, per
apache/datafusion#22395. Say so, and say why the match arm exists anyway --
it keeps this getter compiling when that support lands, and dropping it
would make the match non-exhaustive.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Tighten three small things around output_partitioning

`scheme` returns one of exactly four strings, so annotate it `Literal`
rather than `str`. The promise is safe to make: the Rust getter matches
exhaustively over `Partitioning`, so a new upstream variant is a compile
error here before it can be a lie in the type.

Bind `output_partitioning` once in the scan half of its test. It was read
three times, and each read clones the `Partitioning` and builds a fresh
wrapper.

Validate the partition index in `execute` before building the
`TaskContext`, not after. Nothing observable changes; the rejected call just
stops doing setup work it is about to throw away.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Stop SessionConfig's constructor aborting on an unknown key

`SessionConfig.set` was fixed to raise instead of panicking, but the
constructor still routed its dictionary through `SessionConfig::set`, which
forwards to `set_str` and unwraps. So
`SessionConfig({"datafusion.runtime.memory_limit": "unlimited"})` aborted as a
`PanicException` -- a `BaseException`, past any `except Exception` -- while the
same key passed to `set` raised `ValueError`.

The constructor is the likelier of the two to meet a bad key: it takes a
`dict[str, str]`, which is the shape a replayed set of settings arrives in, and
`information_schema.df_settings` lists eight `datafusion.runtime.*` rows that
cannot be set from a session config at all.

Route each entry through `options_mut().set` and map with
`from_datafusion_error`, so both routes reject the same keys the same way. The
`ScalarValue::Utf8` wrapper went with it: the parameter is already
`HashMap<String, String>`, and upstream only called `to_string()` back on it.

Which entry a dictionary with several bad keys reports is unspecified, since
`HashMap` iteration order is arbitrary. Documented rather than sorted -- a
caller fixes the reported key and runs again.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Say where Range partitioning actually comes from

Two docstrings claimed no plan reports `Partitioning::Range` because upstream
support was unimplemented. Neither half holds for DataFusion 55:
`repartition/mod.rs` routes range partitioning through `RangeExpr`, and
`physical_planner.rs` has a test asserting a planned `RepartitionExec` reports
it. The comment's reasoning was also inverted -- the match arm compiles today
because the variant exists today, not in anticipation of it landing.

What is true is narrower: nothing in this package's own API asks for one.
`repartition` requests round-robin, `repartition_by_hash` requests hash, and
SQL has no range-repartition syntax. But a plan need not have been built here.
`datafusion-proto` encodes and decodes physical range partitioning and
`datafusion-ffi` carries it in both directions, so `ExecutionPlan.from_bytes`
can return a plan reporting it, as can an extension library's query planner.

State that where each reader is: the reachability in the guide beside the rest
of the scheme discussion, one sentence and a `:ref:` in the docstring. Also
note that `hash_expressions` is `None` for `Range`, which leaves the ordering
and split points reachable only through `repr`.

No test: reaching `Range` from Python needs a plan built in Rust, and a Rust
test here would never run in CI.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Describe expr.Partitioning as what it is, not as an argument

Both docstrings introducing `PhysicalPartitioning` distinguished it from
`datafusion.expr.Partitioning` by calling that one "the partitioning
`repartition_by_hash` asks for". No method takes one: `repartition` takes a
count and `repartition_by_hash` takes expressions and a count. The type cannot
be constructed from Python at all -- `Partitioning()` raises `TypeError` -- has
no public members, and is only ever handed back by
`Repartition.partitioning_scheme()`.

Describe it that way instead: the logical partitioning a `Repartition` node
records. The request-versus-result contrast the docstrings were reaching for is
real and worth drawing, so keep it, but attach it to the node that holds the
request rather than to a parameter that does not exist. The Rust comment also
notes that the two enums differ, since the logical one has `DistributeBy` and
no `UnknownPartitioning`.

Pin the contrast with a test rather than only asserting it in prose. A
`repartition_by_hash(num=8)` with nothing above it to consume the redistribution
is dropped by the optimizer, so the request records Hash into 8 while the built
plan reports `UnknownPartitioning(2)` -- disagreeing on both scheme and count,
which is the reason the two types stay separate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Give PhysicalPartitioning equality, and tighten four small things

Drop the guide sentence claiming every config method mutates in place and
returns itself. It is true, but every `with_*` docstring still promises "a new
`SessionConfig` object", and a guide that contradicts the API docs it links to
is worse than a guide that stays quiet. Correcting fifteen docstrings is its
own change.

`PhysicalPartitioning` gains `__eq__` and `__hash__`, computed structurally
from scheme, partition count and hash expressions. Deliberately not delegated
to `Partitioning`'s `PartialEq`, whose match lists no `UnknownPartitioning`
arm and so falls through to `false`: two identical `UnknownPartitioning(2)`
values are unequal there. That is defensible for deciding whether a
partitioning satisfies a distribution requirement, but a non-reflexive
`__eq__` would be a trap in Python, and `UnknownPartitioning` is what an
ordinary file scan reports. `__hash__` comes along so the class stays usable
in a set.

`test_output_partitioning_reports_round_robin` asserted set equality over
every scheme in the tree, pinning optimizer output that is not the property
under test. Membership instead, and 50 rows rather than 2000 -- the round
robin appears either way, so the larger file bought nothing.

The PyO3 parameter is now `partition` to match the wrapper, so
`ctx.execute(plan, partition=0)` works on the internal binding too, and the
test covers the keyword form alongside the positional one.

`from_bytes` had its false memory-table sentence removed but got nothing back,
so it no longer said that a decoding session need share nothing with the
encoder -- the property that makes it useful. Restored from the reader's side.

Also cover the `OverflowError` a negative index raises, which was documented
on `execute` but never exercised.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* File the SessionConfig tests with the other SessionConfig tests

The two `SessionConfig.set` tests went into `test_plans.py` because they were
committed alongside the `execute` bounds check, not because they have anything
to do with plans. `test_context.py` is where `SessionConfig` construction is
already covered, and it now also holds the three constructor tests for the
other half of the same panic defect.

Move them there, ahead of the constructor cases so the method they refer back
to is read first, and fold the duplicated note about `PanicException` deriving
from `BaseException` into the first of the five. `test_plans.py` keeps its
`SessionConfig` import for the `with_target_partitions` calls in the
partitioning tests. No assertion changed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* Import SessionContext in the SessionConfig constructor doctest

The example used SessionContext but only imported SessionConfig, so it
passed under --doctest-modules yet failed when copy-pasted.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs: explain S3 object-store configuration for SQL

Signed-off-by: Yifan Chen <emecii23@gmail.com>

* docs: demonstrate SQL reads through registered object stores

* docs: simplify S3 SQL example wording

---------

Signed-off-by: Yifan Chen <emecii23@gmail.com>

The `pub mod indexed_field;` declaration and the PyGetIndexedField class
registration were dropped from crates/core/src/expr.rs in b5446ef (apache#728)
when DataFusion 39 replaced Expr::GetIndexField with the FieldAccessor
trait, but the file stayed on disk. It has not been compiled since, and it
imports GetIndexedField, which no longer exists upstream.

The nightly pre-commit rust-fmt hook formats files by path, so it keeps
rewriting the orphan; stable `cargo fmt --check` in CI walks the module
tree and never sees it. Deleting the file ends that churn.

Closes apache#1745

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

* chore: harden the extension API before it ships

`SessionContext.with_extensions` and `SessionExtensionComponents` landed
in apache#1679 and have not shipped in a release yet. The bundle stack
(apache#1738-apache#1741) reshapes them substantially, and a release would freeze
three surfaces in their current form.

`PhysicalOptimizerRuleExportable` was defined in `datafusion.context` and
not exported from the package root, so `datafusion.context` would become
its canonical import path. Move it to `datafusion.extensions` beside the
rest of the `*Exportable` family, re-export it from `datafusion.context`
so the old path keeps working, and export it from the package root. The
move brings it under `test_extension_api_has_a_doctest`, which drives off
`extensions.__all__`, so it gains the example it was missing.

`SessionExtensionComponents` was positionally constructible with two
fields. The stack takes it to nine, three of them pair-shaped. Make
construction keyword-only so every later field addition is additive; no
call site in the repository constructed it positionally. This is a new
convention rather than a backport, so it has to be applied forward to the
stack as well.

The ordering that makes `with_extensions` transactional was stated in
three docstrings with no canonical home to point at. Record it under
`ffi_internals_commit_order` in the contributor guide, and label the
existing "Failure and rollback" section `extension_bundles_transaction`,
matching the names the stack links to.

No released behaviour changes, so no `api change` label and no
upgrade-guide entry.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* refactor: narrow the extension protocol surface

The capsule-getter protocols are annotations, never arguments: a bundle
author constructs a SessionExtensionComponents but only ever names
SessionComponentsExportable in a type hint. Exporting the hints from the
package root made this one family the exception among sixteen such
protocols, every other one of which is reached through its defining
module.

Drop PhysicalOptimizerRuleExportable, QueryPlannerExportable,
SessionComponentsExportable, and SessionPlannerExportable from the root
(__all__ 58 -> 54), keeping SessionExtensionComponents, which is the one
name a bundle constructs. The three bundle protocols are new in 55.0.0,
so no import path is lost.

PhysicalOptimizerRuleExportable shipped in 54.0.0 from datafusion.context,
so its move is a break: context.py now imports it under TYPE_CHECKING only,
and the upgrade guide records the new path. Nothing else changes for a rule
author -- the protocol is structural and not runtime-checkable, and
add_physical_optimizer_rule is untouched.

Also removes three now-dead autoapi skip entries, repoints three doctests
that imported from the root, and fixes the add_physical_optimizer_rule
cross-reference, which stopped resolving once the class left context.py.

test_extension_protocols_are_exported_together asserted the premise this
reverses, so it goes. The five doctests in extensions.py already prove the
classes exist there, and SessionExtensionComponents' own docstring pins
the remaining root export.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs: state the commit-order invariant correctly

The ordering rule said step 4 "cannot raise part-way through", and the
comment in `with_extensions` said "everything above is allowed to raise;
this is not". Neither holds: `_install_extension_planner` runs
`ffi_query_planner_from_pycapsule` before it calls
`set_session_query_planner`, so the commit step has fallible work of its
own.

The guarantee survives, because that import happens before the write. But
the passage is written as a rule for whoever adds the next component kind,
and as phrased it asks them to preserve a property the code does not have.
Restate it as what actually holds: every fallible operation, including the
ones inside the commit, completes before the first write.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test: verify the physical optimizer rule docstring example

The `+SKIP` block in `PhysicalOptimizerRuleExportable` named
`test_ffi_physical_optimizer_rule_runs_during_planning` as the test that
runs it for real, but that test never reads the docstring. It is a
separately written test that happens to call the same two APIs, so it
catches a renamed method or module only by coincidence, and cannot see an
edit to the docstring at all.

Add the mirror the convention actually asks for, in the shape of
`test_with_extensions_docstring_example_still_runs`: parse the live
docstring, keep only the skipped statements, drop the skip, and run them.
Only `ctx` is supplied, because the skipped statements go on using the
context the runnable block above them opened.

Verified by mutation. Renaming the imported class in the docstring fails
with `NameError: name 'MyPhysicalOptimizerRule' is not defined`, and
deleting the block fails the `assert examples` guard rather than passing
vacuously.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* refactor: keep bundle protocols out of datafusion.context

datafusion.context imported QueryPlannerExportable,
SessionComponentsExportable and SessionPlannerExportable at runtime,
so they were reachable as datafusion.context.* despite
datafusion.extensions being their one home. Move them under
TYPE_CHECKING and route the runtime isinstance checks through a
private _extensions module alias. The protocols are new in 55.0.0,
so no released import path is dropped.

Add a test pinning that all four capsule-getter protocols live only
in datafusion.extensions, and list PhysicalOptimizerRuleExportable
in llms.txt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* test: pin keyword-only construction; scope the rollback promise

Add a test that SessionExtensionComponents rejects positional
arguments, so dropping kw_only=True fails the suite. Every existing
caller passes keywords and would stay green without it.

Drop the absence assertions from the extension protocol import test.
Re-exporting a name is additive and breaks no caller, so asserting a
name is missing only adds friction for a later deliberate export.
Keep the positive check that each protocol imports from
datafusion.extensions.

The contributor guide promised that a raising bundle leaves the
session as it was without the carve-out for writes a hook makes to
the context it is handed. Add it with a ref to the bundles guide,
which is where the exception is explained.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

ci.yml and release.yml only matched main, so PRs to and merges into
branch-* release branches ran no build or tests.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

…ptions (apache#1761)

`write_parquet` takes `write_options`, documents it and declares it in
the `@overload` for the `ParquetWriterOptions` form, but that branch
calls `write_parquet_with_options(path, compression)` and drops it. The
destination accepts the parameter, so everything in
`DataFrameWriteOptions` (`partition_by`, `single_file_output`,
`insert_operation`, `sort_by`) was silently ignored for that one
spelling.

Measured with the same `DataFrameWriteOptions(partition_by="part")`:

    write_parquet(path, ParquetWriterOptions(), write_options=wo)
        -> ['IDuOjvMa3pdEDotb_0.parquet']     not partitioned
    write_parquet_with_options(path, ParquetWriterOptions(), write_options=wo)
        -> ['part=a', 'part=b']
    write_parquet(path, "zstd", write_options=wo)
        -> ['part=a', 'part=b']

No error and no warning: the files just land in the wrong layout.

The branch arrived in ef62fa8 (apache#1169) while `write_options` came
earlier in apache#857, so the new delegation path was written without carrying
the existing parameter over. The same `if` refuses `compression_level`
with an explicit `ValueError`, which shows that arguments incompatible
with this branch get rejected on purpose; `write_options` was not
rejected, only forgotten.

* fix: Make default sort order nulls last

* Add entry to upgrade guide

Add ty (pinned to 0.0.84, since it is pre-1.0) to the dev dependency
group, configure it in pyproject.toml to check python/datafusion, and
run it in the lint-python CI job and as a local pre-commit hook.
datafusion._internal ships no stubs, and pandas/polars are optional
TYPE_CHECKING-only imports, so they are allowed to stay unresolved.

Fix the diagnostics ty reported:

- Make the internal udtf decorator helper require `name`, matching the
  public overloads. `@udtf()` previously passed None into Rust and
  failed with "'None' is not an instance of 'str'".
- Give AggregateUDF.__init__ defaults matching its FFI overload, and add
  the same FFI overload to ScalarUDF and WindowUDF.
- Add None/non-None overloads to expr_list_to_raw_expr_list and
  sort_list_to_raw_sort_list so callers that unpack the result type
  check.
- Stop rebinding typed *args and parameters to values of other types.
- Import warnings.deprecated behind a sys.version_info check and drop
  the unreachable importlib_metadata fallback.
- Fix smaller annotation mismatches (LogicalPlan.__eq__,
  spark._coerce_i32, CSV file_compression_type).

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* Cargo update

* build(deps): roll up open dependabot updates

Bump GitHub Actions (taiki-e/install-action, astral-sh/setup-uv,
github/codeql-action init+analyze) and uv.lock packages (virtualenv,
urllib3, tornado, pyjwt, soupsieve). CodeQL init and analyze must be
bumped together; separately they fail with a config version mismatch.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>