perf(akita): keep auxiliary commitment buffers compact and reuse their owner - #2185

📏 PR diff

_{Every changed line is classified once.}

Benchmark comparison (crates)

Same-runner A/B: base_run = 894ed05 (merge target) vs pr_run = 1b2c250 (PR head merged onto it). Bold = faster run; Δ colored when the difference exceeds the ±10% noise threshold.

2 of 92 benchmarks differ by more than ±10%:

Review: changes requested (merge blocked on the Akita fork pin)

Reviewed head 90bdaf0ccc656487861083dca510306294615278 (draft).

The compact-prefix path is correct. It gives byte-identical proofs to main for the inputs tested. It cannot merge yet because it pins a personal Akita fork and needs a fork-only API. Both independent reviews request changes for that reason: Claude Opus and GPT-6 Astra (high).

Blocker: the Akita fork pin

Cargo.toml:306-314 points all nine Akita crates at GUJustin/akita-layerzero@ec0efc73. The Jolt code calls DensePoly::from_field_prefix (scheme.rs:721, 735), which exists only on that fork. So repinning to upstream main (038db2d1) does not compile.

Path to merge:

1. Land 1a86a8cc and ec0efc73 in LayerZero-Labs/akita. They apply cleanly on upstream main 038db2d1.
2. Repin all nine crates to the landed upstream revision.
3. Re-run the compact/fallback, Akita, field-inline and verifier-fixture checks.

ec0efc73 is needed to compile. 1a86a8cc is not needed for correctness, but without it there is no "reuse their owner" saving: a same-owner import would get a fresh id and recompute.

What this PR changes in Akita (f594bfc to ec0efc73)

Is it protocol-breaking?

• For Jolt, no. Jolt's shipped schedules are full width, so ed90d495 accepts them unchanged. The Opus review compared main (akita f594bfc) against this head on three programs:

  • field_ops, field_ops with 8 words of untrusted advice, and muldiv;
  • optimized backend, Akita, field-inline.

  It serialized the proof, the verifier preprocessing and public_io for each: 9 of 9 files are byte-identical. Separately, the Astra review found that compact and full inputs give identical commitments across 9 prefix lengths, and identical proof bytes for a fixed input at this head.

• For Akita on its own, ed90d495 strictly shrinks the set of setups the verifier accepts. It now rejects shallow setup-prefix digit layouts (for example q128 with log_basis=1, num_digits=1) that were accepted before. It is a soundness fix: it closes a truncated-setup-commitment hole, and no shipped artifact changes. It is worth calling out explicitly when repinning rather than describing the bump as "protocol unchanged".

• API: additive only. The new trait methods TransparentObjectSetup::commit_bounded_object and MultilinearPoly::dense_evaluation_prefix both have defaults, so no downstream implementor breaks.

What was checked and holds

• Zero suffix. The implicit zero suffix is equivalent to materialized zeros. Every fork consumer clips to the non-zero rings but keeps the logical block count and matrix row width. dense_evaluation_prefix is Some only when t_stride == 1, which is exactly the contiguous prefix.
• Generic PCS fallback. commit_bounded_object's default zero-extends and calls commit, which is the old FusedInc body. It now gives an explicit error where resize used to silently truncate.
• Bounds stay separate. The bounded and full-width catalogs are still distinct even when they share a backend, and a compact 2^80 coefficient is still rejected on the bounded path.
• Same-owner import. It clones an immutable Arc, so there is no shared-mutation hazard. release_post_commit_ntt_residency skips backends that are Arc::ptr_eq to the one-hot owner. Foreign handles are still recomputed.
• Verifier. None of the changed Jolt paths is reachable from the verifier.

Lower-priority notes

1. Stale comment. native_batching.rs:307-312 still says auxiliary objects are committed on their own dense backends and that every import re-derives the commitment. After this PR, FusedInc and full-width objects are committed on the one-hot owner, and the import is an Arc clone.
2. Repeated decoding. commit_bounded_object calls schedule_artifacts.bounded_dense_scheme() (scheme.rs:718), which re-decodes and audits the dense artifact on every call. setup.verifier.dense_scheme() already holds the decoded scheme. This is still cheaper than before, so it is not a regression.
3. No shape check. commit_bounded_object skips validate_commit_shape. The backend still rejects bad arity, so this is about symmetry with commit, not correctness.
4. Fork, prover side. ValidatedSparseChallenges::new now validates only the challenges for live rings. These are prover-derived, so there is no soundness impact.
5. Conflict with perf(field-inline): retain only the increment prefix #2184. Both PRs add an accessor at the same spot in field_inline.rs. When resolving it, keep perf(field-inline): retain only the increment prefix #2184's storage bound.

Prepared with Claude Code (Opus 5.5), an independent Claude Opus review and an independent GPT-6 Astra (high) review; findings were reconciled.

Pushed: Akita pin moved to LayerZero, merge with main, comment fix

With Justin's OK:

• Upstream Akita PR. Opened LayerZero-Labs/akita#292 from the LayerZero-owned branch astra/compact-dense-prefix. It carries Justin's two commits unchanged: 1a86a8cc (same-owner import) and ec0efc73 (compact dense prefix).
• Pushed to this branch (a fast-forward, with no force):
  1. Merge branch 'main'. The only conflict was perf(field-inline): retain only the increment prefix #2184's accessor in field_inline.rs. Both accessors are kept: prefix_len() is the convention across the kernels, and evaluation_prefix() is what dense_evaluation_prefix needs.
  2. chore(akita): pin the compact-source commits on LayerZero-Labs/akita.
     • All nine Akita crates (and the 16 lockfile entries) now point at https://github.com/LayerZero-Labs/akita.git with the same rev = ec0efc73, the head of akita#292. So this PR no longer depends on the personal fork, and the Akita source is byte-identical to what was reviewed.
     • The stale native_batching.rs comment is updated. FusedInc and full-width objects are committed on the trace backend, so only foreign handles are recomputed.

Still needed before merge: once akita#292 merges, repin from ec0efc73 to the resulting main commit, because a squash merge changes the SHA. After that the PR can leave draft.

Not changed here: the bounded_dense_scheme() re-decode, and the validate_commit_shape symmetry in commit_bounded_object. Both are Low and are not regressions.

Validation on the pushed head:

• cargo metadata --locked resolves the new pin;
• clippy -D warnings on jolt-akita, jolt-kernels and jolt-openings with --all-features: clean;
• cargo nextest -p jolt-kernels -p jolt-akita -p jolt-openings --all-features: 319/319 passed, with one test skipped (the ~8-minute production-grid case);
• CI is re-running on the new head.

Prepared with Claude Code (Opus 5.5).

Verifier classifier: risk-4

Head 1b2c2509fd097d77a427e02f93f002646d438703

The Akita dependency upgrade reaches verifier, PCS, transcript, and serialization code, so acceptance changes cannot be ruled out. Catalog admission also feeds trusted verifier preprocessing. Other executable changes are confined to prover commitment generation and backend ownership; the native-batching change is comment-only.

Risk: risk-4

• Cargo.lock: risk-4 — Updates resolved Akita verifier, PCS, transcript, and serialization dependencies used during verification.
• Cargo.toml: risk-4 — Changes the pinned Akita revision, including verifier and protocol dependencies; acceptance equivalence cannot be established.
• crates/jolt-akita/src/adapters.rs: risk-2 — Changes prover-only backend ownership and cache trimming, not verifier setup data or acceptance checks.
• crates/jolt-akita/src/native_batching.rs: risk-1 — Only updates a comment explaining prover handle imports.
• crates/jolt-akita/src/schedule_artifacts.rs: risk-4 — Changes the catalog-admission constructor feeding verifier_setup_from_params and trusted grouped schedules; equivalence of the upstream constructors cannot be established.
• crates/jolt-akita/src/scheme.rs: risk-2 — Changes prover backend reuse and commitment-source construction. Verifier metadata construction, verification methods, and proof encoding remain unchanged locally.
• crates/jolt-kernels/src/field_inline.rs: risk-2 — Adds a prefix accessor used only by the prover's compact commitment-source path.
• crates/jolt-kernels/src/reference/akita/increments.rs: risk-2 — Routes witness commitment generation through the bounded-object helper; verifier claims and checks are unchanged.
• crates/jolt-kernels/src/trace_column.rs: risk-2 — Exposes compact evaluations only to the prover's full-width commitment path; polynomial evaluation and verifier checks are unchanged.
• crates/jolt-openings/src/schemes.rs: risk-2 — Adds a prover-only commitment helper whose default implementation zero-pads and invokes existing setup and commit methods.
• crates/jolt-poly/src/multilinear.rs: risk-2 — Adds an optional prefix hook consumed only during prover commitment generation; existing evaluation behavior is unchanged.

gpt-6.1-sol, reasoning effort xhigh