Visitar URL original
benchmarks: one stage generator, make_2611_stages.py; delete the 2026-10 generators (#430) by tae898 · Pull Request #431 · humemai/arcadedb-embedded-python · GitHub
Skip to content

benchmarks: one stage generator, make_2611_stages.py; delete the 2026-10 generators (#430) - #431

Merged
tae898 merged 2 commits into
mainfrom
one-stage-generator
Oct 21, 2026
Merged

tae898 merged 2 commits into
mainfrom
one-stage-generator

Conversation

@tae898

@tae898 tae898 commented Oct 21, 2026 •

Copy link
Copy Markdown

make_2611_stages.py is now the only stage generator. It was a profile of make_2610_stages.py, itself a profile of make_october_stages.py, and rewrote the October template with _sub text anchors. The template (HEAD) now takes the instrument, REPS and the banner as parameters (stage_params), the stage table holds only the lanes the 2026-11 subset stages (graph, cross-model, documents, dense, sparse, time series, lifecycle, plus the restricted-arm stages derived from runner.ARM_RUNS), and nothing is rewritten after the fact. DECISIONS #196: "october was dev".

Proof that no stage moved, apart from two intended lines: before editing I recorded sha256 of every draft three generations write with the old chain (comparators at the default order and 3 reps, comparators in paper order at 5 reps, ArcadeDB alone on a release for every lane with the second list). The first commit's drafts, stdout and --check output were identical byte for byte; the second commit then changes exactly two things in every lane stage (the pair-install script is unchanged): the banner no longer names the deleted generators, and the grep -q 'def instrument_at_least' bench_common.py check is gone (it was asked for by #429's owner; bench_common keeps the function). The diff of one draft (qS03.sh, old to new):

4,5c4,5
< # GENERATED by make_2611_stages.py (a profile of make_2610_stages.py and make_october_stages.py) -- edit
< # those files, not this one, and
---
> # GENERATED by make_2611_stages.py -- edit
> # that file, not this one, and
35d34
< grep -q 'def instrument_at_least' bench_common.py || { say "$ID ABORT: no BENCH_INSTRUMENT selector in this tree"; exit 1; }

stage_drafts_2611.sha256 holds the 32 hashes of the new drafts and test_the_drafts_are_the_ones_on_file compares them on every run (regenerate with python3 test_stage_generator_2611.py --rewrite after reviewing a deliberate change).

Deleted: make_october_stages.py, make_2610_stages.py, pass2_roster.py and the tests that only exercised 26.10.x passes (test_pass2_roster.py, test_stage_pass_tiers.py, test_stage_rep_pass.py, test_stage_top_up.py, test_stage_tiers.py, 82 tests), oct_repin_smoke.sh, oct_skeleton_arcadedb.sh, structure_check.check_stage_chain (it held CAMPAIGN.md's October chain against the deleted table), the october_stages/ ignore entry, the host-side Python-cost, e4 and restart stage templates that the 2026-11 generator never emitted, and the CAMPAIGN.md rows and paragraphs that only describe the 2026-10 passes (rows 45, 46, 53, the 26.10.1 measurement paragraph, the October chain drawing). check_coverage stays, now for the lanes this table stages, and --check runs it. Re-pointed: cap_rule.known_backends (reads runner.LANES), test_cap_rule.py, test_jpype_pin.py, test_cell_cost_backends.py, test_jvm_defaults_arm.py, test_stage_generator_after.py, and the comments in bench_common.py, Dockerfile.bench, build_images.sh, queue_lint.py; the docs lines in CAMPAIGN.md, PROTOCOL.md, PUBLISHING.md, READING-RESULTS.md and QUANTIZATION.md.

git grep for the deleted generators' names is now empty.

--arcadedb-release X.Y.Z already emits an ArcadeDB-only stage set on a new release (the pair install, then every ArcadeDB arm with the exactly-once proof), which is what #326 asked of the old generator, so this closes it.

Tests (benchmarks/experiments, nice -n 10 taskset -c 12-15): before 1007 passed, 1 failed, 24 skipped; after 918 passed, 1 failed, 24 skipped. The 89 fewer are the 82 deleted tests, 2 from test_stage_generator_after.py, 4 host-stage jpype tests, 2 fewer cap-rule modes, and 1 added (the sha test). The one failure, test_dense_ef_sweep.py::test_export_adds_the_matched_columns_only_for_2026_11, fails identically on origin/main in this worktree (the worktree lacks the untracked bench data that main's checkout has, which I copied in read-only for the run); it is not touched here. The same suite on this branch merged into #429's branch (instrument-2611-only): 850 passed, same 1 failure, 24 skipped; test_check_refuses_another_instrument_in_the_environment now accepts the refusal from either layer (the generator's own check or bench_common at import). None of my tests read untracked results/. No Maven, Docker, benchmark cells or timed runs. pre-commit passes on every changed file.

Closes #430
Closes #326

🤖 Generated with Claude Code

tae898 and others added 2 commits October 22, 2026 00:02
make_2611_stages.py was a profile of make_2610_stages.py, itself a profile of
make_october_stages.py, and rewrote the October template with text anchors.
It is now the only generator: the template takes the instrument, REPS and the
banner as parameters, and the stage table holds the lanes the 2026-11 subset
stages. The drafts it writes are byte-identical to the ones the chain wrote;
stage_drafts_2611.sha256 and a test hold them.

Deleted: make_october_stages.py, make_2610_stages.py, pass2_roster.py, the
tests of the 26.10.x passes (--rep-pass, --rep-from, --roster, --pass-tiers),
the two October re-pin smoke scripts, structure_check.check_stage_chain, the
october_stages/ ignore entry and every doc row that only described them.
Re-pointed: cap_rule.known_backends, the tests that imported the old
generators, the comments that named them.

Closes #430
Closes #326

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…anner from the drafts (#430)

The two intended differences from the pre-PR drafts: the banner no longer names
the deleted generators, and the stage no longer greps bench_common for
instrument_at_least. The refusal test accepts the bench_common layer too.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

benchmarks: one stage generator (make_2611_stages.py); delete the 2026-10 generators and their pass tooling benchmarks: make_2610_stages.py cannot emit an ArcadeDB-only pass on a new release

1 participant