Repository navigation
tests/baselines/parser-corpus.manifest is stale on main; drift accumulates until a parser PR carries it #2186
Description
Found while regenerating for #2160/#2185. Independent of that change.
tests/baselines/parser-corpus.manifest does not match what the current
main produces. Running scripts/regen-corpus-baselines.sh on a clean main
checkout yields:
- 3 modified hashes —
apyb-financeiro/financeiro_main.beancount,
donearm-ledger/accounts.beancount,fava/tests_data_example.beancount - 20 new entries — corpus files with no baseline at all, presumably added
byfetch-compat-test-files.shafter the manifest was last regenerated
Why it went unnoticed
The Corpus output baseline job is path-filtered: it runs when the parser or
formatter changes. Nothing had touched those paths in a while, so the drift
accumulated silently and only surfaced on the next parser change — which then
has to carry it.
That is the problem worth fixing. The gate is sound, but a PR that legitimately
shifts parser output cannot regenerate only its own files: the script
regenerates the whole manifest, so unrelated drift rides along in the diff and
has to be explained by whoever happens to be next.
Suggestion
Either regenerate on a cadence that does not depend on someone touching the
parser, or have the job report drift (without failing) on every PR so it cannot
accumulate. The 20 missing entries in particular mean those corpus files have
had no baseline coverage at all since they were added.
Note
#2185 regenerates the manifest and therefore includes this drift. Its
description separates the two: 10 modified hashes are from that change (all
files containing note directives), the other 3 and all 20 additions are this.
Activity
fix(core): collect a note's tags and links in the visitors
fix(core): keep a note's tags and links (#2185)
ci: report corpus baseline drift on a cadence instead of by accident (#…
robcohen commentedon Sep 1, 2026
Closing — both halves are resolved and verified on main today.
Part 1, the stale manifest. Gone. Against a freshly fetched corpus on current main:
The 3 modified hashes and 20 missing entries this issue listed were carried by #2185 and #2189.
Part 2, the accumulation, which was the real problem. Addressed by #2196.
One correction to this issue's diagnosis: it attributes the drift to a path filter on the baseline gate. There isn't one —
parser-baselines.ymlhas nopaths:key and runs on every PR. The actual mechanism is two deliberate decisions meeting:hashFiles('scripts/fetch-compat-test-files.sh'). That key changes when the script changes, never when upstream does, so CI replays a frozen corpus and every PR agrees there is no drift.Neither is wrong alone, and tightening either costs what its own comment explains. #2196 adds the missing third thing: a weekly job that fetches fresh, no cache and turns drift into an issue.
It has already run and filed #2199, which surfaced a limitation now fixed in #2211 — a failed clone and a deleted file are indistinguishable after the fact, so the fetch now records its failure count and the report drops the "file is gone" group when clones failed.