Repository navigation
Next pass: Ledger composition variance; overlay silences from the loop-engineering confirmation regens #35
Description
Filed from the guide-loop-engineering-residuals promotion pass (matrix:
docs/research/guide-loop-engineering-residuals-matrix.md). Nothing here is applied; every item
below has its proposed fix recorded but not made (fix and test travel together).
1. Ledger composition varies between regenerations of the same mode (headline)
Nine Opus 5.5 clean-room regens of the loop-engineering course on near-identical guides:
agnostic 10 / 13 / 11 / 14 rows; lab 5 / 5 / 5 / 5 / 6. The stable core is the same everywhere
(six learner rows in standalone mode; instruction placement; grader realization; acceptance-suite
model). The variable set: model-per-role (an L4 option, a design-argued row, or a lever),
persistent store (row or §2 default), hosted tracing (row or Not Included), event-intake surface
and "what a triggered run executes" (row, rule, or narrative). Every reading is defensible under
§5.5's surfacing bar; the bar does not force one answer on these subjects. This is #28 F1's
problem in its general form, and the three tie-breakers added so far fixed only the three
subjects that split then.
What the pass did about it: §4 Pass B now ships the mined trade-off table in CTX-C (every spoken
trade-off, lesson, disposition), so each regen's reading is written down and an omission is
visible. Proposal for the next pass: decide the variable subjects one by one from the nine
recorded dispositions (which is the course's own answer? which reading do testers prefer?) and
encode each as a rule, rather than widening the litmus again. Candidate mechanical rule: a
subject that appears in the trade-off table as row in one mode must not be narrative in the
other for the same course — the table makes this checkable.
2. Overlay silences reported by the lab regens (precedence rule 3)
- Name the spec section that carries the
model roles:checklist line (currently inferred: §2
"Model roles" row + §6 Always). - Where the Python version is recorded (regen chose
acceptance-report.md). - The served app's default model mode when RR1 covers only the suite (regen chose
live). - Whether a preinstalled web framework is preferred over the course's unpinned one (regen kept
Flask per base §5 step 5; another regen switched to FastAPI). - Proxied-preview request timeouts for long synchronous runs.
- A course
.envloaded with override clobbers the ambient keys — one regen added a rule and an
environment failure mode on its own; the overlay should state it. - Whether "never ask the learner for a key" extends to non-model keyed tools (LangSmith).
- Add the overlay's three resolution words (ANSWERED / CONSTRAINED / KEPT) and "keyed
dependency" to the required CTX-C how-to vocabulary.
3. Base-guide tensions noticed
- §0 step 3's "(course default)" labelling duty has no target on the scope-boundary row (the
course made no scope choice). Say so in §6.0 or the scope-row rule. - Code-only evidence that carries the deliverable's main surface (the event app has no transcript
lesson) has no stated weight beyond "notebook dump". - §8's fixed citation form says nothing about position; the checker counted 0 ACs when the
citation sat inside the AC-id cell. Regex widened on the branch; the rule could name the
position.
4. Documented residuals (not defects of the guide)
P-A4 (one to three unreached non-enum schema constraints per regen) and P-AU3 (one or two AC
replays resting on an unstated assumption per regen) recur at low intensity in every regen of
both modes. They are generator-diligence slips of the class the /generate-spec step 4b
independent audit catches at handoff; no further guide wording is proposed for them.
Activity
Carr1005 commentedon Oct 12, 2026
Addendum: PR #36 also pins the opening blockquote (§6: four elements, under 320 words, one-sentence provenance) and the orientation lead (artifact noun phrase + promise first, then the demo moment in second person; property P-LEAD). P-LEAD was validated by one regen (regen-E); its N=2 confirmation rides the next pass's regens. Matrix:
docs/research/guide-loop-engineering-residuals-matrix.md, the two post-confirmation sections.Carr1005 commentedon Oct 12, 2026
Two more items from PR #36's post-confirmation work, with evidence:
Standalone mode pins no deliverable shape. regen-E (agnostic, Opus 5.5) demoted the course's Flask web app to a restorable Not Included item and specified a command-line reviewer; regens A–D, F and the committed spec keep the web app. The lab overlay has a shape rule (ADD 4); the base guide has none — §3's project precedence says "re-express the course's example scenario shape" without saying whether a user surface the course shipped is part of that shape. Design decision for the next pass: the project row's default keeps the course's user-facing surface when the course shipped one; dropping it to Not Included is not a default. Evidence:
experiments/loop-engineering/guide-loop-engineering-residuals/regen-E/vs the others; matrix "Post-confirmation addition 2".Orientation lead (P-LEAD), N=2 owed. The first wording of the rule caused the regen-E regression above (the lead was written first and §1 bent to it). The anchored wording — written last, names the project row's Default as §1 states it, decides nothing — passed one regen (regen-F) by eye. Its second confirmation rides the next pass.
Carr1005 commentedon Oct 12, 2026
Checklist item for every promotion pass (from PR #36's review): grep the guide lines added by the pass for course vocabulary (
git diff <base>..HEAD -- spec-generation-guide.md | grep '^+' | grep -i -E '<course terms>'). On PR #36 it found one leak — the §3 scripted-model rule named one course's four loops instead of 'every stage the pattern wires around the model' — fixed on the branch. Illustrative examples in the guide's existing style are fine; rules must be stated in guide concepts (wording-discipline rule 2).Carr1005 commentedon Oct 12, 2026
From PR #37 (the lab spec regenerated through the real
/generate-specwith steps 4a/4b):spec.coding-agent-lab.md§7 chains A/B/C oncourse/loop-engineering-regen.design-structuralwith two options instead of arealizationrow. Say what to do when the course's realization is already the stand-in.AGENTS.md: the runtime's subfolder clause covered it; state the collision explicitly. (b) OVERRIDE 2 routes a role-assignment trade-off (separate grader model, "a common production choice") to a lever + Ask First where the base guide would make adesign-arguedrow; confirm that is the intended reading or carve out role assignments.Observation, not a defect: a fresh generation on the confirmed guide still needed three audit rounds (17 → 7 → 3 hard). The mechanical class is gone (checker 0 at handoff); what remains is cross-AC state and fixture/script coverage, both judgement-class. The audit gate is doing its job; the guide can still shrink the first-round count.
Loop-engineering lab spec: clean-room regeneration on guide 4e1b71e /…
Carr1005 commentedon Oct 18, 2026
Note (2026-10-19): the
/promote-to-guideskill that "promotion pass" refers to is not maintained right now; agents no longer run it on their own (PR #54). Known gaps are in #53. This issue's findings are unaffected.Carr1005 commentedon Oct 19, 2026
Closed in favour of one list. Resolved here: model-roles line and scope-row label (PR #57), blockquote pin (PR #36), §4 residuals. Open items moved to #48 as N6–N20 (#48 (comment)), plus effects on U5, U6, U8, U14; the promotion-pass grep step is moot (#53).