Repository navigation
promote-to-guide: known gaps (skill not maintained) #53
Description
Status: not scheduled. Records what would need fixing to make /promote-to-guide useful again.
Why the skill is marked not maintained. SKILL.md was last updated on 2026-09-08. The skill was invoked once as a whole (2026-09-07). Since 2026-10-13, guide changes have been made by a different process (handoff doc docs/handoff/2026-10-14/README.md §1 and §3, on branch handoff/2026-10-14, not on main). The skill now has disable-model-invocation: true, so agents no longer see or run it on their own; it can still be run as /promote-to-guide.
Still in use — do not move or edit without telling whoever relies on it: references/regen-prompt.md. Clean-room regenerations on 2026-10-17/18 (R1–R16) read its prompt by path and line numbers (lines 17 onward).
Known gaps (skill text vs current practice)
| # | Skill says | Current practice | Source |
|---|---|---|---|
| 1 | Triage a set of candidates and promote them in one pass (Phases 1–2) | One guide/overlay rule per PR, only what the owner asked for | handoff §1 rule 1 |
| 2 | Fix and regenerate for "2–3 rounds" without stopping (SKILL.md Phase 4) |
Stop and report after each generation; next round only on the owner's go | handoff §1 rules 2–3 |
| 3 | No overlay step | Every guide commit is followed by an overlay "Written against" re-pin (e.g. 05b26d7, 4ed71bd) |
git log |
| 4 | No checkpoint after a generation | Read-only checkpoint (checker, pins, row set, trade-off table) before any audit | handoff §3; U16 in #48 |
| 5 | "Never hand-patch a regeneration's output" | Independent audit, then a repair pass that fixes consistency defects only, never decisions | handoff §3; generate-spec step 4b |
| 6 | On failure, "identify the specific rule sentence" | First grep the whole guide and overlay for the concept and read the generator's own reasoning | a mis-aimed rule cost one regeneration on 2026-10-13 (handoff §1 rule 5) |
| 7 | No prediction step | Predict each run's outcome from existing regenerations before spending ~300k tokens per run | 2026-10-17 null-result runs |
| 8 | Phase 5 adopts a regeneration as the course spec | On 2026-10-12 the owner chose to regenerate fresh instead; an adopted regeneration also skips the independent audit | session record 2026-10-12 |
| 9 | Specs are pinned and "never silently regenerated" | generate-spec/SKILL.md:72 overwrites spec.md ("this is regeneration, not editing"); README.md loop shows "tune guide, regenerate" |
those lines |
| 10 | Regenerations on "a cheaper model, not the orchestrator's" | Since 2026-10-18 all runs use Opus 5.5, usually the orchestrator's model too | owner decision 2026-10-18 |
Not a decision record: each row needs the owner's call before any change to the skill.
Activity