|
| 1 | +# gen_simple — JIT generator-creation overhead (Phoenix-stable known-cost) |
| 2 | + |
| 3 | +**Status:** KNOWN-COST, fix-class workstream queued post-terminal-goal |
| 4 | +pure-C completion (per Alex priority order). NOT a recent regression; |
| 5 | +stable across 11+ days, 6+ sessions, multiple unrelated commit |
| 6 | +surfaces. Documented per supervisor 2026-04-27T15:15:20Z disposition |
| 7 | +following pythia #195 catch on prior "pre-existing structural causal |
| 8 | +absence" framing. |
| 9 | + |
| 10 | +## Symptom |
| 11 | + |
| 12 | +`Tools/benchmark_phoenix.py jit --only=gen_simple` reports Phoenix |
| 13 | +(JIT_ON, `cinderjit.auto`) at ~0.70-0.76x of vanilla CPython on the |
| 14 | +4-bench ABBA harness — i.e. JIT is ~30-37% slower than vanilla on |
| 15 | +this single benchmark, while other benchmarks in the same run land at |
| 16 | +positive speedup (fibonacci ~2.27x, nqueens ~1.59x, func_calls |
| 17 | +~1.00x). |
| 18 | + |
| 19 | +## Workload shape |
| 20 | + |
| 21 | +`bench_gen_simple(n_iter=8_700_000)` per-rep (`PER_BENCH_ITERS` in |
| 22 | +`Tools/benchmark_phoenix.py`): |
| 23 | + |
| 24 | +```python |
| 25 | +def bench_gen_simple(n_iter): |
| 26 | + def gen(n): |
| 27 | + for i in range(n): |
| 28 | + yield i |
| 29 | + total = 0 |
| 30 | + for _ in range(n_iter // 100): # 87,000 outer iterations |
| 31 | + for v in gen(100): # 87,000 generator creations |
| 32 | + total += v # 8,700,000 inner yields |
| 33 | + return total |
| 34 | +``` |
| 35 | + |
| 36 | +87,000 short-lived generators are created and exhausted per rep. The |
| 37 | +inner `gen()` function runs ~100 yields each before the generator |
| 38 | +object is discarded. |
| 39 | + |
| 40 | +## Reproduction (testkeeper 2026-04-27T15:14:16Z, `/tmp/gen_simple_match_harness.py`) |
| 41 | + |
| 42 | +3-warmup + 5-measure pattern matching `_worker_jit` in |
| 43 | +`Tools/benchmark_phoenix.py`, `n_iter=8_700_000`, `mean_ms` of 5 |
| 44 | +measurements: |
| 45 | + |
| 46 | +| Configuration | mean_ms | |
| 47 | +|---|---| |
| 48 | +| vanilla CPython `-S` | 476.1 | |
| 49 | +| Phoenix `JIT_ENABLE=0 PYTHONJITAUTO=0 -S` | 491.2 | |
| 50 | +| Phoenix `cinderjit.auto()` `-S` | 633.2 | |
| 51 | + |
| 52 | +`cinderjit.auto()` is the cost driver. Phoenix-with-JIT-disabled is |
| 53 | +within 3% of vanilla (harmless CPython-branch overhead). Phoenix-JIT |
| 54 | +adds 142ms (29% on top of JIT_OFF) on this workload. |
| 55 | + |
| 56 | +## Hypothesis (mechanism candidate, not yet confirmed) |
| 57 | + |
| 58 | +The JIT-compiled outer loop pays per-generator-creation prologue + |
| 59 | +epilogue cost on each of the 87,000 generators. Vanilla CPython's |
| 60 | +interpreter has a more compact path for short-lived generator |
| 61 | +dispatch. Specific candidates worth investigating when fix-class |
| 62 | +workstream opens: |
| 63 | + |
| 64 | +- JIT generator frame setup vs interpreter `_PyEvalFramePushAndInit` |
| 65 | +- Reference-count / LOAD_DEREF cost in compiled generator frames |
| 66 | +- `GEN_START` opcode emit weight relative to interpreter |
| 67 | +- Inliner heuristic: does Phoenix inline `gen(100)` into the outer |
| 68 | + loop? If not, every `gen(100)` call is full prologue. |
| 69 | + |
| 70 | +## Stability evidence (NOT a recent regression) |
| 71 | + |
| 72 | +| Date | Benchmark log | gen_simple ratio | |
| 73 | +|---|---|---| |
| 74 | +| 2026-04-16 | `docs/benchmarks/abba_x86_64_20260416.md` | 0.73x | |
| 75 | +| 2026-04-16 | `docs/benchmarks/abba_arm64_20260416.md` | 0.76x | |
| 76 | +| 2026-04-22 | `docs/benchmarks/abba_push45_20260422_040152.md` | 0.72x | |
| 77 | +| 2026-04-22 | `docs/benchmarks/abba_pre_push46_20260422_044949.md` | 0.72-0.74x | |
| 78 | +| 2026-04-24 | `docs/benchmarks/abba_w27c2a_2026-04-24.md` | 0.74x | |
| 79 | +| 2026-04-25 | `docs/benchmarks/abba_24bench_push59_2026-04-25.md` | 0.70x | |
| 80 | +| 2026-04-27 | `docs/benchmarks/abba_blme_b2_2026-04-27.md` | 0.74x | |
| 81 | +| 2026-04-27 | `docs/benchmarks/abba_kwnames_probe_2026-04-27.md` | 0.75x | |
| 82 | +| 2026-04-27 | `docs/benchmarks/abba_preloader_probe_2026-04-27.md` | 0.73x | |
| 83 | + |
| 84 | +Ratio range 0.70-0.76x over 11+ days, 9 logged runs, 6+ commit |
| 85 | +surfaces. The variance is consistent with same-session ABBA noise |
| 86 | +floor (~3-7% per `feedback_abba_cross_session.md`); no commit in this |
| 87 | +window introduced or removed the cost. |
| 88 | + |
| 89 | +## Anti-pattern guard |
| 90 | + |
| 91 | +When per-commit ABBA reports `gen_simple` in this 0.70-0.76x range, |
| 92 | +gate reports MUST cite this doc + use the framing |
| 93 | +`Phoenix-stable known-cost, fix-class workstream queued |
| 94 | +post-terminal-goal` rather than the prior dismissal language |
| 95 | +`pre-existing, structural causal absence` (which sounded like |
| 96 | +no-action accounting and was caught by pythia #195 |
| 97 | +2026-04-27T15:10:01Z + medic 15:15:15Z motivated-reasoning warning). |
| 98 | + |
| 99 | +If `gen_simple` drops BELOW 0.70x or rises ABOVE 0.76x in a single |
| 100 | +ABBA, that IS a new signal — investigate the proximate commit, do |
| 101 | +NOT cite this doc. |
| 102 | + |
| 103 | +## Resumption gate (when fix-class work opens) |
| 104 | + |
| 105 | +1. Re-baseline `gen_simple` standalone (vanilla / Phoenix-JIT-off / |
| 106 | + Phoenix-JIT-on) to confirm the ~0.73x ratio still holds; if not, |
| 107 | + reframe. |
| 108 | +2. Profile JIT-compiled `bench_gen_simple` under `perf record` or |
| 109 | + `cProfile` (note cProfile distorts vanilla baseline by ~3.8x; use |
| 110 | + `perf` for clean comparison). |
| 111 | +3. Diff JIT-emitted code for `bench_gen_simple` outer loop vs |
| 112 | + interpreter dispatch — look for per-iteration generator-frame |
| 113 | + prologue weight. |
| 114 | +4. Falsify each hypothesis-class candidate above; document which is |
| 115 | + load-bearing. |
| 116 | +5. Heavy-tier instrumentation (analogous to W-PYTORCH-CM (ii) |
| 117 | + tp_alloc-watchpoint pattern) only after lighter probes converge or |
| 118 | + exhaust. |
| 119 | + |
| 120 | +## Related |
| 121 | + |
| 122 | +- `feedback_assume_phoenix_regression.md` — Alex 2026-04-24 directive: |
| 123 | + all bugs presumed Phoenix-introduced. Satisfied here (cost IS |
| 124 | + Phoenix's, just not recent). |
| 125 | +- `feedback_falsifier_convergent_negative.md` — convergent-negative |
| 126 | + ABBA delta needs `investigated-no-quick-fix` framing, not |
| 127 | + `pre-existing` escape. This doc IS that framing. |
| 128 | +- `feedback_no_workarounds.md` — documenting + deferring fix is NOT a |
| 129 | + workaround; only bail-out / deopt is the prohibited pattern. |
| 130 | + Deferred-fix-with-workstream-anchor is transparent accounting. |
| 131 | +- pythia #195 (D-1777300601 2026-04-27T15:10:01Z) — caught the |
| 132 | + framing drift across 4 commits. |
| 133 | +- medic 2026-04-27T15:15:15Z [MEDIC-WARNING] — caught generalist |
| 134 | + follow-up framing as motivated reasoning vs pythia. |
0 commit comments