Sandbox::https://github.com/macintacos/gh-prism-extension/pull/124

macintacos · 2026-11-11T23:56:27Z

Part of: EXC-2294

Change Kind: test

What changed

This is the acceptance run of the structural diff epic (EXC-2276). It adds a live parity bench, an engine caps bench, and a live performance bench, runs them in Chromium and Firefox, tunes the engine's caps from the numbers, and adds e2e coverage for what can be automated. Along the way it fixes two small defects the runs exposed: on-screen tracking that died on large virtualized pages (also broken on main), and a wall-clock refusal that the background cached as if it were a size refusal. Nothing a person reads changed, so no design work was involved.

What to test

With Structural never chosen, open a large pull request and a commit page and confirm they look and behave as on main; the segmented control is the only addition.
Confirm the Structural segment reads "Structural Beta" on a pull request page and a commit page.
Scroll a commit of 300 or more files in Chromium (for example nodejs/node@4eb3908) and confirm the sidebar's on-screen highlight keeps following and the console shows no IntersectionObserver error.
Read the "Awaiting your sign-off" table below: every row is a check this unattended run could not pass or could not judge.

Verdict 1: mode off is trunk

Passes on behaviour, layout, storage, and Chromium performance. One Firefox scroll bar failed on 5 runs and awaits your sign-off.

Parity check 0a, both browsers: with main's build and with this build and the mode off, sandbox PR #3 and commit 8d807ee have the same page signature (every diff table's markup, the sidebar text, file order, header controls, an accessibility snapshot), every toolbar neighbour of the control keeps its width and height, extension storage is unchanged across the visit, and there is no structural attribute, notice, stylesheet, highlight range, or offscreen document (bench/structural-parity-live/results/*.txt).
bench/structural-mode-live.ts pass 1 prints mode off: ok in both browsers.
e2e: "creates no structural engine context while Structural is off" and "paints no structural highlight while Structural is off" pass in both browsers.
Performance, mode-off against main on deps: upgrade npm to 12.1.0 nodejs/node#66212: in Chromium all three bars pass (load 2,217 against 2,363 ms, scroll p95 16.8 against 16.8 ms, slow frames 41.9% against 41.8%). In Firefox load and slow frames pass (2,129 against 2,182 ms; 3.0% against 2.8%) and scroll p95 fails: 15.2 against 10.7 ms, where the bar allows 2 ms. Four of five runs show it (mode off 15.2, 15.0, 16.7, 10.3, 16.6; main 13.9, 10.7, 10.9, 10.3, 9.3). It is not explained; see the sign-off table.
Residuals as built: the Chrome zip is 866,965 bytes and the Firefox zip 865,270 bytes (main: 230,233 and 230,314; the difference is about 636 KB, the tree-sitter runtime and four grammars), both under the 1,000,000-byte ceiling. The manifest differs from main only by content_security_policy.extension_pages (with 'wasm-unsafe-eval') in both browsers and offscreen in Chrome's permissions; bench/structural-host.ts chromium reports noNewWarning: true.
The one change to mode-off behaviour is the on-screen tracking fix (coveredTop never returns a negative band; followOnScreen builds its next observer before dropping the old one and retries a band whose build failed). It changes behaviour only where main throws today. Live proof in Chromium on a 568-file commit: main raises 5 IntersectionObserver page errors, this build raises none with the mode off or on. Firefox did not reproduce the error on main (GitHub rendered a short page there).

Verdict 2: Structural reads as experimental

Passes. Parity check 0b finds exactly one visible segment named Structural Beta with visible text Beta on sandbox PR #4 and commit 8d807ee, in both browsers. The e2e test "the diff view control switches Split, Structural, and Unified on … agreeing with the gear" and a unit test in tests/content/diffs/view-control.test.ts now assert the same. How the tag looks in your own browser is a reviewer check.

Parity checklist

Final run at the branch tip: Chromium 64 ok, 0 failed, 18 not driven. Firefox 62 ok, 0 failed, 20 not driven. Pages: sandbox pull requests macintacos/gh-prism-sandbox#3 and macintacos/gh-prism-sandbox#4, sandbox commits 8d807ee and 4630717, the code-owners sandbox pull request, a gh-prism-extension commit line link, and nodejs/node#66212. Every line is in bench/structural-parity-live/results/chromium.txt and firefox.txt.

#	Check	Chromium	Firefox
0a, 20	Mode off is trunk	pass	pass
0b	Beta tag	pass	pass
0c, 14	Split and Structural signatures equal (five pages)	pass	pass
1, 19	Line comments, painted and context rows, PR and commit	pass; quiet rows not driven	pass; quiet rows not driven
2	Multi-line comment by Shift-click	pass	pass
3	Suggested change	pass; batched not driven	pass; batched not driven
4	File-level comment	pass	pass
5	Pending review, then discarded	pass; submitting not driven	pass; submitting not driven
6	Thread reply and resolve (page paints nothing)	pass	pass
7	Outdated threads	not driven	not driven
8	Viewed, GitHub's toggle and the sidebar box	pass; unmark on change not driven	pass
9	Expand context up and between	pass; down and all not driven	pass
10	Load diff on nodejs/node#66212	pass (unsupported-language notice)	not driven
11	Line links, R and L, and 10 loads of a commit link	pass	pass
12	Switch away and back	pass	pass
13	Hide whitespace	pass	not driven
15	Copy and find	pass	pass
16	Grid keys	pass; screen reader not driven	pass; screen reader not driven
17	`j`, `k`, sidebar reveal, code-owner shield across switches	pass	pass
18	Rich diff, renamed file	pass; binary and one-sided not driven	pass
21	Notices: one sentence per unsupported file; too-large on a big file	pass	pass

Recorded, never failures: the loading line was not seen in either browser; notice density on Rust-heavy sandbox PR #3 was 4 notices mounted, at most 3 in one viewport (Chromium), and 5 and 4 (Firefox); patch mismatches after a full scroll of nodejs/node#66212 were 0 of 85 (Chromium) and 0 of 38 (Firefox); a sidebar reveal after a Layout switch never reloaded the page (10 of 10, both browsers); the code-owner shield count stayed at 1 across Split, Structural, Unified, Split.

The Unified to Structural to Unified to Split round trip leaves GitHub's Layout at Split, Unified, Split with the mode on only after the first press, on all four sandbox URLs, in both browsers (bench/structural-mode-live/results/*.txt).

Awaiting your sign-off

Nobody signed anything off during this run. Each row is yours to accept, or to send back.

Check	State	What a person would do
Firefox scroll p95 with the mode off, against `main`	failed on 5 runs: 15.2 against 10.7 ms (bar: at most 2 ms over). Both stay under one 16.7 ms frame; the share of frames over 16.7 ms passes (3.0% against 2.8%) and frames over 50 ms are equal (11 and 11). Not root-caused, and the 9-run rerun was skipped	This is the one number against "mode off performs as `main`". Rerun with `--runs 9`; if it holds, find what the mode-off build does per frame on that stretch that `main` does not (the on-screen tracking fix keeps tracking alive where `main` may have stopped; the control itself is idle). An earlier version of the bench, timing the page's first 12%, showed no difference (17.0 against 16.8 ms).
Firefox scroll smoothness with Structural on	passes now on the stretch that paints (16.1 against 15.2 ms). On the page's first 12%, which is all unsupported-language files, an earlier version of the bench failed it at 24.0 against 17.0 ms	Decide whether one notice per unsupported file is worth its scroll cost in Firefox. Measured cause (before the bench was reworked): inserting one notice per file inside GitHub's file card costs about 25 extra slow frames per 20 s scroll; with notice insertion disabled the bar passes (17.7 against 17.1 ms), and hiding the notices with `display: none` does not help. Chromium is unaffected. A fix changes what people see (for example one page-level notice), so it was not made here. Owner: the sub-issue that added the notices and loading line.
Performance bars decided on 5 runs only	every other bar passed in both browsers; the Firefox bar above failed	The plan allows one 9-run rerun that decides; it was skipped at your request to keep this run short. Rerun `bun bench/structural-perf-live.ts <browser> <state> --runs 9 --trunk <main build>` on a quiet machine if a bar matters to you.
Tuned caps refuse real files	`maxTextLength` 256,000 to 100,000 and `maxEditLength` 1,000 to 750, by the plan's tighten-only rules	Accept, or relax. On nodejs/node#66212 the text cap now refuses `deps/npm/tap-snapshots/test/lib/docs.js.test.cjs` (166,092 units, though it parses in 8 to 18 ms) and the edit cap refuses two minified `lru-cache` files. `too-slow` fired 0 of 1,520 runs under a 12-core load at the old caps and the new, so the tightening buys headroom, not a fix. The driver for the text cap is a token-dense synthetic Python file in Firefox, whose numbers include about 10 ms of messaging overhead.
Moved-line mark's focus ring in Firefox	not driven	No live page marks a moved line, and the e2e fixture carries none of GitHub's stylesheet. Focus a moved line's code cell in Firefox and compare its ring with an unmarked cell's. e2e does assert the dashed outline survives focus in both browsers.
Screen reader's reading of a painted table and of the notices	not judged	Read a painted file and a notice with VoiceOver or NVDA. The accessibility snapshot is equal with Split and Structural.
Look of the Beta tag	not judged	Look at the control in your own browser, light and dark.
Whether notices on a Rust-heavy page are noise	not judged	Open macintacos/gh-prism-sandbox#3 with Structural on: at most 3 to 4 notices share a viewport.
Batched suggestions; submitting a review as Comment, Approve, Request changes; Minimize comments; Copilot actions	not driven	Each leaves something on the pull request that cannot be removed, or costs account state. Try them once on the sandbox.
Outdated threads; unmark Viewed on change	not driven	No outdated thread exists on the sandbox, and unmark-on-change needs a push.
github.dev (`.`) and codespace (`,`) keys	not driven	They leave the page. Press each with Structural on.
Quiet rows, expand down and expand all, binary and one-sided files	not driven	The sandbox pages have none.
Hide whitespace, and Load diff, in Firefox	not driven	Firefox denies one content script another's highlight ranges, so the probe reads only a total; and six "Load diff" presses there mounted only one-sided tables. Both pass in Chromium.
Loading line	never seen	No request stayed pending past its 150 ms hold in any run, including a first press on nodejs/node#66212 in a fresh profile.
`LARGE_PR_READ_TIMEOUT` raised from 15 s to 30 s	changed	Under a 12-core load, four large-fixture tests' first wait passed 15 s in Chromium (three of them mode-off tests this branch does not touch). 30 s equals Playwright's default test timeout, so a slower start now fails on the test timeout.
Mode-live bench in Chromium	passed on its second run	Its first run at the branch tip printed every check `ok` and then timed out clicking GitHub's diff-settings gear in its closing restore step; the rerun exited 0 with `layout as found: ok`. Not root-caused; the saved Layout was `unified` before and after.

Bench results

Caps (utils/structural/limits.ts, from bench/structural-caps.ts, 76 samples a browser):

Cap	Before	After	Rule that set it
`maxTextLength`	256,000	100,000	Firefox parsed a 150,000-unit side in over 50 ms, half the parse deadline; the cap is the ladder step below
`maxTokens`	20,000	20,000	no side under both size caps parsed in over 50 ms
`maxEditLength`	1,000	750	at 750 Firefox gives up on unrelated sides within the 50 ms diff deadline (42 to 43.5 ms)
`maxParseMs`, `maxDiffMs`	100, 50	unchanged	the 100 ms stage bound, a default awaiting your confirmation

too-slow (the new wall-clock refusal) fired 0 times quiet and 0 of 1,520 runs under a 12-core load, in both browsers. The loaded host run kept pingWithinBound: true (maximum ping 5.1 ms in Chromium, 16 ms in Firefox; those two figures are from before the machine restarted and their raw output was lost). Peak offscreen JS heap in Chromium: about 0.6 MB before the first sample, 83.6 MB at its highest reading (one reading per sample, not a continuous peak); Firefox has no measure. Cold run from a fresh profile: 47 ms in Chromium, 60 ms in Firefox, and every host criterion is true in both.

Performance on nodejs/node#66212 (headless frame timing, 5 runs, medians; the timed 20 s scroll starts at the npm tap snapshots, where Structural paints):

Browser	Variant	first diff	load	p50	p95	frames over 16.7 ms	over 50 ms
Chromium	`main`	2,279 ms	2,363 ms	16.7	16.8	41.8%	6
Chromium	mode off	2,133 ms	2,217 ms	16.7	16.8	41.9%	6
Chromium	mode on	2,114 ms	2,216 ms	16.7	16.8	42.1%	10
Firefox	`main`	2,030 ms	2,182 ms	8.3	10.7	2.8%	11
Firefox	mode off	2,079 ms	2,129 ms	8.3	15.2	3.0%	11
Firefox	mode on	2,123 ms	2,201 ms	8.3	16.1	3.6%	13

Chromium passes all eight bars plus the new mode-on painted bar (a median of 9 files painted inside the timed window). Firefox passes every bar but mode off against main scroll p95 (median of 10 files painted). At 3,000 px/s the engine answers about 8 to 14 files in the window; about 65 paint on that page given time. GitHub stopped short of the full file list on 9 of 39 loads, and the bench loaded those again. The record is bench/structural-perf-live/results/*.txt.

e2e under load (30 repeats of every large-fixture test, 12 busy loops): baseline 299 of 300 in Chromium (one browser-launch flake) and 300 of 300 in Firefox; with the new tests and the 30 s first wait, 330 of 330 in both. These counts are from before the machine restarted and their raw output was lost. The final unloaded e2e run at the branch tip: 370 passed, 6 skipped, both browsers.

Epic defaults still awaiting your confirmation

Set by the orchestrator after the EXC-2277 spike: tree-sitter rather than a lexer; four grammars (TypeScript, TSX, JavaScript, Python) under a 1,000,000-byte download ceiling; a 100 ms bound per uninterruptible stage; no avoidable mode-off residual.

Follow-ups, not done here

IntentSources.headOutline is set by nothing, so the hunk rules classify as on main in every mode (EXC-2296's outline is unwired), and headOutline has no production caller.
CRLF files, and a BOM file edited on line 1, are refused as a mismatch, safely, and are untested. No mismatch was seen live.
The content side keeps a file's answer for the visit, a slow refusal included. This branch stops the background caching a too-slow refusal across visits; it does not retry within one.
The notice copy exists in two bench files and one e2e selector because structural-notices.ts imports a stylesheet a bench cannot load; a constants-only module would remove the copies.
The benches restate some GitHub selectors that files-page.ts owns.
The performance bench retries a load when GitHub stops short of the full file list on a very large pull request; that happened about once in seven loads here.
If you want more of the engine off the main thread, EXC-2297 is the direction; this run changed no Linear state.

How this was reviewed

Two reviewers, a spirit check, and a stale-reference finder read the branch; their accepted findings are the commit "fix(bench): make the acceptance evidence fail". The machine restarted mid-run: the review panel was dispatched again from scratch, and figures marked above as from before the restart could not be re-read from their raw output.

linear-code · 2026-11-11T23:56:34Z

EXC-2294

A virtualized page that unmounts its first file reports a negative sticky top, and the observer was rebuilt with an invalid rootMargin after the old one was already disconnected. - coveredTop never returns a negative band - followOnScreen builds the next observer before dropping the old one Refs: EXC-2294

A stage past its deadline depends on machine load, so one slow parse refused a file until the background cache evicted it. - the engine and the outline refuse a deadline as too-slow - the background answers it as too-large and does not cache it Refs: EXC-2294

- the Structural segment is named and tagged Beta - a line link lands in view with Structural selected - a moved line keeps its dashed outline while focused - a large page highlights on-screen files with Structural on - a round trip back to Structural restores the first paint - LARGE_PR_READ_TIMEOUT is 30 s: under a 12-core load the first wait of four large-fixture tests passed 15 s in Chromium Refs: EXC-2294

bench/structural-caps.ts runs fixture pairs, the largest files of nodejs/node#66212, a size ladder, and unrelated sides through the engine in the built extension, in both browsers. - maxTextLength 256,000 -> 100,000: Firefox parses a token-dense 150,000-unit side in over half the parse deadline - maxEditLength 1,000 -> 750: Firefox spent about 78 ms giving up at 1,000, past the 50 ms diff deadline - the host bench's helpers move to a shared harness Refs: EXC-2294

bench/structural-parity-live.ts drives the epic's checklist against the built extension, signed in, and compares a page signature with Split against Structural and with main's build against mode off. - Chromium: 64 ok, 0 failed, 18 not driven - Firefox: 62 ok, 0 failed, 20 not driven - the mode-live bench round-trips Unified, Structural, Unified, Split - the live benches share their launch and comment-form helpers Refs: EXC-2294

bench/structural-perf-live.ts loads and scrolls nodejs/node#66212 with main's build, the mode off, and the mode on, interleaved, and holds each pair to the no-slowdown bars. - Chromium passes every bar on its deciding run - Firefox fails mode-on scroll p95 (24.0 vs 17.0 ms): one notice inserted per file costs about 25 slow frames per scroll - Firefox fails mode-off load on medians (2684 vs 2486 ms); a 15-run supplemental run passes it (2659 vs 2631 ms) - on a 568-file commit main raises the rootMargin page error and this branch raises none Refs: EXC-2294

Review found parity and performance lines that printed ok without being able to fail, and cleanup that reached past the run's own changes. - the perf bench times a stretch that paints, bars a mode-on run that painted nothing, rotates its variants, and fails a load that never settles - the parity bench reopens only threads and discards only reviews the run made, checks gh is the page's account, and re-reads the state its restore lines report - followOnScreen retries a covered band whose observer failed - DEVELOPMENT.md's five top-level sections are sections again Refs: EXC-2294

Refs: EXC-2294

The result files are the output of the benches at this tip. - the perf bench loads again when GitHub stops short of the file list on a very large pull request, and waits 30 s for the list to grow before calling the page's end - Chromium passes every performance bar; Firefox fails mode off against main on scroll p95 (15.2 vs 10.7 ms over 5 runs) - parity: Chromium 64 ok, Firefox 62 ok, none failed Refs: EXC-2294

macintacos added this pull request to stack #111 November 11, 2026 23:57

macintacos added 9 commits November 12, 2026 00:36

refactor(bench): name four parity locals

8c73349

Refs: EXC-2294

macintacos force-pushed the EXC-2294/verify-structural-parity-performance branch from 6d77c0c to bb3d580 Compare November 12, 2026 06:03

macintacos merged commit 1800d0e into main Nov 12, 2026
8 checks passed

macintacos deleted the EXC-2294/verify-structural-parity-performance branch November 12, 2026 07:23

Navigation Menu

Repository navigation

EXC-2294 Verify structural mode's parity and performance - #124

EXC-2294 Verify structural mode's parity and performance#124
macintacos merged 9 commits into
EXC-2292/unified-split-structural-controlfrom
EXC-2294/verify-structural-parity-performance

macintacos commented Nov 11, 2026

Uh oh!

linear-code Bot commented Nov 11, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

macintacos commented Nov 11, 2026

Change Kind: test

What changed

What to test

Verdict 1: mode off is trunk

Verdict 2: Structural reads as experimental

Parity checklist

Awaiting your sign-off

Bench results

Epic defaults still awaiting your confirmation

Follow-ups, not done here

How this was reviewed

Uh oh!

linear-code Bot commented Nov 11, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant