Four suites live here or are driven from here. Each has its own CI job; the two snapshot suites are rendered by CI and their baselines are committed by CI, which is the one rule that matters when you touch them (below).

cd e2e
./run              # render every scene and compare
./run dataGradient # only scenes whose name contains "dataGradient"

A fixture that says settings.execution: "interpreted" is rendered by the interpreter only (its code nodes are JavaScript with no Nim twin, so there is no compiled half to compare; the render is <id>.png, kept like a pair's when within the threshold of the committed one), and a fixture's e2eEvents list is POSTed to the frame's /event/<name> after the first render, so an input scene (inputTouch, inputKeyboard) is snapshotted showing what it heard: the last upload of the burst, taken once nothing more arrives for FRAMEOS_E2E_EVENT_SETTLE seconds (3), never the first render, which on a slow runner shows the first key alone. An entry {"wait": "render"} instead of an event waits for that settled render before the events after it are sent: a tap on the keyboard on the screen comes after the render that drew it (jsxSquareForm). The reset scene drawn between two fixtures is waited for as well, so its late upload is never mistaken for the next scene's.

Every fixture is drawn on frame.json's 320x480 frame unless it says "e2eFrame": {"width": 800, "height": 480}: each size gets a frameos of its own, one after the other (the JSX interfaces, scenes/jsx*.json, are 800x480, 480x800 and 480x480).

./run hands off to the Makefile: makescenes.py compiles the fixture scenes in scenes/ into generated/, a copy of frameos/ is built with them (make build in tmp/frameos), then makesnapshots.py runs that binary once per scene and diffs the PNGs against snapshots/. Pixie renders these in software from fonts in the repo, so the output is platform-independent as long as the app is deterministic: fixtures must pin now, seeds and any network input (the harness serves fixtures from a local port, E2E_FIXTURE_PORT), or the snapshot will flap.

CI runs the same thing in two shards and — on pull requests from this repository — commits any changed PNGs back to the PR branch as "FrameOS Bot" ("Commit Visual Regression Snapshots"). Pull --rebase before you push again.

The Playwright suite. e2e/frontend-visual/README.md has the page/state catalogue (tests/visual-cases.ts) and how to add one; the short version:

pnpm dev:redis                                  # once, in another shell
pnpm --dir frontend run build                   # the backend serves frontend/dist
pnpm --dir cloud-frontend run build             # only for the visual-cloud-* specs
pnpm exec playwright install chromium           # once
pnpm test:frontend-e2e                          # the @e2e interaction specs only
pnpm test:frontend-visual                       # everything, with pixel comparison

Playwright starts its own backend on 127.0.0.1:8989 with a disposable SQLite database (.tmp/frontend-visual.db, seeded by scripts/seed_backend.py) and Redis db 15. Gotchas that have cost real time:

• 8989 is usually taken by your dev backend. Override with FRONTEND_VISUAL_PORT=8990 FRONTEND_VISUAL_BASE_URL=http://127.0.0.1:8990 instead of killing it.
• Pixel comparisons always fail on macOS. The committed snapshots are rendered on CI's Linux; local font rasterisation differs by a few percent. Run with FRONTEND_VISUAL_MAX_DIFF=1 to skip the pixel diff and still exercise every interaction and the "no frontend errors" assertion. Never run --update-snapshots locally — it overwrites the CI-rendered baselines with macOS ones.
• Do not keep one backend alive across runs (FRONTEND_VISUAL_REUSE_SERVER=1): tests append frame logs and mutate settings, and fixture-dependent assertions (a logs view that expects a literal line count) drift. Let Playwright own the server, or delete .tmp/frontend-visual.db between runs. FRONTEND_VISUAL_SKIP_WEBSERVER=1 points at a backend you run yourself.
• The cloud specs need no backend but do need cloud-frontend/dist; they share tests/cloud-workspace-fixture.ts (a fixed clock, route interception for the bundle). Register a spec's own page.route mocks after serveCloudWorkspace so they win, and answer /api/fonts with {fonts: []} on frame pages.
• A frames-home test asserts zero /frames/{id}/states requests. Anything rendered per frame on that page must not mount controlLogic or scenesLogic (they fetch states on mount); use a lightweight logic.

CI runs four shards. Same-repo pull requests run with --update-snapshots and "Commit Frontend Visual Snapshots" pushes changed PNGs to the PR branch as "FrameOS Bot"; forked PRs and pushes to main only compare. So a visual job failing on a PR is never a pixel diff: read the log for the frontend-error assertion (any console error, e.g. a 404'd resource) or the prepare step that threw. On main it is the opposite: a UI change committed straight to main leaves "Run Tests" red there until a same-repo PR carries the refreshed PNGs. Open one from main, let the bot commit the snapshots to it, merge.

• Do not run the Playwright suite during normal iteration; run it when the change is visual or the user asks. It takes minutes and owns a port.
• Never commit snapshot PNGs rendered locally, for either suite. CI is the source of truth; if a baseline needs to change, push the code and let the bot commit it.
• If a visual test you did not touch fails, check what else was running: a second backend on the port, or a reused server with drifted fixtures.

Opt-in, because it writes real system paths on its target. e2e/deploy/README.md has the two ways to run it (the bundled Docker SSH target, or your own throwaway host) and the flags it covers.