Visitar URL original
perf(test): remove redundant serial PTY idle waits by benvinegar · Pull Request #1276 · modem-dev/hunk · GitHub
Skip to content

perf(test): remove redundant serial PTY idle waits - #1276

Open
benvinegar wants to merge 3 commits into
mainfrom
perf/pty-ci-critical-path
Open

benvinegar wants to merge 3 commits into
mainfrom
perf/pty-ci-critical-path

Conversation

@benvinegar

@benvinegar benvinegar commented Nov 10, 2026 •

Copy link
Copy Markdown
Member

Summary

Optimize the existing single-runner, serial PTY suite. The sharding experiment has been removed completely: no workflow changes, matrix, extra runners, or concurrent PTYs remain in this PR. Other jobs, release branches/tags, and release workflows are untouched.

  • Replace redundant shell-completion idle waits with a unique fixture acknowledgement emitted only after each captured-arguments file closes. Send input/Tab/Enter in Readline order and retain every Bash/Zsh quoting, Unicode, option-boundary, and loading-mode assertion.
  • Deliver intermediate pointer movements without 500ms waits when they cause no repaint. Still observe real hover controls, selection, and saved-card state.
  • Replace composer/paging sleeps with code, keyboard-readiness, and full-footer predicates.
  • Carry the PTY hunks from coordinator commit 0d16237f forward: chore(release): prepare v0.25.0 #1274 merged without that commit. These cover CSS palette menu closure, CJK reflow, wheel/header layout, exact-line jitter, feedback readiness, and isolated default working directories. The Windows metadata fix is outside this PR's scope.
  • Empty maintenance Changeset and profiling/maintenance notes in test/pty/README.md.

Controlled real CI measurements

Both runs are on this pushed branch, the same 62064bcf source base plus the same coordinator stabilization fixes. No retries, no difference in job definition or test inventory:

  • Before: 43053286536, b720f828, single-runner control before optimization.
  • After: 43054778196, 1575d0c1, three-file optimization plus measurement docs.

Job durations are completed_at - started_at from gh api repos/modem-dev/hunk/actions/runs/<id>/jobs, including setup and cleanup.

Job Before After
PTY integration — one runner 7m23s 6m11s
Linux validation and packaging (untouched) 3m31s 3m50s
Windows compatibility (untouched) 5m27s 6m03s
Compiled headless, macOS (untouched) 20s 24s
Compiled headless, Windows (untouched) 38s 40s
Release-note changeset (untouched) 13s 11s
Validation gate (untouched) 4s 3s
Detect code changes (untouched) 6s 6s

Observed PTY reduction: 72s / 16.3%. PR workflow wall time: 7m52s → 6m29s. PTY remains the critical path, now close to Windows; no claim that the untouched jobs improved.

From individual test-duration sums in the real PTY job logs:

Targeted file Before After
Shell completion 26.98s 2.42s
Notes 46.34s 36.91s

About 34 seconds disappeared directly from the modified files. Other file durations also varied, so the entire observed 72-second job improvement is not promised on every runner. Both jobs executed all 291 tests across 41 files: 287 pass, 4 identical existing environment skips, 1,214 assertions. No functional assertions or integration tests were deleted. The prior sharding-only helper/tests were removed with that rejected experiment, not from the original suite.

Confirmation on the final documentation head 43a59868: 43056271912, all PR CI workflow jobs green, no retries. The single PTY job took 6m20s (14.2% below the 7m23s control). Untouched jobs: Linux 3m19s, Windows 5m33s, compiled headless Windows/macOS 46s/23s, changeset 9s, validation gate 2s, change detection 5s. The observed serial PTY range across the two after runs is 6m11s–6m20s, not the superseded matrix timings.

Validation and rejected alternatives

  • bun run test:integration before pushing: 290 pass, 1 unchanged platform skip, 1,252 assertions, 339.57s locally, versus a 391.19s baseline on the refreshed source base (13.2% less local time). CI lacks local zsh/Neovim, explaining its different pass/assertion counts.
  • Four local completion tests: 50.90s → 3.94s, same 76 assertions.
  • Threaded-note test: ~7.9s → 4.08s, same 15 assertions.
  • bun run typecheck, targeted oxlint, formatting, and git diff --check: pass. Controlled before/after PR CI workflows both passed, including Windows and Linux validation.
  • A precompiled executable did not improve the sampled notes/navigation/file-view tests (64.38s versus ~62.23s from source); no build/artifact dependency added.
  • A broad event-only screen-wait rewrite exposed a multi-frame resize/reveal race and was discarded. General screen-settling behavior is unchanged.
  • Keep the deliberate 62-second suspension check, fixed wheel/drag cadence, and negative/stability checks where timing is part of the tested behavior. Builds on perf(test): reduce fixture process overhead #1092 without repeating its fixture-process refactor.

Explicit performance-policy change and risks

The coordinator's dialog fix replaces the noisy cold-minus-warm differential assertion with independent absolute cold/warm gates. Cold limits remain 150ms idle / 200ms first-frame, and warm reopen has an explicit 150ms limit. A very fast warm reopen no longer invalidates an otherwise in-budget cold open. No timeout is blindly increased; this is the existing coordinator fix, called out rather than hidden as a speed optimization.

Acknowledgements belong to the shell fixture and are emitted after closing the captured file; they do not substitute dispatch ACKs for Hunk screen assertions. Faster pointer delivery relies on ordered SGR events, followed by destination-state checks. Shared-runner timing varies; the PR reports both targeted-file gains and full-job observations, not a guaranteed speedup. All original platform gates and coverage remain.

This PR description was generated by Pi using GPT-6.1-sol

@benvinegar benvinegar changed the title perf(ci): balance PTY integration across parallel runners perf(test): remove redundant serial PTY idle waits Nov 11, 2026
@benvinegar
benvinegar force-pushed the perf/pty-ci-critical-path branch from 2b85d74 to b720f82 Compare November 11, 2026 15:56
@benvinegar
benvinegar marked this pull request as ready for review November 11, 2026 16:21
@greptile-apps

greptile-apps Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Low impact] The PR appears safe to merge; no actionable issues were found.

Summary

Speeds up the serial PTY suite by replacing redundant waits with fixture acknowledgements and screen-state checks.

  • Shell completion tests read arguments after each command finishes.
  • Note tests wait for finished screen states instead of idle pauses.
  • Dialog timing checks give cold and warm opens separate limits.
  • PTY tests wait for usable screen states before continuing.

benvinegar explicitly identifies the switch from a cold-minus-warm dialog check to independent absolute limits as intentional.

Reviews (1) · Last reviewed commit: "docs(test): record controlled serial PTY..." · Reviewed by Greptile

@squad-robert squad-robert left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Independently checked 43a59868 against PR base 62064bcf on Linux/Bun 1.4.4. No actionable defect found in the paths exercised.

The faster waits still detect meaningful failures: a partially written capture file with a delayed close passes all 19 Bash argument assertions; repeating an old completion ACK fails on the next command; omitting Tab fails the completion assertion; and omitting the note-save key times out rather than accepting the text already visible in the editor. These were private copies of the real PTY tests, with production code unchanged.

Three serialized base/head comparisons of Bash standalone, Bash helper and the threaded-note case retained 53 assertions each. Median wall time fell 34.78s → 6.37s for those cases. The complete serial PTY suite passed on both revisions: 287 passing, 4 unchanged skips, 1,214 assertions. Its single before/after sample was 373.25s → 342.32s; that isn't a repeatable whole-suite speedup guarantee or confirmation of the hosted-CI timings. Head type/lint/format checks and 11 real-TTY smoke cases also pass.

The cold/warm dialog-budget change is an intentional policy change, not identical assertion retention; it's explicitly documented. My execution covers Linux/Bash/source CLI, not Zsh, macOS, Windows, Bash 3.2 or a compiled binary.

This comment was generated by Pi using GPT-6.1 Sol

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants