Visitar URL original
v6 perf audit: large-scale profile (1–5M atoms, 1.7k–11k units); pipeline bench in the repo by dsehnal · Pull Request #1997 · molstar/molstar · GitHub
Skip to content

v6 perf audit: large-scale profile (1–5M atoms, 1.7k–11k units); pipeline bench in the repo - #1997

Draft
dsehnal wants to merge 1 commit into
masterfrom
claude/perf-large-scale
Draft

dsehnal wants to merge 1 commit into
masterfrom
claude/perf-large-scale

Conversation

@dsehnal

@dsehnal dsehnal commented Oct 19, 2026

Copy link
Copy Markdown
Member

Extends the perf audit (.v6/perf-audit/00-profile.md stopped at 186k atoms and 416 units) to large structures and ranks what breaks first. Read-only on packages/.

What's here

  • .v6/perf-audit/13-large-scale.md: the pipeline on 1tup expanded to 32 / 192 / 864 copies (186k / 1.1M / 5M atoms, 288–11,232 units), both as an assembly (9 shared symmetry groups, the capsid case) and as one unshared model (large entries, cellPACK).
    • Stage tables at default quality and at pinned low quality, plus isolated reruns.
    • Render-buffer sizes and the interactive paths: hover and select marks, overpaint, transparency, components.
    • Scaling with super-linear stages called out, what breaks first, hotspots, new findings 13/P1–P6 and 13/B1, cross-references to existing ids, and a ranked list of wins.
  • tests/src/pipeline/ and scripts/pipeline-bench.mjs: the audit's scratch bench, moved into the repo (step 0 of the audit README).
    • fixtures builds the large fixtures locally, with no network.
    • run produces the stage table and analyze the CPU or allocation hotspots mapped back to the TS sources.
    • New columns: GC pause time per stage, peak RSS, and a release stage per representation.
    • Prerequisites run unmeasured under --only.
    • It is a developer tool, so there is no .v6/changelogs entry.

Headlines (16 GB M4)

  1. 13/P1: overpaint or transparency on one chain of an instanced surface smooths across every instance. On 864 copies that is 12 s and 4 GB allocated; at ≥1M atoms the auto → lowest quality switch hides it.
  2. 08/P7 + 07/P7 (frozen part): unshared ball-and-stick reaches 4.5 GB of render buffers and 10.6 GB retained at 5M atoms, and takes 42–191 s.
  3. 07/P3: marking at 11k unshared units. Selecting a chain in every copy is super-linear (0.2–0.7 s); hover takes 65–191 ms.
  4. Symmetric assemblies stay cheap up to 5M atoms (representations 20–341 ms), except for the properties that ignore symmetry: interactions 13 s, ASA 31 s, inter-unit bonds 2 s / 3.6 GB.
  5. 13/B1: text mmCIF fails past about 4.8M atoms (V8's 512 MB string limit).

Caveats

  • The machine was loaded by other sessions while measuring, so the report bases its super-linear verdicts on isolated reruns and profiles.
  • An independent review round with a fresh subagent ran before pushing. It confirmed about 150 table cells against the raw data. It also found the delayed array-buffer accounting, prerequisites leaking into --only stages, a stale source map in analyze, and overstated noise and scaling claims. All of these are fixed, and the affected rows were rerun.
  • pnpm check:workspace fails locally on @molstar/plugin-ui: Export pattern ./*.css has no matching files, which is unrelated to this change (that package is not built in this worktree).

🤖 Generated with Claude Code

…he pipeline bench in the repo

- .v6/perf-audit/13-large-scale.md: the 00-profile pipeline on 1tup expanded to 32/192/864 copies, as an assembly
  (9 shared symmetry groups) and as one unshared model; stage tables at default and pinned 'low' quality, isolated
  reruns, render buffers, interactive paths (hover/select marks, overpaint, transparency, components), scaling,
  what breaks first, hotspots, new findings 13/P1–P6 and 13/B1, and a ranking of the wins at large scale
- tests/src/pipeline (bench.ts, fixtures.ts) and scripts/pipeline-bench.mjs: the headless pipeline bench moved out
  of the audit's scratch folder (fixtures, run, analyze for CPU and allocation profiles); GC pause time, peak RSS,
  release stages per representation, prerequisites run unmeasured under --only
- README and 00-profile point at the bench

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dsehnal
dsehnal marked this pull request as draft October 19, 2026 18:35
dsehnal added a commit that referenced this pull request Oct 20, 2026
* v6 perf bench: the headless pipeline bench from #1997 (verbatim)

scripts/pipeline-bench.mjs and tests/src/pipeline/{bench,common,fixtures}.ts as in #1997
(f06cd8b), so that the many-structures bench can build on them; identical content, so whichever PR lands
first, the other rebases cleanly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* v6 perf bench: many structures in one scene (many.ts, many-fixtures.ts)

- tests/src/pipeline/many-fixtures.ts: N local bcif files (default 500) cycled from six entries of 1.2k-11.5k
  atoms (data/examples, the Prism sequence fixtures), each moved to its own grid cell
- tests/src/pipeline/many.ts: a PluginContext with the default spec loads N structures (rawData, parse, the
  default hierarchy and auto representation presets), then theme change, hover, select, overpaint,
  visibility and removal, for one structure and for all; --transaction none|each|stage (plugin-API scripts,
  one transaction per file, one per stage as OpenFiles and Prism's load.files)
- plugin.canvas3d is a stub that does Canvas3D's work on representations (add, mark every representation)
  and counts what would reach the frozen scene
- tests/src/pipeline/stage.ts: the stage measurement of bench.ts as a module, with a per-item column
- scripts/pipeline-bench.mjs: many-fixtures and many commands; bundles define NODE_ENV=production as the app
  builds do (no development-only checks such as mutative's draft check)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* v6 perf plugin: structure info walks the state only after root model or structure changes (14/F1)

StructureInfo selected every root model and structure of the state (four walks over the whole tree) on every
object created: components, representations, overpaint layers. The max indices depend only on the root model and
structure objects (rootsOfType), so the walk runs again only after one of those was created or updated (an object
with no ancestor of its type: not a component or a decorator), or a model or structure was removed. The result is
the same, at the same moment (the next created object). Loading N structures was O(N^2) in cells.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* v6 perf plugin: the measurement manager finds its group once per state change (14/F2)

sync() looked for the measurement group (a walk over the whole state tree until the tagged node) once for each
of the six measurement kinds, on every state change outside a transaction: six walks per change. It now looks
once and passes the ref; the lists are the same.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* v6 perf plugin: the focus representation update walks the state once, commits only edits (14/F3)

Every representation preset updates the focus representation's theme params (updateFocusRepr), once per
structure loaded. The behavior's update selected the whole state three times (one per focus tag) and then
committed its builder even when empty, which diffs the whole state tree. It now selects once and partitions by
tag (the same cells in the same order) and skips the empty commit, as setOptions of the measurement manager does.

The empty commit was not quite a no-op: as an update without canUndo it cleared the data state's undo history,
toggled isUpdating, and waited for queued data updates. Those no longer happen for it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* v6 perf core: state updates diff a builder's edits instead of both trees (14/F4)

A builder commit with more than one edit walked the old tree (findDeletes) and the new one (findUpdateRoots), and
presets commit a few edits per object (a component and its representation; an empty component deleted): loading N
structures was O(N^2) in cells even inside one transaction.

- TransientTree records what a builder's tree sets, moves and removes (TransientTree.Edits, editInfo.edits), writes
  to currentTree included
- a builder on the current tree that moved nothing and removed at most one node takes its deletes (the removed node's
  subtree in the old tree, post-order) and roots (edited refs with another version, below no node the diff stops at,
  in pre-order) from those edits: the same as the whole-tree diff, in the same order
- only while the cells are in line with the tree (cellsInSync): false after an update that threw, or a fast track
  whose builder's tree changed outside its node, until a whole-tree pass completes
- state/diff.ts holds both diffs; _test/diff.test.ts compares them before every commit of random builders (Null
  objects, errors, flags, tags, writes to currentTree, removals) and pins both fallbacks
- reconcile.md §2, §8, tree.md §4, README invariant 2, testing.md; TODO: a builder's removal can leak into the state's
  tree with generated refs (pre-existing, found by the random generator)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* v6 perf plugin: updateFocusRepr skips themes that are already set (14/F5)

Every representation preset calls updateFocusRepr, once per structure loaded, mostly with the color themes the
focus representation already has. The behavior update then applied the same params again: it walked the whole
state for the focus representations and updated each with identical props. It now returns when both themes are
deep-equal (core's deepEqual) to those of the last update started (the behavior tree), and never while an earlier
call's update is still in flight, so the last of overlapping calls wins as before. A different theme updates as
before. A skipped update also skips what that update did besides (see 14/F3: no data-state commit), and the
Log's transforms lose its "Updated" line (Prism's log-timing test loads one file more to pass its 100 cap).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* v6 perf plugin: structure info keeps its root models and structures instead of walking (14/F6)

After 14/F1 the behavior still selected the root models and structures of the state (rootsOfType, a walk of the
tree) about nine times per structure loaded: for a new model's index and chain offsets, a new structure's index,
and the max indices. It now keeps those roots from the object events (the same root test as rootsOfType: no
ancestor of the same type), seeded by one walk when registered, plus the cells new in the current update whose
events have not come yet (as the walk would see them). A cell's object is checked when read, and a ref leaves on
any update or removal, whatever the old object (an update in place to a Null object carries none). The values are
maxima, so the order does not matter; the results are the same (tests recorded on master's behavior).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* v6 perf plugin: SelectLoci does not mark an empty selection on a new representation (14/F7)

For every representation object created, SelectLoci marked its structure's selection (Select) through the
canvas, which asks every representation in the scene: O(representations) per representation, O(N^2) for a scene
of N structures. With nothing selected the mark changed nothing (every representation kind returns false for an
empty loci), so it is skipped; a non-empty selection is marked as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* v6 perf plugin: volume and particle hierarchies skip the walk without any volume or particle list (14/F8)

The volume and particle hierarchy managers rebuilt their hierarchies with a walk over the whole state on every
state change outside a transaction, also in a scene of structures only, where they can find nothing: at 500
structures 13 of the 40 ms of a single theme change or removal. Each now keeps the refs of its root objects (volume
data and lazy volumes; particle lists) from the object events (trackObjectRefs, a superset) and returns before the
walk when there are none and its hierarchy is empty, as the walk would (no change).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* v6 perf audit: many structures in one scene (14), changelog

- .v6/perf-audit/14-many-structures.md: the scenario (up to 500 separate 1–12k-atom entries), the bench, what
  dominated (all plugin and state work, O(N^2)), the fixes 14/F1–F8 with per-commit and scaling numbers, and the
  ranked findings not fixed (marks over every representation, the structure hierarchy rebuilt per change, scripts
  without transactions, the cartoon mesh estimate, 13/P4 re-measured, a core state bug)
- .v6/changelogs/perf-many-structures.md

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* v6 prism stories: 50 separate structures in one scene (Viewer/Data)

ManyStructures50: six local entries (1.2k–11.5k atoms) cycled into 50 files, opened in one drop (load.files) and
moved onto a grid (structure.transform) in one data transaction; the load and arrange times show in the corner.
No network.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* v6 perf audit 14, changelog: after merging master's 10/P4 and 10/P5 (#2008)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* v6 prism stories: the 50-structure story's atom count as shown (default assemblies)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* v6 review fixes: the 50-structure story arranges as one step; StructureInfo keeps roots of an update that threw

- data.stories.ts: the 50 moves run in api.group (one history step, all or nothing) around a data transaction that
  rethrows, so a failed move restores the scene instead of logging and reporting it arranged
- structure-info.ts: when an update ends, a pending cell with an object and no event (its update threw after making
  it) is kept as a root, as the walk would find it
- tests: stale comments after #2008's semantics; the measurement test that duplicated master's measurement-sync test
  dropped
- changelog: its numbers predate the merge of #2008

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant