Visitar URL original
fix(cloud): autosave every online session with a retry queue, and replace the Save button with a status chip (#2615) by erseco · Pull Request #2616 · exelearning/exelearning · GitHub
Skip to content

fix(cloud): autosave every online session with a retry queue, and replace the Save button with a status chip (#2615) - #2616

Open
erseco wants to merge 13 commits into
mainfrom
feature/issue-2615-cloud-autosave
Open

erseco wants to merge 13 commits into
mainfrom
feature/issue-2615-cloud-autosave

Conversation

@erseco

@erseco erseco commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #2615

Builds on #2618 (merged): convergence with the stored snapshot on open/reconnection and server-side save validation are owned by ADR-2615-01. This PR adds automatic saving for every online session, the retry queue, keeping unsaved local work, save-before-export and the status chip. Its record is ADR-2615-02 (Proposed).

Problem

Reported from the Extremadura cloud (v4.0.5): a REA edited on 13–14 October, exported and published on the 15th, reopened on the 22nd without those edits. Besides the stale-client overwrite fixed in #2618, the persistence model itself let edits go missing:

  • Single-user online sessions only persisted the Yjs document on a manual Save. The autosave from Collaborative editing can cause data loss/confusion when the last editor does not save before closing #1592/fix(collaboration): autosave shared Yjs changes in collaborative sessions (#1592) #1985 was gated on another collaborator being present, a scope decision with no technical reason recorded.
  • The beforeunload safety net sends the full encodeStateAsUpdate() with navigator.sendBeacon, which Chromium rejects above 64 KiB. Every real project is above that (fixtures: 73 KiB, 880 KiB, 1.76 MiB), so it silently does nothing.
  • Closing the last tab schedules an IndexedDB wipe; the next fresh tab loads the server snapshot and unsaved edits are gone.
  • Exports are built from the browser Y.Doc, so a published package could be newer than what the server stores.
  • SaveManager.save() encodes the snapshot early and calls markClean() at the end: an edit made while the upload runs was marked as saved without ever being sent. With autosave this window happens on every idle save, so it is fixed here too.

Solution

Cloud mode now persists like Google Docs: edits are saved automatically after the idle period, and nothing is exported that the server has not received.

  • Always-on autosave. CollaborativeAutosaveManager no longer requires a collaborator. Eligibility is remote storage + collaboration enabled + a live WebSocket, so static, offline and Electron are untouched. It also listens to sync/connectionChange, so a dirty state restored after a reload is autosaved once connected, and it announces the current phase before any edit.
  • Retry queue that never gives up. While the document is dirty it retries with exponential backoff (5 s → 80 s) and then keeps retrying at the capped interval. A reconnection retries at once. A failed manual or pre-export save (which cancels the queued autosave when it starts) hands the document back to the queue through notifyExternalSaveFailed().
  • A save only cleans what it uploaded. YjsDocumentManager.getChangeSeq() counts tracked edits; SaveManager reads it before encoding and passes it to markClean(), which keeps the document dirty (and re-emits dirty) if anything changed meanwhile. The save's own bookkeeping does not count: the uploaded-asset flag is written with an asset-upload origin, and the post-save thumbnail only re-cleans an already clean document.
  • Unsaved local work survives closing the last tab. The deferred IndexedDB wipe is skipped when the dirty flag is set; the server snapshot is merged on top (idempotent Yjs merge) and the restored dirty state is autosaved once connected. Clean sessions are wiped as before.
  • Save before export. YjsProjectBridge.flushPendingSave(reason) is called from exportToElpx() (after the version and screenshot metadata writes, so they are saved with the export) and from NavbarFile.exportViaYjs() (every client-side format). It waits for a save already in flight, never throws and never overlaps. _updateVersionMetadata() no longer rewrites an unchanged version, so exporting does not leave the document dirty.
  • Status chip instead of a Save button (cloud only). #head-top-save-button keeps its id and saved/unsaved/saving classes; with cloud-autosave (emitted only when the installation is not offline) and once a phase has been announced, it renders as a chip. A session that never becomes eligible (collaboration disabled, WebSocket blocked) keeps a working Save button.
Phase Chip Tooltip / live region
clean cloud_done Saved All changes are saved.
pending cloud_upload Unsaved changes Changes will be saved automatically.
saving cloud_sync (pulsing) Saving... Saving changes...
failed cloud_off Not saved (red) Autosave failed. Retrying automatically. Click the status to retry now.

A click on the chip still saves immediately (manual retry); Ctrl+S and the File menu are unchanged. Narrow screens show the icon only, prefers-reduced-motion disables the pulse, and each phase has a distinct icon, not only a colour. Wording no longer says "collaborative" (new English source strings, no catalog edits). doc/development/real-time.md corrects the "explicit saves only" claim and documents the beacon limitation.

What a browser cannot guarantee: a tab killed by the OS or a crashed browser in the seconds between an edit and the idle autosave. Even then the edit is in IndexedDB for the next open.

Screenshots

Saved Unsaved changes
Saved Unsaved changes
Not saved (save endpoint returning 503) Phone width (icon only)
Not saved Phone width

Deliberately left out

  • The sendBeacon fallback is left as is (harmless for small documents; autosave, the retry queue and the kept local copy make it moot).
  • The pre-export save is a full SaveManager.save() (pending assets included): that is what keeps the server and the exported package in step.
  • Two clients saving divergent edits within the same seconds is handled by the server-side merge of ADR-2615-01; version history ([Proposal] Keep a configurable Yjs version history for administrator recovery #2163) remains a separate proposal.

Tests

Unit (Vitest):

  • CollaborativeAutosaveManager.test.js: single-user eligibility, no autosave once the socket closed (stale synced flag), connection-aware start, unbounded retry and reconnection recovery, re-arming after a failed external save (including an autosave deferred behind it).
  • SaveManager.test.js: change counter read before encoding and passed to markClean; a failed manual save notifies the autosave manager, its own failures do not.
  • YjsDocumentManager.test.js: markClean(seq) keeps a document changed after the snapshot dirty; asset-upload and suppressed updates are not counted; unchanged version not rewritten; deferred cleanup keeps unsaved work / wipes clean sessions / offline unchanged.
  • YjsProjectBridge.test.js: flushPendingSave (waits for an in-flight save, gives up without overlapping, clean doc, no remote storage, failures); export order version → screenshot → flush → export; thumbnail does not re-clean a dirty document.
  • AssetManager.test.js: uploaded flag written with the asset-upload origin. navbarFile.test.js, CollaborativeSaveStatusView.test.js, unsavedChangesHelper.test.js: export flush, chip rendering, wording.

E2E (Playwright), cloud-autosave.spec.ts: single-user edits reach the server (verified after a real IndexedDB wipe); pending → clean chip and live region, no legacy red dot over the chip; clicking the chip cancels the queued autosave; an untouched blank project never saves; an edit made while an autosave uploads is saved by the next one; a burst of edits reaches the server with its last value; the Save button is the fallback when no phase was announced; reload with a pending edit; ELPX export saves first and ends clean; unsaved edits survive closing the last tab; a 503 yields failed and the queue recovers by itself. cloud-autosave-stale-client.spec.ts: a tab offline while a collaborator saved converges on reconnection and its autosave keeps the collaborator's node. indexeddb-tab-close-cleanup.spec.ts covers the keep and wipe paths. The manual-flow specs (save-detection*, open-project-clean-state, indexeddb-tab-close-cleanup) pin a long idle.

Red → green for the review fixes: the concurrent-edit, Save-fallback and red-dot E2E tests failed on the previous build and pass now; the new unit tests cover methods and guards that did not exist.

Results (local, after the review fixes): make fix clean · make test-frontend 18447 pass (316 files) · make test-unit 10981 pass / 0 fail (the per-file 90 % threshold reports 14 src/ files this PR does not touch) · make architecture-check OK · E2E Chromium 36/36 for the autosave, stale-client, collaborative-autosave, tab-close cleanup, clean-state and save-detection specs · E2E Firefox 26/26 for cloud-autosave, stale-client, collaborative-autosave, tab-close cleanup and save-detection.

Single-user cloud sessions only persisted the Yjs document on a manual
Save, and the beforeunload sendBeacon fallback silently fails above 64 KiB,
so edits made without saving were lost once the next fresh tab wiped the
local IndexedDB copy. Exports were built from the browser document, so a
published package could be newer than the server snapshot.

- CollaborativeAutosaveManager no longer requires another collaborator to
  have been present; it runs in every online session with remote storage
  and a live connection, and re-arms when the connection syncs so a dirty
  state restored after a reload is persisted too.
- YjsProjectBridge.flushPendingSave() saves silently before building an
  ELPX; NavbarFile.exportViaYjs() does the same for every client export.
- Save-status wording no longer talks about "collaborative" changes.
- The manual Save button is kept on purpose until autosave has proven
  itself in production.
- New Playwright spec covers persistence across the IndexedDB wipe, the
  pending/clean states, blank projects, debounce, reload, export and
  failure recovery; dirty-state specs pin a long idle so they stay
  deterministic.

Closes #2615
@erseco erseco added bug Something isn't working realtime cloud labels Oct 23, 2026
@erseco erseco self-assigned this Oct 23, 2026
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Deploying exelearning with  Cloudflare Pages  Cloudflare Pages

Latest commit: ef4fffa
Status: ✅  Deploy successful!
Preview URL: https://0607cc95.exelearning.pages.dev
Branch Preview URL: https://feature-issue-2615-cloud-aut.exelearning.pages.dev

View logs

@codecov

codecov Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Bundle Report

Changes will increase total bundle size by 99 bytes (0.01%) ⬆️. This is within the configured threshold ✅

Detailed changes
Bundle name Size Change
app-bundle 991.49kB 99 bytes (0.01%) ⬆️

Affected Assets, Files, and Routes:

view changes for bundle: app-bundle

Assets Changed:

Asset Name Size Change Total Size Change (%)
app.bundle.js 99 bytes 991.49kB 0.01%

@codecov

codecov Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.27%. Comparing base (bce8f99) to head (ef4fffa).
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2616      +/-   ##
==========================================
+ Coverage   92.25%   92.27%   +0.01%     
==========================================
  Files         479      479              
  Lines       89517    89593      +76     
==========================================
+ Hits        82584    82669      +85     
+ Misses       6933     6924       -9     
Flag Coverage Δ
frontend 92.27% <100.00%> (+0.01%) ⬆️
unittests 92.27% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

The manual Save button stays; what changes is that an autosave in flight
can no longer be lost:

- CollaborativeAutosaveManager retries forever while the document is
  dirty: exponential backoff for the first `maxRetries` attempts, then a
  capped steady interval. A reconnection (`sync`/`connectionChange`) drops
  the backoff and retries at once. The 'failed' notice stays until a save
  actually succeeds.
- YjsDocumentManager no longer wipes IndexedDB and the dirty flag on the
  next fresh tab when the last tab closed with unsaved work. The local copy
  is kept, the server snapshot is merged on top (Yjs merges are
  idempotent, so saves made elsewhere are not overwritten) and the restored
  dirty state is autosaved once the connection is live. Clean sessions are
  wiped exactly as before.
- `window.__EXE_COLLAB_AUTOSAVE_RETRY_MS__` mirrors the idle override so
  tests can shorten the backoff.

Tests: manager retry/recovery suites; document-manager deferred-cleanup
suite (keep/wipe/offline/same-tab); E2E "unsaved edits survive closing the
last tab" and "recovers by itself once the endpoint is back"; the
tab-close cleanup spec now covers both the keep and the wipe paths.

Refs #2615
With autosave on in every online session and a retry queue that never
gives up, the manual Save button is redundant in cloud mode. The toolbar
control now carries `cloud-autosave` (server-rendered only when the
installation is not offline; the static build strips the block and
Electron never gets it) and CollaborativeSaveStatusView renders it as a
Google-Docs-style status chip: cloud_done "Saved", cloud_upload "Unsaved
changes", cloud_sync "Saving...", cloud_off "Not saved", with the full
sentence as tooltip and the live region unchanged. A click still saves
immediately, which doubles as the manual retry. The leave warning no
longer tells users to click Save.

Tests: chip rendering suite for the view; Playwright asserts the Save
label/icon are hidden and the chip states in the single-user and
collaborative specs; existing class-based save-detection specs unchanged.

Refs #2615
@erseco erseco changed the title fix(cloud): autosave every online session and persist before exporting (#2615) fix(cloud): autosave every online session with a retry queue, and replace the Save button with a status chip (#2615) Oct 23, 2026
Records the decision to autosave every online session with an unbounded
retry queue, keep unsaved local work across the last-tab cleanup, save
before exporting and replace the Save button with a status chip in cloud
mode, with the sendBeacon 64 KiB measurements as evidence.

Refs #2615
The WebSocket layer is a relay with no document state, and the stored
snapshot was read only when IndexedDB was empty. A tab that was offline
while other users saved (laptop asleep, restored tab) therefore kept its
old copy on reconnection and its next save replaced the server snapshot
with it. This is the mechanism behind the field report in #2615: the
publisher saw the edits on the 15th, so the server had them, and a later
stale save erased them.

YjsDocumentManager now merges the server snapshot on every online open,
whether or not a local copy exists, and again after a WebSocket drop
before re-announcing the connection (so the autosave re-arm sees the
merged document). The merge uses the 'initial' origin so it never marks
the document dirty or enters the undo stack, and concurrent merges share
one request.

Tests: document-manager suite for merge on open, offline, origin, reconnect
ordering, failure and dedupe; Playwright cloud-autosave-stale-client.spec
reproduces both the offline-tab and stale-copy scenarios (red without this
change, green with it).

Refs #2615
loadFromServer() swallows its own errors, but the reconnect hook chained
.finally() which re-threw a mocked rejection as an unhandled error and
failed the Vitest run. Announce the connection on both branches instead.

Refs #2615
Firefox's offline emulation does not close an already-open WebSocket, so
the scenario timed out in CI. Disconnecting and reconnecting the
y-websocket provider exercises the same status handler deterministically
in every browser.

Refs #2615
ADR-2615-01 now belongs to PR #2618 (convergence with the stored snapshot
and server-side save validation). This record builds on it, drops the
convergence point it restated, and names Claude Mythos 5.1 as the model.

Refs #2615
@erseco erseco added this to the v4.1.0 milestone Oct 24, 2026
Combine with #2618 (validated saves): keep both the unsaved-local-copy recovery and serverVersion tracking, and both sets of #2615 tests.
stale-client-save.spec.ts (#2618) already covers the open-time merge of a stale local copy. Keep only the reconnection scenario here, which checks that the autosave, not an explicit Save, carries the collaborator's edits.
- A save only marks the document clean when nothing changed after its
  snapshot was encoded (YjsDocumentManager.getChangeSeq). Edits made while
  an autosave uploads stay dirty and are saved next. The uploaded-asset
  flag uses its own origin and the post-save thumbnail only re-cleans an
  already clean document, so neither hides a real edit.
- A failed manual or pre-export save re-arms the autosave retry queue it
  had cancelled.
- Eligibility relies on the provider's live flags only; the document
  manager's synced flag is never reset after a disconnect.
- The pre-export flush waits for an in-flight save and runs after the
  version and screenshot metadata writes; an unchanged version is not
  rewritten, so an export no longer leaves the document dirty.
- The chip only replaces the Save button once a phase was announced, and
  the legacy red dot no longer overlaps the chip icon.
- Drop the unused _recoverUnsavedLocalCopy flag and merge isEligible().
@erseco erseco added this to the v4.0.7 milestone Oct 29, 2026

This branch had an error being deployed

1 failed deployment
feature/issue-2615-cloud-autosave - exelearning PR #2616 — ef4fffa6 Deployed Oct 29, 2026 by erseco
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working cloud realtime render-preview Create a render.om preview

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cloud: unsaved edits are silently lost; make persistence automatic (Google Docs model) and remove the manual Save button in cloud mode

2 participants