After a vendored-to-hosted takeover is interrupted once its commit journal is written, the next rollback replays the journal, then exits 0 having restored nothing, and the project stays hosted-patched (remove says "No patch found") #1241

[agent] Found by the scheduled Yarn Berry (2+) bug-hunt routine (ledger #305).

Summary

A vendored→hosted takeover (scan --mode hosted over a vendored project, #1039) commits through the roll-forward journal .socket/vendor/.commit-journal.json. Suppose the process dies after the journal is durable but before any file is replaced, and rollback is the next command. Then:

1. rollback discovers the hosted pins before it takes the apply lock (crates/socket-patch-cli/src/commands/rollback.rs:908-915, hosted_inventory). At that moment yarn.lock and package.json still hold the vendored file: wiring, so it finds no hosted pins.
2. acquire_or_emit (rollback.rs:1008) takes the lock, and apply_lock::acquire → recover_group_commit replays the journal. The hosted resolutions pins and re-keyed lock entries land, and .socket/vendor/state.json is deleted.
3. The vendored leg loads the ledger under the lock and finds nothing. The hosted leg uses the stale pre-lock list, which is empty.

rollback exits 0 and prints nothing at all (human mode). JSON reports status: success, rolledBack: 0 and hosted.reverted: []. The project ends up hosted-pinned: every fresh yarn install --immutable installs the patched tarball. A second rollback does restore it.

remove <uuid> and remove <purl> from the same state replay the journal too, then fail with Error: No patch found matching identifier: … (exit 1), although the patch is pinned in the lock they just rewrote. remove.rs has the same order: hosted_inventory at :367, lock at :406.

Impact

The user asked to remove every patch, got exit 0 and no error, and the project still installs Socket's patched bytes. CI or scripted unwinds (rollback && git commit) commit a hosted-pinned lock. The window is narrow (a crash, SIGKILL or OOM between the journal write and the first file replace), but this is exactly the case the journal exists for. The CLI contract promises that the next locked command "replays the journal before reading anything … so a locked command never observes a half-committed run" (CLI_CONTRACT.md, "Vendored group commit"), and that "a commit interrupted after its journal was written is finished by the next command that takes the apply lock" ("Staged takeover").

The cause is in shared CLI code, not in the yarn rewriter, so every ecosystem with a vendored→hosted takeover should be affected. I proved it only with real Yarn Berry.

Repro (real yarn 4.18.1 / 4.0.2, release build, Linux)

This uses a local mock of the patch API (/v0/orgs/org/patches/{batch,package,view,by-package} plus /artifacts/<uuid>/<name>-<ver>.tgz with a yarn-berry-zip yarnBerry10c0 artifact), as in #1157. No failpoint build is needed: a real SIGKILL timed into the commit window hits it within a few tries.

SP="target/release/socket-patch"; API="--api-url http://127.0.0.1:8790 --api-token x --org org --patch-server-url http://127.0.0.1:8790"
mkdir base && cd base
echo '{"name":"proj","dependencies":{"left-pad":"^1.3.0","is-number":"7.0.0"}}' > package.json
echo 'nodeLinker: node-modules' > .yarnrc.yml && yarn install
# hand-stage .socket/manifest.json + blobs for left-pad@1.3.0 and is-number@7.0.0, then:
$SP vendor $API && rm -rf .socket/manifest.json .socket/blobs
git init -q && git add -A && git commit -qm vendored && cd ..
# Kill the takeover inside its commit window. Retry until the journal exists and yarn.lock is still vendored.
for t in 0.040 0.042 0.044 0.046 0.048 0.050; do
  rm -rf w && cp -r base w && cd w
  $SP scan --mode hosted $API >/dev/null 2>&1 & p=$!; sleep $t; kill -9 $p; wait $p
  if [ -f .socket/vendor/.commit-journal.json ] && ! grep -q 8790 yarn.lock; then
    $SP rollback $API; echo "rollback exit=$?"
    grep -E '^"(left-pad|is-number)@' yarn.lock; grep -c 8790 package.json
    break
  fi; cd ..
done

Output (the same on every hit):

rollback exit=0
"is-number@http://127.0.0.1:8790/artifacts/3333…/is-number-7.0.0.tgz":
"left-pad@http://127.0.0.1:8790/artifacts/1111…/left-pad-1.3.0.tgz":
2

Apart from the token warning, rollback printed nothing. A fresh checkout's yarn install --immutable (cold cache) then installs /* SOCKET-PATCHED … */ left-pad. Running rollback a second time prints Restored pkg:npm/is-number@7.0.0 to its upstream registry entry (and the same for left-pad), and yarn.lock and package.json are byte-identical to the pre-vendor commit.

State at the kill: git status shows only ?? .socket/apply.lock and ?? .socket/vendor/.commit-journal.json. The journal names package.json, yarn.lock (both with after bytes) and .socket/vendor/state.json (deletion).

From a copy of the same post-kill snapshot:

Expected vs actual

• Expected: rollback (and remove) plan against the project as it is after the journal replay. CLI_CONTRACT says a locked command replays the journal "before reading anything", so the replayed hosted pins are restored to the registry entries and the run reports them. Concretely: run the hosted-pin discovery (hosted_inventory) and the truly-empty probes after acquire_or_emit, or re-run them once the lock is held when a journal was replayed.
• Actual: discovery runs before the lock. The replay changes what is pinned under it, and the run exits 0 having unwound nothing. remove refuses a patch that is pinned.

Matrix (Linux, Node 22, main f3c6313)

macOS and Windows weren't probed; this is OS-independent CLI ordering. Release 4.0.0 isn't affected: it has no journaled takeover. The first bad commit is probably 823810a (#1039, which made the takeover a journaled group commit); not bisected.

Suspect code

• crates/socket-patch-cli/src/commands/rollback.rs:908-917: hosted_inventory and the truly-empty decision run before the lock. The comment says only cheap existence probes run pre-lock, but the hosted pins are the leg's work list.
• crates/socket-patch-cli/src/commands/rollback.rs:1008: acquire_or_emit → socket_patch_core::patch::apply_lock::acquire → recover_group_commit (crates/socket-patch-core/src/patch/apply_lock.rs:576).
• crates/socket-patch-cli/src/commands/remove.rs:367 / :406: the same order.

Related but different: #1157 (the same crash window leaves the vendored artifact directory behind; there a scan was the next command) and #809 (lock-free readers see the pending journal).