feat(mcp): scoped, expiring tokens for the HTTP MCP server (#3) - #259

Closes #3.

Problem

boxery-mcp --http had a single bearer token, checked once at mcp/boxery-mcp.mjs. Whoever held it could call every tool on every VM of every registered host. To confirm, I ran vm_bash on a VM the client never created, offline with a stub host. There was no expiry, and the only way to revoke the token was to rotate it. Rotating didn't work either: --token-file was read once at startup, so the old token kept working until the server restarted.

What this does

1. --token-file is read again when it changes (fix(mcp): …). The file gets a stat per request and is re-read when it changes, as boxery-proxy does for tokens.json. Rotating it revokes the old token at the next request. While the file is missing or empty, every request is refused, and the server won't start without a readable one.

2. Scoped tokens (boxery mcp-token new|ls|rm, documented in docs/mcp.md#scoped-tokens):

boxery mcp-token new ci-agent --ttl 7d                    # its own VMs only, on the server's --host
boxery mcp-token new viewer --vm 'demo-*' --read-only
boxery mcp-token new ops --host hornet --tools vm_bash,vm_info --vm 'web-*'
boxery mcp-token rm viewer                                # refused from its next request

• Tools: all by default, or --tools TOOL[:ACTION],…, or --read-only (vm_read, vm_screenshot, vm_info). Anything not listed is refused.
• Admin work needs --admin: build-base, host_add, host_rm, pool leases, and any other host-wide command.
• Hosts: the server's --host by default, or each --host NAME.
• VMs: those matching --vm patterns, plus those the token created itself.
• Ownership: a created VM is recorded with its ssh host key, so a VM someone else later makes under the same name isn't the token's. On boat and OpenShell, which manage host keys themselves, the name alone identifies it, and ls says so.
• Expiry: 30 days by default (--ttl), or never.
• Storage: tokens are stored hashed in ~/.boxery/mcp-tokens.json (0600), which vm_copy and vm_grant already refuse to read.

Design choices (please push back)

• Tokens live on the controller, not in boxery-host as the issue suggested. The HTTP server is a controller-side front-end that can drive several hosts, and the host side shouldn't import the core.
• Scope is checked where boxery acts, not only when a tool starts. A tool's arguments name VMs many ways: vm, VM@SNAP, HOST/VM:path, fork targets, generated names. So core/mcp-auth.mjs checks the scope again where a name finally reaches ssh (sshRun, vmProblem, applyGrants, sync's git push, the desktop tunnel, both vm_copy endpoints), and before every host command (hostRun, by command). runTool checks the tool and action first, only for a clearer message. A test fails when a new ssh or host-command call site appears without a check.
• Fails closed. The grant rides in async context next to the current host. The HTTP server calls enforce(), so an operation that lost its grant is refused. stdio, pi and the CLI never call it, so they're unchanged.
• Follows the MCP spec's security guidance: every request is verified, and a session belongs to the token and Tailscale user that opened it, so another can't use or end it. An invalid token gets 401 with WWW-Authenticate: Bearer error="invalid_token". I didn't implement the OAuth 2.1 flow: the spec makes authorization optional, and it would be a lot of machinery for a server reached over a tailnet.
• What the agent sees: tools/list shows only the tools the token allows, and the initialize instructions add one line describing the scope. Refusals read … is outside this token's scope (token 'x'); ask the user for a token that covers it, the same whether or not the VM exists. vm_info list, images and hosts are filtered too. Other output, such as "name taken" or a host error, can still reveal that a name exists, but not let the token use it (documented).
• Existing setups keep working. The BOXERY_MCP_TOKEN / --token-file token keeps full access, and the server logs a warning about it. The server can now start with scoped tokens alone.

Commits

1. fix(mcp): re-read --token-file when it changes.
2. style(mcp): whitespace.
3. feat(core): token store and scope rules (core/mcp-tokens.mjs), and the checks where boxery acts (core/mcp-auth.mjs). Nothing is scoped yet, so behaviour doesn't change.
4. feat(mcp): the HTTP server accepts scoped tokens, and sessions belong to their token.
5. feat(cli): boxery mcp-token, plus docs: docs/mcp.md, docs/security.md, ARCHITECTURE.md (a design decision, Scoped, expiring tokens for the HTTP MCP server #3 removed from the roadmap), the CLI reference and help goldens.

Each commit passes typecheck and the related tests on its own. I can split them into separate PRs if you prefer.

Tests

All offline, using the existing stub host and fake VM.

• test/mcp-tokens.test.mjs:
  • store: hashing, mode 0600, expiry, revocation, a broken store refuses everyone.
  • scope rules, patterns, and that every admin action is a real action.
  • the guarded host commands are real boxery-host commands, and the name-only backends match those declaring backend_platform_host_key.
  • the call-site guard.
  • a fail-closed check: no grant means refused once enforced.
• test/mcp.test.mjs, server with scoped tokens only:
  • a token without a VM is refused 17 ways of reaching one: vm, use, VM:path and HOST/VM:path copies, VM@SNAP, fork, rm/snap/revert/stop, vm_jobs, vm_grant, another host, admin actions. Only listings reach the host, and nothing runs in the VM.
  • a created VM is the token's until its host key changes, and rm forgets it.
  • --vm, --read-only and --tools behave as documented.
  • session ownership, revocation and expiry.
  • --token-file rotation.
• The tests catch real gaps: with the checks where boxery acts disabled, two of them fail.

Run locally on macOS:

• Typecheck: passes.
• Lint: all of test/lint.sh except shellcheck (not installed here; no shell files changed).
• npm test: every file passes when run alone. A full parallel run on this machine had a few timing failures (copy cancellation, background jobs) while it had about 56 MB of free memory, and they pass when rerun alone.
• npm run test:shell: every file passes except common/controller-local.sh, which hangs on this machine at the attach w1 bogus pty case. Unmodified master hangs at the same place, and the 75 cases before it pass identically.
• Evals: not run. Default and unscoped sessions get the same tool list and instructions as before; only scoped sessions get the extra scope line.

Not in this PR (follow-ups)

• Tokens bound to a lease.
• A per-VM id in the host's vm_records, which would remove the name-only fallback on boat and OpenShell.
• Per-token budgets.
• Tailscale app capabilities as a second source of scopes.
• OAuth 2.1, if hosted multi-user use becomes a goal.
• Runtime argument validation (Bound HTTP MCP sessions and requests, and validate tool arguments at runtime #21 / fix(mcp): bound HTTP session retention and concurrent tool work #205): the scope check should run right after it once it lands.