feat(cli): SQL divergence — integration-scoped, measurable, with optional local-model triage - #559

Wilson measures how lopsided a split is. It cannot measure whether the two forms
were ever supposed to agree, and that is the question that decides whether a
finding is worth anyone's time. Whole classes of false positive are invisible to a
lexer: a name collision, `x` versus `t.x`, `count(1)` versus `count(*)`, a table
pair joined two ways because the two joins answer different questions.

Normalising harder is not the fix and is deliberately not used as one — stripping
table qualifiers would collapse a genuine finding, a table rename that only half
the workspace followed. Canonicalisation stays semantics-preserving and the
judgement call goes to a model.

`deepnote audit --triage`. The guarantees, each with a test:

- **Off by default.** No provider means the deterministic path verbatim: the JSON
  is byte-identical and `details.signalSource` reads `prior`.
- **No endpoint, no call.** There is no default URL. `--triage` without
  `DEEPNOTE_TRIAGE_BASE_URL` exits 2 with instructions for pointing it at Ollama,
  LM Studio or vLLM. The resolved endpoint is printed before the first request.
- **The model never sees the corpus.** It gets pre-grouped variant forms, never a
  block and never an output, each passed through the same secret and subject
  redaction as the report. A metric variant keeps string literals verbatim, which
  is the actual leak path, so the redaction test is built on that case and a
  guard test asserts the fixture really is leaky.
- **Bounded by finding count, not corpus size**, asserted by comparing a
  four-query workspace against a three-hundred-query one.
- **Cached** under `.deepnote/`, keyed by model, so CI is free and offline on a
  hit. `--no-triage-cache` busts it.
- **Both numbers recorded.** A verdict replaces the per-kind prior in `signal`,
  and `details` carries `signalSource`, the verdict, its reason and the prior it
  displaced. `false-positive` leaves the ranking but stays in `suppressed` and is
  printed with its reason — suppression nobody can read is indistinguishable from
  a check that quietly stopped working.
- **Never fatal.** Any error, timeout or malformed verdict warns once and the
  deterministic score stands.

One deviation worth flagging. `generateObject` from `ai` was the intended route,
but `ai` and `@ai-sdk/openai` are transitive dependencies of
`@deepnote/runtime-core` and pnpm's strict layout does not resolve them from
`packages/cli`; declaring them would make this the only branch in the stack that
touches a dependency file. The provider is therefore plain `fetch` against an
OpenAI-compatible `/chat/completions` — which is what Ollama, LM Studio and vLLM
expose anyway — isolated in `createOpenAiCompatibleProvider` so adopting
`resolveAgentModel` from #549 is a change to that one function.

Follow-up, not here: the same interface can triage the heuristic half of
`credential-hardcoded` and classify `egress-external` destinations.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…easurable

Three changes to the consensus check, plus the three review findings on it.

**Anchors are scoped by integration.** `sql_integration_id` was read for the ingress
inventory and ignored by everything else, so a table called `users` behind one
connection was compared against a table called `users` behind another — a
disagreement manufactured between systems that never shared a schema. Anchors are
now keyed by integration, `--divergence-scope type` relaxes to the integration type
for teams each holding their own connection to one warehouse, and `none` restores
the old behaviour so the cost of the scoping stays measurable. A block with no
integration goes in its own bucket and is never compared against a known warehouse;
the count is in the report. On the fixture's new second integration this is the
difference between 6 observations and 5, and between a finding and nothing.

Scoping is also what makes dialect folding safe: within one integration type `nvl`,
`ifnull` and `coalesce` are one intent written three ways, and reporting that as a
disagreement is noise. Across two warehouses it would have been a claim about an
agreement nobody tested.

The short-name table decision was re-read rather than left alone. Scoping removes
the worse half of its cost — the cross-warehouse comparison — and the residual,
`staging.users` against `prod.users` inside one warehouse, is narrower than what
keying on the qualified name would cost: the same table written two ways would stop
being one subject, which is the common case rather than the exotic one. Short name
stays, with the reasoning written down.

**Precision is now measurable instead of asserted.** `--export-review` writes every
group with a blank verdict and the locations to judge it against, best-attested
first; `--import-review` reads the verdicts back, reports precision per kind, and
uses it in place of the default once ten entries of that kind are judged — below
that the measurement is noisier than the guess it would replace, so it is reported
and not used. With both a review file and a triage run, the audit reports how often
the model agreed with the reviewer, per kind. That is the number that decides
whether the model earns its place.

The per-kind weights are reworded and regenericised: they encode one structural
claim — a table pair means exactly one thing, an output name is a convention — and
the magnitudes are arbitrary starting points, no longer presented as drawn from a
corpus.

Review findings fixed:
- `!=` and `<>` yielded join keys, so an anti-join produced the same key as the
  equi-join beside it and two genuinely different queries counted as agreeing,
  suppressing the divergence rather than reporting it. Only `=` and `<=>` now do.
- `--divergence-kind` and `--min-confidence` are validated; a NaN confidence
  compared false against everything and made the flag look like it did nothing.
- `N querys diverges`.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…parseable

Two things broke the loop this PR exists to enable.

The review export rebuilt candidate ids from the raw variant labels, while
triage hashes the redacted ones. The two agreed on every group whose SQL
happened to carry nothing redactable and disagreed silently on the ones that
did, so a filled-in file measured precision over no entries and reported
that as an empty result rather than as a mismatch. It also wrote whatever
literals the labels carried — addresses, tokens — into the one file whose
purpose is to be handed to somebody else. Both the id and the text now come
from toCandidate, so a reviewer and a model judge the same text under the
same id, and `locations` is how you get to the real query.

`--export-review` also printed its four lines of instructions under
`-o json`, in front of the document, with the command still exiting 0 — a
pipeline read a successful run and unparseable output.

Also derives the consensus floor in the audit help text from the constant
rather than restating it, which is how it came to disagree with the docs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…t measures nothing

Two more of the same shape as the review export.

`triageDivergence` printed its endpoint disclosure, redaction notice and
stats through `output` unconditionally, so `--triage -o json` put four lines
in front of the document and still exited 0. They move to stderr rather than
being dropped: saying where the SQL is going, before it goes, is a guarantee
this command makes, and the automated path is the one where nobody is
watching. The injected-provider path tests use returns before any of this
runs, which is why nothing caught it — the new tests point a configured
endpoint at a refused loopback port, which reaches the real path offline and
exercises fail-soft at the same time.

`parseReviewFile` checked verdicts and nothing else. A null entry threw a
TypeError from inside; an entry with no id could never match a group; an
entry with an unknown kind was counted toward `reviewed` and then skipped by
`measurePrecision`, so the file reported progress over a measurement that
covered nothing. That is the one outcome this file exists to rule out.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

`redactIssueMetadata` ran over `scoredIssues`, but `suppressed` was split off
`allScored` before that line and went into the report untouched. A finding the
triage model called a false positive therefore carried raw `notebookName`, `path`
and `projectName` — which is exactly where an address or a customer name ends up,
and a `pii-subject-scatter` finding that withholds the subject's fingerprint by
design could name the person in the field beside it.

The comment above that block already claimed the redaction was "one pass over every
finding rather than per check, so the next code added here cannot forget". It was
one pass over every *ranked* finding. Suppression is not deletion: the whole point
of keeping a false positive in the report is that someone can inspect it, which
means it is published, which means it is redacted like anything else.

The pass now runs over `allScored`, before the split. Scoring still happens first,
because `scoreIssue` looks an asset's age up by notebook name and a redacted name
matches nothing.

Also replaces the per-kind priors restated in the skill reference (join 0.55,
filter 0.5, metric 0.29) with a pointer to `KIND_PRECISION_PRIOR`, which has been
0.6 / 0.5 / 0.35 since the constants were genericised. Those three figures were
survivors of that pass — the last place the original internal numbers were still
written down. Not restating them is what stops them drifting again. The Wilson
figures in the same paragraph were checked against the implementation and are
correct; they are computed, not measured.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…bles

`makeResolver` builds one flat alias map for the whole statement, because there is
no grammar here to build query scopes from. When a subquery reuses an outer alias,
the inner declaration overwrote the outer one and every reference resolved to the
wrong table:

    SELECT * FROM orders o JOIN customers c ON o.customer_id = c.id
    WHERE EXISTS (SELECT 1 FROM users o WHERE o.id = c.id)

reported a single `customers ↔ users` join carrying keys merged from both query
blocks. The real `orders ↔ customers` join was gone, and a join between two tables
that never met was in its place.

That direction of error is the expensive one for this check. A missing claim is a
claim nobody measures; a phantom claim becomes an anchor, and other queries get
scored against it as dissenters from a consensus that does not exist.

Full lexical scoping would mean parsing, which this module deliberately does not
do. A shadowed alias now resolves to nothing instead — the same treatment a
subquery alias already gets. The query above yields no join rather than a wrong
one, and queries where an alias means one thing throughout are unaffected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…clared SQL blocks

Four fixes to the divergence surface.

**`candidateId` omitted the scope key.** Groups are scoped per integration,
so two warehouses can each hold the same table pair diverging the same way;
with kind, subject and variant forms identical they hashed to one id. Both
consequences were silent: `--import-review` keys verdicts by id, so one
reviewer's judgement governed both groups and the other was dropped from
the precision denominator, and `TriageCache` keys `${model}:${id}`, so a
verdict obtained about one warehouse was served from cache for another.
This invalidates existing triage caches, which is correct — the old entries
were keyed ambiguously.

**`filter` is no longer a default anchor.** It is the lowest-precision of
the three and produces most of the output, because every anchor carries one
finding per query that merely omits the filter. What it reliably catches
that is genuinely wrong, a comparison against NULL, is already caught per
query and without consensus by `sql-null-comparison`, so the default loses
no real coverage. `--divergence-kind filter` still runs it.

**Blocks that declare no integration are attributed where the project
leaves no choice.** When a project declares exactly one integration, its
SQL has nothing else to run against, so leaving those blocks in the unknown
bucket excluded them from their own warehouse's consensus and pooled them
with unrelated projects. Projects declaring none or several keep their
blocks unscoped. Every finding records which rule placed its query in
`details.integrationSource`, and the inventory still counts only what a
block actually names — an inferred attribution is not evidence that an
integration was used, and counting it would make an orphan look used.

**Triage resolves its configuration before asking whether there is anything
to send.** Resolution was lazy, so on a workspace that yielded no groups
`--triage` with nothing configured exited 0 and never mentioned triage: a
CI job misconfigured for weeks read as a clean pass. The existing failure
when an endpoint is missing and groups do exist is unchanged. `--triage`
now judges metric anchors by default — they are the bulk of the output and
the ones where the question is about intent — and every family the audit
looked for when `--divergence-kind` is explicit.

`KIND_PRECISION_PRIOR` is deliberately untouched: the only measurement
available is from a sample far too small to move a prior.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

`tables` holds one row per (short name, integration) since the identity was
shared with the divergence anchors. The reach index then merged those rows
by short name and took the maximum across integrations, so a disagreement
in a six-project staging warehouse was scored with the reach of a
thirty-project production table of the same name — blast radius 1.0 instead
of 0.86. That is exactly the cross-warehouse mixing integration scoping
exists to prevent, and it mis-ranks the two warehouses against each other.

An `integration`-scoped group now reads the row for its own warehouse.
`type` and `none` groups genuinely span integrations and keep the merged
index; that still over-counts for `type`, which merges every integration
rather than every integration of that type, but those are the deliberate
relaxations and the comment says so.

The comment above the index was also stale: it described a key folded from
qualified names, which stopped being true when `canonicalTableKey` took
over.

Also removes three leftover merge-conflict markers from
`docs/deepnote-cli-audit.md`. They survived every check because prettier
reformatted `>>>>>>>` into nested blockquotes, which is valid markdown —
worth knowing, since it means a conflicted `.md` file can pass format and
spell checks and still show raw conflict text to a reader.

`packages/cli/README.md` and the skill reference still documented
`--divergence-kind` as defaulting to all three anchors.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>