Visitar URL original
feat(tasks): align scenarios with the Final-merged SEP-2663 spec by panyam · Pull Request #2 · panyam/mcpconformance · GitHub
Skip to content

feat(tasks): align scenarios with the Final-merged SEP-2663 spec - #2

Merged
panyam merged 1 commit into
feat/tasks-mrtr-extensionfrom
feat/tasks-mrtr-final-merged-alignment
May 18, 2026
Merged

panyam merged 1 commit into
feat/tasks-mrtr-extensionfrom
feat/tasks-mrtr-final-merged-alignment

Conversation

@panyam

@panyam panyam commented May 18, 2026 •

Copy link
Copy Markdown
Owner

What changes

SEP-2663 merged Final on 2026-05-15. Four normative wire changes baked in at merge time were not yet covered by the tasks scenarios in this branch (feat/tasks-mrtr-extension, the source of modelcontextprotocol/conformance PR 262). This PR adds FAIL-on-divergence assertions for each, so the suite validates the merged spec rather than tolerating divergence as INFO/MAY.

The four spec changes:

  1. notifications/tasks/status → notifications/tasks.
  2. notifications/progress and notifications/message MUST NOT be sent for tasks.
  3. requestState removed from the tasks-v2 wire. SEP-2322's InputRequiredResult on the MRTR surface still carries it and is unchanged.
  4. -32003 (Missing Required Client Capability) for tools whose task support is required when the client did not negotiate the extension.

Reviewer's guide

Read in this order:

  1. src/scenarios/server/tasks/notifications.ts — start here. The status-notifications scenario grew from 1 check to 5: new wire method name (FAIL on the legacy method); absence-asserts for notifications/progress and notifications/message on task streams; the existing shape check on the new method's payloads; and an absence-assert for requestState on notification payloads (reuses the SSE-observation buffer so the harness isn't duplicated in request-state.ts).
  2. src/scenarios/server/tasks/request-state.ts — full rewrite. The prior file tested the deleted "Request State Management" section; replaced with absence-asserts on CreateTaskResult and the tasks/get DetailedTask response. Scenario class renamed TasksRequestStateScenario → TasksRequestStateRemovalScenario.
  3. src/scenarios/server/tasks/required-task-error.ts — new file. Scenario TasksRequiredTaskErrorScenario initializes a client without declaring the tasks extension, calls a TaskSupport=required tool, and asserts the rejection code is -32003 plus the structured requiredCapabilities.extensions[...] shape. Documents failing_job as a fixture requirement.
  4. src/scenarios/server/tasks/all-scenarios.test.ts and src/scenarios/index.ts — registry updates. The renamed scenario class and new scenario class are added to both runners; src/scenarios/index.ts also has a stale "SEP is still in draft" comment swept to current framing since the area was already being touched.

Skim or skip: nothing — every file is load-bearing for the alignment.

Decision log

  • FAIL on divergence, not INFO on miss. The prior status-notifications scenario emitted INFO when no notifications were received, which made the rename invisible to the suite. The post-merge spec language is normative: when a server emits the legacy method name on the v2 surface, that is FAILURE. INFO is retained only when an absence-assert is genuinely not exercisable (e.g. no notifications/tasks observed, so the no-requestState-in-notif check has nothing to inspect).
  • Reused the notifications.ts SSE-observation harness for the requestState absence-assert on notification payloads. Duplicating the raw fetch + SSE parsing in request-state.ts would have made future SSE-harness changes a 2-file edit; the absence-assert is a few lines once the buffer exists.
  • failing_job as fixture name, matching panyam/mcpkit's examples/tasks-v2. Other implementations adopting the suite need to expose a tool registered with task support required; the name is documented in the scenario file's JSDoc fixture block.
  • Cited only SEP-2663 on the requestState-removal scenario. The field history is SEP-2322, but the removal is SEP-2663. Only SEP-2663 is cited so a reader doesn't have to triage cross-SEP attribution.

Risk

  • This PR is stacked on feat/tasks-mrtr-extension (the source of modelcontextprotocol/conformance PR 262). Merging here updates PR 262's diff to include these scenarios.
  • The new assertions intentionally FAIL against the current panyam/mcpkit main. Specifically: panyam/mcpkit still emits requestState on the tasks/get DetailedTask response. The next mcpkit catch-up PR (requestState removal from the v2 wire) closes that gap.
  • Other server implementations adopting the suite (everything-server, third parties) will also see FAILUREs until they catch up to the merged spec. That is the intended signal.

Verification

Ran locally against panyam/mcpkit's examples/tasks-v2 fixture (post-rename and post-progress-filter catch-up) via make testconf-tasks-v2: 8 of 9 scenarios pass; only tasks-request-state-removal fails, and only on the tasks-get-detailed-no-request-state check (mcpkit still emits the field on DetailedTask). The notification absence-asserts pass (mcpkit already filters progress/message on tasks); the rename check passes (mcpkit already emits the new method name); the -32003 checks pass (mcpkit already returns the structured error).

Out of scope

  • panyam/mcpkit requestState-removal PR — a follow-up on mcpkit, which will close the deliberate failing window opened by this PR.
  • Re-engagement on modelcontextprotocol/conformance PR 262 — once this stack merges into feat/tasks-mrtr-extension, a comment on PR 262 noting the Final-merged alignment is the natural next step.

Test plan

  • Pre-push lefthook (npm run lint:fix && git diff --exit-code + vitest run over 118 tests) green
  • make testconf-tasks-v2 (mcpkit side) shows 8 of 9 scenarios pass; the failing one is the requestState-removal scenario, by design
  • CI green

SEP-2663 merged Final on 2026-05-15. Four normative wire changes baked in at merge time. This commit updates the tasks server-conformance scenarios so the suite validates each one as a MUST rather than tolerating divergence as INFO/MAY.

1. notifications/tasks/status renamed to notifications/tasks. The status-notifications scenario now FAILs on observing the legacy method on the v2 surface, and validates the new name's payload shape when emitted.

2. notifications/progress and notifications/message MUST NOT be sent for tasks. Two new absence-asserts on the status-notifications scenario, reusing the existing SSE-observation harness.

3. requestState removed from the tasks-v2 wire. The previous request-state scenario tested the deleted "Request State Management" section; replaced with absence-asserts on CreateTaskResult, DetailedTask, and notifications/tasks payloads. SEP-2322's InputRequiredResult on the MRTR surface still carries requestState and is unchanged.

4. -32003 (Missing Required Client Capability) for required-task tools when the client did not negotiate the extension. New scenario TasksRequiredTaskErrorScenario initializes without declaring the extension, calls a TaskSupport=required tool, and asserts the rejection code plus the structured requiredCapabilities payload. Documents `failing_job` as a fixture requirement.

Verified locally against panyam/mcpkit examples/tasks-v2: 8/9 scenarios pass; the request-state-removal scenario intentionally FAILs against the current mcpkit main because the implementation still emits requestState on DetailedTask. That gap is the next item on the mcpkit catch-up plan.
@panyam
panyam force-pushed the feat/tasks-mrtr-final-merged-alignment branch from 934532f to 28c0ff1 Compare May 18, 2026 04:44
@panyam
panyam merged commit c37e496 into feat/tasks-mrtr-extension May 18, 2026
@panyam
panyam deleted the feat/tasks-mrtr-final-merged-alignment branch May 18, 2026 04:48
panyam pushed a commit that referenced this pull request Aug 4, 2026
…1933) (modelcontextprotocol#268)

* feat(auth): add JWT-bearer helper for WIF client conformance (SEP-1933)

Adds createWorkloadJwt and generateWorkloadKeypair to provide reusable,
tested JWT signing infrastructure for the upcoming wif-jwt-bearer scenario
(PR #2). Also extracts JWT_BEARER_GRANT_TYPE constant and migrates
cross-app-access.ts to use it.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(auth): add JWT-bearer helper for WIF client conformance (SEP-1933)

Adds createWorkloadJwt and generateWorkloadKeypair to provide reusable,
tested JWT signing infrastructure for the upcoming wif-jwt-bearer scenario
(PR #2). Also extracts JWT_BEARER_GRANT_TYPE constant and migrates
cross-app-access.ts to use it.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(auth): add WIF JWT-bearer scenario and negative tests (SEP-1933)

Adds the auth/wif-jwt-bearer client conformance scenario using the RFC 7523
JWT-bearer grant (urn:ietf:params:oauth:grant-type:jwt-bearer).

The scenario pre-signs valid, wrong-audience, and expired JWTs on start()
to simulate cloud workload identity tokens. The conformance AS verifies the
assertion and emits per-class checks (wif-assertion-verified,
wif-assertion-missing, wif-assertion-audience, wif-assertion-expired,
wif-assertion-malformed). Broken example clients exercise the missing-assertion
and wrong-audience failure paths.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* style: apply prettier formatting to wif-jwt-bearer.ts

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(auth): add expired-assertion negative test; clarify wif-jwt-bearer comments

- Add runWifJwtBearerExpiredAssertion + auth-test-wif-expired-assertion.ts
  to exercise the wif-assertion-expired check path (was dead code)
- Clarify WifAssertionVerified description to note iss is not validated
- Add comment explaining clockTolerance: 5 is intentional (same-run keypair)
- Add comment explaining numeric expiresIn is absolute epoch seconds

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wif-jwt-bearer): disable DCR, detect retries, improve getChecks description

- Add WIF_CLIENT_ID constant; pre-seed provider's clientInformation() so
  the SDK skips Dynamic Client Registration entirely (disableDynamicRegistration
  on the auth server + pre-seeded client_id on the provider side)
- Add failedOnce/tokenRequestReceived tracking on the scenario; reject and
  record wif-no-retry FAILURE if the client attempts a second token request
  after the first fails
- Add hasAttempted guard in WifJwtBearerProvider.prepareTokenRequest() to
  throw on retry from the client side
- Fix getChecks() sentinel description: distinguish "no request made" from
  "request made but verification failed"
- Add client_id to context and Zod schema

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* style: prettier formatting on everything-client.ts

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wif-jwt-bearer): let errors propagate from broken example clients

The try/catch blocks were silently swallowing auth failures, so the
negative tests passed purely because expectedFailureSlugs found the
AS-emitted check — not because client error-surfacing was verified.
allowClientError: true on the test cases handles the non-zero exit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wif-jwt-bearer): tag as draft, add scope-rejected check, add SEP traceability

- Change specVersions from ['extension'] to [DRAFT_PROTOCOL_VERSION] and add
  source = { introducedIn: DRAFT_PROTOCOL_VERSION } so the scenario is
  reachable via --spec-version draft (extension tag excluded it from all
  spec-version runs)
- Move registration from extensionScenariosList to draftScenariosList in
  index.ts to match the tag/list convention enforced by spec-version.test.ts
- Add wif-assertion-scope-rejected check: AS returns invalid_scope for a
  valid JWT when the client requests the reserved 'wif.rejected' scope;
  verify client surfaces the error and does not retry
- Add runWifJwtBearerScopeRejected to everything-client.ts; add optional
  scope param to WifJwtBearerProvider; add CLI entry point and vitest case
- Add src/seps/sep-1933.yaml with requirements mapped to each check ID,
  covering the three deferred checks (iss, sub, jti) with exclusion rationale

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* style: prettier formatting

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(wif-jwt-bearer): add wif-grant-fallback check for unauthorized_client fallback

Client bug class: WIF client receives unauthorized_client from JWT-bearer grant
and silently switches to authorization_code instead of surfacing the error.

The MCP SDK retries after UnauthorizedClientError (auth.js:152-154), calling
prepareTokenRequest() a second time. WifGrantFallbackProvider exploits this by
returning authorization_code params on the second call.

Spec anchor: RFC 7523 §2.1 — clients MUST use the JWT-bearer grant type;
silent grant-type switching hides authentication failures.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* style: rename unused scope param in WifGrantFallbackProvider

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(wif-jwt-bearer): use realistic SPIFFE and K8s PSAT token formats

valid_jwt uses a SPIFFE JWT-SVID subject (spiffe://conformance-test.local/...)
issued by a SPIRE-style issuer. wrong_audience_jwt uses a Kubernetes PSAT
subject and kubernetes.io claims, mirroring how K8s projected service-account
tokens look in production. Token verification and client behaviour are
unchanged; the formats ground the scenario in real workload identity platforms.

k8s_issuer and k8s_subject added to context and schema for external clients
that want to construct their own K8s-style assertions.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* style: prettier formatting on wif-jwt-bearer.ts

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* chore(wif-jwt-bearer): drop informational k8s_issuer/k8s_subject from context

These fields were not used by any client handler and are redundant since
the K8s claims are already baked into wrong_audience_jwt.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wif-jwt-bearer): address pre-merge review feedback

- Downgrade wif-no-retry, wif-grant-fallback, and wif-assertion-scope-rejected
  to WARNING; RFC 7523 is silent on client retry and grant-type switching, so
  FAILURE status was not traceable to spec text
- Add WifRetryProvider broken client that re-sends JWT-bearer after
  unauthorized_client, making wif-no-retry actually exercisable as a WARNING
- Drop getChecks() sentinel (non-standard pattern that overloaded the
  wif-assertion-verified check ID); test harness detects empty check sets
- Add comment on DRAFT_PROTOCOL_VERSION workaround for runner limitation
- Add comment explaining why both slash/no-slash audience forms are accepted
- Tighten schema comments for audience URL constraint and ES256 literal

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wif-jwt-bearer): drop decorative fields and neutralise JWT fixtures

- Remove issuer and subject from context schema and scenario return value;
  neither is validated by the AS or used by the client, making them
  misleading. The iss exclusion rationale is updated to explain why.
- Replace SPIFFE/K8s-PSAT constants and the kubernetes.io additionalClaims
  with neutral values across all three JWT fixtures; no check inspects
  token format, so format flavour was decorative and invited false
  assumptions about fixture semantics.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wif-jwt-bearer): remove workaround comment, drop signing_algorithm, annotate retry provider

- Remove self-admitted workaround comment on DRAFT_PROTOCOL_VERSION; the
  classification stands on its own and the comment was inviting challenge
- Drop signing_algorithm from context schema and scenario return; the field
  was redundant once the value was fixed to ES256, and the assertion is opaque
  to the client regardless
- Add comment to WifRetryProvider explaining it deliberately omits the
  hasAttempted guard so the SDK retry reaches the AS

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wif-jwt-bearer): export scope constants, remove unused AS field, annotate state machine

- Export WIF_TRIGGER_UNAUTHORIZED_SCOPE and WIF_REJECTED_SCOPE from
  createWorkloadJwt.ts and import them in everything-client.ts; removes
  duplicate bare-string declarations that could silently diverge
- Remove tokenEndpointAuthSigningAlgValuesSupported from mock AS config;
  the SDK does not consume this field in the JWT-bearer flow
- Add comment to onTokenRequest clarifying wif-no-retry and wif-grant-fallback
  fire on any second request after any failure, not only post-unauthorized_client

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wif-jwt-bearer): address pcarleton review comments

- Move all broken WIF provider classes and runners to wif-broken-clients.ts;
  everything-client.ts now contains only the conformant WifJwtBearerProvider
- Add sentinel in getChecks() for !tokenRequestReceived to prevent a silent
  pass when the client never sends a request
- Drop wif-assertion-scope-rejected check; the no-retry behaviour after
  invalid_scope is already covered by wif-no-retry, and sep-1933.yaml updated
  accordingly
- Remove audience from context schema and scenario return; the field was
  unused by any client implementation
- Expand scenario description to tell a client implementer what to implement
- Apply suggested description fix to wif-grant-fallback (does not assume
  unauthorized_client was the specific error that preceded the fallback)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(wif-jwt-bearer): drop number type from expiresIn, use jose duration strings

Jose supports negative duration strings ('-60s', '60 seconds ago') that produce
already-expired tokens without needing absolute epoch arithmetic. Drop the
number overload, update the expired_jwt call site to '-60s', and update the
unit test accordingly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
panyam pushed a commit that referenced this pull request Aug 4, 2026
…rotocol#394)

* Add DPoP client conformance scenario + shared DPoP helpers (SEP-1932, modelcontextprotocol#368)

Client-anchored packaging: carries the shared DPoP foundation (dpopProof /
dpopToken helpers, the createAuthServer DPoP core, and sep-1932.yaml) together
with the client scenario, so it can merge first; the server (modelcontextprotocol#369) and
authorization-server (modelcontextprotocol#370) scenarios are follow-ups that depend on it.

DPoP is treated as a draft/not-yet-official feature (mirroring the WIF SEP-1933
scenario): the scenario's source is `introducedIn: DRAFT_PROTOCOL_VERSION` and
it is registered in draftScenariosList (informational, not scored). It is NOT
registered in EXTENSION_IDS, since the ext-auth DPoP extension is still on a
branch rather than merged.

Client checks (the two extension MUSTs, RFC 9449 §7.1 / §4.2-4.3):
- sep-1932-client-dpop-auth-scheme — token presented with the DPoP scheme.
- sep-1932-client-fresh-proof — a fresh, well-formed DPoP proof per request.

The test MCP server judges the client via an opt-in DPoP middleware
(dpopResourceAuth) passed through createServer's authMiddleware hook; a compliant
example client (covered by the Client Draft Scenarios loop) plus deliberately-
broken bearer/replay variants drive the pass/fail acceptance tests.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* client/dpop: split into nonce-less baseline + nonce-required scenario

Server-provided nonces are optional in RFC 9449 (AS §8 "MAY", RS §9 "can
also choose"), and the nonce-less flow is the common case. The single
auth/dpop scenario hard-wired nonce enforcement on both the test AS and
MCP server, so a plain nonce-less proof was never accepted end-to-end and
a client that only does plain proofs was never exercised on its happy path.

Parameterize DPoPClientScenario with `requireNonce` and register it twice:
- auth/dpop        (requireNonce=false) — nonce-less baseline; emits the
  three baseline checks (token-request-proof, dpop-auth-scheme, fresh-proof).
- auth/dpop-nonce  (requireNonce=true)  — prior behavior; adds as-nonce and
  rs-nonce (five checks).

Check IDs are reused (no new IDs, no sep-1932.yaml change); the two nonce
checks are emitted only by auth/dpop-nonce, so traceability stays intact.

Add auth-test-dpop-no-nonce.ts (nonce-incapable but otherwise compliant)
asserting SUCCESS against auth/dpop — proof that a client with no nonce
handling still completes DPoP when the server does not require a nonce.
Baseline negatives stay on auth/dpop (nonce-independent); the two nonce
negatives are re-pointed to auth/dpop-nonce.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* client/dpop: round-4 nonce-surface correctness fixes

Address review findings on the auth/dpop-nonce posture:

- Record the token-request proof observation BEFORE the §8 nonce challenge,
  so a client that is challenged but does not retry is no longer mis-reported
  by token-request-proof as "never completed a token request" (its proof was
  just validated). (#2)
- Record the resource-request jti/replay on ANY valid proof (before the §9
  nonce gate), so fresh-proof is never asserted on zero evidence and a client
  that never honours the RS nonce fails ONLY rs-nonce. (#5)
- Require challengeIssued && honored for as-nonce/rs-nonce SUCCESS, closing a
  vacuous-pass path (a client that pre-sends the nonce is never challenged). (#3a)
- Gate the AS nonce observation on the authorization_code exchange, matching
  recordTokenRequestProof (a refresh exchange must not satisfy §8). (#3c)
- Collapse duplicate shared token-flow check IDs (token-request, pkce-*) that
  the §8 round-trip re-POST would otherwise emit twice. (#4)
- Example client: retry the token request only on a use_dpop_nonce error code,
  not any 400 carrying a DPoP-Nonce header (consistent with the RS path). (#6)
- Fix the DpopTokenRequestObservation docstring ("last write wins" →
  sticky-failure). (#7)

Adds an expectedSuccessSlugs option to the client test helper and tightens the
two nonce negative tests to pin the now-clean behavior (token-request-proof
SUCCESS for no-as-nonce; only rs-nonce fails for no-rs-nonce).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* client/dpop: round-5 fixes — dedupe masking, wording, replay decision

- dedupeSharedChecks was masking failures: it kept the last occurrence of a
  duplicate check ID regardless of status and ran in BOTH postures, so a
  FAILURE on a challenged POST (or an earlier attempt in a restarted flow) was
  silently dropped. Replace with an exported, unit-tested collapseDuplicateChecks
  that prefers the MOST-SEVERE occurrence, and only apply it in the nonce
  posture (the baseline keeps genuinely distinct repeated attempts). (R5 #1)
- Reword the token-request-proof description: it can be SUCCESS for a client
  that presented a valid proof but never obtained a token (challenged, no
  retry), so it no longer claims "obtaining a DPoP-bound access token". (R5 #2)
- Reword the as-nonce/rs-nonce failure messages to cover a client that DID
  retry but whose retry proof was rejected (not only "did not retry"). (R5 #3)
- Document the deliberate replay decision: recording jti on challenged requests
  means re-sending the identical proof across the challenge/retry boundary trips
  replay detection — a genuine RFC 9449 §4.2 jti-uniqueness violation. (R5 #5)

Adds src/scenarios/client/auth/dpop.test.ts unit-testing collapseDuplicateChecks
(prefers failures, keeps INFO, preserves order) — the dedupe fix is now pinned.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* client/dpop: round-6 polish — pin collapse wiring; docstring

- Add an integration assertion that the §8/§9 double-POST is collapsed to one
  token-request/pkce-* entry each in auth/dpop-nonce, so the requireNonce gating
  of collapseDuplicateChecks can't be silently deleted or inverted (previously
  mutation-survivable). runClientAgainstScenario now returns the emitted checks
  so callers can make occurrence-level assertions (backward-compatible).
- Docstring: note the severity ladder ranks SKIPPED below SUCCESS, and that
  equal-severity ties keep the last occurrence.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant