Anthropic server-side fallback: summed iteration usage makes the next step see about twice the real prompt size #60792

Activity

1. opencodejr commented on Oct 31, 2026

   

   opencodejrBot

   on Oct 31, 2026

   More actions

   Thanks for the detailed report and the fixture. I reproduced this on v2.2.4, and it still happens on the latest dev (e8238ba).

   The script below feeds the fallback usage block from Anthropic's docs through LLMClient.generate, then through core's SessionUsage.tokens. On both versions:

   • inputTokens and nonCachedInputTokens come out as 947, and contextTokens is undefined.
   • The stored message tokens.input is 947, so input + cache.read + cache.write (the figure estimatePrompt treats as the measured prompt) is 947.
   • The response's own top-level usage reports 412.

   The cause matches your analysis: mapUsage sums every iteration and only sets contextTokens when the last iteration is message, not fallback_message. I only checked the usage mapping. I didn't run a full session to see the next step's auto-compaction or output limit react to the inflated number.

   Related: #46285 (the PR that added iteration summing for Anthropic usage).

   Reproduction script (anthropic-fallback-usage.test.ts)

   // Repro for anomalyco/opencode#60792: Anthropic server-side fallback usage.
   // Run from packages/ai: bun test test/provider/anthropic-fallback-usage.test.ts
   import { expect } from "bun:test"
   import { Effect } from "effect"
   import { ai } from "../../src/effect.js"
   import { LLMClient } from "../../src/route/client.js"
   import { Anthropic } from "../../src/providers/index.js"
   import { SessionUsage } from "../../../core/src/session/usage.js"
   import { testEffect } from "../lib/effect.js"
   import { fixedResponse } from "../lib/http.js"
   import { sseEvents } from "../lib/sse.js"
   
   // Usage block from Anthropic's "Refusals and fallback" docs example.
   const usage = {
     input_tokens: 412,
     output_tokens: 264,
     cache_read_input_tokens: 0,
     cache_creation_input_tokens: 0,
     iterations: [
       { type: "message", model: "claude-fable-5", input_tokens: 535, output_tokens: 0, cache_read_input_tokens: 0, cache_creation_input_tokens: 0 },
       { type: "fallback_message", model: "claude-opus-4-8", input_tokens: 412, output_tokens: 264, cache_read_input_tokens: 0, cache_creation_input_tokens: 0 },
     ],
   }
   
   testEffect(
     fixedResponse(
       sseEvents(
         { type: "message_start", message: { usage } },
         { type: "message_delta", delta: { stop_reason: "end_turn" }, usage },
         { type: "message_stop" },
       ),
     ),
   ).effect("fallback turn: prompt size should be the answering attempt (412), not the sum", () =>
     Effect.gen(function* () {
       const result = yield* LLMClient.generate(
         ai.request({ model: Anthropic.configure({ apiKey: "test" }).model("claude-opus-4-8"), prompt: "hello" }),
       )
       const u = result.usage!
       const tokens = SessionUsage.tokens(u)
       const measuredPrompt = tokens.input + tokens.cache.read + tokens.cache.write // what estimatePrompt reads
       console.log({
         inputTokens: u.inputTokens,
         nonCachedInputTokens: u.nonCachedInputTokens,
         contextTokens: u.contextTokens,
         storedTokens: tokens,
         measuredPrompt,
       })
       // The response's top-level usage reports 412 input tokens for the message returned.
       expect(u.contextTokens).toBe(412)
       expect(measuredPrompt).toBe(412)
     }),
   )

   Run it: bun anthropic-fallback-usage.test.ts

   👍React with 👍1Reacted by Raxxoor
2. opencodejr

   added

   reproduced

   on Oct 31, 2026
3. dhaern commented on Oct 31, 2026

   

   dhaern

   on Oct 31, 2026

   Author

   More actions

   Another case from today on v2.2.4, same session, which shows how far the inflated number travels.

   One Opus 5.5 step at 04:56:35 UTC went through a server-side fallback (the stored message carries the fallback marker). The stored usage for that step is:

   {"input":4,"output":1590,"reasoning":0,"cache":{"read":416217,"write":1439}}

   The step before it reported 217,121 input tokens. Magic Context read the next value as inputTokens=417660 against a 302,222 limit (138%) and went into emergency mode. Its emergency drop then refused to run because the "fixed floor ≈391071 already exceeds ceiling 250000", and that floor is derived from the same summed figure. The turn was cut twice in a row (04:58:01 and 05:00:21) with EmergencyFailClosedError before a third attempt went through at 58,845 tokens. The Magic Context side is tracked in cortexkit/magic-context#828.

   For anyone hitting this with @cortexkit/opencode-anthropic-auth: the plugin adds fallbacks: "default" and the server-side-fallback-2026-07-01 beta to Opus requests by default. Setting OPENCODE_ANTHROPIC_AUTH_CONTENT_FILTER_FALLBACK=legacy in the server's environment stops it from sending them, so there are no extra iterations for mapUsage to add up. A refusal then comes back as a normal content-filter finish instead of being answered by claude-opus-4-8 in the same response. That avoids the trigger, but the summing in mapUsage is still wrong for every client that uses server-side fallback.