fix(models): replay OpenRouter reasoning_details in LiteLlm histories - #8111

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

• Fixes: [LiteLLM] Non-Anthropic reasoning_details (encrypted/signed) are dropped on replay #8109

Note: I saw @its-amann's offer to pick this up — this branch was already complete and locally verified when that comment landed, so submitting it now to avoid duplicated effort. Happy to coordinate if reviewing is easier another way.

Problem

Non-Anthropic reasoning models routed through OpenRouter return their reasoning as an opaque reasoning_details array — encrypted entries and signed texts included — which the provider requires clients to send back unchanged between tool calls. LiteLlm flattened this to a plain reasoning_content string when replaying an assistant turn, so the model lost its reasoning state on every tool round trip, and encrypted-only reasoning was dropped entirely. Anthropic thinking_blocks and Gemini thought_signature already round-trip; this gives the OpenRouter-style format the same treatment.

Solution

• Inbound: _message_to_generate_content_response reads message.provider_specific_fields["reasoning_details"] and stores the raw array as a JSON envelope ({"adk_reasoning_details": [...]}) on a thought part's thought_signature — the same slot that already preserves Anthropic thinking signatures through session persistence. Encrypted-only reasoning (nothing displayable) gets a textless carrier thought part.
• Outbound: _content_to_message_param replays the stored array unchanged and in order as a top-level reasoning_details field on the assistant message, but only while the outgoing request still targets the same OpenRouter model. The envelope records the model that produced the details, and the replay is scoped to OpenRouter routes (a openrouter/ model prefix, litellm_proxy/openrouter/... transports, or custom_llm_provider="openrouter") with the model compared after stripping the routing prefixes, so a session that moves to another provider or model sends the plain reasoning_content replay instead of a field the new route would reject. Envelopes recorded before the model was stored (or with no model reported) stay replayable to any OpenRouter model. Text-only reasoning (no captured details) keeps the existing reasoning_content replay, so nothing changes for providers that never return reasoning_details.
• Anthropic routes are untouched: openrouter/anthropic/claude-* model strings do not match _is_anthropic_model, so OpenRouter-routed Claude models take the new replay path (covered by a test) while native anthropic/... keeps thinking_blocks. When a session does switch from an OpenRouter-routed Claude model to the direct anthropic/... route, the stored envelope is kept out of the Anthropic thinking signatures instead of leaking into them.

Known limitation: streamed responses are not covered. LiteLLM currently drops reasoning_details-only deltas before they reach this adapter (BerriAI/litellm#48401), so there is nothing to capture on the streaming path yet; streaming sessions keep today's behavior. The inbound helper already accepts the delta-level shape, so capture starts working once LiteLLM delivers the deltas.

Testing Plan

Unit Tests:

• I have added or updated unit tests for my change.
• All unit tests pass locally.

20 new tests in tests/unittests/models/test_litellm.py: helper extraction (including empty/invalid lists), inbound capture (text+details, encrypted-only carrier, no-details unchanged, producing model stored), outbound replay (identical array next to tool calls, the openrouter/anthropic routing edge, text-only fallback unchanged, an opaque Gemini-style signature not misdecoded as details), replay scoping (cross-provider skip, cross-model skip, same-model replay, response-form model without the routing prefix, custom_llm_provider route, litellm_proxy transport, envelopes without a stored model), no envelope leak into Anthropic thinking signatures after a provider switch, and two-turn tests through generate_content_async with a mocked client asserting the second request carries the identical reasoning_details array next to the original tool call — and does not carry it after the request switches to a different model.

pytest summaries (at head 101d0ea):

• tests/unittests/models/test_litellm.py: 834 passed.
• tests/unittests/models: 2063 passed.

tox (py311/py312/py313/py314, full suite; run at e4bd38a, the head before the replay-scoping commit — that commit only touches lite_llm.py and its test file, both covered by the models/ run above): 24520/24511/24511/24513 passed. 13 tests fail in every environment — all in network-path tools tests (SSRF/DNS pinning in test_rest_api_tool.py, test_load_web_page.py NAT64/pinning, one MCP google-auth transport test). The same 13 fail identically on pristine main in the same virtualenv with the same command (verified by A/B), so they are pre-existing local-environment failures, not regressions; zero failures in models/ or anywhere this change touches.

Manual End-to-End (E2E) Tests:

• Red→green: with the source change reverted, the outbound replay test fails with KeyError: 'reasoning_details' (the reported drop); restored, it passes. Against the pre-scoping head, the five scoping tests fail exactly as designed (model never stored, replay sent to another provider, replay sent to another model, envelope leaked into an Anthropic signature, replay after a model switch); at the current head all 834 pass.
• LiteLLM forwarding check (litellm 1.96.2, the version uv.lock resolves): built the assistant message this change emits (top-level reasoning_details + tool_calls) and passed it through OpenrouterConfig().transform_request(...); the outgoing provider request JSON carries the array unchanged, element for element and in order, including the reasoning.encrypted data and signatures.
• To verify against a live endpoint (needs an OpenRouter key, not available locally): run an LlmAgent with model=LiteLlm(model="openrouter/<reasoning model>") and one function tool, capture the second request via litellm._turn_on_debug() or an HTTP proxy, and check the replayed assistant message now carries the full reasoning_details array.

Checklist

• I have read the CONTRIBUTING.md document.
• I have performed a self-review of my own code.
• I have commented my code, particularly in hard-to-understand areas.
• I have added tests that prove my fix is effective or that my feature works.
• New and existing unit tests pass locally with my changes.
• I have manually tested my changes end-to-end.
• Any dependent changes have been merged and published in downstream modules.

Additional context

Sibling report #8110 (OpenAIResponsesLlm never replaying reasoning items) describes the same class of loss in the OpenAI Responses adapter; this PR only changes the LiteLLM adapter.