Repository navigation
fix(combo): time a rotated attempt from its own send, not the combo target's start - #8749
Draft
andrew05060414 wants to merge 2 commits into
Draft
andrew05060414 wants to merge 2 commits into
andrew05060414 wants to merge 2 commits into
Conversation
…arget's start A key or account rotation inside one combo target seals the earlier segment as its own attempt and moves activeAttemptStartedAt forward. The combo still finished the live segment from the target's start, so the last segment re-counted every sealed one and the request's attempt durations summed to more than the request took. Also adds a regression showing a failover combo relays the surviving target's deltas at the upstream's pace rather than in one burst.
Contributor
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. |
Contributor
|
✅ Deterministic PR hygiene checks passed. |
Contributor
⏳ DRAFT
What to do
Review readiness checklist
0/4 boxes ticked. This PR stays in draft until every box above is ticked. Hygiene✅ Deterministic PR hygiene checks passed. |
…fixed milliseconds The rotation case asserted each attempt stayed under an absolute bound, which a heavily loaded machine exceeded with the fix in place. It now checks the second key's attempt is shorter than the child's own measured run, which the old target-start timing can never satisfy.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The problem. When a combo target rotates to another key or OAuth account before it gives up (for example a multi-account Antigravity target answering 429 on each account), the request log recorded the last account's attempt as if it had started when the combo target started. That attempt's
durationMs(andfirstOutputMs, when it succeeded) silently included every earlier account's time, so a request's attempts summed to more time than the request itself took.What changes. The combo now times the live attempt from the moment the rotation sealed the previous one (
childLog.activeAttemptStartedAt, whichnoteProviderAttemptSendalready moves forward), falling back to the target's start when nothing rotated. The native combo lane gets the same correction in itsfinish.Impact for users. Request logs, the usage ledger, and any dashboard that splits a request into "earlier attempts / waiting for upstream / streaming" stop double-counting failed accounts. Example from a real log row before the fix: three Antigravity accounts recorded
268 + 370 + 1017 ms(the third really took about 380 ms) before the request failed over to Anthropic, inside a request that lasted 1962 ms in total. Routing and delivery are unchanged.What this is not. I investigated a report that combo failover buffers the whole upstream stream before relaying it. It does not:
preflightComboStreamResponseholds bytes only until the first output event or a terminal, then relays live. The new test pins that down, so a future change that turns the relay into a burst fails the suite.Technical details:
src/server/responses/core-combo.ts: asegmentStartedAt()helper replaces the fixedstartedin everyfinishRequestAttempt/firstOutputMscomputation and in theactiveAttemptStartedAthanded to the parent log at commit. The[combo] ... failed after Nmsconsole line still reports the whole target's time, on purpose.src/server/responses/core-combo-native.ts: the attempt handle'sfinishuses the same segment start.tests/responses/combo-stream-attempt-timing.test.ts(new, registered inlayout-combos.jsonand the expected-layout fixture):[429 target, paced streaming target]combo delivers five deltas spread over their real 120 ms gaps;noteProviderAttemptSendseam records two attempts, each about one key's latency, and all attempts sum to no more than the elapsed time. Without the fix this case fails (Expected: < 260, Received: 372).Verification
bun run typecheck: passes.bun run structure:check: passes.bun test tests/responses/combo-stream-attempt-timing.test.ts: 2 pass; the rotation case fails onupstream/devwithout the source change.tests/test-layout.test.ts(2 pass),tests/ci-workflows/file-size-ratchet.test.ts(9 pass),tests/responses/responses-combo-fallback-off.test.ts(12 pass),tests/responses/chat-native-combo.test.ts(27 pass),tests/responses/responses-combo-bridge-usage.test.ts(5 pass),tests/responses/combo-codex-ws-overload-failover.test.ts(2 pass),tests/responses/responses-account-label.test.ts(10 pass with--timeout 60000; four cases hit the default 5 s timeout under CPU contention).tests/server/server-combo-failover-e2e.test.tsdid not finish within 500 s on this machine both with and without this change, so it is left to CI.Checklist
Review readiness checklist
This PR stays in draft until every box below is ticked. Tick all four boxes once the requirements are met:
Required local validation passed; commands, results, and any full-suite exception are documented.
I pushed my PR to a recent dev commit (at most 10 behind; a maintainer may still ask for the exact tip before merge).
I resolved all correct Codex and CodeRabbit findings.
My PR is ready for review.