Repository navigation
[Feature] Return token ids under /chat/completion endpoint (Prompt + Response) #18378
Description
Checklist
- If this is not a feature request but a general question, please start a discussion at https://github.com/sgl-project/sglang/discussions. Otherwise, it will be closed.
- Please use English. Otherwise, it will be closed.
Motivation
Context: For RL, we need token consistency between training and inference. Previously people used /generate to get the token ids + logprobs. However, in nowadays agentic workflows, many users prefer using /chat/completion endpoint.
Gap:
- Under /chat/completion, sglang currently only returns response token text + logprobs, but does not return the exact token ids. (Retokenize the text may not reproduce the exact token id).
- Sglang does not return any prompt token meta info.
Request:
- Please add support for returning token ids for both prompt and response in
/chat/completion. - Currently there is no need for returning prompt's token logprobs.
Thanks!
Related resources
No response
Activity
guapisolo commentedon Feb 6, 2026
btw, I have a draft implementation here for this feature included in
sglang.patch. It includes some additional changes that are not related to this request.https://github.com/radixark/miles/pull/502/changes#diff-c6217d91cb59fab575767e4171efdc3c43071fefcc800321289fba7a659108f0R77
[-][Feature] Token-level Metadata for /chat/completion in sglang (Prompt + Response)[/-][+][Feature] Return token ids under /chat/completion endpoint (Prompt + Response)[/+]CatherineSue commentedon Feb 9, 2026
I think we could support this with a default option to turn it off. As it is not OpenAI standard.
guapisolo commentedon Feb 10, 2026
Good idea.
CSWYF3634076 commentedon Mar 9, 2026
May I ask if this feature has been implemented? I look forward to supporting it. If you don't have time, I would be more than happy to develop it @CatherineSue @guapisolo
Superskyyy commentedon Mar 18, 2026
Really this feature from Slime side, please merge it
cafeii commentedon Apr 5, 2026
is this feature added in the newest sglang?
MLKoz2 commentedon Apr 7, 2026
@cafeii rather not, and the PR is freezing for last two months
billxbf commentedon Apr 8, 2026
Same, can this feautre be merged?
feat: add return_token_ids to /v1/chat/completions and /v1/completions
ianliuy commentedon Apr 12, 2026
I've opened PR #22610 (#22610) which implements this feature.
What it does:
Adds
eturn_token_ids: bool = False to both /v1/chat/completions and /v1/completions. When enabled, the response includes prompt and completion token IDs in the sglext field.
Usage:
\\python
response = client.chat.completions.create(
model='...',
messages=[{'role': 'user', 'content': 'Hello!'}],
extra_body={'return_token_ids': True},
)
sglext = response.model_extra.get('sglext', {})
print(sglext['prompt_token_ids']) # [1, 2, 3, ...]
print(sglext['completion_token_ids']) # [[4, 5, 6, ...]] (one list per choice)
\\
Key design decisions:
Files changed: \protocol.py, \serving_chat.py, \serving_completions.py, and 7 new unit tests in \ est_protocol.py.
github-actions commentedon Jun 12, 2026
This issue has been automatically closed due to inactivity. Please feel free to reopen it if needed.
capture: accept SGLang as an engine, not just vLLM
Harbor integration: serve Harbor task datasets through OpenEnv as tra…