Repository navigation
Description
Problem Statement
When nativeStreaming: true is set, the Telegram adapter streams to private chats with sendRichMessageDraft (or sendMessageDraft as a fallback) and saves the result with sendRichMessage. Users have no way to stop a long answer while it streams.
Both draft methods accept can_stop, which shows the user a stop button. When the user presses it, Telegram sends the bot a stopped_message_generation update (MessageGenerationStopped, with chat, optional message_thread_id, and draft_id). keep_on_stop keeps the partial draft visible after the press, for a short time or until the bot sends a message.
The adapter doesn't send can_stop or keep_on_stop, and it doesn't handle stopped_message_generation, so the stream keeps running until the model finishes.
Proposed Solution
Add opt-in stop support to native draft streaming in @chat-adapter/telegram.
- Config: an option such as
streamStopButton: true, off by default. It applies only whennativeStreamingis on, because post-and-edit streams (the default) don't use drafts. - Sending:
nativeDraftStreampassescan_stop: trueandkeep_on_stop: truetosendRichMessageDraft, and tosendMessageDraftin the plain fallback. - Receiving: add
stopped_message_generationto the update types the adapter handles, and toTELEGRAM_DEFAULT_ALLOWED_UPDATES, which the adapter sends as an explicit list in business mode. - Stopping: match the update to the running stream by chat id,
message_thread_id, anddraft_id, then stop readingtextStream. - Saving: send the text received so far with
sendRichMessage, as when a stream ends normally, so the partial answer stays in the chat.keep_on_stopkeeps the draft visible until then, so the message doesn't flicker.
Open questions
- How does the bot learn about the stop? Stopping the adapter's read loop doesn't cancel the model request.
StreamOptions.signalalready lets the bot cancel a stream, but the adapter can't abort a signal the bot owns. Options:- The adapter aborts an
AbortSignalit exposes for the stream, which the bot passes tostreamText. - A new handler, such as
onStreamStop, receives the thread and the partial text.
- The adapter aborts an
- How does a stop reach the right process? On serverless deployments, the webhook with
stopped_message_generationcan arrive at a different instance from the one running the stream. One approach is to write a stop flag to the state adapter and check it on each draft flush. - Draft ids come from a per-process counter (
createDraftId), so two instances can use the samedraft_idin one chat. Should draft ids become random, or is matching on the chat plus the id enough?
Roadmap
- Stop button and update handling. Add the option, send
can_stopandkeep_on_stop, add theMessageGenerationStoppedtype totypes.ts, routestopped_message_generation, stop the read loop, and save the partial text. Cover a stop arriving in the same process. Update the Telegram docs page and add a changeset. - Bot notification and cross-instance stops. Implement the answers to open questions 1 and 2.
Alternatives Considered
- Handling
stopped_message_generationin bot code. The adapter doesn't route the update, so a bot would need to subclassTelegramAdapterand overrideprocessUpdate, and it couldn't reach the stream's draft id. - Stopping only the read loop, without telling the bot. The stream ends in the chat, but the model keeps generating tokens.
Use Case
import { Chat } from "chat";
import { createTelegramAdapter } from "@chat-adapter/telegram";
import { createMemoryState } from "@chat-adapter/state-memory";
import { streamText } from "ai";
import { model } from "./model"; // any AI SDK model
const bot = new Chat({
userName: "mybot",
adapters: {
telegram: createTelegramAdapter({
nativeStreaming: true,
// Proposed option
streamStopButton: true,
}),
},
state: createMemoryState(),
});
bot.onNewMention(async (thread, message) => {
const result = streamText({ model, prompt: message.text });
// Private chat: the draft shows a stop button. Pressing it ends the
// stream and keeps the text received so far.
await thread.post(result.fullStream);
});Priority
Important
Contribution
- I am willing to help implement this feature
Additional Context
sendRichMessageDraftandsendMessageDraft:can_stopandkeep_on_stopMessageGenerationStopped- fix(telegram): make native draft streaming opt-in #822 made native draft streaming opt-in (
nativeStreaming) - Telegram: send cards as HTML widgets (Bot API 10.4) #1138 proposes HTML widget cards for the same adapter
Activity