puter.ai.convo()

The conversation API prepares a realtime session, verifies Puter sign-in, and lets you connect when your app is ready. Realtime conversations require a paid Puter plan; usage is billed to the signed-in user. Native OpenAI events and convenience methods use the same session. Microphone capture and speaker playback start when your app requests them.

In Node.js, the SDK requires a global WebSocket, available in Node.js 22 or later.

Beta: This API is incomplete. Its methods, options, events, and behavior may change abruptly, including breaking changes without notice.

To connect without puter.js, see the Realtime WebSocket API.

Syntax

const conversation = await puter.ai.convo(options = {});
await conversation.connect();

Parameters

The options object accepts these fields.

Return value

A Promise resolves to a prepared conversation after options and sign-in are checked. In a browser, preparation opens the Puter sign-in prompt if needed. Preparation opens no realtime connection and requests no microphone access.

The conversation exposes these properties and methods.

Attach listeners before connecting to receive initial session events. Closing during connection rejects the pending attempt.

If the connection drops because of network loss, a server restart or a provider error, the conversation reconnects on its own. While it reconnects, state is connecting, a reconnecting event fires, and pending operations reject with connection_lost. The new session gets every setting the provider acknowledged, from the factory options, configure() and native session.update events. Then ready fires again with reconnected: true. Listeners, the microphone and the speaker carry over. Microphone audio captured while reconnecting is dropped.

The conversation also restores recent context. Before ready fires, it adds back up to 40 of the most recent conversation items, 48 KiB in total. Restored items are user text, user speech that has a transcript, assistant replies as text, and function calls with their outputs. User speech has a transcript only when input transcription is configured through native session settings. An assistant reply that was interrupted, or still playing when the connection dropped, is cut to what the user saw or heard. The cut is estimated from how much of the audio played, so it can be off by a few words. Images, speech without a transcript, deleted items, and replies still generating when the connection dropped are left out. Restored items get new item IDs, don't fire item events, and count as input tokens on the next response. Item IDs from before the reconnect don't work in later events. If the provider refuses a restored item, an error event fires with code context_restore_failed and the conversation stays connected.

When Puter knows ahead of time that the server a session runs on will restart, it sends a puter.reconnect event, which also reaches your listeners. Without that notice, a restart drops the connection and the conversation reconnects as above. Within a few seconds, the conversation opens a replacement session in the background while the current one keeps working. It switches once no reply is generating or playing, no user turn is being spoken or transcribed, no function call is waiting for its output, and nothing you sent is waiting on the provider. The switch goes through the same steps as a reconnect, and the reconnecting event has code 1012. If the replacement can't open yet, the conversation keeps the current session and tries again later. A refusal that retrying can't fix, such as insufficient_funds, stops the attempts. If the server closes the session first, the conversation switches to the replacement, or reconnects if none is open.

The conversation makes up to ten attempts over about a minute. In a browser that reports being offline, it first waits up to two minutes for the connection to come back. If every attempt fails, or the server refuses with an error such as insufficient_funds, the conversation closes. A closed conversation cannot reconnect. To start another session, prepare a new one.

Native event definitions are listed in the OpenAI realtime reference. Native events retain their provider fields. Sending native events preserves direct control of the session. Puter refuses conversation.item.retrieve events and MCP tools ("type": "mcp") in session.update and response.create events. Sending one fires an error event with code invalid_event, and the conversation closes with code 1008. Audio helpers expect the session to use PCM16 at 24 kHz.

Convenience events

The conversation emits these events alongside native events.

Each event includes its type. When the provider event leaves out an ID, responseId and itemId are empty strings and contentIndex is 0. The interrupted event signals local interruption immediately. Its corresponding operation resolves after the provider acknowledges cancellation and any requested truncation. Late audio from a canceled response remains visible as native events and is excluded from convenience audio and built-in playback.

Browser media

The optional microphone and speaker helpers require a browser with Web Audio support. Microphone capture also requires media permissions and a secure context. Start audio from a user action so the browser can allow playback.

Both start methods require a connected session. Capture and playback are independent, so custom audio input can use the built-in speaker, and built-in capture can use custom playback.

Errors and limits

Failures use objects containing code and message. Preparation can reject with invalid_argument, model_not_found, auth_required, auth_canceled, or auth_failed. Connection can reject with subscription_required or insufficient_funds, which also show the Puter upgrade prompt. Connection and asynchronous operations can reject with provider errors, session_closed, connection_lost, operation_timeout, or operation_in_progress. Sending before readiness, or while reconnecting, throws session_not_ready.

Invalid input throws invalid_argument, invalid_event, or input_too_large. A full SDK send queue throws queue_full. Media helpers report unsupported_environment, permission_denied, media_error, media_timeout, or queue_full. Runtime media errors and context_restore_failed arrive through the error event. Provider errors also arrive as native error events with a nested error object.

Each native event is capped at 64 KiB. The SDK bounds JSON traversal, outgoing queues, acknowledgment waits, microphone work, and playback. All numeric limits are listed on rate limits and quotas.

Examples

A text conversation uses convenience methods for input and streamed output.

const conversation = await puter.ai.convo({
    output: ['text'],
    instructions: 'Keep replies short.',
});

conversation.on('text', ({ delta }) => console.log(delta));
conversation.on('turn.done', () => conversation.close());
conversation.on('error', (event) => console.error(event));

await conversation.connect();
conversation.sendText('Say hello in three words.');

A voice conversation enables the microphone and speaker after connection.

const conversation = await puter.ai.convo();
conversation.on('transcript', ({ role, text }) => console.log(role, text));
conversation.on('error', (event) => console.error(event));

await conversation.connect();
await conversation.speaker.start();
await conversation.microphone.start();

// To interrupt a response and submit a replacement text turn:
await conversation.interrupt();
conversation.sendText('Actually, make it Wednesday.');

// When the user ends the conversation:
conversation.close();