HQ Chat WebSocket

Every chat client talks to an HQ through one WebSocket path, /ws/chat. The socket carries JSON text frames in both directions. The client sends commands, for example to start a turn, add input to a turn, stream microphone audio, or control a browser. The HQ sends back the turn lifecycle, agent events, voice state, and browser frames.

The HQ does not listen on the network. Remote clients connect through the hosted relay. The HQ dials out to the relay and connects each relayed socket to its loopback channel server. For this reason, the frames and rules on this page are the same for a relayed socket and a loopback socket.

The chat token (DASHSQUAD_HQ_CHAT_TOKEN) is also the bearer of the HQ Mobile API. The HQ accepts it on /ws/chat and /mobile/v1. The HQ never accepts the management token (DASHSQUAD_HQ_TOKEN) on /ws/chat.

AuthenticationCopy Anchor Link

The HQ checks the credentials when it receives the upgrade request. These rules apply on every /ws/chat endpoint:

1. If the request has a non-empty Authorization header, the header must be Bearer <chat-token>.
2. If the request has an Authorization header, the HQ ignores the query parameters.
3. If there is no Authorization header, the HQ accepts ?token=<chat-token> or a valid ?ticket=<ticket>.
4. If the credentials are not correct, the HQ completes the upgrade and closes the socket with code 4001.

The reason of the 4001 close is Unauthorized. If the HQ starts without a chat token, it does not check credentials on /ws/chat.

Native clients send the header, so the token is not in the URL:

GET /ws/chat HTTP/1.1
Upgrade: websocket
Authorization: Bearer <your-chat-token>

Desktop on loopback can use the query parameter:

ws://127.0.0.1:9200/ws/chat?token=<your-chat-token>

WebSocket ticketsCopy Anchor Link

A browser cannot set headers on a WebSocket upgrade, and the web app must not put the chat token in a URL. For this reason, the web app gets a single-use ticket over HTTP with the chat token:

POST /mobile/v1/ws-ticket
Authorization: Bearer <your-chat-token>

{"ticket":"0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef","expiresAt":"2026-08-29T12:00:30.000Z"}

Then the web app connects with wss://<host>/ws/chat?ticket=<ticket>. A ticket has these properties:

• It has 64 hexadecimal characters.
• It expires 30 seconds after the HQ makes it.
• The first try to redeem it uses it, also when that try is too late.
• It works on any /ws/chat endpoint of the HQ that made it. One ticket store serves the full HQ.
• If the upgrade also has an Authorization header, the HQ uses the header and does not redeem the ticket.

See HQ Mobile API for the other /mobile/v1 routes.

HandshakeCopy Anchor Link

Before it opens the socket, a client reads GET /mobile/v1/health and requires contractVersion: 1. See HQ Mobile API.

The first frame on every socket must be hello. It names contract version 1. It can also list the client capabilities:

{"type":"hello","contractVersion":1,"capabilities":["provider-recovery-v1","chat-archive-v1"]}

The HQ answers with hello_ack. It has the same two fields:

{"type":"hello_ack","contractVersion":1}

Wait for hello_ack before you send another frame. The HQ closes the socket with code 1002 in these conditions:

• The first frame is not hello (hello_required).
• contractVersion is not 1 (unsupported_version). The HQ checks the version before the other fields.
• hello has an unknown field (invalid_frame). The known fields are type, contractVersion, and capabilities.
• capabilities has an unknown value, or the same value two times (invalid_frame).
• A second hello arrives (unexpected_hello).

The HQ sends every server frame to every socket that subscribes to the conversation. This includes browser_session and subagent_progress. A client does not ask for these frames.

Voice mode needs the speech-v1 capability in GET /mobile/v1/health. No capability advertises the browser live view. A client sends browser_view_start and handles browser_view_error with code: "unavailable".

Compatibility rules:

• Unknown server frame types. Clients reject a server frame whose type they do not know. iOS closes the socket and shows Update required. Desktop stops the active turn and tells you to update dashsquad. Thus a new server frame type breaks older clients. Update the clients before the HQ that sends it.
• Unknown event variants. Clients must preserve or safely ignore agent events they do not recognize, so newer HQs remain compatible.
• Unknown enum values. Clients decode voice_state.state, voice_stopped.reason, and the browser_view_* enums leniently, so a new value does not break the socket.

Connection lifecycleCopy Anchor Link

One socket carries at most one conversation subscription, plus one voice session and one browser viewer. A new subscribe_conversation replaces the earlier subscription of that socket. To follow a second conversation, open a second socket.

Subscribe before you sendCopy Anchor Link

1. Read GET /mobile/v1/conversations/:id/bootstrap for the conversation, its messages, its queue, and throughSeq.
2. Open the socket, send hello, and wait for hello_ack.
3. Send subscribe_conversation with the throughSeq from the bootstrap, or with the last seq you applied.
4. The HQ replays every sequenced frame after that cursor, then sends conversation_subscribed. After that, it sends each new frame live.

A run reaches a socket only through its subscription. Every turn on the conversation arrives this way. This includes turns from this device and from other devices, and turns from the queue. It also includes notification turns that the HQ starts (origin: "notification") and sub-agent child turns (origin: "parent"). The subscription lasts across runs. A terminal frame does not end it.

Runs and sequencesCopy Anchor Link

1. The client sends message with a client-generated run id and resumable: true.
2. The HQ persists the user and assistant messages and sends accepted. accepted always precedes the run's first event.
3. The HQ streams event frames: text deltas, tool calls, questions, and a final response.
4. The run ends with exactly one terminal frame: done (completed, cancelled, or interrupted) or error.

A delivered Steer starts a new segment inside the same run. Every frame of a run carries runId and the segmentTurnId of its segment.

Every sequenced frame carries a durable seq. The HQ writes the frame to the conversation's log before it sends it. Sequences are per conversation and increase across runs. Control frames (hello_ack, conversation_subscribed, command_rejected, subagent_progress, browser_session, and the voice and live-view frames) do not carry the conversation sequence and are never replayed.

voice_speech, voice_audio, voice_played, browser_view_ack, and browser_view_frame have a seq of their own. It counts the chunks of one stream. It is not the conversation sequence.

ReconnectCopy Anchor Link

Closing a socket only detaches it. Provider work continues on the HQ. To catch up after a gap, open a new socket, send hello, and send subscribe_conversation with the last seq you applied. Store a cursor only after you apply its frame. You can also read the gap over HTTP with GET /mobile/v1/agents/:agentId/conversations/:conversationId/events?sinceSeq=<last-seq>.

Do not send a message again with a new run id. You can send the same message again with the same id. The HQ does not run it two times.

CancellationCopy Anchor Link

Send cancel with the run id. A cancelled run ends with done and outcome: "cancelled". If Follow Up messages wait, the HQ pauses the queue and sends queue_paused. The next Follow Up starts only after resume_follow_ups.

To stop only the tool call that runs now, send cancel_step. The run continues.

Steer and Follow UpCopy Anchor Link

While a run is active, a client queues more input with enqueue_input:

• behavior: "steer" delivers the message into the active run at the next safe agent boundary.
• behavior: "followUp" queues the message as a later run. Follow Up messages run in order, one at a time.
• behavior: "auto" lets the HQ choose.

edit_follow_up and remove_follow_up change a queued Follow Up message that has not started. A failed run pauses the queue. A cancel also pauses it if Follow Ups wait. resume_follow_ups resumes it. Every queue change is a sequenced input_* or queue_* frame, sent to every subscriber.

For the limits, the auto-delivery rules, and how to show these messages in a client, see Steer and Follow Up.

Heartbeats and timeoutsCopy Anchor Link

The HQ sends no application-level heartbeat on /ws/chat and enforces no idle timeout on it. A client that needs liveness detection must provide its own, such as WebSocket pings. Two timers apply inside features: the voice drain timer (8 seconds) and the browser-view safety acknowledgement (5 seconds).

Client framesCopy Anchor Link

Every client frame is a JSON object with a string type. Every frame except hello also has a string id.

Common field rules:

• id is client-generated. Commands and subscriptions use a UUID. For message, answer, cancel, and cancel_step it is the run ID. For voice and browser frames it is the session or viewer ID.
• conversationId is a UUID, or sub_ plus a 26-character ULID for a sub-agent conversation.
• images is an array of { "mediaType", "data" } objects, where data is standard base64. The HQ accepts at most four images of type image/jpeg, image/png, image/gif, or image/webp. Each decoded image is at most 5 MiB, and all decoded images together are at most 12 MiB.

helloCopy Anchor Link

Opens the protocol. Must be the first frame. See Handshake.

Send a capability only when GET /mobile/v1/health lists it. An HQ that does not know a value closes the socket with 1002 invalid_frame. Without a capability, the HQ sends this socket the older shapes. See Client capabilities.

subscribe_conversationCopy Anchor Link

Replays a conversation from a cursor, then delivers its frames live.

{"type":"subscribe_conversation","id":"00000000-0000-4000-8000-000000000002","agentId":"agent-01","conversationId":"018f0f4a-5c42-7a8b-9c01-1234567890ab","sinceSeq":0}

messageCopy Anchor Link

Starts a run when the conversation is idle.

location fields:

JavaScript's getTimezoneOffset() is west-positive and must be negated.

{"type":"message","id":"018f0f4a-5c42-7a8b-9c01-2234567890ab","agentId":"agent-01","channelId":"mobile-ios","conversationId":"018f0f4a-5c42-7a8b-9c01-1234567890ab","text":"Is the mobile connection ready?","location":{"timezone":"Asia/Singapore","utcOffsetMinutes":480,"locale":"en-SG","region":"SG"},"resumable":true}

If the conversation already has an active run, the HQ answers with command_rejected, code conversation_busy. Use enqueue_input instead.

A retry with retryOf is safe only when the failed turn did not run tools. The HQ checks providerFailure.toolEffects of that turn (see error):

• If retryOf does not name a failed turn of this conversation, the HQ answers command_rejected with code: "validation_failed".
• If toolEffects is not none, the HQ answers command_rejected with code: "validation_failed" and details.reason: "unsafe_retry". Some actions of the turn can be complete already. Tell the user to check the chat and send again.

If the text is a /goal control, such as /goal pause, the HQ starts no run. It answers command_rejected with code: "validation_failed" and details.goalCommand: true. error holds the answer to show to the user.

enqueue_inputCopy Anchor Link

Queues a Steer or Follow Up message.

{"type":"enqueue_input","id":"218f0f4a-5c42-7a8b-9c01-1234567890ab","inputId":"818f0f4a-5c42-7a8b-9c01-1234567890ab","agentId":"agent-01","channelId":"mobile-ios","conversationId":"018f0f4a-5c42-7a8b-9c01-1234567890ab","text":"Also compare the migration cost.","behavior":"followUp"}

When you retry an uncertain command, reuse both id and inputId with exactly the same payload. The HQ rejects a reused ID with a different payload.

edit_follow_upCopy Anchor Link

Replaces the text and images of a queued Follow Up message that has not started.

remove_follow_upCopy Anchor Link

Removes a queued Follow Up message that has not started.

resume_follow_upsCopy Anchor Link

Resumes a paused queue.

answerCopy Anchor Link

Answers a question event from the active run.

{"type":"answer","id":"018f0f4a-5c42-7a8b-9c01-2234567890ab","questionId":"question-01","answer":"Yes"}

cancelCopy Anchor Link

Stops the active run.

{"type":"cancel","id":"018f0f4a-5c42-7a8b-9c01-2234567890ab"}

cancel_stepCopy Anchor Link

Stops the tool call (or calls) that the active run does now, but not the run. Needs cancel-step-v1 in GET /mobile/v1/health. The model gets "Stopped by the user." as the error of the call. Then the run continues. A Steer message that waits goes in next.

{"type":"cancel_step","id":"018f0f4a-5c42-7a8b-9c01-2234567890ab"}

If no tool call runs, the HQ sends command_rejected with code: "validation_failed" and No step is running. to this socket only. If the run is not active, the code is not_found.

Voice and browser client framesCopy Anchor Link

The voice_* client frames are documented under Voice mode, and the browser_view_* client frames under Browser live view.

Server framesCopy Anchor Link

Server frames are JSON objects with a string type. There are two groups:

• Sequenced frames carry seq, are stored in the conversation's log, and are replayed: accepted, event, done, error, input_accepted, input_updated, input_removed, input_delivered, input_failed, queue_paused, and queue_resumed.
• Control frames do not carry the conversation sequence and are never replayed: hello_ack, conversation_subscribed, command_rejected, subagent_progress, browser_session, and the voice and live-view frames.

hello_ackCopy Anchor Link

Answers hello. It has exactly two fields: type and contractVersion (1).

conversation_subscribedCopy Anchor Link

Sent after the replay of a subscribe_conversation.

If the agent has a browser open in the conversation, the HQ then sends a browser_session frame with state: "open" to this socket.

acceptedCopy Anchor Link

The HQ admitted a run, or a new segment of a run, and persisted its user and assistant messages.

Replayed accepted frames never carry origin, kind, requestId, or retryOf. Read the durable origin and kind from the message and the conversation.

{"type":"accepted","id":"018f0f4a-5c42-7a8b-9c01-2234567890ab","conversationId":"018f0f4a-5c42-7a8b-9c01-1234567890ab","runId":"018f0f4a-5c42-7a8b-9c01-2234567890ab","segmentTurnId":"018f0f4a-5c42-7a8b-9c01-2234567890ab","seq":1,"userMessageId":"018f0f4a-5c42-7a8b-9c01-3234567890ab","assistantMessageId":"018f0f4a-5c42-7a8b-9c01-4234567890ab","revision":2}

eventCopy Anchor Link

One agent event from the run. Has id, conversationId, runId, segmentTurnId, seq, and event. event is an object with a string type.

{"type":"event","id":"018f0f4a-5c42-7a8b-9c01-2234567890ab","conversationId":"018f0f4a-5c42-7a8b-9c01-1234567890ab","runId":"018f0f4a-5c42-7a8b-9c01-2234567890ab","segmentTurnId":"018f0f4a-5c42-7a8b-9c01-2234567890ab","seq":2,"event":{"type":"text_delta","text":"Ready "}}

Agent event types:

A model error does not arrive as an error event. The run ends with an error frame. Sub-agent heartbeats arrive as subagent_progress frames. The HQ can add new event types at any time, so ignore the types that you do not know.

doneCopy Anchor Link

The run ended. Has id, conversationId, runId, segmentTurnId, seq, and outcome: completed, cancelled, or interrupted (an HQ restart stopped the run).

{"type":"done","id":"018f0f4a-5c42-7a8b-9c01-2234567890ab","conversationId":"018f0f4a-5c42-7a8b-9c01-1234567890ab","runId":"018f0f4a-5c42-7a8b-9c01-2234567890ab","segmentTurnId":"018f0f4a-5c42-7a8b-9c01-2234567890ab","seq":5,"outcome":"completed"}

errorCopy Anchor Link

The run failed. Has id, conversationId, runId, segmentTurnId, seq, and error (a human-readable message), and optionally code and retryable. It ends the run.

With provider-recovery-v1, the frame can also have providerFailure. It tells why the provider refused the turn, so a client can offer the correct repair. The stored error event of the assistant message has the same object. It never holds a secret, a header, or a response body.

client_outdated means that the provider refuses this version of dashsquad. Only a newer version of dashsquad can fix it. The other value lists in providerFailure are closed.

Input and queue framesCopy Anchor Link

Every input frame has id (the command ID), conversationId, seq, queueRevision, and input, the full input in the same shape as pendingInputs on the bootstrap. Use queueRevision to discard stale queue state.

command_rejectedCopy Anchor Link

The HQ refused a command. Sent only to the socket that sent it. A rejection does not close the socket or end the subscription.

{"type":"command_rejected","id":"018f0f4a-5c42-7a8b-9c01-5234567890ab","conversationId":"018f0f4a-5c42-7a8b-9c01-1234567890ab","code":"conversation_busy","error":"Conversation has an active turn","retryable":true,"details":{"activeTurnId":"018f0f4a-5c42-7a8b-9c01-2234567890ab"}}

subagent_progressCopy Anchor Link

A heartbeat from a sub-agent that runs under the active run. Sent to every socket that is subscribed to the conversation.

Voice and browser server framesCopy Anchor Link

The voice_* server frames are documented under Voice mode, and the browser_view_* and browser_session frames under Browser live view.

Question cardsCopy Anchor Link

An agent asks the user multiple-choice questions with the ask_user_question tool. The questions are in the input of the tool_use_start event for that tool. A client shows them as a card. The tool ends the turn. The answers of the user start the next turn.

{"type":"tool_use_start","id":"toolu_01abc","name":"ask_user_question","input":{"questions":[{"question":"Which DB?","header":"Database","multiSelect":false,"options":[{"label":"Postgres","description":"Use the managed Postgres service.","recommended":true},{"label":"SQLite","description":"Keep a local file database."}]}]}}

The tool input has these rules:

• 1 to 4 questions. Each question has question, header (a maximum of 12 characters), multiSelect, and options.
• 2 to 4 options for each question. Each option has label, description, and optional recommended.
• Question text has a maximum of 2,000 Unicode code points. Each option label has a maximum of 500.
• A maximum of one recommended option for each question. The HQ puts it first.
• No option with the label "Other". The user can always type free text.

If the input is correct, the tool_result has details with kind: "question_card", version: 1, and the questions. If the input is not correct, the tool_result has isError: true and the turn continues.

To answer, send a new message. Put a readable summary in text and the answers in questionAnswers:

{"type":"message","id":"00000000-0000-4000-8000-000000000301","agentId":"agent-01","channelId":"mobile","conversationId":"00000000-0000-4000-8000-000000000001","text":"Which DB?\n→ Postgres\n\nWhich clients?\n→ iOS, Web; Android later","questionAnswers":{"toolCallId":"toolu_01abc","answers":[{"question":"Which DB?","selected":["Postgres"]},{"question":"Which clients?","selected":["iOS","Web"],"other":"Android later"}]},"resumable":true}

The HQ counts lengths in Unicode code points. questionAnswers must have only the fields in this table.

• To skip the questions, send a message with the text Skipped your questions. and no questionAnswers.
• A card has the answered state if the next user message has the same toolCallId. Any other next user message replaces the card.
• Message reads always include questionAnswers when a reply has them.

Voice modeCopy Anchor Link

Voice mode runs a hands-free spoken conversation on the same socket. The client streams microphone audio. The HQ finds each utterance, transcribes it, runs it as a turn, and streams the spoken reply back. Voice mode requires the speech-v1 capability.

Every voice_* frame in both directions has the session id. The client selects this id in voice_start. A socket holds a maximum of one voice session.

Session lifecycleCopy Anchor Link

1. Send voice_start. The HQ checks the speech configuration, the conversation, and the speech provider.
2. Read voice_state with state: "listening".
3. Stream voice_audio. The HQ drops audio that arrives before listening.
4. When the user stops, read voice_state transcribing and a final voice_transcript with a turnId.
5. Read voice_state thinking and the accepted, event, and done frames of the turn.
6. Read voice_state speaking and the voice_speech parts of the reply.
7. Send voice_played as playback finishes.
8. Read voice_state listening again, or voice_stopped when the session ends.

The voice turn is a turn with that turnId, channelId: "ios", and modality: "voice". The voice_transcript with the turnId always arrives before the accepted of the turn. For this reason, the client can show the user row first. The turn frames arrive through the conversation subscription, so subscribe to the conversation before voice_start. The accepted usually arrives just before thinking.

After the HQ speaks for 300 ms or more, speech from the user interrupts it (barge-in). The HQ cancels the active turn, discards the queued speech, and goes back to listening. That utterance becomes the next turn.

An utterance during a turn that does not interrupt it goes into a queue. The HQ sends its voice_transcript without a turnId when it hears the utterance. It sends the transcript again with a turnId when the turn of that utterance starts.

If the agent asks a question, the HQ speaks it and sends the next utterance as the answer. That transcript has no turnId.

voice_startCopy Anchor Link

Client frame. Starts a session. A second voice_start on the same socket first stops the active session with voice_stopped and reason: "replaced".

{"type":"voice_start","id":"018f0f4a-5c42-7a8b-9c01-6234567890ab","agentId":"agent-01","conversationId":"018f0f4a-5c42-7a8b-9c01-1234567890ab"}

The HQ reports failures with voice_error. The code is unavailable when the HQ has no speech configuration or no available provider. The code is invalid when the conversation does not exist or belongs to a different agent.

voice_audioCopy Anchor Link

Client frame. One part of the microphone capture.

{"type":"voice_audio","id":"018f0f4a-5c42-7a8b-9c01-6234567890ab","seq":42,"pcm":"AAAAAAAAAAA="}

If pcm is too large, the HQ answers with an ordinary error frame and code validation_failed, not with voice_error. The HQ drops audio while the session is muted.

voice_muteCopy Anchor Link

Client frame. Mutes or unmutes the microphone. Playback continues.

{"type":"voice_mute","id":"018f0f4a-5c42-7a8b-9c01-6234567890ab","muted":true}

When you mute, the HQ sends voice_state muted. It sends no other voice_state while the session is muted. When you unmute, the HQ sends the state that the session got to while it was muted.

voice_playedCopy Anchor Link

Client frame. Each voice_speech up to and with this seq has finished playback.

{"type":"voice_played","id":"018f0f4a-5c42-7a8b-9c01-6234567890ab","seq":4}

After the last voice_speech of a turn, the HQ stays in speaking. It waits for a voice_played with a seq that is the same or higher, or for an 8-second safety timer. If a client stops playback when the session leaves speaking, it must send voice_played. If it does not, the user does not hear the last sentence of each reply.

The HQ ignores an old or out-of-order seq. Do not acknowledge audio that a barge-in discarded.

voice_stopCopy Anchor Link

Client frame. Stops the session. If no session is active, this is not an error.

{"type":"voice_stop","id":"018f0f4a-5c42-7a8b-9c01-6234567890ab"}

The HQ cancels the active voice turn and sends voice_stopped with reason: "client". If the session stops before its first listening, the HQ sends no voice_stopped.

If there is no session on the socket, the HQ answers voice_audio, voice_mute, and voice_played with voice_error and code invalid.

voice_stateCopy Anchor Link

Server frame. The phase of the session.

{"type":"voice_state","id":"018f0f4a-5c42-7a8b-9c01-6234567890ab","state":"thinking","turnId":"018f0f4a-5c42-7a8b-9c01-7234567890ab"}

stopped is a valid value, but the HQ never sends it. Each session ends with a voice_stopped frame.

voice_transcriptCopy Anchor Link

Server frame. The text that the HQ recognized from the user's speech.

{"type":"voice_transcript","id":"018f0f4a-5c42-7a8b-9c01-6234567890ab","text":"What's the weather like today?","final":true,"turnId":"018f0f4a-5c42-7a8b-9c01-7234567890ab"}

If an utterance gives no text, the HQ sends no transcript and starts no turn.

voice_speechCopy Anchor Link

Server frame. A part of the spoken reply.

{"type":"voice_speech","id":"018f0f4a-5c42-7a8b-9c01-6234567890ab","seq":3,"audio":"AAAAAAAAAAAAAAAAAAAAAA==","format":"pcm16","sampleRate":24000,"text":"It's sunny and 24 degrees."}

voice_state speaking always comes before the first voice_speech of a turn.

voice_errorCopy Anchor Link

Server frame. A speech failure or a session failure.

{"type":"voice_error","id":"018f0f4a-5c42-7a8b-9c01-6234567890ab","code":"provider","error":"The speech provider failed to transcribe the clip"}

Most errors do not stop the session:

• If a transcription fails or takes more than 20 seconds (network), the HQ drops that utterance and goes back to listening.
• If synthesis fails with unavailable or network, the HQ skips that sentence.
• If a turn fails, the HQ reports provider after its partial speech plays.

These failures stop the session with voice_stopped and reason: "provider":

• Three synthesis failures in sequence, or one synthesis failure with a different code.
• A turn that the HQ rejects because the conversation is busy (unavailable), missing, or not authorized (invalid).

voice_stoppedCopy Anchor Link

Server frame. The session ended. No more voice_* frames come for this id.

{"type":"voice_stopped","id":"018f0f4a-5c42-7a8b-9c01-6234567890ab","reason":"client"}

Browser live viewCopy Anchor Link

When an agent opens a browser in a conversation, a client can watch it live and take control of the input. The view is a JPEG screencast of the active tab of the session. The frames and the viewer input are for the person who watches. The agent never sees them.

Every browser_view_* frame has the viewer id that the client selected in browser_view_start. There is no capability flag. If the HQ has no live view, it answers every browser_view_* frame with browser_view_error and code: "unavailable".

Viewer rulesCopy Anchor Link

• One viewer for each socket. A second browser_view_start on the same socket stops the first viewer with reason: "replaced".
• One viewer for each browser session, on the full HQ. A viewer on a different socket replaces this viewer in the same way.
• Flow control. A maximum of one browser_view_frame waits for an acknowledgement at a time. While it waits, the HQ keeps only the newest frame and drops the frames between.
• Acknowledgements. Acknowledge each frame with browser_view_ack. If the client does not acknowledge a frame in 5 seconds, the HQ acknowledges it and sends the next frame.
• Control. control is "agent" while a turn is active in the conversation, and "user" at other times. The HQ examines it every second and before each input.
• Control changes. The HQ sends browser_view_state when control changes. The HQ drops input that arrives while control is "agent".
• Stop. The view stops with reason: "session_closed" when the browser closes. It stops with "socket_closed" when the socket closes.

browser_view_startCopy Anchor Link

Client frame. Starts a view of the browser of the conversation.

{"type":"browser_view_start","id":"018f0f4a-5c42-7a8b-9c01-8234567890ab","agentId":"agent-01","conversationId":"018f0f4a-5c42-7a8b-9c01-1234567890ab","maxWidth":960}

The HQ answers with browser_view_started or with browser_view_error:

• invalid when the conversation does not belong to the agent.
• no_session when no browser is open in the conversation.
• unavailable when the view does not start.

browser_view_ackCopy Anchor Link

Client frame. Acknowledges a browser_view_frame.

{"type":"browser_view_ack","id":"018f0f4a-5c42-7a8b-9c01-8234567890ab","seq":42}

The HQ ignores an acknowledgement for a frame other than the frame that waits.

browser_view_inputCopy Anchor Link

Client frame. Sends pointer, keyboard, text, or scroll input to the page. It has an effect only while control is "user".

The HQ scales frame coordinates and scroll distances to the page. modifiers uses the Chrome DevTools Protocol bitmask: Alt 1, Ctrl 2, Meta 4, Shift 8.

{"type":"browser_view_input","id":"018f0f4a-5c42-7a8b-9c01-8234567890ab","input":{"kind":"pointer","x":120.5,"y":340,"phase":"down","button":"left"}}

{"type":"browser_view_input","id":"018f0f4a-5c42-7a8b-9c01-8234567890ab","input":{"kind":"text","text":"hello dash"}}

browser_view_stopCopy Anchor Link

Client frame. Stops the view with this viewer id. If no view with this id is active, the HQ ignores the frame. This is not an error.

{"type":"browser_view_stop","id":"018f0f4a-5c42-7a8b-9c01-8234567890ab"}

If the id names the active view, the HQ answers with browser_view_stopped and reason: "client".

If the viewer id is not active, the HQ answers browser_view_ack and browser_view_input with browser_view_error and code invalid.

browser_view_startedCopy Anchor Link

Server frame. The view is active.

{"type":"browser_view_started","id":"018f0f4a-5c42-7a8b-9c01-8234567890ab","width":960,"height":600,"url":"https://example.com/","title":"Example Domain","control":"agent"}

No other frame for this viewer comes before it.

browser_view_frameCopy Anchor Link

Server frame. One screencast frame.

{"type":"browser_view_frame","id":"018f0f4a-5c42-7a8b-9c01-8234567890ab","seq":7,"data":"AAAAAAAAAAA=","width":960,"height":600}

browser_view_stateCopy Anchor Link

Server frame. The page or the control changed.

{"type":"browser_view_state","id":"018f0f4a-5c42-7a8b-9c01-8234567890ab","url":"https://example.com/login","title":"Sign in","control":"user","dialog":"Leave site? Changes you made may not be saved."}

browser_view_stoppedCopy Anchor Link

Server frame. The view ended.

{"type":"browser_view_stopped","id":"018f0f4a-5c42-7a8b-9c01-8234567890ab","reason":"client"}

browser_view_errorCopy Anchor Link

Server frame. A live-view request failed.

{"type":"browser_view_error","id":"018f0f4a-5c42-7a8b-9c01-8234567890ab","code":"no_session","error":"No browser session is open on this conversation."}

browser_sessionCopy Anchor Link

Server frame. The agent opened or closed a browser in a conversation. The HQ sends it to every socket that subscribes to the conversation. It often arrives during a turn. It has no sequence, and the HQ does not replay it.

When a socket subscribes to a conversation with an open browser, the HQ sends one browser_session with state: "open" to that socket, after conversation_subscribed. Thus a client that connects again can offer the live view immediately.

{"type":"browser_session","id":"018f0f4a-5c42-7a8b-9c01-9234567890ab","agentId":"agent-01","conversationId":"018f0f4a-5c42-7a8b-9c01-1234567890ab","state":"open","profile":"e2e"}

The HQ does not send a separate browser_session for the browser of a sub-agent.

Errors and close codesCopy Anchor Link

Rejected commandsCopy Anchor Link

A well-formed frame that the HQ cannot act on gets a command_rejected frame. The socket stays open and keeps its subscription.

Codes on command_rejected:

The contract's MobileApiErrorCode list defines further codes shared with the HTTP API, so treat any unknown code as a generic failure.

Close codesCopy Anchor Link

Through the relay, a device can also see the relay's own 4401 and 4429 closes. See Relay API.

When a socket closes, the HQ stops that socket's voice session (reason: "socket"), stops its browser viewer (reason: "socket_closed"), and drops its subscription. Runs continue.