Actions

Actions are service-specific messages that are dispatched to your Pipecat bot to trigger specific pipeline behaviors. The client implements the RTVI standard for these communications.

Some examples of actions include:

• tts:say - Speak a message using text-to-speech.
• tts:interrupt - Interrupt the current text-to-speech message.
• llm:append_to_messages - Append a context to the current LLM messages.

Under the hood, actions are data objects sent via the transport layer to the bot. The bot listens for and processes these actions to perform the necessary operations.

Actions do not directly trigger events or callbacks in the client, but instead return a Promise that resolves once the bot has processed the action. They provide a way to extend bot functionality without modifying the client.

Actions differ from messages in that:

• They are service specific
• They do not trigger callbacks or events
• They return a Promise that resolves once the bot has processed the action.

​

Obtaining available actions

To obtain a list of available actions, you can use the describeActions method on the RTVI client.

const actions = await rtviClient.describeActions();

This will return an array of action objects that you can use to determine which actions are available.

{
  "label": "rtvi-ai",
  "type": "actions-available",
  "id": "UNIQUE_ID_FROM_REQUEST",
  "data": {
    "actions": [
      {
	      "service": "tts",
	      "action": "say",
	      "arguments": [
		      { "name": "text", "type": "string" },
		      { "name": "interrupt", "type": "bool" },
		      ...
	      ]
      },
      ...
    ]
  }
}

​

Anatomy of an action

An action object has the following properties:

• service - The service that the action belongs to.
• action - The name of the action, as defined by the bot.
• arguments - An array of argument objects that the action accepts.

When the client dispatches an action, it will pass the action name and arguments to the bot, alongside a unique ID that is referenced on response to resolve the Promise.

RTVI clients maintain an internal queue of actions that are dispatched to the bot. The action is sent as JSON data via the transport layer to the bot, which processes the action.

Once the action has been processed the bot will send a response message with the same unique ID that the client uses to reference the action queue to resolve the Promise.

​

Dispatching an action

const someAction = await rtviClient.action({
  service: "tts",
  action: "say",
  arguments: [
    { name: "text", value: "Hello, world!" },
    { name: "interrupt", value: false },
  ],
});

// > Promise<RTVIActionResponse>

An action is resolved or rejected with the following:

{
  "label": "rtvi-ai",
  "type": "action-response",
  "id": "UNIQUE_ID_FROM_REQUEST",
  "data": {
    "result": BOOL | NUMBER | STRING | ARRAY | OBJECT
  }
}

If an action is unable to be processed, the bot will return a error-response typed message with the same unique ID assigned by the client, triggering a rejection.

You can handle error responses in multiple ways:

• onMessageError callback
• MessageError event
• try / catching the Promise

try {
	const someAction = await rtviClient.action({...});
} catch(e) {
	console.error(e);
}

​

Action response data

Some actions resolve with data. This data is specific to the action and is defined by the bot.

A successful action will resolve with RTVIActionResponse:

{
  "label": "rtvi-ai",
  "type": "action-response",
  "id": "UNIQUE_ID_FROM_REQUEST",
  "data": {
	"result": "Hello, world!"
  }
}

Awaiting an action will return the RTVIActionResponse object, which contains a result property with any associated data.

try {
	const someAction = await rtviClient.action({...});
	console.log(someAction.data.result);
} catch(e) {
	console.error(e);
}