WebRTC transport implementation using Daily for real-time audio/video communication
DailyTransport
provides real-time audio and video communication capabilities using Daily’s WebRTC platform. It supports bidirectional audio/video streams, transcription, voice activity detection (VAD), and participant management.
To use DailyTransport
, install the required dependencies:
You’ll also need to set up your Daily API key as an environment variable:
DAILY_API_KEY
.
You can obtain a Daily API key by signing up at Daily.
Daily room URL
Daily room token
Name of the bot in the room
Transport configuration parameters
Daily API endpoint URL
Daily API key for authentication
Enable audio output capabilities
Enable live audio streaming mode
Audio output sample rate in Hz
Number of audio output channels
Audio output bitrate in bits per second
Enable audio input capabilities
When enabled, incoming audio frames are pushed downstream
Audio input sample rate in Hz
Number of audio input channels
Audio filter for input processing. Supported filters are: KrispFilter()
and
NoisereduceFilter()
. See the KrispFilter()
reference
docs and NoisereduceFilter()
reference docs` for more
information.
Enable video output capabilities
Enable live video streaming mode
Video output width in pixels
Video output height in pixels
Video output bitrate in bits per second
Video output frame rate
Video color format (RGB, BGR, etc.)
Voice Activity Detection analyzer. You can set this to either
SileroVADAnalyzer()
or WebRTCVADAnalyzer()
. SileroVADAnalyzer is the
recommended option. Learn more about the
SileroVADAnalyzer.
Configuration for dial-in functionality
Enable real-time transcription
Configuration for transcription features
DailyTransport provides a comprehensive callback system for handling various events. Register callbacks using the @transport.event_handler()
decorator.
Called when the bot successfully joins the room.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing room join informationCalled when the bot leaves the room.
Parameters:
transport
: The DailyTransport instanceCalled when the first participant joins the room. Useful for initializing conversations.
Parameters:
transport
: The DailyTransport instanceparticipant
: Dictionary containing participant informationCalled when any participant joins the room.
Parameters:
transport
: The DailyTransport instanceparticipant
: Dictionary containing participant informationCalled when a participant leaves the room.
Parameters:
transport
: The DailyTransport instanceparticipant
: Dictionary containing participant informationreason
: String describing why the participant left, “leftCall” | “hidden”
Event emitted when a participant is updated. This can mean either the participant’s metadata was updated, or the tracks belonging to the participant changed.
Parameters:
transport
: The DailyTransport instanceparticipant
: Dictionary containing updated participant informationEvent emitted when a custom app message is received from another participant or via the REST API.
Parameters:
transport
: The DailyTransport instancemessage
: The message content (any type)sender
: String identifier of the message senderEvent emitted when the call state changes, normally as a consequence of joining or leaving the call.
Parameters:
transport
: The DailyTransport instancestate
: String representing the new call state. Learn more about call states.Event emitted when dial-in is ready. This happens after the room has connected to the SIP endpoint and the system is ready to receive dial-in calls.
Parameters:
transport
: The DailyTransport instancesip_endpoint
: String containing the SIP endpoint informationEvent emitted when the session with the dial-in remote end is established (i.e. SIP endpoint or PSTN are connectd to the Daily room).
Note: connected does not mean media (audio or video) has started flowing between the room and PSTN, it means the room received the connection request and both endpoints are negotiating the media flow.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing connection information. See DialinConnectedEvent.Event emitted when the dial-in remote end disconnects the call.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing connection information. See DialinStoppedEvent.Event emitted in the case of dial-in errors which are fatal and the service cannot proceed. For example, an error in SDP negotiation is fatal to the media/SIP pipeline and will result in dialin-error being triggered.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing error information. See DialinEvent.Event emitted there is a dial-in non-fatal error, such as the selected codec not being used and a fallback codec being utilized.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing warning information. See DialinEvent.Event emitted when the session with the dial-out remote end is answered.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing call information. Learn more.Event emitted when the session with the dial-out remote end is established.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing connection information. Learn more.Event emitted when the dial-out session is stopped.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing connection information. Learn more.Event emitted in the case of dial-out errors which are fatal and the service cannot proceed. For example, an error in SDP negotiation is fatal to the media/SIP pipeline and will result in dialout-error being triggered.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing error information.Learn more.Event emitted there is a dial-out non-fatal error, such as the selected codec not being used and a fallback codec being utilized.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing warning information. Learn more.Called when a transcription message is received. This includes both interim and final transcriptions.
Parameters:
transport
: The DailyTransport instancemessage
: Dictionary containing transcription data including. Learn more.Called when a room recording starts successfully.
Parameters:
transport
: The DailyTransport instancestatus
: Dictionary containing recording status information. Learn more.Called when a room recording stops.
Parameters:
transport
: The DailyTransport instancestream_id
: String identifier of the stopped recording streamCalled when an error occurs during recording.
Parameters:
transport
: The DailyTransport instancestream_id
: String identifier of the recording streammessage
: Error message describing what went wrongCalled when a transport error occurs.
Parameters:
transport
: The DailyTransport instanceerror
: String containing error detailsRaw audio data from participants
Video frames from participants
Audio data to be sent
Video frames to be sent
Sends a message to participants in the Daily room. Messages can be directed to all participants or targeted to a specific participant.
Parameters:
frame
: A transport message frame containing the message to send. Can be either:
TransportMessageFrame
: Standard messageTransportMessageUrgentFrame
: High-priority messageMessage Structure:
Messages should conform to the expected format for the receiving system. For RTVI messages, use:
Example - Standard Message:
Example - Urgent Message to Specific Participant:
The message will be delivered to:
on_app_message
handlerUse TransportMessageUrgentFrame
for high-priority messages that need
immediate delivery.
Sends an image frame to the room.
Parameters:
frame
: Image frame to send, either raw image data or sprite animationSends an audio frame to the room.
Parameters:
frame
: Audio frame to sendStarts capturing video from a specific participant.
Parameters:
participant_id
: ID of the participant to captureframerate
: Target frame rate (default: 30)video_source
: Video source type (default: “camera”)color_format
: Color format of the video (default: “RGB”)To request an image from the camera stream, set the framerate
to 0 and push
a UserImageRequestFrame
.
Starts capturing and transcribing audio from a specific participant.
Parameters:
participant_id
: ID of the participant to transcribeUpdates the transcription configuration for specific participants or instances.
Parameters:
participants
: Optional list of participant IDs to transcribe. If None, transcribes all participants.instance_id
: Optional specific transcription instance ID to updateExample:
Starts recording the room session.
Parameters:
streaming_settings
: Recording configuration settingsstream_id
: Optional stream identifierforce_new
: Force start a new recordingSee start_recording for more details.
Stops an active recording.
Parameters:
stream_id
: Optional stream identifier to stop specific recordingSee stop_recording for more details.
Initiates a dial-out call.
Parameters:
settings
: Dial-out configuration settingsSee start_dialout for more details.
Stops an active dial-out call.
Parameters:
participant_id
: ID of the participant to disconnectSee stop_dialout for more details.
Sends DTMF tones in an existing dial-out session.
Parameters:
settings
: DTMF settingsSee send_dtmf for more details.
Updates subscriptions and subscription profiles. This function allows you to update subscription profiles and at the same time assign specific subscription profiles to a participant and even change specific settings for some participants.
Parameters:
participant_settings
: Per-participant subscription settingsprofile_settings
: Global subscription profile settingsSee update_subscriptions for more details.
When using the DailyTransport, if your bot is given a meeting token with owner or participant admin permissions, it can remote-control other participants’ capabilities and permissions. With this control, the bot can modify settings for how others send and receive media, how they are seen in a session, and whether they have administrative controls.
Updates permissions and settings for specific participants in the room.
Parameters:
remote_participants
: Dictionary mapping participant IDs to their updated settingsThis method allows fine-grained control over participant capabilities and permissions in the Daily room.
Examples:
Permissions Settings:
For comprehensive details on permissions settings, see the Daily API documentation:
update_remote_participants()
documentationupdateParticipant()
->updatePermission
and participants().permissions
When testing with Daily Prebuilt, you can send messages to the Prebuilt chat using this method.
Parameters:
message
(str): The chat message to senduser_name
(str): The name of the user sending the messageSee send_prebuilt_chat_message for more details.
Returns the transport’s participant ID in the room.
Returns the transport’s room URL for the room currently in use.
WebRTC transport implementation using Daily for real-time audio/video communication
DailyTransport
provides real-time audio and video communication capabilities using Daily’s WebRTC platform. It supports bidirectional audio/video streams, transcription, voice activity detection (VAD), and participant management.
To use DailyTransport
, install the required dependencies:
You’ll also need to set up your Daily API key as an environment variable:
DAILY_API_KEY
.
You can obtain a Daily API key by signing up at Daily.
Daily room URL
Daily room token
Name of the bot in the room
Transport configuration parameters
Daily API endpoint URL
Daily API key for authentication
Enable audio output capabilities
Enable live audio streaming mode
Audio output sample rate in Hz
Number of audio output channels
Audio output bitrate in bits per second
Enable audio input capabilities
When enabled, incoming audio frames are pushed downstream
Audio input sample rate in Hz
Number of audio input channels
Audio filter for input processing. Supported filters are: KrispFilter()
and
NoisereduceFilter()
. See the KrispFilter()
reference
docs and NoisereduceFilter()
reference docs` for more
information.
Enable video output capabilities
Enable live video streaming mode
Video output width in pixels
Video output height in pixels
Video output bitrate in bits per second
Video output frame rate
Video color format (RGB, BGR, etc.)
Voice Activity Detection analyzer. You can set this to either
SileroVADAnalyzer()
or WebRTCVADAnalyzer()
. SileroVADAnalyzer is the
recommended option. Learn more about the
SileroVADAnalyzer.
Configuration for dial-in functionality
Enable real-time transcription
Configuration for transcription features
DailyTransport provides a comprehensive callback system for handling various events. Register callbacks using the @transport.event_handler()
decorator.
Called when the bot successfully joins the room.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing room join informationCalled when the bot leaves the room.
Parameters:
transport
: The DailyTransport instanceCalled when the first participant joins the room. Useful for initializing conversations.
Parameters:
transport
: The DailyTransport instanceparticipant
: Dictionary containing participant informationCalled when any participant joins the room.
Parameters:
transport
: The DailyTransport instanceparticipant
: Dictionary containing participant informationCalled when a participant leaves the room.
Parameters:
transport
: The DailyTransport instanceparticipant
: Dictionary containing participant informationreason
: String describing why the participant left, “leftCall” | “hidden”
Event emitted when a participant is updated. This can mean either the participant’s metadata was updated, or the tracks belonging to the participant changed.
Parameters:
transport
: The DailyTransport instanceparticipant
: Dictionary containing updated participant informationEvent emitted when a custom app message is received from another participant or via the REST API.
Parameters:
transport
: The DailyTransport instancemessage
: The message content (any type)sender
: String identifier of the message senderEvent emitted when the call state changes, normally as a consequence of joining or leaving the call.
Parameters:
transport
: The DailyTransport instancestate
: String representing the new call state. Learn more about call states.Event emitted when dial-in is ready. This happens after the room has connected to the SIP endpoint and the system is ready to receive dial-in calls.
Parameters:
transport
: The DailyTransport instancesip_endpoint
: String containing the SIP endpoint informationEvent emitted when the session with the dial-in remote end is established (i.e. SIP endpoint or PSTN are connectd to the Daily room).
Note: connected does not mean media (audio or video) has started flowing between the room and PSTN, it means the room received the connection request and both endpoints are negotiating the media flow.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing connection information. See DialinConnectedEvent.Event emitted when the dial-in remote end disconnects the call.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing connection information. See DialinStoppedEvent.Event emitted in the case of dial-in errors which are fatal and the service cannot proceed. For example, an error in SDP negotiation is fatal to the media/SIP pipeline and will result in dialin-error being triggered.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing error information. See DialinEvent.Event emitted there is a dial-in non-fatal error, such as the selected codec not being used and a fallback codec being utilized.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing warning information. See DialinEvent.Event emitted when the session with the dial-out remote end is answered.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing call information. Learn more.Event emitted when the session with the dial-out remote end is established.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing connection information. Learn more.Event emitted when the dial-out session is stopped.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing connection information. Learn more.Event emitted in the case of dial-out errors which are fatal and the service cannot proceed. For example, an error in SDP negotiation is fatal to the media/SIP pipeline and will result in dialout-error being triggered.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing error information.Learn more.Event emitted there is a dial-out non-fatal error, such as the selected codec not being used and a fallback codec being utilized.
Parameters:
transport
: The DailyTransport instancedata
: Dictionary containing warning information. Learn more.Called when a transcription message is received. This includes both interim and final transcriptions.
Parameters:
transport
: The DailyTransport instancemessage
: Dictionary containing transcription data including. Learn more.Called when a room recording starts successfully.
Parameters:
transport
: The DailyTransport instancestatus
: Dictionary containing recording status information. Learn more.Called when a room recording stops.
Parameters:
transport
: The DailyTransport instancestream_id
: String identifier of the stopped recording streamCalled when an error occurs during recording.
Parameters:
transport
: The DailyTransport instancestream_id
: String identifier of the recording streammessage
: Error message describing what went wrongCalled when a transport error occurs.
Parameters:
transport
: The DailyTransport instanceerror
: String containing error detailsRaw audio data from participants
Video frames from participants
Audio data to be sent
Video frames to be sent
Sends a message to participants in the Daily room. Messages can be directed to all participants or targeted to a specific participant.
Parameters:
frame
: A transport message frame containing the message to send. Can be either:
TransportMessageFrame
: Standard messageTransportMessageUrgentFrame
: High-priority messageMessage Structure:
Messages should conform to the expected format for the receiving system. For RTVI messages, use:
Example - Standard Message:
Example - Urgent Message to Specific Participant:
The message will be delivered to:
on_app_message
handlerUse TransportMessageUrgentFrame
for high-priority messages that need
immediate delivery.
Sends an image frame to the room.
Parameters:
frame
: Image frame to send, either raw image data or sprite animationSends an audio frame to the room.
Parameters:
frame
: Audio frame to sendStarts capturing video from a specific participant.
Parameters:
participant_id
: ID of the participant to captureframerate
: Target frame rate (default: 30)video_source
: Video source type (default: “camera”)color_format
: Color format of the video (default: “RGB”)To request an image from the camera stream, set the framerate
to 0 and push
a UserImageRequestFrame
.
Starts capturing and transcribing audio from a specific participant.
Parameters:
participant_id
: ID of the participant to transcribeUpdates the transcription configuration for specific participants or instances.
Parameters:
participants
: Optional list of participant IDs to transcribe. If None, transcribes all participants.instance_id
: Optional specific transcription instance ID to updateExample:
Starts recording the room session.
Parameters:
streaming_settings
: Recording configuration settingsstream_id
: Optional stream identifierforce_new
: Force start a new recordingSee start_recording for more details.
Stops an active recording.
Parameters:
stream_id
: Optional stream identifier to stop specific recordingSee stop_recording for more details.
Initiates a dial-out call.
Parameters:
settings
: Dial-out configuration settingsSee start_dialout for more details.
Stops an active dial-out call.
Parameters:
participant_id
: ID of the participant to disconnectSee stop_dialout for more details.
Sends DTMF tones in an existing dial-out session.
Parameters:
settings
: DTMF settingsSee send_dtmf for more details.
Updates subscriptions and subscription profiles. This function allows you to update subscription profiles and at the same time assign specific subscription profiles to a participant and even change specific settings for some participants.
Parameters:
participant_settings
: Per-participant subscription settingsprofile_settings
: Global subscription profile settingsSee update_subscriptions for more details.
When using the DailyTransport, if your bot is given a meeting token with owner or participant admin permissions, it can remote-control other participants’ capabilities and permissions. With this control, the bot can modify settings for how others send and receive media, how they are seen in a session, and whether they have administrative controls.
Updates permissions and settings for specific participants in the room.
Parameters:
remote_participants
: Dictionary mapping participant IDs to their updated settingsThis method allows fine-grained control over participant capabilities and permissions in the Daily room.
Examples:
Permissions Settings:
For comprehensive details on permissions settings, see the Daily API documentation:
update_remote_participants()
documentationupdateParticipant()
->updatePermission
and participants().permissions
When testing with Daily Prebuilt, you can send messages to the Prebuilt chat using this method.
Parameters:
message
(str): The chat message to senduser_name
(str): The name of the user sending the messageSee send_prebuilt_chat_message for more details.
Returns the transport’s participant ID in the room.
Returns the transport’s room URL for the room currently in use.