Overview
TogetherLLMService
provides access to Together AI’s language models, including Meta’s Llama 3.1 and 3.2 models, through an OpenAI-compatible interface. It inherits from OpenAILLMService
and supports streaming responses, function calling, and context management.
API Reference
Complete API documentation and method details
Together AI Docs
Official Together AI API documentation and features
Example Code
Working example with function calling
Installation
To useTogetherLLMService
, install the required dependencies:
TOGETHER_API_KEY
.
Get your API key from Together AI Console.
Frames
Input
OpenAILLMContextFrame
- Conversation context and historyLLMMessagesFrame
- Direct message listVisionImageRawFrame
- Images for vision processing (select models)LLMUpdateSettingsFrame
- Runtime parameter updates
Output
LLMFullResponseStartFrame
/LLMFullResponseEndFrame
- Response boundariesLLMTextFrame
- Streamed completion chunksFunctionCallInProgressFrame
/FunctionCallResultFrame
- Function call lifecycleErrorFrame
- API or processing errors
Function Calling
Function Calling Guide
Learn how to implement function calling with standardized schemas, register
handlers, manage context properly, and control execution flow in your
conversational AI applications.
Context Management
Context Management Guide
Learn how to manage conversation context, handle message history, and
integrate context aggregators for consistent conversational experiences.
Usage Example
Metrics
Inherits all OpenAI metrics capabilities:- Time to First Byte (TTFB) - Response latency measurement
- Processing Duration - Total request processing time
- Token Usage - Prompt tokens, completion tokens, and totals
Learn how to enable Metrics in your Pipeline.
Additional Notes
- OpenAI Compatibility: Full compatibility with OpenAI API features and parameters
- Open Source Models: Access to cutting-edge open-source models like Llama
- Vision Support: Select models support multimodal image and text understanding
- Competitive Pricing: Cost-effective alternative to proprietary model APIs
- Flexible Scaling: Choose model size based on performance vs cost requirements