Azure
Large Language Model service implementation using Azure OpenAI API
Overview
AzureLLMService
provides access to Azure OpenAI’s language models through an OpenAI-compatible interface. It inherits from OpenAILLMService
and supports streaming responses, function calling, and context management.
Installation
To use AzureLLMService
, install the required dependencies:
You’ll need to set up:
- Azure OpenAI API key
- Azure OpenAI endpoint URL
- Azure OpenAI deployment name (used as the model identifier)
Configuration
Constructor Parameters
Azure OpenAI API key
Azure OpenAI endpoint URL
Model deployment name
Azure OpenAI API version
Input Parameters
Inherits all input parameters from OpenAILLMService:
Additional parameters to pass to the model
Reduces likelihood of repeating tokens based on their frequency. Range: [-2.0, 2.0]
Maximum number of tokens in the completion. Must be greater than or equal to 1
Maximum number of tokens to generate. Must be greater than or equal to 1
Reduces likelihood of repeating any tokens that have appeared. Range: [-2.0, 2.0]
Random seed for deterministic generation. Must be greater than or equal to 0
Controls randomness in the output. Range: [0.0, 2.0]
Controls diversity via nucleus sampling. Range: [0.0, 1.0]
Input Frames
Contains OpenAI-specific conversation context
Contains conversation messages
Contains image for vision model processing
Updates model settings
Output Frames
Contains generated text chunks
Indicates start of function call
Contains function call results
Methods
See the LLM base class methods for additional functionality.
Usage Example
Function Calling
Supports OpenAI-compatible function calling:
Available Models
Azure OpenAI supports deploying various OpenAI models:
Model Type | Description |
---|---|
GPT-4o | GPT-4o models |
GPT-4 | GPT-4 models |
GPT-3.5 | GPT-3.5 Turbo models |
Embeddings | Text embedding models |
The actual model used is determined by your Azure OpenAI deployment configuration.
See Azure’s OpenAI docs for a complete list of supported models.
Frame Flow
Metrics Support
The service collects the same metrics as OpenAILLMService:
- Token usage (prompt and completion)
- Processing duration
- Time to First Byte (TTFB)
- Function call metrics
Notes
- OpenAI-compatible interface
- Supports streaming responses
- Handles function calling
- Manages conversation context
- Includes token usage tracking
- Thread-safe processing
- Automatic error handling
- Inherits OpenAI service features