Azure OpenAI Service

Overview & Role

Enterprise access to OpenAI models with Azure security, networking and compliance.

Domain Classification & Tags

Read Performance Profile

Throughput: Hundreds to thousands of requests per second (RPS) per deployed model, depending on model type, token limits, and provisioned throughput units (PTUs) or standard pay-as-you-go capacity. For example, GPT-3.5-Turbo can handle hundreds of RPS, while GPT-4 might be tens to hundreds of RPS.

Payload Size: Input prompts can range from a few tokens (e.g., 10-100 tokens for simple queries) to thousands of tokens (e.g., 4,000-32,000+ tokens for complex documents or chat history). Output completions can also vary similarly.

Processing Latency: Typically 100 ms to several seconds. Latency is highly dependent on model complexity (e.g., GPT-3.5-Turbo is faster than GPT-4), input/output token count, and network conditions. Simple requests to GPT-3.5-Turbo might be 100-500 ms, while complex GPT-4 requests could be 1-5+ seconds.

Write Performance Profile

Throughput: N/A (primarily an inference service; 'write' operations are not a primary performance metric in the traditional sense. Model fine-tuning or deployment operations are infrequent and not throughput-oriented.)

Payload Size: N/A

Processing Latency: N/A