LLaMA 3

Overview & Role

LLaMA 3 (Large Language Model Meta AI 3) is a family of large language models developed by Meta AI. It is designed to be open and accessible, offering pre-trained and instruction-tuned versions with varying parameter counts, capable of performing a wide range of natural language processing tasks.

Domain Classification & Tags

Read Performance Profile

Throughput: Varies significantly by hardware (GPU type, quantity) and model size. For LLaMA 3 8B, 50-200 tokens/sec on a single high-end consumer GPU (e.g., RTX 4090). For LLaMA 3 70B, 10-50 tokens/sec on multiple high-end GPUs (e.g., A100s). Cloud APIs can offer higher aggregate throughput.

Payload Size: Up to 8,192 tokens for LLaMA 3 8B and 70B models. Context length can be extended with techniques like RAG or specialized fine-tuning.

Processing Latency: Varies significantly. First token latency: 100-500 ms (depending on model size and hardware). Per-token generation latency: 10-50 ms/token (depending on model size and hardware). Total response latency depends on output length.

Write Performance Profile

Throughput: Not applicable for typical usage of a pre-trained model. Training throughput is measured in tokens/second processed during training, which is orders of magnitude higher and requires massive GPU clusters (e.g., thousands of A100s).

Payload Size: Not applicable for typical usage of a pre-trained model. Fine-tuning involves datasets ranging from hundreds to millions of examples.

Processing Latency: Not applicable for typical usage of a pre-trained model. Fine-tuning can take hours to days depending on dataset size and available compute.