Azure Databricks

Overview & Role

First-party Apache Spark and lakehouse analytics platform on Azure.

Domain Classification & Tags

Read Performance Profile

Throughput: Highly variable, from hundreds of MB/s to several GB/s per cluster, depending on data size, format, cluster size, and query complexity. Can reach tens of TB/hour for large-scale ETL.

Payload Size: From small files (KB) to very large datasets (TB) per query. Typically processes data in blocks/partitions, not individual 'payloads' in a request/response sense.

Processing Latency: Highly variable, from seconds for simple queries on small datasets to minutes or hours for complex analytics on large datasets. Interactive queries often target sub-minute latency.

Write Performance Profile

Throughput: Highly variable, from hundreds of MB/s to several GB/s per cluster, depending on data size, format, cluster size, and transformation complexity. Can ingest tens of TB/hour.

Payload Size: From small files (KB) to very large datasets (TB) per write operation. Typically writes data in partitions/files.

Processing Latency: Highly variable, from seconds for small writes to minutes or hours for large-scale ETL or complex transformations. Streaming writes can achieve near real-time latency (seconds to tens of seconds end-to-end).