Local Inference Benchmarks – Reproducible Performance Measurements
All measurements were collected using the Homegrown Intelligence benchmark harness version 2.3.0 on a test bench running Ubuntu 24.04 LTS with kernel 6.8. Each reported value is the median of fifty consecutive runs after five warm-up iterations. GPU clock frequencies are pinned to manufacturer base specifications to reduce variance. Temperature is stabilized at 70 °C before recording begins. Driver version, CUDA toolkit revision, and model checkpoint SHA-256 are recorded for every row. Raw CSV files are linked from each table footer.
Flux2 Image Generation – 1024×1024, 28 Steps
| GPU | VRAM | Median (s) | P95 (s) | Peak VRAM (GB) | TGP (W) |
|---|---|---|---|---|---|
| RTX 5090 | 32 GB GDDR7 | 8.71 | 8.94 | 18.2 | 575 |
| RTX 4090 | 24 GB GDDR6X | 11.34 | 11.67 | 17.8 | 450 |
| RTX 4080 Super | 16 GB GDDR6X | 15.92 | 16.41 | 15.9 | 320 |
| RTX 4070 Ti Super | 16 GB GDDR6X | 19.47 | 20.12 | 15.8 | 285 |
| RTX 3090 | 24 GB GDDR6X | 14.21 | 14.58 | 17.6 | 350 |
Download Flux2 benchmark summary (CSV, verified) — raw run data in datasets/raw/
WhisperX Transcription – 60-minute Audio File
| GPU | Processing Time (s) | Real-Time Factor | Peak VRAM (GB) |
|---|---|---|---|
| RTX 5090 | 118 | 30.5× | 6.2 |
| RTX 4090 | 142 | 25.3× | 5.9 |
| RTX 4080 Super | 189 | 19.0× | 5.7 |
| RTX 4070 Ti Super | 231 | 15.6× | 5.6 |
LLM Inference – Llama 3.1 8B Q4_K_M
| GPU | Prompt Processing (tok/s) | Generation (tok/s) | TTFT (ms) | Peak VRAM (GB) |
|---|---|---|---|---|
| RTX 5090 | 4820 | 186 | 41 | 7.1 |
| RTX 4090 | 3910 | 152 | 52 | 6.9 |
| RTX 4080 Super | 2870 | 118 | 68 | 6.8 |
| RTX 4070 Ti Super | 2210 | 94 | 83 | 6.7 |
Download LLM benchmark summary (CSV, verified) — raw run data in datasets/raw/
Methodology Notes
The benchmark harness pins GPU clock frequencies to manufacturer base specifications to reduce variance. Each measurement series discards the first five runs as GPU warm-up. Temperature is stabilized at 70 °C before recording begins. Memory measurements capture peak allocated VRAM via nvidia-smi sampled at 100 ms intervals. Power draw is recorded from the GPU power rail sensor averaged over the entire workload duration. All software versions, model SHAs, and driver revisions are documented in the accompanying metadata file for each test run. Full methodology is published in the benchmark methodology guide.