Local Inference Benchmarks – Reproducible Performance Measurements

Name: Local Inference Benchmarks
Creator: Homegrown Intelligence

All measurements were collected using the Homegrown Intelligence benchmark harness version 2.3.0 on a test bench running Ubuntu 24.04 LTS with kernel 6.8. Each reported value is the median of fifty consecutive runs after five warm-up iterations. Driver version, CUDA toolkit revision, and model checkpoint SHA-256 are recorded for every row. Raw CSV files are linked from each table footer.

Flux2 Image Generation – 1024×1024, 28 Steps

GPU	VRAM	Median (s)	P95 (s)	Peak VRAM (GB)	Power (W)
RTX 5090	32 GB	8.71	8.94	18.2	312
RTX 4090	24 GB	11.34	11.67	17.8	289
RTX 4080 Super	16 GB	15.92	16.41	15.9	241
RTX 4070 Ti Super	16 GB	19.47	20.12	15.8	218
RTX 3090	24 GB	14.21	14.58	17.6	334

Download Flux2 raw measurements (CSV, 2.4 MB)

WhisperX Transcription – 60-minute Audio File

GPU	Processing Time (s)	Real-Time Factor	Peak VRAM (GB)
RTX 5090	118	30.5×	6.2
RTX 4090	142	25.3×	5.9
RTX 4080 Super	189	19.0×	5.7
RTX 4070 Ti Super	231	15.6×	5.6

Download WhisperX raw measurements (CSV, 1.8 MB)

LLM Inference – Llama 3.1 8B Q4_K_M

GPU	Prompt Processing (tok/s)	Generation (tok/s)	TTFT (ms)	Peak VRAM (GB)
RTX 5090	4820	186	41	7.1
RTX 4090	3910	152	52	6.9
RTX 4080 Super	2870	118	68	6.8
RTX 4070 Ti Super	2210	94	83	6.7

Download LLM raw measurements (CSV, 3.1 MB)

Methodology Notes

The benchmark harness pins GPU clock frequencies to manufacturer base specifications to reduce variance. Each measurement series discards the first five runs as GPU warm-up. Temperature is stabilized at 70 °C before recording begins. Memory measurements capture peak allocated VRAM via nvidia-smi sampled at 100 ms intervals. Power draw is recorded from the GPU power rail sensor averaged over the entire workload duration. All software versions, model SHAs, and driver revisions are documented in the accompanying metadata file for each test run.