Inference Arena runs the same training and inference workload through every supported ML framework on every available platform, then publishes the results side by side. Each tab below covers one model — pick your model, filter to the frameworks you care about, and compare. Lower numbers are better; bold marks the best matching framework on each platform.
HuggingFaceTB/SmolLM2-135M — 134.5M parameter decoder-only language model.
Benchmark config: seq_len=128, input=[0,1,…,127]. The full forward is a 128-token prefill. The minimal measurement is a stateless one-token forward without a KV cache; it is not decode latency. The historical table below predates the current timing and precision metadata; regenerate it from one pinned revision before using it in a publication.
| Platform | Framework | Compile (s) | Prefill (ms) | Stateless 1-token (ms) | Training (ms) | Loss |
|---|---|---|---|---|---|---|
| Intel Xeon @ 2.10GHz | PyTorch 2.11.0+cu130 (CPU) | 135.63 | 188 | 18 | 486 | 10.98 |
| ONNX Runtime 1.24.4 (CPU) | 65.50 | 118 | 20 | — | 10.98 | |
| JAX 0.9.2 (CPU) | 6.79 | 194 | 31 | 2107 | 10.98 | |
| Candle (CPU) | 0.31 | 453 | 61 | — | 11.11 | |
| Luminal (CPU) | 3.37 | 17006 | — | 14459 | 10.81 | |
| Burn (wgpu/Lavapipe) | ||||||
| Meganeura (Vulkan/Lavapipe) | 7.29 | 3933 | 852 | 3651 | 10.99 | |
| llama.cpp (CPU) | 0.10 | 221 | 24 | — | 10.98 | |
| AMD Radeon 890M Graphics | PyTorch 2.10.0 (ROCm 7.2.53210) | 51.07 | 64 | 27 | 119 | 8.35 |
| Burn (wgpu/vulkan) | ||||||
| Inferi | ✗ | ✗ | ✗ | ✗ | ||
| Meganeura (Vulkan) | 0.59 | 26 | 9.1 | 92 | 8.64 | |
| ONNX Runtime | ✗ | ✗ | ✗ | ✗ | ||
| Apple M3 | PyTorch 2.11.0 (MPS) | 0.00 | 356 | 71 | 699 | 8.35 |
| MLX (MLX) | 0.00 | 97 | — | 253 | 8.64 | |
| Candle (Metal) | ||||||
| Burn (wgpu/metal) | ||||||
| Inferi | ✗ | ✗ | ✗ | ✗ | ||
| Luminal | ✗ | ✗ | ✗ | ✗ | ||
| Meganeura (Metal) | 1.50 | 201 | 9.1 | 464 | 8.65 | |
| GGML (Metal) | 0.38 | 49 | 11 | — | 8.69 | |
| JAX (METAL) | ||||||
| NVIDIA GeForce RTX 5080 | PyTorch 2.13.0+cu130 (CUDA 13.0) | 20.53 | 4.4 | 3.0 | 6.4 | 6.01 |
| Candle (CUDA) | ||||||
| Burn (wgpu) | ||||||
| Meganeura (Vulkan) | 0.62 | 8.5 | 2.2 | 22 | 5.79 | |
| GGML (Vulkan) | ||||||
| ONNX Runtime 1.27.0 (CUDAExecutionProvider) | 20.62 | 5.3 | 3.2 | — | 6.01 | |
| MAX (GPU) | ||||||
| JAX 0.11.0 (GPU) | 21.02 | 3.8 | 1.4 | 11 | 5.79 | |
| NVIDIA GeForce RTX 3050 (Windows) | PyTorch 2.11.0+cu128 (CUDA 12.8) | 0.00 | 11 | 5.1 | 51 | 8.35 |
| Burn (wgpu/vulkan) | ||||||
| Inferi | ✗ | ✗ | ✗ | ✗ | ||
| Meganeura (Vulkan/DX12) | 1.40 | 13 | 3.6 | 58 | 8.63 | |
| GGML (CUDA) | 0.31 | 132 | 5.9 | — | 8.69 | |
| ONNX Runtime (CUDAExecutionProvider) | ||||||
| JAX | ✗ | ✗ | ✗ | ✗ | ||
| Intel(R) Graphics (RPL-U) | PyTorch 2.11.0+xpu (CPU) | 0.00 | 541 | 126 | 1130 | 8.35 |
| Candle (CPU) | ||||||
| Burn (wgpu/vulkan) | ||||||
| Inferi (Vulkan) | ||||||
| Luminal (CPU) | ||||||
| Meganeura (Vulkan) | 1.74 | 172 | 52 | 700 | 8.64 | |
| GGML (CPU) | 0.11 | 433 | 33 | — | 8.69 | |
| ONNX Runtime (CPUExecutionProvider) | ||||||
| MAX | ✗ | ✗ | ✗ | ✗ | ||
| JAX (CPU) | ||||||
| AMD Radeon RX 7900 XT | PyTorch 2.10.0+rocm7.1 (ROCm 7.1.25424) | 73.03 | 10 | 6.8 | 23 | 8.35 |
| Burn (wgpu/vulkan) | ||||||
| Inferi | ✗ | ✗ | ✗ | ✗ | ||
| Meganeura (Vulkan) | 0.99 | 7.3 | 2.1 | 21 | 8.64 | |
| GGML (ROCm) | 0.11 | 259 | 3.4 | — | 8.69 | |
| MAX (GPU) |
Historical correctness note: the old validator sampled the first 16 flattened logits, all from sequence position 0, and could accept a backend when only its loss was close. Those PASS/CLOSE labels are not publication-grade. The current harness samples across the complete output and requires both output and loss agreement, plus gradient-norm agreement when available.
Caveats: - PyTorch and Meganeura load real model weights and run the full architecture — their outputs match.
- Candle runs the real LLaMA architecture with the same safetensors
weights, but its
forward()returns last-position logits only (private fields prevent getting all-position logits). Loss is computed on 1 position vs 128 for others — hence DIFFERENT MODEL in correctness check. Timing is valid. Backward not yet wired. - Burn and Luminal use a simplified model (single-head attention, no RoPE/RMSNorm) with random weights.
- Luminal backward is estimated as a second forward pass.
Run it yourself: git clone https://github.com/kvark/inferena && cd inferena && ./run.sh -m SmolLM2-135M
lerobot/smolvla_base — a matched 99,848,592-parameter SmolVLA-style action expert workload for robotics.
Benchmark config: batch=1, chunk_size=50, vlm_seq_len=16, float32, deterministic matched weights, MSE loss.
The table below predates the current timing and validation metadata. Its
timings and historical CLOSE label are exploratory; regenerate the table
from one revision before using it in a publication.
| Platform | Framework | Compile (s) | Inference (ms) | Latency (ms) | Training (ms) | Loss |
|---|---|---|---|---|---|---|
| Intel Xeon @ 2.10GHz | PyTorch 2.11.0+cu130 (CPU) | 51.63 | 40 | 11 | 116 | 0.00 |
| Meganeura (Vulkan/Lavapipe) | 2.75 | 696 | — | 3850 | 0.01 | |
| ONNX Runtime (CPU) | ✗ | ✗ | ✗ | ✗ | ||
| JAX (CPU) | ✗ | ✗ | ✗ | ✗ | ||
| Candle (CPU) | ✗ | ✗ | ✗ | ✗ | ||
| Burn (wgpu) | ✗ | ✗ | ✗ | ✗ | ||
| Luminal (CPU) | ✗ | ✗ | ✗ | ✗ | ||
| AMD Radeon 890M Graphics | PyTorch 2.10.0 (ROCm 7.2.53210) | 19.72 | 27 | 14 | 49 | 0.00 |
| Candle | — | — | — | — | ||
| Burn | — | — | — | — | ||
| Inferi | ✗ | ✗ | ✗ | ✗ | ||
| Luminal | — | — | — | — | ||
| Meganeura (Vulkan) | 0.12 | 15 | 6.7 | 47 | 0.00 | |
| GGML | — | — | — | — | ||
| ONNX Runtime (MIGraphXExecutionProvider) | 19.38 | 10 | — | — | 0.00 | |
| Apple M3 | PyTorch 2.11.0 (MPS) | 0.00 | 173 | 9.1 | 117 | 0.00 |
| MLX (MLX) | 0.00 | 13 | — | 24 | 0.00 | |
| Candle | — | — | — | — | ||
| Burn | — | — | — | — | ||
| Inferi | ✗ | ✗ | ✗ | ✗ | ||
| Luminal | — | — | — | — | ||
| Meganeura (Metal) | 0.12 | 34 | 6.4 | 170 | 0.00 | |
| GGML | — | — | — | — | ||
| ONNX Runtime (CoreMLExecutionProvider) | 7.96 | 86 | — | — | 0.00 | |
| JAX (METAL) | 1.17 | 15 | — | 147 | 0.00 | |
| NVIDIA GeForce RTX 5080 | PyTorch 2.13.0+cu130 (CUDA 13.0) | 9.30 | 2.0 | 1.1 | 2.6 | 0.00 |
| Candle | — | — | — | — | ||
| Burn | — | — | — | — | ||
| Meganeura (Vulkan) | 0.24 | 3.3 | 1.5 | 8.2 | 0.00 | |
| GGML | — | — | — | — | ||
| ONNX Runtime 1.27.0 (CUDAExecutionProvider) | 2.64 | 1.5 | — | — | 0.00 | |
| MAX (GPU) | 1.12 | 25 | — | — | 0.00 | |
| JAX 0.11.0 (GPU) | 15.82 | 2.4 | — | 3.9 | 0.00 | |
| NVIDIA GeForce RTX 3050 (Windows) | PyTorch 2.11.0+cu128 (CUDA 12.8) | 0.00 | 4.5 | 3.5 | 22 | 0.00 |
| Candle | — | — | — | — | ||
| Burn | — | — | — | — | ||
| Inferi | ✗ | ✗ | ✗ | ✗ | ||
| Luminal | — | — | — | — | ||
| Meganeura (Vulkan/DX12) | 0.70 | 4.9 | 2.9 | 23 | 0.00 | |
| GGML | — | — | — | — | ||
| ONNX Runtime (CUDAExecutionProvider) | 4.14 | 4.9 | — | — | 0.00 | |
| JAX | ✗ | ✗ | ✗ | ✗ | ||
| Intel(R) Graphics (RPL-U) | PyTorch 2.11.0+xpu (CPU) | 0.00 | 183 | 72 | 388 | 0.00 |
| Candle | — | — | — | — | ||
| Burn | — | — | — | — | ||
| Inferi | — | — | — | — | ||
| Luminal | — | — | — | — | ||
| Meganeura (Vulkan) | 0.39 | 73 | 40 | 222 | 0.00 | |
| GGML | — | — | — | — | ||
| ONNX Runtime (CPUExecutionProvider) | 10.33 | 86 | — | — | 0.00 | |
| MAX | ✗ | ✗ | ✗ | ✗ | ||
| JAX (CPU) | 3.86 | 162 | — | 471 | 0.00 | |
| AMD Radeon RX 7900 XT | PyTorch 2.10.0+rocm7.1 (ROCm 7.1.25424) | 9.23 | 4.8 | 4.1 | 8.1 | 0.00 |
| Candle | — | — | — | — | ||
| Burn | — | — | — | — | ||
| Inferi | ✗ | ✗ | ✗ | ✗ | ||
| Luminal | — | — | — | — | ||
| Meganeura (Vulkan) | 0.81 | 5.1 | 1.5 | 9.9 | 0.00 | |
| GGML | — | — | — | — | ||
| MAX (GPU) | 2.20 | 15 | — | — | 0.00 |
Historical correctness: PyTorch vs Meganeura was reported as CLOSE (loss diff 1e-5, max error 4.6e-3) under the old validator. The current harness rechecks output samples distributed across the tensor, loss, total gradient norm, and per-parameter gradient norms.
Caveats: - PyTorch and Meganeura implement the full action expert architecture and should produce matching outputs.
- Burn and Luminal do not implement this architecture yet (reported as ✗).
- Inputs are synthetic: random noisy actions, sinusoidal timestep, random VLM context.
Run it yourself: git clone https://github.com/kvark/inferena && cd inferena && ./run.sh -m SmolVLA
The CLI retains the historical StableDiffusion workload name. The matched
PyTorch/Meganeura workload is now a scaled, conditioned latent-diffusion U-Net,
not the earlier generic convolutional U-Net and not the complete Stable
Diffusion 1.5 pipeline.
It retains the denoiser features that materially change the compiler workload: timestep conditioning in every residual block, spatial self-attention, 77-token text cross-attention, transformer feed-forward blocks, convolutional down/up paths, GroupNorm, and U-Net skip connections.
The previous table measured the superseded 5.29M-parameter convolution-only workload and has been removed to prevent accidental comparison. This empty table preserves the validated results schema and insertion point for the next frozen run.
| Platform | Framework | Compile (s) | Inference (ms) | Latency (ms) | Training (ms) | Loss | |———-|———–|:———–:|:————–:|:————:|:————-:|:—-:|
Freeze one Meganeura revision and rerun both practical-default and strict configurations on every platform before publishing new results:
./run.sh -m StableDiffusion -f pytorch,meganeura
./run.sh -m StableDiffusion -f pytorch,meganeura --strict
During development, strict-f32 cross-engine validation on an NVIDIA RTX 5080
reached approximately 1.2e-5 forward relative L2 error and 8.1e-5
per-parameter gradient-vector relative L2 error. These are correctness smoke
results from a dirty development revision, not frozen performance results.
The legacy Candle runner uses a much larger SD 1.5-like U-Net and is not comparable to this matched workload.
Run it yourself: git clone https://github.com/kvark/inferena && cd inferena && ./run.sh -m StableDiffusion
Classic convolutional neural network for image classification. The matched benchmark graph has 25,530,472 trainable parameters.
Benchmark config: batch=4, 3×224×224, float32, deterministic matched weights, cross-entropy loss.
The table below predates the current timing and validation metadata. Regenerate it before using the numbers or the historical correctness label in a publication.
| Platform | Framework | Compile (s) | Inference (ms) | Latency (ms) | Training (ms) | Loss |
|---|---|---|---|---|---|---|
| Intel Xeon @ 2.10GHz | PyTorch 2.11.0+cu130 (CPU) | 60.61 | 141 | 40 | 284 | 10.10 |
| ONNX Runtime 1.24.4 (CPU) | 0.28 | 76 | 18 | — | 10.37 | |
| Candle (CPU) | ||||||
| Meganeura (Vulkan/Lavapipe) | ||||||
| Burn (wgpu) | ✗ | ✗ | ✗ | ✗ | ||
| JAX (CPU) | ✗ | ✗ | ✗ | ✗ | ||
| AMD Radeon 890M Graphics | PyTorch 2.10.0 (ROCm 7.2.53210) | 36.46 | 48 | 16 | 97 | 6.92 |
| Burn | — | — | — | — | ||
| Inferi | ✗ | ✗ | ✗ | ✗ | ||
| Luminal | — | — | — | — | ||
| Meganeura (Vulkan) | 0.31 | 58 | 19 | 125 | 6.92 | |
| GGML | — | — | — | — | ||
| ONNX Runtime (MIGraphXExecutionProvider) | 3.17 | 29 | 9.3 | — | 6.92 | |
| Apple M3 | PyTorch 2.11.0 (MPS) | 0.00 | 166 | 21 | 274 | 6.92 |
| MLX | — | — | — | — | ||
| Candle (Metal) | 0.01 | 16 | 5.3 | — | 6.91 | |
| Burn | — | — | — | — | ||
| Inferi | ✗ | ✗ | ✗ | ✗ | ||
| Luminal | — | — | — | — | ||
| Meganeura (Metal) | 0.32 | 63 | 23 | 965 | 6.92 | |
| GGML | — | — | — | — | ||
| ONNX Runtime (CoreMLExecutionProvider) | 4.87 | 6.5 | 2.1 | — | 6.92 | |
| JAX (METAL) | 0.90 | 139 | 9.4 | 409 | 6.92 | |
| NVIDIA GeForce RTX 5080 | PyTorch 2.13.0+cu130 (CUDA 13.0) | 9.15 | 2.5 | 1.6 | 4.3 | 6.92 |
| Candle (CUDA) | 0.17 | 63 | 2.3 | — | 6.92 | |
| Burn | — | — | — | — | ||
| Meganeura (Vulkan) | 0.24 | 5.1 | 3.4 | 26 | 6.92 | |
| GGML | — | — | — | — | ||
| ONNX Runtime 1.27.0 (CUDAExecutionProvider) | 1.54 | 2.4 | 1.3 | — | 6.92 | |
| MAX | — | — | — | — | ||
| JAX 0.11.0 (GPU) | 2.65 | 2.0 | 1.3 | 5.3 | 6.92 | |
| NVIDIA GeForce RTX 3050 (Windows) | PyTorch 2.11.0+cu128 (CUDA 12.8) | 0.00 | 12 | 4.1 | 36 | 6.92 |
| Burn | — | — | — | — | ||
| Inferi | ✗ | ✗ | ✗ | ✗ | ||
| Luminal | — | — | — | — | ||
| Meganeura (Vulkan/DX12) | 1.04 | 19 | 6.5 | 52 | 6.92 | |
| GGML | — | — | — | — | ||
| ONNX Runtime (CUDAExecutionProvider) | 3.20 | 13 | 4.5 | — | 6.92 | |
| JAX | ✗ | ✗ | ✗ | ✗ | ||
| Intel(R) Graphics (RPL-U) | PyTorch 2.11.0+xpu (CPU) | 0.00 | 511 | 127 | 1048 | 6.92 |
| Candle (CPU) | 0.36 | 1187 | 368 | — | 6.92 | |
| Burn | — | — | — | — | ||
| Inferi | — | — | — | — | ||
| Luminal | — | — | — | — | ||
| Meganeura (Vulkan) | 0.77 | 225 | 73 | 1049 | 6.92 | |
| GGML | — | — | — | — | ||
| ONNX Runtime (CPUExecutionProvider) | 5.75 | 204 | 53 | — | 6.92 | |
| MAX | ✗ | ✗ | ✗ | ✗ | ||
| JAX (CPU) | 6.99 | 436 | 176 | 2813 | 6.92 | |
| AMD Radeon RX 7900 XT | PyTorch 2.10.0+rocm7.1 (ROCm 7.1.25424) | 17.57 | 7.1 | 3.8 | 17 | 6.92 |
| Burn | — | — | — | — | ||
| Inferi | ✗ | ✗ | ✗ | ✗ | ||
| Luminal | — | — | — | — | ||
| Meganeura (Vulkan) | 0.92 | 6.5 | 3.4 | 26 | 6.92 | |
| GGML | — | — | — | — | ||
| MAX | — | — | — | — |
Historical correctness: the old PyTorch/ONNX Runtime CLOSE label
predates the matched folded-BatchNorm graph and the audited validator.
Caveats: - The benchmark intentionally does not measure training-mode BatchNorm statistics or their backward pass. Both engines run the same inference-folded representation: identity scale plus a trainable channel bias.
- Inputs and labels are synthetic, so this measures systems performance and numerical equivalence, not ImageNet accuracy.
Run it yourself: git clone https://github.com/kvark/inferena && cd inferena && ./run.sh -m ResNet-50
openai/whisper-tiny supplies the configuration, but the audited workload is the four-layer encoder only: 8,208,384 total parameters, of which 7,632,384 are trainable. It is not a full speech-to-text pipeline.
Benchmark config: batch=1, synthetic 30-second mel spectrogram (80×3000), float32, deterministic matched weights, and mean-square encoder-output loss. The positional embedding is frozen in both implementations.
The table below predates the current timing and validation metadata and includes runners that measured older or different Whisper workloads. Regenerate it before using the numbers in a publication.
| Platform | Framework | Compile (s) | Inference (ms) | Latency (ms) | Training (ms) | Loss |
|---|---|---|---|---|---|---|
| Intel Xeon @ 2.10GHz | PyTorch 2.11.0+cu130 (CPU) | 39.88 | 150 | — | 371 | 11.80 |
| ONNX Runtime 1.24.4 (CPU) | 0.84 | 212 | — | — | 11.80 | |
| Candle (CPU) | ||||||
| Meganeura (Vulkan/Lavapipe) | ||||||
| Burn (wgpu) | ✗ | ✗ | ✗ | ✗ | ||
| JAX (CPU) | ✗ | ✗ | ✗ | ✗ | ||
| AMD Radeon 890M Graphics | PyTorch 2.10.0 (ROCm 7.2.53210) | 17.09 | 79 | 63 | 220 | 0.00 |
| Burn | — | — | — | — | ||
| Inferi | ✗ | ✗ | ✗ | ✗ | ||
| Luminal | — | — | — | — | ||
| Meganeura (Vulkan) | 0.20 | 34 | 33 | 101 | 0.01 | |
| GGML | ✗ | ✗ | ✗ | ✗ | ||
| ONNX Runtime (MIGraphXExecutionProvider) | 23.58 | 32 | — | — | 0.01 | |
| Apple M3 | PyTorch 2.11.0 (MPS) | 0.00 | 318 | 41 | 127 | 0.00 |
| MLX | — | — | — | — | ||
| Candle (Metal) | 0.01 | 22 | — | — | 0.00 | |
| Burn | — | — | — | — | ||
| Inferi | ✗ | ✗ | ✗ | ✗ | ||
| Luminal | — | — | — | — | ||
| Meganeura (Metal) | 0.15 | 406 | 415 | 1062 | 0.01 | |
| GGML | ✗ | ✗ | ✗ | ✗ | ||
| ONNX Runtime (CoreMLExecutionProvider) | 7.93 | 440 | — | — | 0.01 | |
| JAX (METAL) | 2.17 | 128 | 315 | 445 | 0.01 | |
| NVIDIA GeForce RTX 5080 | PyTorch 2.13.0+cu130 (CUDA 13.0) | 4.14 | 2.3 | 2.4 | 9.7 | 0.00 |
| Candle (CUDA) | 0.02 | 45 | — | — | 0.00 | |
| Burn | — | — | — | — | ||
| Meganeura (Vulkan) | 0.21 | 3.4 | 3.4 | 12 | 0.01 | |
| GGML (faster-whisper (CTranslate2, CUDA)) | 7.98 | 15 | 15 | — | 0.00 | |
| ONNX Runtime 1.27.0 (CUDAExecutionProvider) | 2.04 | 3.6 | — | — | 0.01 | |
| MAX | — | — | — | — | ||
| JAX 0.11.0 (GPU) | 17.39 | 2.1 | 1.6 | 5.6 | 0.01 | |
| NVIDIA GeForce RTX 3050 (Windows) | PyTorch 2.11.0+cu128 (CUDA 12.8) | 0.00 | 13 | 13 | 43 | 0.00 |
| Burn | — | — | — | — | ||
| Inferi | ✗ | ✗ | ✗ | ✗ | ||
| Luminal | — | — | — | — | ||
| Meganeura (Vulkan/DX12) | 0.66 | 19 | 19 | 43 | 0.01 | |
| GGML (faster-whisper (CTranslate2, CUDA)) | 7.00 | 40 | 45 | — | 0.00 | |
| ONNX Runtime (CUDAExecutionProvider) | 4.82 | 20 | — | — | 0.01 | |
| JAX | ✗ | ✗ | ✗ | ✗ | ||
| Intel(R) Graphics (RPL-U) | PyTorch 2.11.0+xpu (CPU) | 0.00 | 477 | 420 | 899 | 0.00 |
| Candle (CPU) | 0.02 | 795 | — | — | 0.00 | |
| Burn | — | — | — | — | ||
| Inferi | — | — | — | — | ||
| Luminal | — | — | — | — | ||
| Meganeura (Vulkan) | 0.39 | 467 | 466 | 1594 | 0.01 | |
| GGML (faster-whisper (CTranslate2, CPU)) | 14.76 | 1036 | 1104 | — | 0.00 | |
| ONNX Runtime (CPUExecutionProvider) | 6.18 | 333 | — | — | 0.01 | |
| MAX | ✗ | ✗ | ✗ | ✗ | ||
| JAX (CPU) | 5.59 | 717 | 686 | 2681 | 0.01 | |
| AMD Radeon RX 7900 XT | PyTorch 2.10.0+rocm7.1 (ROCm 7.1.25424) | 5.38 | 12 | 6.5 | 44 | 0.00 |
| Burn | — | — | — | — | ||
| Inferi | ✗ | ✗ | ✗ | ✗ | ||
| Luminal | — | — | — | — | ||
| Meganeura (Vulkan) | 0.82 | 4.8 | 4.8 | 21 | 0.01 | |
| MAX | — | — | — | — |
Historical correctness: the old table’s PyTorch/ONNX Runtime PASS
applies to the prior workload and validator, not the audited encoder
comparison.
Caveats: - Uses the Whisper-tiny encoder shape (4 layers, d=384), not its decoder
- Input is synthetic mel spectrogram, not real audio
- The objective is a synthetic differentiable systems workload, not a transcription-quality evaluation
Run it yourself: git clone https://github.com/kvark/inferena && cd inferena && ./run.sh -m Whisper-tiny