Documentation
EdgeBench
Open-weight models measured on the machine that runs them: how fast they read and write, and how often they get real work right.
Overview
EdgeBench is for anyone choosing a model and serving setup to run on their own hardware. Every configuration answers the same tasks with one standard setup per engine, so differences come from the model, its weights and the engine. Open it at qai.io/bench/edgebench.
| Measure | What it means |
|---|---|
| Quality | The share of tasks a configuration solved, graded by running the code or matching the exact answer. Shown with its 95% range. |
| Time per solved task | How long it takes, on average, to reach a right answer. Lower is better. |
| Writing speed | Tokens written per second, at short and long prompts. |
| First token | How long it waits before it starts to answer. |
| Usable depth | How long a prompt it handles before it breaks down. |
| Energy | Joules per token, from the GPU’s own power reading, not power at the wall. |
DGX Spark results
Filter by engine, precision, size and whether speculative decoding is on, or jump to the fastest to a right answer, the highest quality or the fastest writer. The tabs:
- Overview: quality against time per solved task. Up and to the left is better; click a mark to compare it.
- Leaderboard: every configuration with all its measures.
- Compare: configurations side by side, each measure saying whether higher or lower is better.
- Prompt length: how writing speed holds up as the prompt grows.
- Speculative decoding: a small draft model guesses tokens ahead for speed. Each row is checked against the same model without it, to confirm the answers did not change.
- Method: how each model is served, what each measure means and the limits to read before quoting a number.
Recorded tests
Head-to-head runs replayed token by token at the speed they were written, such as one model on two engines on the same DGX Spark. Compare the speeds, not the wording, since each answer is sampled.
For published benchmark scores across all models, see Rankings.
FAQ
Which devices does EdgeBench cover?
The NVIDIA DGX Spark today.
Why do some models show as waiting?
They are queued to be measured. Until then they are not results.
Can I quote a single number?
Quote it with its 95% range. Quality comes from a compact set of tasks, so ranges are wide, and two scores differ only when their ranges do not overlap.
Do I need an account?
No. EdgeBench is open to everyone.