vLLM vs SGLang on a DGX Spark: Breaking the 14.5 tok/s Wall

On one NVIDIA DGX Spark, Qwen3.8-27B writes about 12.6 tokens per second under vLLM, and tuning barely moves it, because the hardware caps it near 14.5. With SGLang and the DFlash2 draft model, the same machine wrote 2 to 3.3 times as fast in our recorded test. A 3D page that took vLLM 464 seconds took SGLang 179.

You can watch both engines race, token by token on QAI EdgeBench.

Why a Spark tops out near 14.5 tokens per second

To write one token, a language model reads all of its weights from memory. The Spark's memory moves 273 GB per second, and Qwen3.8-27B in 4-bit form is about 18.8 GB, so the ceiling is 273 ÷ 18.8, about 14.5 tokens per second (ai-muninn). The engine doesn't change that arithmetic. Without help, community testers measure every engine on the Spark at 8 to 12 tokens per second, and our own vLLM run came in at 12.6.

In an agentic coding session, most of the wait is the model writing: code, edits, tool calls. So writing speed is the lever.

How speculative decoding gets past it

A small draft model guesses several tokens ahead, eight per pass in DFlash2's case. The main model then checks all of them in a single pass, the same pass it would have spent writing one token, and keeps the ones it would have written itself. Each pass now produces several tokens instead of one. Because the main model still approves every token, the answers are as good as without drafting. Only the speed changes.

The gain depends on how many guesses survive, and that depends on the draft model and on how predictable the text is.

What other Spark owners measured

NVIDIA hasn't published figures for this model on the Spark, so the grey bars below are community results, one request at a time on one Spark. Our own test runs are in blue.

Without speculative decoding, the engines are about equal, because the memory limit applies to all of them. What separates them is which draft models each one runs well on the Spark.

What we measured

We recorded both engines on 3 October 2026, on the same Spark, with the same model size, the same prompts and the same settings. They ran one after the other, because both servers don't fit in memory at once. vLLM ran without speculative decoding; SGLang ran with DFlash2.

Editing a file landed at the top of the community's 34 to 40 range, and writing new code fell below it. That fits how drafting works: an edit mostly repeats text the model has just read, so more of the draft survives. The finish times show what that means for waiting.

One result went the other way. Reading a brand-new prompt, vLLM was faster. Community figures had put the two level.

The 3D pages in the race are exactly what each engine wrote, unedited, and each one runs in your browser the moment its engine finishes.

Memory settings

The two engines' memory settings measure different things. vLLM's caps everything vLLM uses. SGLang's covers only the model and its cache, and its temporary working memory comes on top (memory creep thread). The Spark's memory is shared with the processor and the operating system, and going past it freezes the whole machine without an error.

Risks, and how to handle them

Speed isn't free on a chip this new to SGLang. The two crash bugs come from community testing (4× Spark write-up); in our run, SGLang passed all 7 of our known-crash-trigger checks.

If you want to try it

1. Record your current vLLM launch, so you can restart it unchanged.
2. Get the SGLang build made for the Spark, the 4-bit (NVFP4) weights and the DFlash2 draft model, about 4 GB.
3. Start SGLang on the same port and model name as before, with --mem-fraction-static 0.5, the Qwen tool-call and reasoning parsers, and metrics on.
4. Apply the 16-token-minimum patch and add a watchdog that restarts a crashed server.
5. For Claude Code, set CLAUDE_CODE_ATTRIBUTION_HEADER=0 and a long API_TIMEOUT_MS (SGLang docs).
6. Before relying on it, test a normal request, a tool call, a request allowing one output token and a cancelled reply.

The limits of these numbers

• Every speed here except ours comes from community testers, and their setups differ.
• Our test is one recorded run per scene with the model's default sampling, so the two engines wrote different text of different lengths. Compare the speeds, not the wording.
• The engines loaded different 4-bit exports of the same model: NVIDIA's for vLLM and RadixArk's for SGLang, whose output layer is full precision. That makes each SGLang pass slightly heavier, not lighter.
• Most tests, ours included, used prompts far shorter than a long coding session builds up, which may shrink the gain in daily use.

On a Spark, the memory ceiling means drafting, not engine tuning, is what moves writing speed: in our test it took a 3D page from 464 seconds to 179. Watch vLLM and SGLang race on one DGX Spark on QAI EdgeBench, with every chart, the setup and the full method.

Sources

• OpenZeka: Comprehensive Qwen3.8-27B study on DGX Sparks (NVIDIA Developer Forums)
• dgx-spark-qwen38: SGLang + 4-bit + DFlash2 recipe
• kubesimplify: Running Qwen3.8-27B on DGX Spark
• 0xBakeer: Qwen3.8-27B 8-bit on a single DGX Spark
• ai-muninn: the DGX Spark bandwidth ceiling
• 4× DGX Spark with SGLang + DFlash2: crash bugs and workarounds (NVIDIA Developer Forums)
• MiaAI-Lab: Qwen3.8-27B SGLang recipe, model and cache sizes
• Inco AI: DFlash 2 release
• z-lab/Qwen3.8-27B-DFlash2 and RadixArk/Qwen3.8-27B-DSpark on Hugging Face
• SGLang docs: Anthropic API and Claude Code setup
• Memory creep on DGX Spark (NVIDIA Developer Forums)