Liquid AI’s 328M draft model makes its 24B MoE up to 3.1x faster

Liquid AI has published LFM2.5-24B-A2B-DSpark, a small draft model that speeds up its 24B mixture-of-experts model. On an NVIDIA B300 running the FP8 target in SGLang, throughput averages 2.4x the baseline for a single request, and reaches 3.1x at best.

A 328M drafter that the big model checks

The release is not a new chat model. It is a speculative-decoding draft model: it guesses a block of upcoming tokens, and the target, LFM2.5-24B-A2B, verifies them. Because the large model accepts or rejects every proposed token, the idea is to spend less time waiting for one token at a time.

According to the model card, the draft network has 327.7 million parameters in BF16. It uses five full-attention layers, a hidden size of 2048 and grouped-query attention with 32 query heads and 8 key-value heads. Two extra components sit on top: a Markov head (rank 256) and a confidence head. The block size is 9, and the vocabulary is 128,000 tokens.

Liquid AI reports tokens per step averaging 5.16 across eight benchmarks at concurrency 1, counting the target’s bonus token, with a ceiling of 10. Throughput gains over the target alone vary a lot by task:

• MBPP: 3.08x at concurrency 1 and 3.28x at 32, the best results in the table.
• LiveCodeBench: 1.87x at concurrency 1, falling to 1.34x at 256.
• MATH-500: 2.63x at concurrency 1 and 1.79x at 256.
• Arithmetic mean: 2.38x, 2.69x and 1.71x at concurrency 1, 32 and 256.

Each shard used one B300, the FP8 target, a BF16 KV cache and temperature 0, and throughput is given per GPU. The card also cautions that small numerical differences between batched verification and ordinary decoding may alter the generated text, even with greedy sampling, and that the figures measure speed, not task accuracy.

Running it takes a recent SGLang build

The weights are on Hugging Face, but serving them is not yet a one-line affair. The card says you need an up-to-date SGLang build that includes two pull requests: #31041, which adds DSpark handling for LFM2-MoE models, and #43939, which lets SGLang read Hugging Face configurations natively. The launch command points --model-path at LiquidAI/LFM2.5-24B-A2B-FP8, sets --speculative-algorithm DSPARK, attaches the draft with --speculative-draft-model-path, and uses --speculative-dspark-block-size 9 with --speculative-num-draft-tokens 10. Dropping the --speculative-* flags gives the baseline. The server then exposes an OpenAI-compatible endpoint.

This is the fifth member of a DSpark family. The card lists drafters for LFM2.5-1.2B-Instruct, LFM2.5-2.6B, LFM2.5-VL-3B, LFM2.5-8B-A1B and this 24B model. The target, according to its own model page, has about 2.45B active parameters out of 24B, a 131,072-token context and ships under Liquid AI’s LFM 1.0 license. The card for the draft model does not state its own license in the text we could read, so teams planning commercial use should check the repository.

Why a mixture-of-experts model gains less under load

Speculative decoding pays off when generation is limited by memory traffic rather than arithmetic: verifying nine guesses in one pass costs little more than producing one token. That explains the pattern in the table. At concurrency 1 the GPU is underused, so extra verification work is nearly free. At concurrency 256 the batch is already large, the hardware is busy, and rejected guesses become wasted compute, so the average gain falls from 2.38x to 1.71x.

Liquid AI’s earlier DSpark post, covering the smaller models, describes how the design tries to raise acceptance. A single forward pass produces the whole block of draft tokens, and the Markov head models dependencies between neighboring tokens, which the company says improves acceptance at later positions. The confidence head estimates whether each draft token will be accepted so a scheduler could prune weak suffixes, but the company said its benchmark runs skipped it and used a fixed verify window. The B300 results for the 24B model therefore reflect the simpler setup.

The smaller-model numbers help put the new ones in context. In that post Liquid AI reported mean speedups of 2.54x for the 8B-A1B model on an H100 at batch size 1, but only 1.18x for the same model on a MacBook M4 Max with llama.cpp, a gap it blamed on the current mixture-of-experts implementation in llama.cpp’s Metal backend. Sparse models are harder to accelerate than dense ones, and the 24B results are so far shown only on NVIDIA’s data-center hardware.

For the target itself, Liquid AI’s model page cites 184 tokens per second on an Apple M5 Max, 114 tokens per second on an AMD Ryzen AI Max+ 395 and up to 21,000 output tokens per second on a single B300 at high concurrency. The company has not published DSpark results for 24B on laptops, so it is unclear how much of the speedup carries over to the on-device agents the model is pitched at.

What teams can and cannot conclude yet

For anyone serving the 24B model on B300-class GPUs with low or moderate concurrency, a roughly 2.4x to 2.7x gain from a 0.3B add-on is a cheap way to cut latency, and it comes without retraining the target. At very high concurrency the benefit is smaller, in line with the broader push toward cheaper self-hosted serving covered in our report on llm-d 1.0. Because the target model’s weights are downloadable but not rebuildable from scratch, as our explainer on open-weight models describes, a separately released drafter is one of the few speed levers outside teams can pull.

Caveats remain. All figures are Liquid AI’s own and have not been independently reproduced. The tests use one GPU type, one precision and temperature 0, and the card says outputs can differ from plain decoding. Support depends on the SGLang changes the card names as prerequisites, and nothing in the source covers vLLM or llama.cpp for this specific drafter.