NVFP4: How 4-bit numbers let huge AI models fit in far less memory

NVFP4 is a way of writing the numbers inside an AI model using only 4 bits each, instead of the usual 16. Nvidia introduced it with its Blackwell GPUs (graphics processors) in 2025. Fewer bits per number means a smaller model, less memory traffic and faster output, at a small cost in precision.

The format appeared in our weekly roundup because Meta published its Muse Spark Open weights in two versions: BF16 and NVFP4.

Why a model’s size depends on how its numbers are written

A model is mostly a huge list of numbers called weights. Each one takes up space, and the space depends on the format. BF16 (bfloat16) uses 16 bits per number. A 4-bit format uses a quarter of that.

Think of measuring a room. A tape measure marked to the millimeter gives a precise answer but is long to read. A ruler with only a few marks is quick to read, but you must round every measurement. Shrinking weights to 4 bits is the second ruler.

A rough calculation shows the stakes. Meta’s model has about 816B (billion) parameters. At 16 bits, the weights alone come to roughly 1.6 TB. At 4.5 bits per value, the NVFP4 average described below, they come to roughly 460 GB. This is simple arithmetic on one assumption, that every weight is stored this way, and it is not a figure from Meta.

Sixteen possible values, each group with its own ruler

Four bits can only describe 16 different values. NVFP4 uses a layout called E2M1: one sign bit, two exponent bits and one mantissa bit. Nvidia’s blog lists the magnitudes it can represent as 0, 0.5, 1, 1.5, 2, 3, 4 and 6, plus their negatives. That range is far too coarse on its own.

The fix is scaling. NVFP4 splits the weights into small blocks of 16 values. Each block gets one shared scale factor, stored in FP8 (an 8-bit float format). The block’s numbers are stretched or shrunk to fit the tiny 4-bit range, then restored when used.

A second, larger scale sits on top: one FP32 (32-bit float) number per whole tensor, which keeps the block scales in a range where they can be stored well. In the analogy, every group of 16 numbers gets a ruler cut to its own size.

Those extras add up. Nvidia puts the cost at 4.5 bits per value on average. The company reports about 3.5 times less memory than FP16 and about 1.8 times less than FP8.

From the older MXFP4 standard to an Nvidia refinement

Block scaling was not new. An industry format called MXFP4 (microscaling FP4) also uses 4-bit values with shared scales, but its blocks hold 32 values and its scales are limited to powers of two. Nvidia’s blog says NVFP4 halves the block size to 16 and uses an FP8 scale that allows fractional steps, giving a closer fit.

NVFP4 first targeted inference, meaning running a finished model. In a June 24, 2025 technical blog, Nvidia compared DeepSeek-R1-0528 in FP8 and NVFP4 after post-training quantization, which converts an already-trained model. It reported 1% or less accuracy loss on key benchmarks. For example, MMLU-PRO scored 85 in FP8 and 84 in NVFP4.

Training came next. In a paper posted September 29, 2025 (revised March 2026), Nvidia and dozens of coauthors described training a 12-billion-parameter model on 10 trillion tokens in NVFP4. They call it the longest publicly documented 4-bit training run. They report training loss and downstream accuracy comparable to an FP8 baseline.

The recipe needs several safeguards:

• Random Hadamard transforms spread out extreme values within blocks.
• Two-dimensional quantization keeps the weight representation consistent between the forward and backward passes.
• Stochastic rounding makes rounding errors in gradients average out instead of piling up in one direction.
• A few sensitive layers stay in higher precision.

Where it shows up today

Meta’s Muse Spark Open, a sparse mixture-of-experts model with about 816B total parameters, ships NVFP4 and BF16 checkpoints. The same release has community llama.cpp and MLX quantizations, which are other low-bit formats for laptops and desktops. They are separate from NVFP4.

Software support is growing. Nvidia’s 2025 blog lists TensorRT Model Optimizer and LLM Compressor for quantizing, and TensorRT-LLM and vLLM for serving. That list is a year old, so check current documentation. Our weekly roundup also names vLLM and SGLang as ways to deploy Muse Spark Open.

What 4 bits cost, and what to check before trusting a number

The accuracy evidence has limits. Nvidia’s inference results cover one model, one conversion method and seven benchmarks. Its energy and throughput claims are the company’s own, and the blog does not detail how they were measured. Nvidia itself notes that 4-bit formats carry a risk of a noticeable accuracy drop versus FP8, and it says the risk is lower for NVFP4, particularly for larger models.

Hardware matters too. NVFP4 is built around Blackwell-generation Tensor Cores, the units that do the matrix math. On other chips it may need slower software paths or give no speed gain. The memory saving still applies, but the speed gain is not guaranteed.

A smaller file also does not mean a better model. A 4-bit copy can only approximate its 16-bit original, so it is worth testing on your own tasks.

What to watch next

Three things are worth following. First, independent tests of NVFP4 on long-context and agent tasks, where small errors can compound over many steps. Second, whether other chip makers adopt similar block-scaled formats, as MXFP4 already shows some interest. Third, whether more labs train in 4 bits from the start, rather than converting afterwards, since the 2025 paper shows this is possible but demanding.