Meta opens up an 816B-parameter multimodal model: Muse Spark Open is here

Meta has released Muse Spark Open, an open-weight multimodal mixture-of-experts model with about 816 billion total parameters and 42 billion active per token, according to a Hugging Face blog post published on October 15. The post calls it “the most capable US open-weights model to date.”

What is new

Muse Spark Open is built for agentic coding: reading codebases, writing and debugging code, and calling tools. It accepts text, images, audio and video as input, including video with audio tracks, and produces text only.

The architecture is a sparse mixture of experts. Each of its 62 layers holds 256 experts, and 8 are selected for every token. The experts account for roughly 97% of the parameters. Attention uses 128 query heads and 16 key-value heads, with a hidden dimension of 8192.

Vision is handled by a separate 1.8-billion-parameter perception encoder for images and video. The audio encoder emits one token every 80 milliseconds, or 12.5 tokens per second.

The context window reaches 1,048,576 tokens, counting image, video and audio tokens. The post estimates that a full 1M-token key-value cache in BF16 takes about 266 GB of memory, on top of a BF16 checkpoint of about 1.5 TB.

Reasoning effort can be set to low, medium, high or xhigh, with xhigh as the default. Reasoning cannot be switched off. As the post puts it: “Note that turning off reasoning is not possible with Muse Spark Open.”

Checkpoints, hardware and tooling

Several formats are available from day one. Besides the full BF16 weights, there is an NVFP4 version for Nvidia Blackwell GPUs, GGUF quantizations for llama.cpp and 3-bit and 4-bit MLX builds for Apple Silicon.

The post lists the hardware it used. Running the BF16 weights in Transformers took four nodes of eight H200 GPUs each. The NVFP4 weights run on a single B200 node, and SGLang can serve them on four GB300 GPUs. Hugging Face‘s Inference Endpoints offer a pre-validated NVFP4 recipe on eight RTX Pro 6000 cards with a 131,072-token context, or on eight H200s for the full 1M tokens.

At the small end, the post reports about 18 tokens per second from a 2-bit llama.cpp quantization on a Mac M3 Ultra with 512 GB of memory. An MLX 3-bit build on the same machine managed about 21 tokens per second of generation.

Software support spans Transformers, llama.cpp, SGLang and vLLM, plus the OpenAI-compatible Chat Completions and Responses APIs on Inference Endpoints. For fine-tuning, the post demonstrates TRL’s supervised trainer with LoRA on the attention projections while the experts stay frozen. In its example run on eight nodes of eight H100 GPUs, 500 steps took about two hours. Mean token accuracy rose from 72% to 84% and loss fell from 1.26 to 0.57.

What the post does not say

The page reports no benchmark results, so the “most capable” claim cannot be checked against competing open models from the material provided. It also does not state a license, which matters for anyone planning commercial use. Meta’s own announcements on the subject had not covered the release at the time of writing.

Why it matters

The two open questions in the announcement, the license and the benchmarks, were partly answered elsewhere on launch day, and the answers sharpen what to watch next.

On licensing, VentureBeat reports that the weights come under a custom Muse Model License rather than a standard open-source one. It permits running, fine-tuning, distilling and redistributing the model. Model-as-a-service businesses with more than $20 million in gross revenue over any 12 months must notify Meta and sign a separate agreement. Products must also carry a “Built with Muse Spark Open” notice. Internal use and embedding the model in a company’s own product are exempt. The services to watch are therefore the large hosting providers, which are the ones that would need that agreement.

On performance, the “most capable US open-weights model” claim looks narrower once the numbers are on the table. In Meta’s own table, as relayed by VentureBeat, the model leads rivals on MCP-Atlas (89.2) and JobBench (61.8). It trails Kimi K3 on Terminal-Bench 2.1 (76.4 against 85.0), a coding test, even though the model is pitched at agentic coding. Trending Topics notes that Meta’s score of 39 on the Artificial Analysis Intelligence Index had not been independently confirmed, and that Xiaomi’s MiMo-V2.6-Pro scores 46 on that index.

The confirming or contradicting evidence will be an independent Artificial Analysis listing for the 39, evaluations run outside Meta’s own agent harness, and whether third-party hosts start serving the model under the license terms.