ByteDance opens 17 GB sparse autoencoders to probe what Qwen3’s MoE experts do

ByteDance’s Seed team has published three sparse autoencoders for Qwen3-30B-A3B on Hugging Face, each with 2^20 latents, trained on the output of the model’s mixture-of-experts blocks. The files are released under the Apache 2.0 license.

Three layers, one million features each

The release covers layers 4, 30 and 46 of the model, counted from zero. According to the model card, each autoencoder reads the output of the MoE block of its layer, before the residual addition, with no normalization applied. Each one uses a TopK activation with k set to 100, so for every token only the 100 strongest of 1,048,576 latents stay active.

Every checkpoint is about 17.2 GB and holds four float32 tensors: an encoder matrix of 2,048 by 1,048,576, its bias, a decoder matrix with unit-norm rows, and a decoder bias. The 2,048 figure is the width of the hidden state the autoencoder reads. Latents are named by layer and index; the card gives 30-F97783 as an example.

The card includes a short Python recipe. It loads one layer’s weights with safetensors, attaches a forward hook to model.model.layers[LAYER].mlp in the Transformers implementation of Qwen3-30B-A3B, and encodes the captured activations. The card warns that the dense pre-activation costs 4 MiB per token, so long inputs should be processed in chunks.

According to the card, each autoencoder was trained independently on about 3.2 billion tokens of MoE outputs. The data came from the Olmo 3 training corpus: five eighths Dolma 3 pretraining text, a quarter Dolci-Think-SFT reasoning traces and an eighth Dolci-Instruct-SFT instruction data. The k value was lowered from 1,000 to 100 over the first 10,000 steps, and an auxiliary loss was used. Ai2 distributes those datasets under the ODC-BY v1.0 license, which the card notes.

Reconstruction is far from perfect at layer 30

The card reports the fraction of variance unexplained, as measured in the accompanying paper: 12.6% at layer 4, 25.1% at layer 30 and 7.6% at layer 46. In plain terms, the middle layer is the hardest to reconstruct from 100 active latents, and a quarter of its variance is not captured. Researchers who plan to splice the reconstruction back into the model should expect that gap to matter. These figures are ByteDance’s own, and the source does not say how they affect downstream behavior of the model.

The paper, titled “Sparse AutoEncoders Reveal That MoE Experts Exhibit Human-Interpretable Fine-Grained Specialization,” is credited to Anyuan Zhuo, Shaohua Zhang, Yuan Lin and Pinyan Lu. The card lists the paper link and journal as “TODO”, so the findings behind the title cannot yet be checked. The page points to a project page and code, but the claim in the title rests on a paper that is not yet available through the card.

The price of looking inside is memory

The main cost is hardware. The card says one autoencoder takes about 17 GB on top of the model, which suggests a GPU with comfortable headroom even for a single layer. According to the Qwen3-30B-A3B model card on Hugging Face, the model has 30.5 billion parameters, 3.3 billion of them active per token, spread over 48 layers with 128 experts and 8 experts chosen per token. An autoencoder with a million latents for a block that routes among 128 experts gives researchers roughly eight thousand features per expert on average, though the source does not say how the latents map to experts.

For scale, ByteDance published a second interpretability release on the same day, a cross-layer transcoder for the much smaller dense Qwen3-8B-Base. Its page lists 94.2 billion parameters in fp32 and 377 GB in total, with 357 GB of that in decoder weights. Against that, three 17.2 GB files, about 52 GB together, are a lighter download, and the card puts the Qwen3-30B-A3B autoencoders under the same Apache 2.0 terms as the base model.

Before this release, anyone wanting to study expert specialization in this model would have needed to train their own autoencoders on billions of tokens. Now the weights, training recipe summary and loading code are public, though the full recipe sits in the paper’s appendix, which is not yet linked. The release is a research tool, not a product: no inference provider hosts it, and the page shows no download statistics.

What the source does not establish is how well the latents hold up under independent testing, or whether the “human-interpretable” claim in the paper’s title applies at all three layers or only to selected features. Those answers depend on the paper and on outside researchers using the files.