Hot AI News

Tencent opens the base weights of Hy4 flash, a 220B model with 12B active

Tencent has put the pretrained base of Hy4 flash on Hugging Face. It is a 220B mixture-of-experts model with 12B active parameters, a 1M-token context and image input.

Tencent opens the base weights of Hy4 flash, a 220B model with 12B active
Image: Tencent (Hunyuan)

Tencent has published Hy4-flash-base on Hugging Face. It is the pretrained base of Hy4 flash, a mixture-of-experts model with 220B total parameters but only 12B active per token. It takes text and images and handles a context of up to 1M tokens.

The release is not a chat model. Tencent’s model card says the base is not instruction-tuned, and the chat and reasoning examples apply to the separate Hy4 flash and Hy4 flash-FP8 releases. Tencent says all three are open-weight on Hugging Face, ModelScope and GitCode. The license is the Tencent Hy License, tagged as “other” on Hugging Face, and the page is not gated.

MMLU-Pro, 5-shot, base models
Image: Tencent, chart by Aiholics

What is inside the 220B parameters

Of the total, 155.5B are the core model, 63.5B sit in an n-gram embedding table (Tencent calls it Engram), 0.43B in the vision encoder and 0.85B in a built-in drafter. Per token, the card lists 11.3B active text parameters plus 0.43B for vision.

The 42-layer backbone has one dense layer and 41 mixture-of-experts layers. Each token activates 16 of 512 routed experts plus one shared expert. Tencent lists three changes against the larger Hy4: sparse and sliding-window attention interleaved at 1:3, experts that work in a lower-dimensional latent space, and learned bigram and trigram embeddings retrieved by hash lookup. The sparse layers select up to 2,048 tokens, and the sliding-window layers use a 512-token window.

DFly2 drafter architecture reusing the KV cache and indexer-selected positions of sparse attention layers
Image: Tencent (Hunyuan)

Tencent says this cuts the KV cache to 1/24 of Hy3 and 1/6 of Hy4. The drafter, DFly2, is a speculative-decoding model that reuses the main model’s KV cache and sparse indices. Tencent says that keeps drafting cost constant as context grows. For serving, it points to vLLM and SGLang, with prebuilt images, and mentions a GGUF build for end-to-end testing. Transformers support is not yet in the main branch, so the card tells users to install from a pull request.

Advertisements

Base benchmarks: Strong on knowledge, mixed on code

Tencent’s few-shot table compares the base with Hy4 Base, Hy3 preview Base and DeepSeek’s V4-flash and V4-Pro bases. Hy4 flash Base scores 88.7 on MMLU, 71.1 on MMLU-Pro, 57.8 on SuperGPQA and 69.8 on U-MATH. That is above both DeepSeek bases on MMLU-Pro, SuperGPQA and U-MATH, but below the larger Hy4 Base in every knowledge row.

Blind evaluation of user satisfaction, completion time and cost per task
Image: Tencent (Hunyuan)

Code is less clear. On MBPP-plus the flash base scores 73.7, behind Hy3 preview Base (77.4) and both DeepSeek bases, while on BigCodeBench it scores 58.8, ahead of Hy4 Base at 47.9. Tencent warns that heavy synthetic instruction data in mid-training sometimes makes the base ignore the expected answer format, which may distort few-shot scores. On multimodal tests, only the two Hy4 bases are compared, so there is no outside reference.

Cost is the pitch, and the evidence is Tencent’s own

The cost figures on the page come from the instruct model and use an internal benchmark, WorkIndex, built around tasks from Tencent’s WorkBuddy product. Tencent reports 42.1 for Hy4 flash against 39.7 for Hy4 preview and 51.4 for Hy4, with cost per task at 1/13 of Hy4 preview and 1/20 of Hy4. In a blind test with 85 internal users, satisfaction was 2.79/4 against 2.81/4 for Claude Haiku 5.5, at RMB 0.26 per task. Tencent also says reinforcement learning cut average output tokens by 50% and that decoding passes 150 tokens per second. None of this has been independently checked, and the blind test used Tencent’s own staff.

The hosted price tells a similar story. As we reported on November 16, OpenRouter lists Hy4 flash at $0.04375 per million input tokens and $0.1756 per million output tokens, served only by Tencent Cloud. That is roughly a tenth of the discounted Hy4 rate, and a sixth of Baidu’s ERNIE-5.5-Flash-Preview at $0.28 per million input tokens. The open weights add what a hosted API cannot: teams can run the model themselves and fine-tune it.

Advertisements

What a base release does and does not give

A base checkpoint is mainly useful to researchers and companies that want to run their own fine-tuning. Tencent says a finetuning pipeline is documented, and the model needs real hardware: the vLLM example for the FP8 instruct build uses eight GPUs with tensor parallelism. Anyone who simply wants to chat with the model should use the instruct version.

Open questions remain. The card does not publish training data details, the full license terms are only linked, and the headline cost and quality claims rest on internal evaluations. Tencent recommends the flash model for everyday work such as deep search, documents, slides and computer use, and the larger Hy4 for long-horizon software engineering.

Sign Up for the Daily AI Pulse

One email a day. All the stories that matter.

Alex CarterAI News & Big Tech Correspondent

Alex Carter writes for Aiholics, keeping readers updated on the fast-paced world of AI and Big Tech.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may also like