Hot AI News

Tencent’s 770B Hy4-base tops DeepSeek’s V4 Pro base in 18 of 21 tests

Tencent has put the weights of Hy4-base, its flagship 770B mixture-of-experts base model with a 1M-token context, on Hugging Face; the benchmark numbers are its own.

Tencent’s 770B Hy4-base tops DeepSeek’s V4 Pro base in 18 of 21 tests
Image: Tencent (Hunyuan)

Tencent’s Hunyuan team has published Hy4-base on Hugging Face, the pre-trained base of its flagship Hy4 family. The mixture-of-experts model has about 770B parameters, activates 49B of them per text token, and handles a 1M-token context.

The model card describes Hy4 as a model that understands text, image and video. It also states plainly that this release “is a pre-trained base model, not an instruction-tuned chat model.” Tencent lists instruct versions, Hy4 and an FP8 variant, as separate releases in the same collection.

Bits-per-byte chart for the Hy4 base model on internal code and agent validation sets
Image: Tencent (Hunyuan)

A 770B backbone that wakes up 49B per token

According to the card, the 770B figure covers the language backbone only. It leaves out the vision encoder, the adapter and a drafter model. On top of the 49B active text parameters, each token passing through the vision path uses 1.23B more.

The backbone has 78 layers, one dense and 77 mixture-of-experts. Each expert layer holds 256 routed experts plus one shared expert, and 8 routed experts fire per token. Hidden size is 6,144, and the vocabulary is 120,832 tokens.

Diagram of the DFly2 drafter architecture used with Hy4
Image: Tencent (Hunyuan)

For long inputs, Tencent uses a gated version of DeepSeek Sparse Attention with an IndexCache, built on multi-head latent attention. Each token selects the top 2,048 positions to attend to. The card also lists a 40-layer vision encoder with 1.23B parameters and a 3.3B drafter called DFly2, used for speculative decoding. Residual connections use a scheme Tencent calls iHC, with four streams.

The weights are on Hugging Face, with ModelScope and GitCode listed as other hosts. The license is the Tencent Hy License, whose full text sits in the repository. The card links a finetuning guide, a system card PDF and GGUF files from the AngelSlim team, which it labels as meant for end-to-end testing. A model this size is not a laptop download: the card’s own serving example for the instruct model uses tensor parallelism across 8 GPUs.

Tencent's 770B Hy4-base tops DeepSeek's V4 Pro base in 18 of 21 tests
Image: Tencent
Advertisements

Tencent’s own table: 18 wins in 21 text tests

The card compares Hy4 Base with its smaller sibling Hy4 flash Base, the older Hy3 preview Base, and DeepSeek’s V4-flash and V4-Pro bases. In Tencent’s few-shot table, Hy4 Base posts the top or a tied-top score in 18 of the 21 text rows. DeepSeek-V4-Pro-Base leads in three: OCW Courses (68.2 against 66.7), MBPP-plus for code (79.1 against 74.0) and the Belebele multilingual subset (95.2 against 95.0).

The largest gaps are on knowledge questions. On SimpleQA, Hy4 Base scores 57.2 against 52.1 for V4-Pro-Base and 31.3 for V4-flash-Base. On SuperGPQA it reaches 62.7 against 54.8 and 52.5. MMLU-Pro is closer: 73.9 against 73.4 for the V4-Pro base.

Six multimodal benchmarks, such as MMMU-Pro at 59.6 and TextVQA at 78.2, are compared only with Hy4 flash Base, because the DeepSeek and Hy3 columns have no entries there. Tencent also warns that the base model saw a lot of synthetic instruction data during mid-training and sometimes breaks the expected answer format in few-shot tests, which “may distort the scores.” All numbers are Tencent’s and have not been independently reproduced.

A flagship that follows its cheaper sibling by a day

The release completes a pair. Tencent opened the weights of the smaller Hy4-flash-base a day earlier, a 220B model with 12B active parameters that we covered on November 18. The two share the 1M context and the vocabulary, but the architecture differs. The flash model uses a hybrid of sparse and sliding-window attention and 512 routed experts, while the flagship uses sparse attention in every layer with 256 experts. In the card’s table, the flash base trails the flagship on nearly every row, for example 37.5 against 57.2 on SimpleQA.

Price is where the family has been tested in public. Tencent’s Hy4 flash was listed on OpenRouter at $0.04 per million input tokens with the 1M context kept, and an earlier piece on the Hy4 preview listing reported it at half the preview’s price. Those are instruct models served through a third party. A base model has no list price: whoever downloads Hy4-base brings their own hardware and their own post-training.

That is the practical meaning of this release. Base weights suit labs and companies that want to continue pre-training, fine-tune on private data, or study how the model was built. They do not suit someone who wants a chat assistant today.

Advertisements

What the card leaves out

The page gives no pre-training token count and no description of the training data beyond the note about synthetic instruction data. It does not publish scores for the instruct model on the same tests, so the base table says little about how Hy4 behaves in conversation. The vendor figures on the card for the instruct model, such as a hallucination rate falling from 15.76% to 5.45% against the preview, come from Tencent’s own evaluations and are not covered by the base release.

Licensing terms also deserve a read before commercial use. The card names the Tencent Hy License rather than a standard open-source license, and this article does not summarize its restrictions.

Sign Up for the Daily AI Pulse

One email a day. All the stories that matter.

Alex CarterAI News & Big Tech Correspondent

Alex Carter writes for Aiholics, keeping readers updated on the fast-paced world of AI and Big Tech.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may also like