Nvidia’s open speech model takes written prompts, and tuning cut medical errors 62%

Nvidia has released Nemotron 3.5 Transcribe, an open-weight speech model that transcribes 40 languages and follows plain-language instructions, together with Granary v2, a public manifest of 463,062 hours of speech in 48 languages, according to Nvidia’s post on Hugging Face.

A 30B model that activates only 4B parameters

The checkpoint is named NVIDIA-Nemotron-3.5-Transcribe-30B-A4B. Nvidia describes about 30 billion total parameters with roughly 4 billion active per token, although the Hugging Face listing shows 34B. A speech encoder feeds a hybrid Mamba-Transformer mixture-of-experts language model, and speaker cues from Nvidia’s Sortformer component are passed to the language model alongside the acoustic features.

Beyond transcription in 40 languages, the model translates 31 languages to or from English and labels speakers in 13 languages, for up to eight speakers. It also gives word-level timestamps and can answer questions about audio, although that last skill is trained in English. Recordings can run up to an hour. Nvidia says it is built for recorded audio, not live streaming, so it does not replace a real-time captioning engine.

The model is steered with natural-language instructions rather than fixed settings. In a medical test, simply telling it the speech was English written in the Latin alphabet cut word error rate (WER) on the Eka Care benchmark from 30.8% to 23.8%, according to the post.

It runs on a single Nvidia GPU, from Jetson AGX Thor and DGX Spark up to Blackwell data-center cards. On one RTX PRO 6000 Blackwell Server Edition with NVFP4 precision, Nvidia says it processes 100 hours of audio in under three minutes, using 30-second clips at high concurrency.

Benchmarks Nvidia cites, and who checked them

On the Hugging Face Open ASR leaderboard, Nvidia reports an English average WER of 5.33%, first among open-weight models. It also reports 4.62% across seven languages and 10.98% on long-form audio, both the lowest in the model card’s own comparison. Artificial Analysis lists it tied for first among open-weight models at 2.8% WER, with the tied model having about 24B active parameters.

For speaker attribution, David AI’s 21-language test gives a diarization error rate of 18.26%, narrowly ahead of Nvidia’s own Nemotron 3 Diarization at 18.38%. VoiceArena’s English test gives 32.4% cpWER, first of 14 systems. Most of these are third-party leaderboards, but Nvidia chose which to show, and the multilingual comparison comes from its model card.

Fine-tuning on 14 GPU-hours beat larger rivals

The adaptation case study is the most practical part. Using about 153 hours of synthetic speech, 29 hours of accented English from Common Voice and 63 hours of medical recordings, fine-tuning on one GB300 for about 14 GPU-hours lowered full-set WER to 11.8%, roughly 62% fewer errors than out of the box. On held-out speakers the drop was from 22.44% to 12.17%, which Nvidia calls a 45.8% relative reduction.

On the same held-out set, an open medical-specialist model scored 13.79% and a general-purpose ASR model 14.92%. The cost was small but real: WER on LibriSpeech test-clean rose from 1.54% to 1.69%. The study is Nvidia’s own and covers one domain, so it shows the method works rather than proving it will for every accent or industry.

Granary grows from European languages to 48

The dataset side matters for technical reasons. Speech models are limited less by architecture than by clean, labeled audio in many languages. Granary v1, announced in August 2025, covered 25 European languages and about a million hours, with roughly 650,000 hours for recognition and 350,000 for translation, according to Nvidia’s blog at the time. Nvidia said then that its Canary-1b-v2 model extended that family from four to 25 languages. Granary v2 widens the language count to 48 but lists 463,062 hours, and Nvidia notes these figures describe the release, not the full training mix of the new model.

Version 2 also adds task labels such as audio captions, restored loanwords in code-switched speech, speech question answering and contextual biasing terms, and translations with quality scores. A High Fidelity subset filters aggressively: Nvidia says it shrank VoxPopuli from 354,000 to 111,000 hours across 22 European languages. The pipeline is built with NeMo Curator, so others can rerun it on their own audio.

That reflects a wider Nemotron push. Nvidia has recently shown a 4GB Nemotron 3.5 Bolt model for local assistants and a retrieval trio a week earlier. Speech now joins that family.

What you can use today, and what is unclear

The checkpoint is on Hugging Face and ModelScope, with a demo Space and hosted endpoints from Baseten, DeepInfra and Simplismart. Deployment guides cover vLLM, and the NeMo Speech repository has a fine-tuning guide. Granary v2 is manifests only: developers must download source audio separately and follow each corpus’s original license.

The post does not name a license for the model, dataset or code, calling the model only open-weight, so commercial terms should be checked on the model card before use. Pricing for the hosted endpoints is not given, and there is no latency or accuracy data for live use. The leaderboard rankings are Nvidia’s selection, and the medical gains come from a single benchmark.