Nvidia’s Nemotron 3.5 retrieval release: Five open models, an 8B embedder at 61.17 on ViDoRe

Nvidia has released Nemotron 3.5 Embed, Rerank and Parse on Hugging Face: five retrieval models for AI agents. The 8B embedder scores 61.17 nDCG@10 on ViDoRe v3.1, which Nvidia says is first among dense retrieval models.

The release covers three jobs in the pipeline that feeds documents to an agent: turning text and images into searchable vectors, re-ordering the results, and converting documents into structured text. All five models are on Hugging Face, and Nvidia calls them open and commercially available.

Five models, from a 631M parser to an 8B reranker

The family has two embedding tiers. Nemotron-3.5-Embed-8B-BF16 is the quality model for text and image retrieval. It builds on Ministral-3-8B-Instruct. Two 1B versions are distilled from it: a BF16 build and an NVFP4 build with quantized text and image encoders. Nemotron-3.5-Rerank-8B-BF16 is a cross-encoder on the same backbone with an MLP scoring head. Nemotron-3.5-Parse, at 631M parameters, extracts structured text from documents.

Nvidia says the language component stays frozen while the embedders train, so text-only embeddings match those of Nemotron 3 Embed. In practice that means teams with an existing index should be able to keep it. Text retrieval was evaluated across 34 languages, and Parse across 163 language and script combinations.

On ViDoRe v3.1, the 8B embedder beats Nvidia’s earlier Llama Nemotron Embed VL 1B (52.23) by 8.94 points. The 1B BF16 model scores 58.38, which is 95.4% of the 8B teacher. On the multilingual MIRACL Vision benchmark the 8B model reaches 72.6, 4.3 points above Qwen3-VL-Embedding-8B, according to Nvidia.

The reranker scores 60.49 on ViDoRe v3.1 against 59.17 for Qwen-3-VL-Reranker-8B, working on the top 50 candidates from the 1B embedder. In agent tests, adding it cut inference tokens per query by about 14% with the 8B embedder and 13% with the 1B one. Nvidia describes the Parse model as ranking first on ParseBench among models under 1 billion parameters: 72.74 in a single pass at 16.1 pages per second on an H100, and 75.20 with three passes at 10.5 pages per second.

Quantization and serving numbers for production teams

The NVFP4 1B model is aimed at throughput. On a GB200 system it processes 128.8 documents per second against 101.3 for BF16, a 27.1% gain, while keeping 99.85% of the ViDoRe v3 accuracy, per Nvidia. Served through NIM on an RTX PRO 6000 Blackwell Server Edition, the models handle 6,031 query requests per second against 2,099 for vLLM, roughly 2.9 times as many, with comparable passage throughput.

Beyond Hugging Face, the models are listed on build.nvidia.com and NGC, and through Baseten, DeepInfra and FriendliAI. Elastic exposes them through its Elasticsearch Inference API, with Nemotron 3 on its inference service coming soon. For fine-tuning, Nvidia points to NeMo AutoModel and its embed and rerank recipes. On a set of 8,202 chart and infographic images with 8,591 synthetic queries, fine-tuning lifted the 1B model from 37.34 to 44.12 nDCG@10, but the 8B only from 41.44 to 42.75.

Partners report gains too. Vectorize cites 93.8% recall@10 on LoCoMo for its Hindsight memory system, and Turbopuffer says it saw about 5% more relevant results at Recall@20 than the next-best model. Perplexity said the reranker had the highest average on its private 72-task benchmark, without publishing a score.

Open rivals set the bar on license and context

The closest open comparison is the Qwen3-VL-Embedding-8B model that Nvidia benchmarks against. Its Hugging Face model card lists an Apache-2.0 license, a 32K context window and support for 30+ languages, with embeddings up to 4096 dimensions. Nvidia’s post names no license and states no context length, and refers readers to each model card. For teams choosing between the two, those are the details to check first, since terms and input limits can matter as much as a point or two of nDCG.

The comparisons also need care. Qwen reports its own results on a different test, MMEB-V2, where the card gives an overall 77.9, so the two vendors’ headline numbers cannot be set side by side. The head-to-head figures here, MIRACL Vision and ViDoRe, come from Nvidia’s own evaluation and have not been independently verified. The ParseBench claim is limited to models below 1B parameters.

The release also fits a wider Nvidia push into open agent models. The company recently put out its 120B Super VL agent model with open weights, priced at $0.10 per million tokens on hosted access, according to our earlier report. Retrieval models are the other half of that stack: an agent that reads fewer, better documents spends fewer tokens, which is the pitch behind the reranker’s 13% to 14% saving. Nvidia has not said how that saving holds up outside its own agent tests.

Nvidia does not give prices for the NIM and partner endpoints in the post, so cost on hosted access is unclear. Anyone who wants to try the models can download them from Hugging Face now.