Hot AI News

Nvidia’s Nemotron recipe: RL lifted drug-metabolism accuracy 17% past fine-tuning

Nvidia published a recipe for post-training Nemotron 3.5 Super VL on a lab's own experimental data. In a drug-metabolism test, SFT added 12% and RL a further 17%.

Nvidia’s Nemotron recipe: RL lifted drug-metabolism accuracy 17% past fine-tuning
Image: Nvidia

Nvidia has published a step-by-step recipe for adapting Nemotron 3.5 Super VL to a research lab’s own data. On a drug-metabolism task, supervised fine-tuning raised accuracy by 12% and reinforcement learning added a further 17%, Nvidia says.

A lab’s CSV becomes a training environment

The guide is part of what Nvidia calls Nemotron for Science, a collection of models, datasets, tools and recipes for agentic scientific work. It starts from a CSV of experimental records, such as molecular structures written as SMILES strings paired with measured outcomes. Each row is turned into a question and answer pair, and the measured result acts as the verifier. Nvidia describes this as reinforcement learning with verifiable rewards, which needs no human preference labeling.

Chart of mean Tanimoto similarity before and after post-training Nemotron 3.5 Super VL on drug metabolism
Image: Nvidia

The worked example predicts a drug metabolite. The model gets a parent compound and the enzyme that transforms it, reasons through the reaction and submits a predicted structure through a tool call. The verifier compares molecular fingerprints from the prediction and the reference using Tanimoto similarity, a score from 0 to 1. The example data comes from MetxBioDB, a database of experimentally confirmed biotransformation reactions.

Training uses group relative policy optimization (GRPO) with low-rank adapters (LoRA). NeMo Gym supplies the environment and NeMo RL the training framework. The post includes the config and launch pieces. Only the resources server, which holds the scoring function and tool schema, is domain-specific. The example launch runs on 16 GPUs across four nodes with run_grpo_nemo_gym.py, and the policy model is the 120B Nemotron 3.5 Super VL checkpoint.

Chart of four models on 11 science benchmarks, with Nemotron 3.5 Super VL leading on seven
Image: Nvidia
Advertisements

Claude Sonnet 5.5 as an optional teacher

If the base model needs a boost or RL converges slowly, Nvidia suggests a distillation step first. A stronger teacher model runs against the same environment, and only reasoning traces whose answers match the ground truth are kept. The teacher in the example is Anthropic’s Claude Sonnet 5.5. Nvidia kept 1,123 traces from 1,934 training prompts at a Tanimoto threshold of 0.7 or higher. They then fine-tune the smaller model before RL starts, a “warm-start” that Nvidia says typically speeds convergence.

Supervised fine-tuning improved accuracy by 12% and RL lifted it a further 17%, at a 16,384-token output budget.

Nvidia's Nemotron recipe: RL lifted drug-metabolism accuracy 17% past fine-tuning
Image: Nvidia

Nvidia notes that filtering at an exact match (score of 1.0) keeps only traces where the teacher got the metabolite exactly right, while a softer threshold allows more variation. It also advises switching on the importance-sampling correction option in the loss function. According to the post, this narrows the gap between the vLLM engine that generates rollouts and the Megatron engine that trains, a concern that weighs most on mixture-of-experts models.

What 120B parameters cost to adapt

The analysis angle here is cost, and the post itself gives no dollar figures. Context comes from our earlier coverage of an AWS deployment guide: Nemotron 3.5 Super VL has 120B total parameters with 12B active per token, and serving it in BF16 needs at least 16 H100s or 8 B200s, according to that guide. Nvidia’s training example uses 16 GPUs across four nodes, so the recipe targets teams with cluster access rather than a single workstation. The LoRA adapters and a verifier that is cheap to compute, such as a fingerprint comparison, keep the reward side light. The post notes that richer scoring, from simulators to a live autonomous lab, can be plugged in at higher cost.

The distillation step adds a different cost: calls to a frontier model. The example samples a teacher repeatedly per prompt (the script uses --num-repeats 8) over 1,934 prompts, and about 58% of prompts yielded a usable trace at the 0.7 threshold. Teams would pay for that API usage in return for a better starting point, a trade the post does not quantify.

Several things remain unestablished. The 12% and 17% gains are Nvidia’s own results on one task, and the post does not say whether they are relative or absolute, or which score they refer to beyond mean molecular similarity. There is no independent replication, and the post does not say how the gains compare with the teacher model alone or with RL started without SFT. Held-out evaluation is the check Nvidia itself recommends before trusting any gain.

Advertisements

Who can try it today

The tooling is open: Nvidia points readers to the NeMo Gym and NeMo RL repositories and says the recipe can serve as a template for other scientific tasks. Nemotron 3.5 Super VL is described as an open, customizable model, and the post also introduces the Nemotron Litmus-Bench V2.0 chemistry-reasoning dataset and benchmark. The post gives no pricing, license terms or hardware lead times. Nvidia advises starting with a narrow task, validating the scoring method and expanding only as the evidence supports it.

Sign Up for the Daily AI Pulse

One email a day. All the stories that matter.

Alex CarterAI News & Big Tech Correspondent

Alex Carter writes for Aiholics, keeping readers updated on the fast-paced world of AI and Big Tech.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may also like