Nvidia’s Nemotron recipe: RL lifted drug-metabolism accuracy 17% past fine-tuning
Nvidia published a recipe for post-training Nemotron 3.5 Super VL on a lab's own experimental data. In a drug-metabolism test, SFT added 12% and RL a further 17%.
Nvidia has published a step-by-step recipe for adapting Nemotron 3.5 Super VL to a research lab’s own data. On a drug-metabolism task, supervised fine-tuning raised accuracy by 12% and reinforcement learning added a further 17%, Nvidia says.
A lab’s CSV becomes a training environment
The guide is part of what Nvidia calls Nemotron for Science, a collection of models, datasets, tools and recipes for agentic scientific work. It starts from a CSV of experimental records, such as molecular structures written as SMILES strings paired with measured outcomes. Each row is turned into a question and answer pair, and the measured result acts as the verifier. Nvidia describes this as reinforcement learning with verifiable rewards, which needs no human preference labeling.
Image: Nvidia
The worked example predicts a drug metabolite. The model gets a parent compound and the enzyme that transforms it, reasons through the reaction and submits a predicted structure through a tool call. The verifier compares molecular fingerprints from the prediction and the reference using Tanimoto similarity, a score from 0 to 1. The example data comes from MetxBioDB, a database of experimentally confirmed biotransformation reactions.
Training uses group relative policy optimization (GRPO) with low-rank adapters (LoRA). NeMo Gym supplies the environment and NeMo RL the training framework. The post includes the config and launch pieces. Only the resources server, which holds the scoring function and tool schema, is domain-specific. The example launch runs on 16 GPUs across four nodes with run_grpo_nemo_gym.py, and the policy model is the 120B Nemotron 3.5 Super VL checkpoint.
Image: Nvidia
Advertisements
Claude Sonnet 5.5 as an optional teacher
If the base model needs a boost or RL converges slowly, Nvidia suggests a distillation step first. A stronger teacher model runs against the same environment, and only reasoning traces whose answers match the ground truth are kept. The teacher in the example is Anthropic’s Claude Sonnet 5.5. Nvidia kept 1,123 traces from 1,934 training prompts at a Tanimoto threshold of 0.7 or higher. They then fine-tune the smaller model before RL starts, a “warm-start” that Nvidia says typically speeds convergence.
Supervised fine-tuning improved accuracy by 12% and RL lifted it a further 17%, at a 16,384-token output budget.
Image: Nvidia
Nvidia notes that filtering at an exact match (score of 1.0) keeps only traces where the teacher got the metabolite exactly right, while a softer threshold allows more variation. It also advises switching on the importance-sampling correction option in the loss function. According to the post, this narrows the gap between the vLLM engine that generates rollouts and the Megatron engine that trains, a concern that weighs most on mixture-of-experts models.
What 120B parameters cost to adapt
The analysis angle here is cost, and the post itself gives no dollar figures. Context comes from our earlier coverage of an AWS deployment guide: Nemotron 3.5 Super VL has 120B total parameters with 12B active per token, and serving it in BF16 needs at least 16 H100s or 8 B200s, according to that guide. Nvidia’s training example uses 16 GPUs across four nodes, so the recipe targets teams with cluster access rather than a single workstation. The LoRA adapters and a verifier that is cheap to compute, such as a fingerprint comparison, keep the reward side light. The post notes that richer scoring, from simulators to a live autonomous lab, can be plugged in at higher cost.
The distillation step adds a different cost: calls to a frontier model. The example samples a teacher repeatedly per prompt (the script uses --num-repeats 8) over 1,934 prompts, and about 58% of prompts yielded a usable trace at the 0.7 threshold. Teams would pay for that API usage in return for a better starting point, a trade the post does not quantify.
Several things remain unestablished. The 12% and 17% gains are Nvidia’s own results on one task, and the post does not say whether they are relative or absolute, or which score they refer to beyond mean molecular similarity. There is no independent replication, and the post does not say how the gains compare with the teacher model alone or with RL started without SFT. Held-out evaluation is the check Nvidia itself recommends before trusting any gain.
Advertisements
Who can try it today
The tooling is open: Nvidia points readers to the NeMo Gym and NeMo RL repositories and says the recipe can serve as a template for other scientific tasks. Nemotron 3.5 Super VL is described as an open, customizable model, and the post also introduces the Nemotron Litmus-Bench V2.0 chemistry-reasoning dataset and benchmark. The post gives no pricing, license terms or hardware lead times. Nvidia advises starting with a narrow task, validating the scoring method and expanding only as the evidence supports it.
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.