Reinforcement learning with verifiable rewards: How AI learns from checkable answers

Reinforcement learning with verifiable rewards (RLVR) is a way to train an AI model by letting it try a task many times and rewarding it only when a program can confirm the result is correct. The checker might compare a math answer to the known solution or run a piece of code against tests. No human has to judge each attempt.

The idea sits behind many recent reasoning models. Moonshot AI says its open-weight Kimi K3.1 Pro was post-trained from Kimi K3 with large-scale reinforcement learning, as covered in our weekly model roundup.

Learning from a marked answer sheet

Think of a student working through past exam papers with an answer key. The student writes a solution, checks it against the key, and learns which approaches work. Nobody needs to explain why a wrong answer is wrong. The mark alone is the feedback.

RLVR works the same way. The model is given a question and writes a full attempt. A verifier then returns a score, often just 1 for correct and 0 for incorrect. The model’s weights are updated so that attempts like the correct ones become more likely.

The loop: Sample, check, reinforce

Reinforcement learning (RL) means learning by trial and reward instead of by copying examples. In RLVR, one round looks like this:

1. Pick a prompt with a checkable answer, such as a math problem or a coding task.
2. Have the model write several different attempts.
3. Run the verifier on each attempt and assign rewards.
4. Nudge the model toward the attempts that earned higher rewards.

Step 4 needs an update rule. DeepSeek’s papers use one called GRPO (group relative policy optimization). It compares each attempt with the other attempts to the same prompt, so it does not need a separate model to estimate how good a situation is. That makes it cheaper than the older PPO (proximal policy optimization) method, which needs one.

Verifiers can be simple. A rule can extract the final number from an answer and compare it. A test suite can decide whether code passes. A format check can confirm that the model followed an instruction such as a word limit.

Where the idea came from

Earlier chatbot training used reinforcement learning from human feedback (RLHF), where a reward model learned human preferences. A learned judge can be fooled, because the model may find answers the judge likes but that are not actually right.

The Allen Institute for AI (Ai2) named the approach in its Tulu 3 work, first posted on arXiv in November 2024. The paper introduced Reinforcement Learning with Verifiable Rewards as a final training stage after supervised fine-tuning and preference optimization, applied to tasks such as math and instruction following. Ai2 released the training data and code.

In January 2025, DeepSeek published its R1 paper. It reported that pure reinforcement learning on verifiable tasks in math, coding competitions and STEM fields could outperform models trained on human demonstrations. The authors also described behaviors such as self-reflection and verification appearing during training. The paper was later published in Nature in 2025.

Who uses it today

RLVR is now a standard part of post-training for reasoning and coding models. Moonshot AI describes Kimi K3.1 Pro, a mixture-of-experts model with 2.8 trillion total parameters, as post-trained from Kimi K3 with large-scale reinforcement learning. It says the model improves coding, financial research and factual accuracy. Its scores, such as 94.2 on GPQA-Diamond, are self-reported and not independently verified, as noted in our roundup.

Coding is a natural fit because tests give a clear pass or fail. Math is another, because final answers can be compared exactly. This is also why many labs invest in realistic coding environments where an agent can run code and see whether it works.

The limits of a pass or fail mark

The first limit is coverage. Many valuable tasks have no clean checker. Whether an essay is persuasive or a summary is fair cannot be reduced to a rule. DeepSeek’s R1 paper itself notes that the reasoning-focused stage left writing and open-domain question answering weaker, and the team added other training stages to compensate.

The second limit is gaming. A model can find shortcuts that satisfy the verifier without solving the real problem, such as exploiting a weak test. Verifiers have to be built carefully.

The third is a research debate about what RLVR really adds. One line of work, including a 2025 paper on arXiv (2506.14245), argues that it can improve the correctness of reasoning in the base model. Other studies argue it mostly sharpens sampling toward solutions the base model could already reach. The question is not settled.

What to watch

Three areas are moving. Labs are building verifiers for fuzzier domains, such as science and finance. They are mixing verifiable rewards with learned judges. And independent evaluations are checking whether the gains seen on a lab’s own benchmarks hold on tests the lab did not train against.

When a release says a model was trained with large-scale reinforcement learning, a useful question is what the verifier checks. The answer shows where the model is likely to be strong and where it may not be.