Hot AI News

LLM-as-a-judge: How AI models grade other AI models, and where the grading goes wrong

LLM-as-a-judge means asking a strong AI model to score other models' answers. Here is how it works, where it came from, and which biases make the scores less solid than they look.

LLM-as-a-judge: How AI models grade other AI models, and where the grading goes wrong
Photo illustration: Aiholics

LLM-as-a-judge is a way of grading AI output by asking another large language model (LLM) to do the grading. Instead of paying people to read thousands of answers, a strong model reads them and returns a verdict: which answer is better, or what score each one deserves.

It is now common in benchmarks, product testing and model training. It is also imperfect, and the flaws matter when a single score decides which model looks best.

A teacher with a mark scheme, but not enough hours in the day

Picture a thousand essays and one teacher. Marking them all by hand takes weeks, so the teacher hands the pile to a capable assistant along with a mark scheme: one point for a clear thesis, one for evidence, and so on. The assistant is fast and consistent, though not always right.

That is the whole idea. A judge model receives a task, one or more answers, and instructions for scoring. It then writes a verdict, often with a short explanation. Once the setup is done, some tests need no human at all.

The appeal is cost and speed. A human panel can review a few hundred answers. A judge model can review millions, and it can be run again whenever a new model appears.

Advertisements

Three ways a judge can be asked to decide

Three wall-mounted screens in a research lab show abstract interfaces for scoring one AI answer, comparing two answers, and ranking a set of answers.
Photo illustration: Aiholics

Setups differ, but most fall into three styles:

  • Pairwise comparison: the judge sees two answers and picks the better one. Many chatbot leaderboards work this way.
  • Single-answer scoring: the judge gives one answer a grade, for example from 1 to 10.
  • Rubric grading: the judge checks an answer against a list of criteria, each with its own weight, and the weighted sum becomes the score.

Rubrics are the most useful for open-ended work such as software engineering, where there is no single correct output. Test suites can check whether code runs. They struggle to say whether it is clean, maintainable and something a reviewer would approve.

Some setups also give the judge a reference answer or a checklist written by experts. This narrows what the judge has to decide on its own and usually makes verdicts steadier.

Where the idea came from

The approach was popularized in 2023 by Lianmin Zheng and co-authors in the paper Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena, presented at NeurIPS 2023. They introduced MT-bench, a set of multi-turn questions, and compared judge verdicts with human votes from Chatbot Arena, a crowdsourced platform where people pick the better of two anonymous chatbots.

Their main result was that GPT-4 as a judge agreed with human preferences more than 80% of the time. According to the authors, that is about the level at which humans agree with each other. They also released human votes and conversations so others could test judges themselves.

Since then, a large body of work has studied how to prompt, calibrate and audit judges. A 2024 survey of the field (arXiv 2412.05579) collects methods for building judges, improving them and measuring how reliable they are.

The judge has habits of its own

The same 2023 paper named several weaknesses, and later studies have kept probing them:

  • Position bias: the verdict can change when the two answers swap places. The usual fix is to run each comparison twice, in both orders.
  • Verbosity bias: judges can favor longer answers even when length adds nothing. Dubois and co-authors (2024) proposed length-controlled win rates as a correction.
  • Self-preference bias: a judge may favor answers from its own model family. This is hard to separate from real quality, because a stronger model may simply write better.

Other work, such as the 2024 papers on quantifying biases in judges (arXiv 2410.02736) and on self-preference (arXiv 2410.21819), measured these effects across many models.

The evidence is not uniform. A 2026 preprint (arXiv 2604.23178) reports that some earlier findings no longer hold for newer models, with position bias negligible in the models it tested. The safe reading is that these biases depend on the model and the task, and need checking each time.

Advertisements

A 1.1-point gap between Neon and Opus 5.5 is small next to a grader’s quirks

The idea is relevant to our report on Gemini 4 Neon in Devin. Cognition says its FrontierCode 2.0 benchmark judges real engineering tasks by quality and mergeability, and that the score is a weighted aggregate of rubric items. Solutions that fail blocking criteria score 0.

The post, as we read it, does not say who or what applies the rubric. A rubric can be scored by human reviewers, by automated checks, by a judge model, or by a mix. That is not a claim about Cognition’s method. It is a gap in what the post tells readers.

It matters because Neon scores 46.2% and Claude Opus 5.5 scores 47.3%. If a judge model is involved anywhere in the scoring, a gap of 1.1 points sits in the range where grader habits could plausibly matter. Whether they do is something only the benchmark’s authors can show, for example by publishing how often repeated runs disagree.

The cost side of the chart is a different kind of number. Dollars per task are measured, not judged, so the 2.9 times cost gap does not depend on a grader at all.

What a judge cannot do, and what to look for in a benchmark write-up

A judge cannot reliably grade what it cannot do itself. On hard math or subtle bugs, a judge may accept a confident wrong answer. Judges can also be gamed: a model tuned against a judge may learn the judge’s tastes rather than the task.

When reading a benchmark report, these safeguards are worth looking for:

  • Agreement checks between the judge and human reviewers on a sample.
  • Swapped answer order for pairwise tests.
  • Judges from a different model family than the models being graded.
  • Published rubrics and grader prompts, so others can rerun the test.
  • Repeated runs, so readers can see how much a score moves by chance.

Expect the practice to keep growing, because human grading does not scale to the volume of agent runs labs now produce. Expect the scrutiny to grow with it, especially when a vendor grades models on its own benchmark and reports gaps of a point or two.

Sign Up for the Daily AI Pulse

One email a day. All the stories that matter.

Daniel ReedAI Research, Safety & Ethics Analyst

Daniel Reed currently works as an AI Research, Safety & Ethics Analyst at Aiholics, writing about how changes in artificial intelligence are affecting and will affect scholarship,…

Leave a Reply

Your email address will not be published. Required fields are marked *

You may also like