heydeer

Best LLM for code review: what the public benchmarks show

· · 9 min read

Public benchmarks can tell you which models review code well. They also show that the tooling around a model moves the score about as much as the choice between top models.

The best LLM for code review, on public evidence as of November 19, 2026, is a frontier model such as Claude Opus 5.5 or GPT-5.6 Sol running inside a review harness that gathers context and checks findings: those two lead the ReviewBench entries that name their model. The same model moves by several F1 points between harnesses and settings, so test the combination on your own pull requests before you commit to one.

Best LLM for code review: what the leaderboards show

The most direct public evidence is ReviewBench, because some of its entries name the model they run. Here are those entries in the leaderboard’s default view, which counts high-severity findings, as of November 19, 2026:

ModelEntry and settingPrecisionRecallF1
Claude Opus 5.5Laude RevEngine92.8%60.6%73.3%
GPT-5.6 SolCodex · Max96.3%55.3%70.2%
GPT-5.6 SolCodex · Ultra95.3%55.2%69.9%
Claude Opus 5.5Klaussy · 193.3%55.2%69.3%
GPT-5.6 SolCodex · xHigh97.2%51.4%67.2%
deepseek-flashDeepSeek Harness Reviewer · high95.1%49.2%64.8%
deepseek-v4-flashopswiki hybrid review v292.7%48.9%64.0%
Claude Sonnet 5.5Klaussy · 193.5%47.6%63.1%
Claude Sonnet 5.5Klaussy · default95.6%43.9%60.2%

Source: ReviewBench leaderboard, default view, read on November 19, 2026. Entries that name their model; the full board has 24 entries, and the others don’t name a model.

Three things stand out.

  • Claude Opus 5.5 and GPT-5.6 Sol lead. Every model-named entry above 67% F1 runs one of the two. The smaller and cheaper models, Claude Sonnet 5.5 and the DeepSeek flash models, sit a few points behind, not in a different league.
  • The harness moves the score. Claude Opus 5.5 scores 73.3% F1 in one harness and 69.3% in another. Claude Sonnet 5.5 moves from 60.2% to 63.1% between two settings of the same harness. Those gaps are as large as the gap between the two top models.
  • More effort isn’t automatically better. GPT-5.6 Sol in Codex at three effort settings spans 67.2% to 70.2% F1, and the setting with the highest precision (97.2%) has the lowest recall.

The very top of the board belongs to review agents, not bare models: the two highest F1 scores, 83.0% and 82.2%, come from entries that don’t name a model at all. They outscore the best model-named entry by about ten points of F1, which is the clearest public sign that “which model” is only half of the question.

The code review benchmarks worth reading

ReviewBench

GitHub released ReviewBench on October 5, 2026 as a research preview built with Microsoft. It runs each reviewer on 219 public pull requests from 187 repositories in 19 languages, three times, and scores the findings with Claude Sonnet 5 as the judge. Its headline metrics are “grounded”: they count only findings that match the labeled golden set, so a reviewer isn’t rewarded or punished for findings nobody labeled. The catch is the denominator. A golden set goes incomplete as reviewers find issues its authors never anticipated, which the methodology lists among its threats to validity. Our ReviewBench write-up covers what the precision numbers mean in practice.

Martian’s Code Review Bench

Martian’s benchmark has two halves. The offline set is 50 pull requests from Sentry, Grafana, Cal.com, Discourse and Keycloak, with 173 human-curated golden comments, judged by three different models; Martian reports the top five tools are the same under all three judges. The online half samples fresh merged pull requests where review bots commented, then matches bot suggestions with the fixes developers actually made, which sidesteps the risk that a model has seen the offline PRs in training. It ranks tools such as CodeRabbit, Copilot and Greptile rather than models, and because the online pool keeps growing, any ranking you read is a snapshot.

Smaller studies that name models

LangChain’s separate benchmark, also called ReviewBench, turned 59 tasks from real reviewer feedback in its own monorepo into reproducible checks. With a basic harness, the strongest runs recovered about 30% of the issues trusted human reviewers had caught, and one model’s improvement came from a new review prompt alone, with no new tools. HeyDeer’s own Elo evaluation on internal pull requests put GPT-5.6 Sol in Codex at 1,393 and Opus 5 in Claude Code at 1,244 among single-pass reviewers, with the caveat that the top rating rests on the fewest rounds.

Why SWE-bench scores don’t pick a code review model

Model announcements lead with SWE-bench, and it’s tempting to read it as a coding score that covers review too. It doesn’t. Each of its original 2,294 tasks hands a model a Python repository and a GitHub issue and asks it to edit the code until the issue is resolved, checked by running tests. That measures writing a fix toward a known goal.

Review runs the other way. The reviewer gets someone else’s change, no issue telling it what’s wrong, and has to decide whether anything is wrong at all. There’s no test that turns green. The skills overlap, since you need to understand the code either way, but review adds two that SWE-bench never scores: staying quiet when the change is fine, and explaining a problem convincingly enough that a person acts on it. A model can climb SWE-bench and still post noisy reviews.

Older academic work does target review. The CodeReviewer dataset (2022) built tasks from real code reviews in nine languages: estimating whether a change needs comments, generating the comment and refining the code. The newer benchmarks above go a step further and match each finding to a labeled issue, so a comment earns credit only when it points at a real problem.

Why the harness matters as much as the code review model

A model reviewing a diff sees only what it’s given. The harness decides that: whether the model can open the function a changed line calls, whether it reads the repository’s AGENTS.md, how many passes it makes, and whether anything checks a finding before it’s posted. Those choices explain the spreads in the table above.

  • Context. Most real review bugs live at the boundary between the diff and the code it touches. A model that can search the repository confirms or drops a suspicion; one that sees only the diff guesses.
  • Verification. A second step that tries to prove each finding from the code, a caller or an input that triggers it, cuts false positives more than switching models does.
  • Instructions. LangChain’s result above came from the review prompt, not a new model. Asking for evidence, a severity and an explicit “no findings” answer changes output quality at no cost; the code review prompt guide has examples.
  • Task fit. Not every step of a review needs the largest model. HeyDeer’s Antler Gate 1 research post-trained an open-weight model to predict whether a repository check would fail, and it ranked on par with Claude Opus 5.5 at about a sixtieth of the cost per call at list prices.

No single LLM is best for code review in every setup. On ReviewBench, Claude Opus 5.5 and GPT-5.6 Sol lead the entries that name their model, but the same model moves by three to four F1 points between harnesses and settings, and purpose-built review agents score about ten points higher still. Choose the model and the harness together.

Precision, recall and what a review costs

F1 ranks the leaderboard, but it weighs a false alarm and a missed bug equally, and few teams do. Precision is the share of reported problems that are real; recall is the share of known problems found. On ReviewBench, Greptile has 95.2% precision and 20.2% recall, while the top F1 entry has 92.0% precision and 75.6% recall. Both are defensible, for different teams.

Precision matters more as review comments go straight to coding agents. A person can dismiss a wrong comment in a minute; an agent asked to fix it often writes the fix anyway, even after recognizing the request as wrong, as Jaiswal et al. (2026) found for code models, and a false positive becomes a null check for a value that can’t be null. In ReviewBench’s default view, HeyDeer has the highest precision on the board, 99.0%, with 51.4% recall.

Cost is the third axis, and the leaderboard leaves it out. What a review costs depends on how much context the harness reads, the effort setting and how many passes run, so the same model can cost very different amounts in different tools. The useful unit is cost per useful finding: a cheap reviewer that posts mostly noise is expensive once you count the time spent dismissing it. Our AI code review pricing comparison covers what the hosted tools charge.

How to test a code review LLM on your own pull requests

Benchmarks narrow the field; your repository decides. A model that does well on 219 public pull requests can still miss what matters in your codebase, such as a tenant check your team relies on. A small, honest trial takes an afternoon:

1. Pick 20–30 merged pull requests from the last few months,
   including a few that later caused a bug or a revert.
2. Run every candidate on the same commits: same diff, same
   repository access, same instructions (AGENTS.md, REVIEW.md).
3. For each finding, record one label:
     useful       real problem, worth changing
     correct-nit  true, but not worth a comment
     wrong        false positive
4. Per candidate, add up:
     useful findings, wrong findings,
     known bugs caught (from the PRs that caused one),
     cost and minutes per review.
5. Compare cost per useful finding, not cost per review.

Keep the setup identical across candidates, or you end up measuring the harness you gave each one rather than the model. The full method, including how to count seat fees against usage-based pricing, is in the guide to evaluating AI code review on your own pull requests.

Where HeyDeer fits

Picking a model is one decision; running it on every pull request with the right context, verifying what it finds and keeping costs predictable is the rest of the work. HeyDeer handles that part for GitHub pull requests: models from different vendors review each change against the whole repository, and every finding is verified independently before it’s posted.

HeyDeer is an AI code review agent for GitHub pull requests. In ReviewBench’s default view, it has the highest precision of every reviewer on the board, 99.0%, because each finding has to show how the problem happens, from a real caller or input to the consequence, before it reaches your pull request. To compare it with other hosted reviewers, see the best AI code review tools.

Sources

  1. ReviewBench leaderboard
  2. GitHub Blog: ReviewBench, an open benchmark for AI code review
  3. ReviewBench methodology
  4. Martian: Code Review Bench (GitHub repository)
  5. LangChain: Evaluating code review agents with ReviewBench
  6. Jimenez et al.: SWE-bench, Can Language Models Resolve Real-World GitHub Issues?
  7. Li et al.: Automating Code Review Activities by Large-Scale Pre-training (CodeReviewer)
  8. Jaiswal et al.: Blind Obedience to Incorrect Instructions Drives Code LLMs to Irrecoverable Code Semantic Collapse

Frequently asked questions

Which LLM is best for code review?

On ReviewBench, the open code review benchmark from GitHub and Microsoft, the entries that name their model are led by Claude Opus 5.5 and GPT-5.6 Sol as of November 19, 2026, with F1 scores between about 67% and 73% in the default view. The same model scores differently in different harnesses, so the best LLM for code review depends on the tool it runs in as much as on the model.

Is SWE-bench a code review benchmark?

SWE-bench is a benchmark for fixing code, not reviewing it. Each task gives a model a repository and a GitHub issue and asks it to edit the code to resolve the issue, and the result is checked by running tests. Code review asks the opposite question: given someone else’s change, find what is wrong with it without reporting false alarms.

What is ReviewBench?

ReviewBench is an open benchmark for AI code review, released by GitHub on October 5, 2026 as a research preview and built with Microsoft. It runs each reviewer on 219 public pull requests from 187 repositories in 19 languages and scores grounded precision, recall and F1 against labeled findings, with Claude Sonnet 5 as the judge.

What is Martian’s Code Review Bench?

Martian’s Code Review Bench is an open-source benchmark for AI code review tools with two parts: an offline set of 50 pull requests from five open-source repositories with 173 human-curated golden comments, and an online leaderboard that samples fresh merged pull requests where review bots left comments, to avoid training-data leakage.

Should a code review model favor precision or recall?

A code review model should favor precision when people or coding agents act on every comment, because each false positive costs time or turns into unnecessary code. Favor recall for high-risk changes where a missed bug costs more than extra noise. Benchmarks report both; the F1 score weighs them equally, which may not match your team.

Are open-weight models good enough for code review?

Open-weight models can do focused code review tasks well. On ReviewBench, entries running DeepSeek flash models score about 64–65% F1 in the default view, a few points behind the frontier entries, and HeyDeer’s published Antler Gate 1 research shows a post-trained open-weight model matching Claude Opus 5.5 at predicting whether a review check fails.

How do I test an LLM for code review on my own code?

To test an LLM for code review on your own code, run each candidate on the same 20–30 merged pull requests with the same context, label every finding as useful, correct but not worth a comment, or wrong, and compare useful findings, false positives, known bugs caught and cost per useful finding.

Start with your first PR.

Free plan. No credit card required.

Get started free