AI now writes better peer reviews than paid experts. It also learned to be Reviewer 2

Every academic has met Reviewer 2: the anonymous referee who stops reading early, rejects the paper and suggests you cite their own work. A new Harvard working paper says AI reviewers are now better than the best human ones. It also accidentally confirms the AI has picked up the attitude.

The study. Gary King and Steven Worthington (opens in a new tab) of Harvard's Institute for Quantitative Social Science used papers sent to rapidPeer, a Harvard service that gives faculty paid, double-blind reviews of drafts before they submit to a journal. The corpus: 124 papers and 134 human reviews collected from 2019 to 2026. These weren't weak reviews: the reviewers were paid "(well)", and 88% of the 74 authors who answered a survey rated their review high or very high quality.

For each paper, the authors had Claude Opus 4.8 (Anthropic) write a review at its maximum reasoning setting, in two versions: one with the same instructions as the humans and no tools, and one with a purpose-built prompt plus web search.

The scores.

• An AI judge from a different model family, GPT-5.5, preferred the AI review in 132 of 132 decided pairs (two were undecided because the pick flipped with presentation order).
• Three former journal editors, judging 10 random blinded pairs each without knowing AI was involved, picked the AI review in 26 of 30 judgments (87%).
• Accuracy: AI reviews cited no fabricated sources, and 5.5% of their claims about the manuscript were judged incorrect, versus 15.9% for the human reviews.
• Not a length trick: on 44 pairs where the reviews were about the same length, the AI review still won all 44.

And yes, it's Reviewer 2. In discussing how journals might use this, the authors warn that as a paper improves through rounds of AI feedback, "AI sometimes becomes absurdly pedantic even though still technically correct (such as 'yes 2+2 = 4, but you never defined the + sign')." That line is in the paper. We just drew it.

The fine print.

• These were constructive, pre-submission reviews, not journal accept-or-reject decisions. The authors say their results don't speak to how editors should make publication decisions.
• They didn't measure whether papers actually got better after AI feedback.
• The editor sample is small: 10 papers, 30 judgments.
• All social science papers, from one service.
• The authors flag a real risk: if everyone gets reviewed by similar models, science could drift toward "homogenous arguments and conclusions."
• It's a working paper dated October 22, 2026, not yet published in a journal as far as we can tell.

Our take

The surprising part isn't that AI reviews are fast. It's that they beat careful, paid humans on accuracy too. The authors' own suggestion is the sensible one: let authors run AI reviews before submitting, make them answer the AI in writing, and keep the final call with a human editor. Reviewer 2 isn't going away. It's just getting a lot cheaper.

What we did and didn't do: we read the full paper (PDF) and the paper page on October 23, 2026. We didn't rerun the study or test Claude as a reviewer ourselves. Not sponsored; no affiliate links.

MODEL Editors