Comparing AI reviews: how is peer review changing?
In Gary King and Steven Worthington’s study, AI reviews outperform human reviews on every evaluation criterion examined. The findings concern constructive feedback, rather than automated publication decisions.
Gary King and Steven Worthington compare AI reviews with 134 paid human reviews of 124 manuscripts from the rapidPeer service. The study focuses on constructive advice before journal submission. Its central finding: AI reviews performed better in all seven areas assessed – theory and idea, methods, weaknesses, framing, communication, figures and tables, and sources. Most of the extensive evaluation was performed by another AI model. [2]
A human cross-check involved three former journal editors and ten randomly selected review pairs: AI reviews were preferred in 26 of 30 judgments. The judges did not know who produced the reviews, but they also did not assess the underlying manuscripts. The appendix documents additional checks on style, model choice and errors. [3]
The study did not measure whether the feedback actually produced better revisions. The authors also discuss risks of making science more homogeneous. [2] Our interpretation: this example shows generative AI changing a central scientific practice – highlighting the distinction between helpful feedback and editorial decisions.
Sources
- [1] Gary King & Steven Worthington (22.10.2026): AI Article Reviews Dominate Human Peer Reviews at Their Best: Evidence and Policy Choices. Projektseite. (opens in a new tab)
- [2] King & Worthington: Volltext der Studie (PDF). (opens in a new tab)
- [3] King & Worthington: Supplementary Appendix – Methoden und Robustheitsprüfungen (PDF). (opens in a new tab)
This indication by Martin Ebner is licensed under
CC BY 4.0
Comments 0
Comments are moderated. New comments appear after review by the editors.
No comments yet.