AI Peer Review in 2026: The ICLR Problem Isn't Going Away

Researchers analyzed all 70,000 reviews submitted to ICLR 2025 and found that roughly 21% were fully AI-generated. Not polished with AI. Not outlined with AI. Fully written by an LLM, start to finish, for one of the most important machine learning conferences in the world. The analysis was conducted by Pangram Labs, which specializes in detecting AI-generated content.

That number came out in early 2025. About a year later, in 2026, the structural conditions that produced it are worse, not better.

Why 21% Is Just the Start

The ICLR number was shocking because the scale was visible. Conference peer review is more auditable than journal peer review: you can analyze 70,000 reviews from a single venue in one study. Journals are fragmented across thousands of publications, each with its own editorial system. Nobody has done the equivalent analysis for journal reviews, but there's no reason to think journals are cleaner.

GPTZero found over 100 hallucinated citations in papers already accepted at NeurIPS: fake references invented by an LLM and passed by reviewers who apparently didn't check. These aren't edge cases. They're symptoms of a system under pressure it wasn't built to handle.

In 2025, global scientific output crossed 5 million papers per year. The pool of qualified expert reviewers hasn't grown at anything close to that rate. Editors are writing to the same researchers repeatedly. Reviewer fatigue is real and widely documented. When a researcher gets their 40th review request of the year, the temptation to paste the abstract into ChatGPT and clean up the output is obvious: and in most cases, undetectable.

What Makes AI Reviews Fail

The problem isn't that AI can't summarize a paper. It can. The problem is that it can't do the things peer review actually exists to do.

It can't catch what it doesn't know to look for. A reviewer who has spent five years doing single-cell RNA-seq knows that certain clustering algorithms oversplit populations under specific conditions. That knowledge doesn't live in papers: it lives in researchers' hands and heads. An LLM reviewing your scRNA-seq paper doesn't know this. It will produce a competent-sounding summary and miss the methodological issue that anyone in the field would flag.

It can't tell you whether the interpretation is actually right. Your statistics might be technically correct and your conclusions still wrong. A human reviewer who's done the same experiments in a different system can say: "I've seen this artifact before. Here's the simpler explanation you haven't considered." An AI reviewer says: "The authors may wish to consider whether alternative explanations exist." That's not the same thing.

It can't hold itself accountable. When a reviewer publishes in the same field they review, they have skin in the game. Getting something wrong has reputational consequences. An AI has no reputation to protect and no accountability to the scientific community it's supposedly serving.

The result is reviews that look like peer review: structured, formatted, grammatically correct: but function like a summary of the abstract with generic concerns appended. They're the peer review equivalent of a participation trophy.

The Accountability Gap

Here's what makes the structural problem sticky: there's no mechanism to fix it.

Editors can't pay reviewers (or can't pay enough to matter). They can't compel participation. They can't easily verify whether a review was written by a human. Detection tools like GPTZero and Turnitin's AI detector are imperfect and are getting outpaced by increasingly fluent LLMs.

Some journals have added AI disclosure requirements. A handful have banned AI-generated reviews explicitly. These are good policies. They're also largely unenforceable: a researcher who uses ChatGPT and doesn't disclose it faces no meaningful consequence unless caught, and being caught is rare.

The incentive structure hasn't changed. The volume of submissions hasn't dropped. The number of qualified human reviewers hasn't grown. In 2026, those conditions are more pronounced than they were when the ICLR data first surfaced.

What This Means for Your Paper

If your manuscript goes out to peer review, there's a meaningful chance that at least one of your reviewers will use AI to generate some or all of their evaluation. At high-volume journals, the probability is higher. At conferences in AI and computer science specifically, it's highest.

This has a practical implication: the official peer review your paper receives may not give you the substantive, expert-level feedback that would actually strengthen the work. You might get generic criticism that doesn't engage with your specific methods, hallucinated references in reviewer comments, or structural feedback that misses the real issues in your design.

There is one place in the publication pipeline where you can still be confident the feedback is human: before you submit.

Pre-submission review isn't just about improving your acceptance odds. In a system where official peer review is increasingly unreliable, getting real expert feedback on your manuscript before it enters that system is the only way to be sure the feedback reflects genuine scientific judgment.

Manusights reviewers are human scientists who've published in your field and reviewed for the journals you're targeting. Every review is written by a person, not generated by an LLM. For a full breakdown of how to spot AI-generated reviews and protect your work in this environment, see AI-Generated Peer Reviews: How Common Are They and What Researchers Can Do.

How journals are responding

The major journals have adopted different stances. Nature Portfolio journals require reviewers to disclose AI use and prohibit AI-generated review text. PLOS journals ask reviewers to confirm that reviews were not generated by AI. eLife's editorial model (consulting with reviewers before full review) provides some structural protection: the consulted review phase involves direct editor-reviewer dialogue.

Many journals now use AI-detection tools on submitted reviews. These tools are imperfect: they flag probabilistic outputs, not certainties: but they create accountability pressure. Reviewers who use AI extensively now risk being removed from reviewer pools.

The practical effect: highly specialized technical reviews that show no familiarity with the specific methodological details of your paper are more common as a red flag than before. If a review reads like it could apply to any paper in your field, rather than specifically yours, that's the signal.

What to do if you suspect an AI-generated review

You have options, and most journals have formal processes:

1. Flag specific concerns to the editor. Not accusations: observations. "Reviewer 2's comments appear to lack engagement with the specific experimental details in Figure 3, and several points seem to address a different study design than the one we submitted." Concrete and professional.

1. Request editorial assessment. Ask the editor whether the review meets their standards for specificity and engagement. Framing it as a quality question, not an AI accusation, is more effective.

1. Write a thorough response anyway. Even if you suspect AI-generated review, a complete point-by-point response is the right move. It documents your engagement with the concerns and gives the editor a basis for accepting over a weak review.

1. Don't accuse directly in the response letter. Accusations of AI use in a formal response rarely go well. Flag concerns through editor communication, separately from the response letter.

What AI-assisted review actually looks like

There's an important distinction between AI-generated reviews and AI-assisted reviews. AI-assisted means a researcher read your paper, formed their own judgments, and used AI to help draft the text. AI-generated means the AI did the reading and evaluation too.

AI-assisted reviews are increasingly common and increasingly permitted. Most journals allow AI assistance in drafting text as long as the scientific evaluation is the reviewer's own.

AI-generated reviews are what journals are trying to prevent. They typically show:

• Generic praise or concern that doesn't reference specific figures, tables, or data points
• Balanced language that hedges every point without committing to a position
• Unusual uniformity of sentence structure across the review
• Suggestions that contradict specific content in the manuscript

What this means for how you write your paper

If AI-reviewed papers are more common, papers that are resistant to superficial review fare better. Practically:

• Make your key claims figure-specific. "Our main finding is in Figure 3B" forces reviewers to engage with the actual data.
• Reference your own results in the discussion. Link claims to specific data points by figure number. This is good practice regardless, but it makes your paper harder to review superficially.
• Anticipate the reviewers who won't read carefully. Your methods and statistics should be defensible without requiring the reviewer to read the entire paper.

Related reading

• AI peer review: how common is it?: detailed analysis of the 21% finding and what it means
• How to respond to reviewer comments: responding when reviews are thin or off-base
• Desk rejection red flags: what happens before peer review
• The grant success rate crisis
• NIH funding freeze 2025
• How peer review actually works
• How to find a manuscript reviewer
• What pre-submission peer review includes

The Bottom Line

AI in peer review is expanding fast but isn't replacing expert judgment on the questions that matter for publication decisions. The practical takeaway for authors: use AI tools for what they're good at (language, structure, common errors) and human expert review for what they're not (scope fit, methodology validity, novelty assessment).

Sources

• Published editorial guidelines from high-impact journals
• International Committee of Medical Journal Editors (ICMJE) reporting standards
• CONSORT, PRISMA, STROBE, and ARRIVE reporting guidelines
• Pre-Submission Checklist , 25-point audit before you submit

See also

• Is AI Reviewing Your Paper?
• Peer Review Process Explained

• How to Respond to Reviewer Comments