AI Peer Review in 2026: The ICLR Problem Isn't Going Away

The ICLR number was shocking because the scale was visible. Conference peer review is more auditable than journal peer review: you can analyze 70,000 reviews from a single venue in one study. Journals are fragmented across thousands of publications, each with its own editorial system. Nobody has done the equivalent analysis for journal reviews, but there's no reason to think journals are cleaner.

GPTZero found over 100 hallucinated citations in papers already accepted at NeurIPS: fake references invented by an LLM and passed by reviewers who apparently didn't check. These aren't edge cases. They're symptoms of a system under pressure it wasn't built to handle.

In 2025, global scientific output crossed 5 million papers per year. The pool of qualified expert reviewers hasn't grown at anything close to that rate. Editors are writing to the same researchers repeatedly. Reviewer fatigue is real and widely documented. When a researcher gets their 40th review request of the year, the temptation to paste the abstract into ChatGPT and clean up the output is obvious: and in most cases, undetectable.

The problem isn't that AI can't summarize a paper. It can. The problem is that it can't do the things peer review actually exists to do.

It can't catch what it doesn't know to look for. A reviewer who has spent five years doing single-cell RNA-seq knows that certain clustering algorithms oversplit populations under specific conditions. That knowledge doesn't live in papers: it lives in researchers' hands and heads. An LLM reviewing your scRNA-seq paper doesn't know this. It will produce a competent-sounding summary and miss the methodological issue that anyone in the field would flag.

It can't tell you whether the interpretation is actually right. Your statistics might be technically correct and your conclusions still wrong. A human reviewer who's done the same experiments in a different system can say: "I've seen this artifact before. Here's the simpler explanation you haven't considered." An AI reviewer says: "The authors may wish to consider whether alternative explanations exist." That's not the same thing.

It can't hold itself accountable. When a reviewer publishes in the same field they review, they have skin in the game. Getting something wrong has reputational consequences. An AI has no reputation to protect and no accountability to the scientific community it's supposedly serving.

The result is reviews that look like peer review: structured, formatted, grammatically correct: but function like a summary of the abstract with generic concerns appended. They're the peer review equivalent of a participation trophy.

Here's what makes the structural problem sticky: there's no mechanism to fix it.

Editors can't pay reviewers (or can't pay enough to matter). They can't compel participation. They can't easily verify whether a review was written by a human. Detection tools like GPTZero and Turnitin's AI detector are imperfect and are getting outpaced by increasingly fluent LLMs.

Some journals have added AI disclosure requirements. A handful have banned AI-generated reviews explicitly. These are good policies. They're also largely unenforceable: a researcher who uses ChatGPT and doesn't disclose it faces no meaningful consequence unless caught, and being caught is rare.

The incentive structure hasn't changed. The volume of submissions hasn't dropped. The number of qualified human reviewers hasn't grown. In 2026, those conditions are more pronounced than they were when the ICLR data first surfaced.

If your manuscript goes out to peer review, there's a meaningful chance that at least one of your reviewers will use AI to generate some or all of their evaluation. At high-volume journals, the probability is higher. At conferences in AI and computer science specifically, it's highest.

This has a practical implication: the official peer review your paper receives may not give you the substantive, expert-level feedback that would actually strengthen the work. You might get generic criticism that doesn't engage with your specific methods, hallucinated references in reviewer comments, or structural feedback that misses the real issues in your design.

There is one place in the publication pipeline where you can still be confident the feedback is human: before you submit.

Pre-submission review isn't just about improving your acceptance odds. In a system where official peer review is increasingly unreliable, getting real expert feedback on your manuscript before it enters that system is the only way to be sure the feedback reflects genuine scientific judgment.

Manusights reviewers are human scientists who've published in your field and reviewed for the journals you're targeting. Every review is written by a person, not generated by an LLM. For a full breakdown of how to spot AI-generated reviews and protect your work in this environment, see AI-Generated Peer Reviews: How Common Are They and What Researchers Can Do.

The major journals have adopted different stances. Nature Portfolio journals require reviewers to disclose AI use and prohibit AI-generated review text. PLOS journals ask reviewers to confirm that reviews were not generated by AI. eLife's editorial model (consulting with reviewers before full review) provides some structural protection: the consulted review phase involves direct editor-reviewer dialogue.

Many journals now use AI-detection tools on submitted reviews. These tools are imperfect: they flag probabilistic outputs, not certainties: but they create accountability pressure. Reviewers who use AI extensively now risk being removed from reviewer pools.

The practical effect: highly specialized technical reviews that show no familiarity with the specific methodological details of your paper are more common as a red flag than before. If a review reads like it could apply to any paper in your field, rather than specifically yours, that's the signal.

There's an important distinction between AI-generated reviews and AI-assisted reviews. AI-assisted means a researcher read your paper, formed their own judgments, and used AI to help draft the text. AI-generated means the AI did the reading and evaluation too.

AI-assisted reviews are increasingly common and increasingly permitted. Most journals allow AI assistance in drafting text as long as the scientific evaluation is the reviewer's own.

AI-generated reviews are what journals are trying to prevent. They typically show:

• Generic praise or concern that doesn't reference specific figures, tables, or data points
• Balanced language that hedges every point without committing to a position
• Unusual uniformity of sentence structure across the review
• Suggestions that contradict specific content in the manuscript

If AI-reviewed papers are more common, papers that are resistant to superficial review fare better. Practically:

• Make your key claims figure-specific. "Our main finding is in Figure 3B" forces reviewers to engage with the actual data.

• Reference your own results in the discussion. Link claims to specific data points by figure number. This is good practice regardless, but it makes your paper harder to review superficially.

• Anticipate the reviewers who won't read carefully. Your methods and statistics should be defensible without requiring the reviewer to read the entire paper.

• AI peer review: how common is it?: detailed analysis of the 21% finding and what it means
• How to respond to reviewer comments: responding when reviews are thin or off-base
• Desk rejection red flags: what happens before peer review
• The grant success rate crisis
• NIH funding freeze 2025
• How peer review actually works
• How to find a manuscript reviewer
• What pre-submission peer review includes

Act on this if:

• You use AI tools in manuscript preparation or review
• Your target journal has specific AI disclosure policies
• You want to understand the current landscape before choosing tools

Less urgent if:

• You do not use AI tools in your research workflow
• Your institution has not yet implemented AI use policies

AI tools and human reviewers aren't interchangeable, they're good at different things. Here's a practical framework:

Use AI tools when:

• You need a fast structural check (section flow, missing methods details, formatting compliance)
• You want language polish and grammar cleanup across the full manuscript
• You're screening for common statistical reporting errors or reference formatting issues
• You need a first pass before sending to human reviewers, so their time is spent on substance

Use human expert review when:

• You need someone to evaluate whether your conclusions actually follow from your data
• The journal-fit question matters, an AI tool can't tell you whether your paper reads like a PNAS paper vs. a PLOS ONE paper
• Your methodology involves judgment calls that require hands-on experience with the technique
• You're unsure whether the novelty claim holds up against recent work in the field
• The stakes are high enough that a generic "looks good" isn't useful

The gap to watch: AI tools are getting better at surface-level feedback, which makes it tempting to skip human review entirely. That's a mistake. The 21% AI-generated review finding from ICLR shows what happens when surface-level evaluation replaces expert judgment. Don't replicate that problem in your own prep workflow.

If you want AI-assisted structural feedback combined with human expert scientific review, that's what manuscript readiness check is built for.

Last verified: April 20, 2026. ICLR statistics from Pangram Labs and current public policy language from ICLR and Nature Portfolio materials.