Is AI Reviewing Your Paper? How Common It Is and What Researchers Can Do

In our pre-submission review work, the practical problem is not just that AI-generated reviews exist. It is that they often fail in predictable ways that matter to authors: generic praise, no figure-level engagement, no sign the reviewer understands the specific method, and no clear judgment about whether the central claim is actually exposed.

That changes what authors should do before submission. If there is a reasonable chance the official review will be shallow or AI-assisted, then the manuscript needs a stronger human screening layer before it enters that system. Otherwise the first real expert judgment may arrive too late, after the paper has already burned a review cycle.

When you get your reviews back, read them with fresh eyes. Not for what they say about your paper, but for how they say it.

Generic praise that could apply to any paper. "The authors present an interesting study on an important topic." "The methodology is generally sound." If you could swap the review onto a completely different manuscript and it would still make sense, that's a red flag.

Cookie-cutter structure with no real engagement. AI reviews love templates. Summary paragraph, then "Strengths" (three bullet points), then "Weaknesses" (three bullet points), then "Minor Comments." The problem is when every section feels like it was generated from the abstract alone, without evidence that the reviewer read the actual paper.

Hallucinated citations in the review itself. This is the smoking gun. If a reviewer says "as shown in Smith et al. (2023)" or "consistent with the findings of Chen and colleagues (Nature, 2024)," check those references. AI-generated reviews sometimes cite papers that don't exist. A human reviewer might misremember a citation. Only an AI invents one from scratch.

Suspiciously fast turnaround. If a journal's average review time is 4-6 weeks and you get a detailed review back in 48 hours, someone might have outsourced the work.

Absence of field-specific judgment. Human reviewers bring their own experience. They'll say things like "This approach has been tried before by the Johnson lab with mixed results" or "The authors should consider whether batch effects could explain Figure 3." AI reviews rarely contain this kind of domain-grounded specificity.

Weird hedging patterns. LLMs default to diplomatic nothingness. "While the authors present some evidence, it would be beneficial to further elaborate on..." Human reviewers tend to be more direct, even blunt.

An AI can summarize your paper. It can check whether your references are formatted correctly. It can probably flag obvious statistical errors. But that's not what peer review is supposed to do.

Peer review exists to answer questions that require actual expertise:

Is this actually new, and does the methodology work in practice? An LLM can search its training data for similar-sounding abstracts, but it can't tell you whether the specific combination of approach, model system, and finding genuinely advances the field. A reviewer who has personally run Western blots on this protein knows that it's notoriously difficult to detect at low expression levels. A reviewer who has done single-cell RNA-seq knows that the clustering algorithm the authors used tends to oversplit populations. This practical knowledge doesn't live in papers. It lives in researchers' hands and heads.

Is the interpretation reasonable, and does the framing match the evidence? The data might be real and the statistics correct, but the conclusion might still be wrong. A human reviewer can say "I've seen this artifact before" or "there's a simpler explanation you haven't considered." Authors oversell, and a good reviewer pushes back: "You're claiming this drives metastasis, but you've only shown it in vitro." An AI might parrot a generic "consider toning down the claims" but can't tell you which claims are overreaching and why.

The result is that AI reviews tend to be generically correct but specifically useless. They'll tell you to "strengthen the statistical analysis" without telling you what's wrong with it. They're the review equivalent of a participation trophy.

Here's the part that actually worries me.

Peer review has always been imperfect. But it had one thing going for it: a human being with relevant expertise made a judgment call about whether your science held up. That judgment was grounded in experience, and the possibility of being called on it kept people honest.

What happens when that human drops out of the loop? We're already seeing the answer. Authors use AI to write papers. Reviewers use AI to review them. The AI reviewing the paper can't tell that the AI-written paper has hallucinated data, because the reviewer-AI doesn't actually know anything. It's pattern-matching text.

In July 2025, Nature reported that researchers had started hiding prompt injection attacks in their papers. Invisible text instructions designed to manipulate AI reviewers into writing positive reviews. Authors are optimizing their manuscripts not for human readers, but for the LLMs they expect will review them.

The NeurIPS hallucinated citations scandal makes more sense in this context. Over 100 fake references made it into accepted papers. That doesn't happen if humans are carefully checking citations. It happens when the entire chain has been partially automated. AI writing reviewing AI writing.

The answer isn't AI that reviews better. It's humans who review carefully.

You can't control who reviews your paper or whether they use AI. But you can take steps to protect your work.

If you receive a review that's clearly generic, contains hallucinated references, or shows no evidence of engagement with your actual data, write to the editor. Be specific. Say "The reviewer cites Smith et al. (2024) in their comment about Figure 3, but this paper doesn't exist."

Some journals have taken concrete steps: Science has explicitly banned AI-generated reviews; Springer Nature requires disclosure of AI use. When choosing where to submit, consider this. A journal that takes review integrity seriously is more likely to give your paper a fair evaluation.

Papers that are highly specific, tightly argued, and deeply technical are harder for AI to review poorly. If your methods section is precise enough that only someone who's actually done the experiment would understand the decisions you made, an AI-generated review will expose itself by missing those nuances.

If you get back comments that are clearly shallow or AI-generated, you can note where a reviewer's comments don't engage with the specifics of your work. Editors notice when a reviewer's comment is generic and the author's response reveals that the reviewer didn't actually read the paper.

The system is more resilient than its critics think, mostly because the people in it genuinely care about getting things right. But if you care about the quality of feedback your paper receives, be proactive. Get expert eyes on your manuscript before you submit. Choose journals that take integrity seriously. The peer review system is only as good as the people participating in it.

Not every manuscript needs the same level of feedback. Here's a practical framework:

AI review is probably sufficient when:

• You're checking formatting, structure, and language before submission
• You want a fast first pass to catch obvious gaps in methods or references
• The manuscript has already been through internal lab review and you're polishing
• You need to verify journal-specific requirements (word counts, figure specs, reference style)

You need human expert review when:

• The methodology is novel or unconventional and you're unsure whether reviewers will accept it
• Your claims are strong and you need someone who knows the field to pressure-test them
• You're targeting a high-rejection journal (Cell, Nature, Science tier) where desk rejection is 60-80%
• The paper crosses disciplines and you're not sure the framing works for your target audience
• You've been rejected once and need to understand why before resubmitting

Use both when:

• AI catches the structural and formatting issues first (fast, cheap), then a domain expert evaluates the science, framing, and journal fit. This is the most efficient workflow for high-stakes submissions.

AI tool features, pricing, and journal policies on AI-generated reviews verified against each provider's website, April 2026. The ICLR and NeurIPS studies cited are from published analyses by Pangram Labs and GPTZero. Journal policies on AI use were checked against Nature, Springer Nature, and Science editorial policy pages.