Statistical Review Red Flags: What Reviewers Notice Fast

Elsevier's quick guide to common statistical errors is unusually blunt and practical.

It warns that:

1. many studies are too small to detect even large effects
2. clinical trials should always report sample-size calculations
3. authors with negative results should not claim equivalence unless sufficiently powered
4. post-hoc subgroup and risk-factor analyses should be treated as speculative
5. comparing groups at multiple time points with repeated simple tests should be avoided

That already covers most of the failure modes I see in manuscripts sent for statistical review.

The same guide also flags design-level bias issues such as:

• treatment-by-indication bias
• historical controls
• retrospective data-collection bias
• selection bias from low response or high refusal
• informative dropout
• inadequate randomization or lack of concealment

This is the right frame. The biggest statistical problems often start before any model is fit.

Underpowered studies are dangerous for two opposite reasons.

They can miss real effects, and they can also produce unstable positive-looking findings that the paper then overinterprets.

Elsevier's guide is especially useful here because it says directly: authors with negative results should not report equivalence unless the study was sufficiently powered. "Absence of evidence is not evidence of absence" is not a cliché in this context. It is a review criterion.

What reviewers want:

• sample-size calculation or design justification
• realistic effect assumptions
• clarity on primary endpoint
• acknowledgement of precision limits when the study is small

Many papers discuss bias in the Discussion only after the central analyses are already locked in. Statistical reviewers usually want to see bias handled upstream in design and analysis.

Elsevier's reviewer sheet highlights several recurring bias problems:

• historical controls often exaggerate treatment effect
• retrospective data collection can vary systematically by group
• informative dropout can skew outcome comparisons
• poor randomization methods are not acceptable substitutes for allocation concealment

If a paper seems to assume that adjustment later can rescue a biased design fully, reviewers become skeptical quickly.

This is one of the oldest and still one of the most effective ways to weaken a paper.

Post-hoc subgroup analyses can generate seductive stories because they often look specific and mechanistic. But if enough slices are tested, something will look significant by chance alone.

Elsevier's guidance says conclusions should be drawn from a small number of clear, predefined hypotheses, and that post-hoc subgroup or risk-factor analyses should be treated as speculative. That sentence alone could rescue a lot of discussion sections.

If you ran subgroup analyses after seeing the data:

• call them exploratory
• reduce the rhetoric
• do not promote them to headline results

Many manuscripts do not fail because multiple testing occurred. They fail because authors behave as if it did not matter.

Reviewers want to know:

• how many related tests were run
• which analyses were primary
• whether multiplicity adjustment was used
• if not, how the exploratory nature is being framed

The bigger the analysis garden, the more discipline the reporting needs.

Missing data are often where optimistic manuscripts quietly break.

Reviewers want answers to:

• how much data were missing
• whether missingness differed by group
• whether dropout might be outcome-related
• what imputation or sensitivity methods were used, if any

Elsevier's guide explicitly flags informative dropout because it can bias comparisons when follow-up ends for reasons connected to the primary outcome.

If your Methods and Results do not show the missing-data story, reviewers may assume you do not understand its importance.

This shows up in papers that treat repeated measurements, clustered data, or grouped observations as if they were independent datapoints.

Even when the statistical code technically runs, the inference can still be wrong if the unit of analysis does not match the design.

Statistical reviewers often catch this by asking simple questions:

• what exactly counts as one observation
• were repeated measures summarized or modeled appropriately
• were patients, samples, or sites clustered

If the answer is unclear, the model probably is too.

Elsevier's guide says this plainly: correlation is not agreement.

That warning matters because manuscripts often use correlation coefficients to imply interchangeability between methods when the real question is whether two methods agree closely enough for practical use. These are different judgments.

The same category error appears when authors use association to imply prediction or causation without the necessary design support.

Poor statistical reporting is not only about formulas. It is also about what the reader can inspect.

Common reviewer discomfort points:

• only bar charts with means and whiskers
• no sense of sample distribution
• no confidence intervals where effect estimation matters
• no denominators or attrition counts

If the figures make it hard to see the data's structure, reviewers trust the analysis less.

This is often where a statistically decent paper turns into a statistically weak one.

Examples:

• observational data written up with causal language
• non-significant findings reframed as proof of no difference
• exploratory models presented as confirmatory
• adjusted analyses described as if confounding is solved

Statistical reviewers care about language because language reveals whether the authors understand the inferential boundaries of their own work.

Statistical red flags make reviewers distrust everything else.

Once a reviewer believes the design or analysis logic is shaky, even strong domain expertise in the rest of the paper stops carrying the same weight. That is why statistical concerns so often trigger major revision rather than a few minor comments.

For many manuscripts, the most efficient workflow is:

1. fix the methods and statistics section
2. recast the headline claims to the true evidentiary limit
3. then polish the prose

That sequence is more rational than the reverse.

If you want connected guidance, pair this with how to write a methods section, how to respond to reviewer comments, and a final Manusights AI Review.

Statistical review is not about catching clever mistakes. It is about catching ordinary ways that manuscripts overstate what the data can support.

If you want to survive it, make the design defensible, the analysis explicit, the exploratory work clearly labeled, and the claims smaller than the data can bear rather than larger.