Back to AuditAbout

About Manusights Audit

Audit is a reimplementation, not original research. The four consistency checks — statcheck, GRIM, GRIMMER, DEBIT — were developed and published by independent researchers over the last decade. This page credits them, links their original papers, and explains what we built on top.

What Audit adds, and what it doesn’t

The math behind every Audit flag is published, peer-reviewed, and reproducible. Audit’s contribution is not the algorithms themselves but the integration:

  • LLM-based extraction from prose (statcheck’s regex extraction has documented ~60% recall on real psychology papers)
  • Single paste-and-go flow that runs all four checks together (each original tool runs only one)
  • Plain-English flag explanations, alternative-cause UX (one-tailed, correction)
  • Shareable result URL (7-day cached, content-addressed)

For batch checking of full PDF or DOCX manuscripts, the original statcheck R package remains the reference tool. Audit is built for the “paste my Results paragraph and tell me what’s wrong” pre-submission loop.

statcheck

p-value recompute from t / F / χ² / r / z reports

Developed by Michèle B. Nuijten (Tilburg University) with co-authors Chris H. J. Hartgerink, Marcel A. L. M. van Assen, Sacha Epskamp, and Jelte M. Wicherts. The original 2016 paper scanned 30,000+ psychology articles and found that ~50% contained at least one inconsistent reported p-value, with ~13% containing a decision-flipping inconsistency.

Original tool: the statcheck R package (PDF / DOCX batch processor) and statcheck.io (web upload).

  • Nuijten, M. B., Hartgerink, C. H. J., van Assen, M. A. L. M., Epskamp, S., & Wicherts, J. M. (2016). The prevalence of statistical reporting errors in psychology (1985–2013). Behavior Research Methods, 48(4), 1205–1226. https://doi.org/10.3758/s13428-015-0664-2
  • Nuijten, M. B., & Polanin, J. R. (2020). statcheck: Automatically detect statistical reporting inconsistencies. Research Synthesis Methods, 11(5), 574–579. https://doi.org/10.1002/jrsm.1408

GRIM

Granularity-Related Inconsistency of Means

Developed by Nicholas J. L. Brown (University of Groningen) and James A. J. Heathers (Northeastern University). GRIM identified numerous reporting anomalies in published psychology research and demonstrated that arithmetic-impossible means are surprisingly common.

  • Brown, N. J. L., & Heathers, J. A. J. (2017). The GRIM test: A simple technique detects numerous anomalies in the reporting of results in psychology. Social Psychological and Personality Science, 8(4), 363–369. https://doi.org/10.1177/1948550616673876

GRIMMER

Granularity-Related Inconsistency of Means Mapped to Error Repeats

Developed by Jordan Anaya (independent researcher), with subsequent formalization by Heathers and Brown. GRIMMER extends GRIM’s mean-consistency check to the standard deviation: given an integer-bounded scale and N, only certain SD values are mathematically achievable.

DEBIT

Descriptive Binary data Inconsistency Test

Developed by James A. J. Heathers, Tim van der Zee (Leiden University), and Andrew Jung, with subsequent formalization by Heathers and Brown. DEBIT tests whether a reported (proportion, SD, N) triple is mathematically consistent for binary outcomes.

  • Heathers, J. A. J., van der Zee, T., & Jung, A. (2018). DEBIT method development. OSF.
  • Heathers, J. A. J., & Brown, N. J. L. (2019). DEBIT: A simple consistency test for binary data. OSF Preprints. https://osf.io/5vb3u/

Acknowledgements

We are grateful to the broader meta-science and statistical-integrity research community whose work made this tool possible — including but not limited to Daniël Lakens, Sacha Epskamp, and Uri Simonsohn for adjacent contributions to reproducibility tooling, and to the maintainers of the Data Colada blog and stat-twitter for keeping research-integrity discourse accessible.

If you developed an algorithm Audit uses and feel the credit on this page is incomplete or inaccurate, please email us and we will fix it within 24 hours.

About Manusights

Manusights is a manuscript-readiness platform for researchers preparing papers for submission. Audit is one of three free tools we offer; the others are Compass (journal fit predictor) and Verify (citation claim checker).

Our paid product is a $29 manuscript readiness scan that runs the full statistical-rigor suite alongside methodology, citation, novelty, and journal-fit checks against your entire manuscript. The free preview is unlocked without payment; payment is only required for the full report.