Pre-Submission Review for Machine Learning Papers

Machine learning reviewers often ask:

• what is the exact technical contribution?

• are the baselines current, strong, and fairly tuned?

• do ablations isolate the proposed contribution?

• are train, validation, and test splits clean?

• is there data leakage, benchmark contamination, or evaluation shortcutting?

• are compute, seeds, hyperparameters, and code details sufficient for reproduction?

• do results include uncertainty, multiple runs, or statistical comparison where needed?

• are limitations, failure modes, ethics, and societal impact handled honestly?

• does the venue match the claim level and artifact quality?

The paper has to survive a reviewer who tries to reproduce the logic, not only a reader who likes the idea.

Across Manusights submission reviews, ML papers most often fail when the claimed advance depends on an evaluation setup that reviewers do not trust.

Baseline weakness: the comparison set omits a recent or stronger method, or the baselines are not tuned fairly.

Ablation gap: the paper does not prove which part of the method creates the gain.

Leakage risk: preprocessing, splitting, prompt construction, feature engineering, or benchmark reuse lets information cross the evaluation boundary.

Reproducibility gap: code, environment, seeds, compute, data, or hyperparameters are too thin for another lab to rerun the core result.

Generality overclaim: a result on one benchmark family is written as a broad method claim.

A useful review should identify the first experiment a skeptical ML reviewer would ask for. In our pre-submission review work on machine-learning manuscripts, the gap between a paper that reads well and one that survives review is almost always the evidence behind the headline number, not the writing. When the baselines are the issue, we push authors to add the most recent competitive method and to report a fair hyperparameter search for it, because a reviewer who knows the field registers a missing or under-tuned baseline immediately and discounts the whole comparison. When the ablation is the issue, we ask for the table that isolates each component's contribution, since reviewers will not accept that the full method is necessary until they see what happens without each part. When leakage is the risk, we trace the split, the preprocessing, and any benchmark reuse end to end, because a single information path across the evaluation boundary invalidates the result no matter how large the gain looks. And when the reproducibility package is thin, we treat the code, seeds, environment, compute budget, and data access notes as part of the contribution, not an appendix, because an ML reviewer in 2026 reads the artifact story as evidence the result is real. The recurring lesson across these manuscripts is that the claim has to be matched to the strength of the evidence: a single-benchmark result stated as a broad method advance is the overclaim reviewers cut first.

NeurIPS says its paper checklist is designed to encourage responsible ML research, including reproducibility, transparency, ethics, and societal impact. Its guidance also says papers that do not include the checklist will be desk rejected. JMLR guidance tells authors to situate work in the broader ML literature and notes that articles may be accompanied by online appendices with data, demonstrations, source-code instructions, or source code.

Reproducibility research from the NeurIPS program identifies code availability, checklists, and reproducibility challenges as core infrastructure for ML publishing. That means pre-submission review cannot stop at prose. It needs to inspect the evidence and artifact story.

Send the manuscript, target venue, code repository or archive, environment file, data access notes, evaluation scripts, baseline implementation notes, hyperparameter search details, seed strategy, ablation tables, compute budget, model cards or dataset cards if relevant, and prior reviews if available.

If the paper uses proprietary data or large models that cannot be fully released, include the exact reproducibility compromise and what artifacts can be shared.

A useful ML pre-submission review should include:

• ML contribution verdict

• baseline and ablation critique

• evaluation and leakage-risk check

• reproducibility artifact review

• limitations and ethics review

• venue-fit recommendation

• submit, revise, retarget, or diagnose deeper call

The review should not only say "add experiments." It should name the experiment or artifact gap that will decide reviewer trust.

Before submission, authors often need to:

• add or strengthen baselines

• rerun a cleaner split or leakage check

• add ablations tied to the method claim

• report uncertainty across seeds or folds

• explain hyperparameter tuning and compute budget

• document code, environment, data, and scripts

• narrow claims from "general" to the tested setting

• retarget from a top ML conference to JMLR, an applied ML journal, or a domain venue

These fixes are often more valuable than another round of wording polish.

A new-model paper needs strong baselines, ablations, compute transparency, and error analysis. A benchmark paper needs dataset construction, leakage control, annotation or generation quality, baseline suite, and maintenance plan. A theory paper needs assumptions, proof clarity, and examples that show relevance. An applied ML paper needs domain validity, deployment context, and evaluation that matches the real decision. A generative-model paper needs contamination checks, safety limitations, and task-specific evaluation that cannot be exhausted by cherry-picked examples.

The AI manuscript review can flag whether the blocking risk is baselines, leakage, reproducibility, ethics, or venue fit.

Use this page when the submission risk depends on ML experiment quality, model contribution, benchmark design, or reproducibility. Use artificial intelligence review when the paper is broader AI systems, AI policy, human-AI interaction, robotics, or deployment governance. Use data science review when the contribution is data pipeline, analytics workflow, or applied statistical insight rather than a machine-learning method.

Do not submit an ML paper if the main result depends on a benchmark setup that a reviewer can plausibly call unfair. A small improvement can matter when evaluation is clean. A large improvement can fail if the comparison, split, or tuning protocol is suspect.

Also pause if the code story is not credible. Some venues and papers cannot release everything, but the manuscript should still explain what is reproducible, what is constrained, and how the reader can audit the core claim.

For LLM, diffusion, or foundation-model papers, pause again if the evaluation depends on examples selected by the authors. Reviewer trust improves when qualitative examples are paired with defined sampling rules, benchmark results, failure cases, and a clear explanation of what the model was not tested on.

For ML papers whose novelty is a graphics method, not a general learning method, venue fit changes. A differentiable rendering, geometry processing, simulation, animation, or computational-display paper may need the ACM Transactions on Graphics submission guide before the authors decide between TOG, SIGGRAPH, CVPR, NeurIPS, or ICLR.

For applied-ML papers already submitted to ESWA, the Expert Systems with Applications Under Review status guide explains how to read Elsevier status language without overreacting to a normal reviewer-search delay.

Ready to submit if:

• the ML contribution is precise

• baselines and ablations are strong

• evaluation avoids leakage and shortcutting

• reproducibility materials are organized

• limitations and ethics are honest

• venue fit matches the claim level

Pause first if:

• the best baseline is missing

• one benchmark carries the whole paper

• code or data cannot support the claim

• the paper overgeneralizes from narrow experiments