Journal of Statistical Software Response to Reviewers: How to Rebut When the Reviewer Runs Your Code (2026)
How to write a point-by-point response to reviewers for the Journal of Statistical Software, where two reviewers install and run your software, so a revision must fix the code, docs, and replication script, not just the paper.
Readiness scan
Find out if this manuscript is ready to submit.
Run the Free Readiness Scan before you submit. Catch the issues editors reject on first read.
How to use this page well
These pages work best when they behave like tools, not essays. Use the quick structure first, then apply it to the exact journal and manuscript situation.
Question | What to do |
|---|---|
Use this page for | Building a point-by-point response that is easy for reviewers and editors to trust. |
Start with | State the reviewer concern clearly, then pair each response with the exact evidence or revision. |
Common mistake | Sounding defensive or abstract instead of specific about what changed. |
Best next step | Turn the response into a visible checklist or matrix before you finalize the letter. |
Quick answer: A Journal of Statistical Software response to reviewers is a mandatory point-to-point reply that travels with every revision, and the work it documents is mostly code, not prose, because the two reviewers selected by the section editor install and run your software and verify that every result is exactly reproducible. Treat a reproducibility comment as a request to fix the package, not to explain it.
Structure it like the reviewers read it: open with a short letter to the section editor, answer under Reviewer 1 and Reviewer 2, and for each change cite the page and line to reference in the revised paper plus the file and function in the revised package.
Start with the Journal of Statistical Software rebuttal readiness check before you resubmit, or work through this guide by hand. For broader cluster context, see the Journal of Statistical Software submission guide, and if the decision was a reject rather than a revise, the Journal of Statistical Software alternative venues guide routes the work elsewhere.
What does a Journal of Statistical Software response to reviewers require?
The Manusights Journal of Statistical Software rebuttal scan. This guide tells you what the section editor and the two reviewers look for in a JSS rebuttal, where the rebuttal is judged partly by re-running your code. The scan tells you whether YOUR response, your revised package, and your replication script pass that check before you resubmit. We have reviewed statistical-software manuscripts and their rebuttals targeting the Journal of Statistical Software and peer software venues; the patterns below are the same ones reviewers flag at re-review. We do not train AI on your manuscript and delete it within 24 hours.
Three things make a Journal of Statistical Software rebuttal different from a generic one:
- The point-to-point reply is mandatory. JSS requires that every revision, in any state of the review process, travel with a point-to-point reply to the reviewers and editors. The rebuttal is a hard gate, not a courtesy. 2. The reviewers run the software. The section editor selects two reviewers who evaluate correctness and usefulness, install the package, and verify that the replication material regenerates every figure, table, and number.
A fix lives in the code and the replication script, not only in the paper. 3. Acceptance stays conditional on the manuscript and package satisfying JSS style and reproducibility requirements. A reviewer recommends acceptance only after the reproducibility problems are fully resolved.
Our methodology for this guide: we reviewed the JSS information-for-authors and review-process documentation, checked it against SciRev community reports, and compared it to our own pre-submission reviews of statistical-software rebuttals. Every claim below traces to a primary source or our review corpus.
So use this guide to fix the package, the docs, and the point-to-point reply before you resubmit. At JSS a slow queue means a second round costs months you do not get back.
Element | What the Journal of Statistical Software expects | What reviewers flag at re-review |
|---|---|---|
Reply format | Mandatory point-to-point reply with every revision | Free-form prose answering all comments together |
Code fixes | Bug fixed in the package, change cited by file and function | "We explain the cause of the error in the text" with no code change |
Reproducibility | Updated replication script regenerates every result, runs in about an hour | Old replication script that no longer matches the revised paper |
Documentation | Help files, vignette, and examples updated to match the revision | New feature added instead of the requested tests or documentation |
Location | Page and line in the revised paper, file and function in the package | "We have updated the package accordingly" with no location |
Tone | Substantive on the code, gracious on style | Defensive about a reviewer who could not install the software |
Source: Journal of Statistical Software information-for-authors and review-process documentation, accessed June 2026.
The copyable Journal of Statistical Software rebuttal template
JSS reviewers read your point-to-point reply while they re-run your code, so every reply needs to point them to two places: the change in the manuscript and the change in the package. Copy this skeleton, then replace the bracketed text. Keep the reviewer text and your reply in two distinct fonts or colors.
Dear Section Editor,
Thank you for the opportunity to revise our manuscript and software, "the manuscript title" (JSS [ID]). We are grateful to the two reviewers for installing and testing the package. In response, we have fixed the [installation / example / reproducibility] issue in the package, updated the standalone replication script, expanded the [unit tests / help files / vignette], and revised the manuscript.
A point-to-point reply follows; reviewer comments are in bold and our replies in plain text, with revised-manuscript page and line numbers and the package file and function for every change. The updated replication script regenerates all results in under one hour on a standard machine.
----------------------------------------------------------------
Reviewer 1
Comment 1.1: "The package failed to install on a clean R 4.x session
because of an undeclared dependency."
Response: We agree and have fixed this. We added the missing import to
DESCRIPTION (Imports field) and to NAMESPACE, and we re-tested a clean
install on Linux, macOS, and Windows. See file DESCRIPTION and
NAMESPACE; the install step is documented in README, page 4, lines 9
to 14.
Comment 1.2: "Figure 3 did not reproduce from the supplied script."
Response: We have corrected this. The seed was set after the sampling
call in the previous script; we moved set.seed() before it in
replication.R (lines 22 to 40). Figure 3 now reproduces exactly. The
output log is provided as output.txt.
----------------------------------------------------------------
Reviewer 2
Comment 2.1: "The summary() method is undocumented and the example in
Section 4 errors."
Response: We have added a formatted help file for summary.[class]
(man/summary.[class].Rd) and corrected the Section 4 example, which
called a renamed argument. Revised example is on page 11, lines 3 to
18, and runs cleanly in the replication script.
Comment 2.2: "Add unit tests for the edge case where n = 1."
Response: We have added tests covering n = 1 and the empty-input case
(tests/testthat/test-edgecases.R) rather than the new plotting feature
we had considered, per the reviewer's priority. All tests pass.
We believe the revised manuscript and package now address each comment
and reproduce every result, and we look forward to your decision.
Sincerely,
[Corresponding author, on behalf of all authors]The template carries the four tokens that JSS reviewers scan for: a letter to the section editor, a Reviewer 1 / 2 structure, explicit action language ("we have fixed", "we have added", "we have updated"), and a page and line reference for every change, paired here with the package file and function.
The page-and-line rule, plus the file-and-function rule
Most rebuttal advice stops at one coordinate: the exact page and line for each manuscript change, plus the figure, table, or section you revised. A JSS reviewer needs a second coordinate the other journals never ask for, because they have your package open and are about to re-run it: the file and function where the code change lives.
Missing that second coordinate is the single most-cited rebuttal failure across software venues. A reviewer who has to hunt for your fix in a thirty-file package reads it as evasion, and at JSS they re-run the code anyway, so a missing location just stretches the round.
Two rules close the gap:
- Never write "we have updated the package accordingly" without naming the file and the function.
- Use line numbers from the revised manuscript, not the original, and point to the exact line in the updated replication script whenever a result changed.
Reviewer-text vs author-response typography
Make the reviewer's words and your reply visually distinct. Put each reviewer comment in bold or a colored text box, and keep your response in plain regular text directly beneath it.
Here the layout earns its keep more than at most journals. The section editor and the two reviewers read your point-to-point reply with the package open in another window, cross-referencing each reply against an actual install and run. A rebuttal where comment and reply blur together costs you exactly the attention you need while they are also testing your code.
So a clean two-font or two-color layout is not cosmetic at JSS. It is the difference between a reply the reviewer can follow line by line and one they have to untangle while a build is running.
Tone calibration: how to phrase the hard replies
The two reviewers see your tone across every comment, and one of them may be the section editor. A defensive reply to a reviewer who reported that your code would not install reads worse than the bug itself. Calibrate.
Bad (defensive or prose-only) | Better (code fixed, then pointed to) |
|---|---|
"The reviewer's environment must be misconfigured; it installs fine for us." | "We reproduced the install failure and fixed the undeclared dependency in DESCRIPTION and NAMESPACE; clean installs now pass on Linux, macOS, and Windows." |
"We explain in the text why Figure 2 may differ slightly." | "We fixed the seed ordering in replication.R (lines 22 to 40); Figure 2 now reproduces exactly, and we include the output log." |
"Documentation for that function is in the vignette." | "We have added a formatted help file (man/predict.[class].Rd) and cross-linked it from the vignette; see page 9, lines 4 to 12." |
"Adding tests is outside the scope of this revision." | "We have added the requested unit tests (tests/testthat/), covering the edge cases the reviewer named; all pass." |
"We have improved the package." | "We refactored the core solver (R/solver.R) for the numerical-stability issue the reviewer raised; results are unchanged and the test suite passes." |
The pattern that works: reproduce the problem the reviewer hit, fix it in the code, point to the exact file and line, and push back only on a request that is genuinely out of scope, with a reason and an alternative.
The Journal of Statistical Software reviewer culture you are writing into
Who reviews, and what they actually do
JSS is run by the statistical-software community, and the review is unusual because the artifact is checked as rigorously as the paper. The chain is short: the editor-in-chief assigns a section editor, an associate editor role that coordinates the review, and the section editor selects two reviewers, one of whom may be the section editor.
Those reviewers do not just read the manuscript. They install the package, run the examples, and confirm that every figure, table, and number is exactly reproducible from your replication material.
The working guideline puts a clock on it: anyone with sufficient technical background should be able to install the software and run the replication script within about one hour on a regular PC. A revision that breaks installation or slows the run materially is itself a regression the reviewers will catch.
SciRev community data, drawn from author-reported reviews, puts the first review round at roughly 9 months and total handling near 15.1 months across about three review rounds, with a self-reported handling rating near the bottom of the scale. A volunteer board and a steadily growing submission queue drive the wait.
That timeline sets a hard planning reality: a second revision round is not a few weeks, it is potentially another half-year. The fastest path through the queue is a revision that is complete the first time.
The defining feature is the point-to-point reply requirement. Revisions, in any state of the review process, must always travel with a point-to-point reply to the points made by reviewers and editors. There is no informal "we made some changes" path.
Acceptance is conditional even after a positive decision. It stays contingent on delivering a manuscript and package that satisfy the JSS style and reproducibility requirements, and a reviewer recommends conditional acceptance only once reproducibility problems are fully resolved.
The practical consequence: your JSS response to reviewers is judged partly by whether the code now does what you say it does, not only by whether your prose is persuasive.
How a JSS rebuttal differs from other venues
How this compares to the rest of the field matters for calibration. JSS sits at a strict intersection that flagship and methods journals do not occupy.
- At a flagship like Nature, Science, or PNAS, editors evaluate your rebuttal and reviewers re-read the paper, but nobody installs and re-runs your software as a condition of acceptance.
The artifact is supplementary, so a persuasive letter plus new figures can carry a revision.
- A methods journal like the Journal of Computational and Graphical Statistics or Computational Statistics also reads the response against the paper alone.
- The Journal of Open Source Software reviews openly on GitHub and centers the software, but the bar is "does it work and is it documented," not a full statistical-methods contribution.
- The R Journal reviews packages too, yet is generally faster and lighter on the manuscript.
JSS demands both at once: a real statistical-methods contribution AND production-grade, exactly-reproducible software, both re-checked on revision. Because the reviewers re-run your code, the bar for a JSS rebuttal is closer to shipping a tested software release than to writing a persuasive letter.
Key Insight
At JSS the reviewer is not just reading your rebuttal, they are re-installing your package and re-running your replication script. A reply that explains a bug without fixing it in the code fails on re-review no matter how well it is written.
What our Journal of Statistical Software rebuttal reviews surface
In our pre-submission review work with Journal of Statistical Software submissions, the rebuttals that trigger an extra revision round share a small set of recurring weaknesses, and almost all of them come from treating a code-and-reproducibility review as if it were a prose review. Across our analysis of statistical-software rebuttals, each weakness below maps to a specific, named failure pattern in the JSS review culture, and each is testable against your own draft response, revised package, and replication script before you resubmit.
Answering a "the code failed to install or run" report with a prose explanation. The most common and most expensive pattern in our Journal of Statistical Software pre-submission reviews is a rebuttal that responds to a reviewer who could not install the package, or who hit an error in an example, with a paragraph explaining why.
The reviewer will re-install on revision. If the package still fails, the explanation is irrelevant. The only adequate response is a fixed DESCRIPTION, NAMESPACE, or function, re-tested across platforms, and cited by file. Across our JSS rebuttal reviews, this mismatch between a code problem and a prose answer is the single strongest predictor of a second round.
Adding a new feature instead of the requested tests or documentation. When a reviewer asks for unit tests on an edge case, or for documentation of an exported method, a rebuttal that instead ships a new plotting option or extra functionality reads as not listening.
In our Journal of Statistical Software pre-submission reviews we routinely find authors who answered a "add tests for n = 1" comment by expanding scope, which lengthens the review without addressing the concern. Do exactly what was asked, in the methods and help files, before adding anything new.
Not updating the replication script to match the revised paper. A revision that changes a figure, a table, or a number in the manuscript but ships the old replication script forces a reproducibility failure the moment the reviewer runs it.
In our pre-submission review work with JSS manuscripts, responses that revise the paper but leave the standalone replication script stale consistently draw a re-review comment that a figure or table no longer reproduces. That adds a round at a journal where a round is months. Every manuscript change that touches a result needs a matching change in the replication material and the output log.
Letting the documentation drift from the package and the paper. A rebuttal that fixes the code but leaves the help files, vignette, or reference to functions out of sync with the revised manuscript fails the JSS consistency bar.
In our Journal of Statistical Software pre-submission reviews, the rebuttals we flag hardest are the ones where the paper, the package documentation, and the replication examples each describe a slightly different interface, because the reviewer testing reproducibility trips on the gap immediately. Reconcile the paper, the help files, the vignette, and the examples to one interface before resubmission.
Fix the code, update the replication script, do exactly what was asked, and reconcile the docs. That four-part discipline is what separates a Journal of Statistical Software rebuttal that clears one revision round from one that stalls into a second, at a journal where a second round can cost half a year. Check your JSS point-to-point reply and replication package for these patterns before you resubmit.
Readiness check
Run the scan to see how your manuscript scores on these criteria.
See score, top issues, and what to fix before you submit.
When to comply and when to push back
Situation | Recommended approach at the Journal of Statistical Software |
|---|---|
Reviewer reports the package failed to install | Comply. Fix the dependency or build, re-test across platforms, cite the file. |
Reviewer says a result did not reproduce from the script | Comply. This is the highest-leverage fix; correct the script and include the output log. |
Reviewer asks for unit tests on an edge case | Comply. Add exactly those tests; do not substitute a new feature. |
Reviewer requests a method the package genuinely should not implement | Push back with a reason, point to the design rationale, note the limitation in the paper. |
Reviewer flags documentation that does not match the paper | Comply. Reconcile help files, vignette, and examples to one interface. |
Reviewer questions whether the contribution is a true statistical-method advance | Engage substantively. If it is a re-implementation, that is a fit problem, not a wording fix. |
Source: Manusights pre-submission reviews of Journal of Statistical Software-targeted resubmissions, 2025 cohort.
How much work a Journal of Statistical Software rebuttal actually takes
Authors consistently underestimate the code-and-reproducibility effort and overestimate the writing effort. This breakdown is about workload, not the journal's decision clock; for the end-to-end picture before you ever submit, see the Journal of Statistical Software submission guide.
Rebuttal task | Where the effort goes | What it costs you |
|---|---|---|
Reproducing the reviewer's environment and failures | Confirming the install or run problem the reviewer hit | A real day, because "works on my machine" is the trap |
Fixing the package and re-testing across platforms | The actual bar for a JSS revision | The bulk of the work, often a week or more of code |
Updating the standalone replication script and output log | Making every revised result regenerate exactly | Skipped most often, and the reviewer catches it on re-run |
Reconciling paper, help files, vignette, and examples | One consistent interface across all four | More than authors expect once a function is renamed |
Writing the point-to-point reply | One reply plus page, line, file, and function per comment | Less than authors fear once the code is fixed |
Source: Manusights pre-submission reviews of Journal of Statistical Software resubmissions, 2025 cohort, last updated June 7, 2026.
Budget the code, not the letter
The writing is the cheap part of a JSS rebuttal. The expensive part is fixing the package, re-testing across platforms, and making the replication script regenerate every revised result before a reviewer runs it for you.
Honest friction: rejection on revision is real at JSS
An invitation to revise at the Journal of Statistical Software is not a soft acceptance. Acceptance stays conditional on the package and manuscript meeting JSS reproducibility and style requirements.
The revised manuscript, the code, and your point-to-point reply go back to the reviewers, who re-run the replication material. The paper can still end in rejection after re-review if the code still does not reproduce the results, or the contribution still reads as a re-implementation of existing functionality.
Most rejections at this stage trace to one cause: the author answered a code or reproducibility comment with prose instead of a fix. The second most common is a stale replication script that no longer matches the revised paper.
Think twice before you resubmit if any of these are true:
- The reply uses generic "we have updated the package" language with no file and function.
- A reviewer said the code failed to install or run and you answered with an explanation rather than a fix.
- You changed a figure or number in the paper but did not update the replication script.
- The package documentation, the vignette, and the manuscript still describe different interfaces.
- The contribution is a re-implementation of what an existing package already does, which is a fit problem no rebuttal can resolve.
Fixing the code-side issues before resubmission is what keeps a second round, and another stretch of that 15-month queue, from becoming a rejection.
Red flags a Journal of Statistical Software reviewer spots in seconds
Before you resubmit, scan your own rebuttal, revised package, and replication script for the patterns that draw an immediate re-review comment. Each is a specific, checkable thing, not a vague quality dimension.
- A prose answer to a code failure. Any reply that explains an install or runtime error without a fixed file and a re-test reads as evasion the moment the reviewer re-runs the package.
- A stale replication script. A revised figure or number in the paper that the supplied script no longer regenerates is a guaranteed reproducibility comment.
- A new feature where tests were requested. Expanding scope instead of adding the unit tests or documentation the reviewer named signals you did not listen.
- Drifting documentation. Help files, vignette, and the paper describing different argument names or methods, which the reviewer hits the instant they run the example.
How does this guide go beyond the Journal of Statistical Software author guidelines?
The official guidelines tell you that a point-to-point reply must accompany every revision and that the software must install, run, be documented, and be exactly reproducible. What they leave out is the part that changes how you write every reply:
- The reviewers re-run your code on revision, not just re-read the paper.
- A prose answer to a code failure is the single most expensive mistake you can make.
- The replication script must be updated in lockstep with every revised result.
- The volunteer queue means a second round can cost half a year.
The patterns above come from our pre-submission reviews of statistical-software rebuttals. They are testable against your own draft, package, and replication script today, not theoretical concerns.
- [Foundation for Open Access Statistics:
JSS](https://www.foastat.org/jss.html) (accessed June 2026)
- Reviews for Journal of Statistical Software, SciRev (accessed June 2026)
- Ten simple rules for writing a response to reviewers, William Stafford Noble, PLOS Computational Biology (accessed June 2026)
- Journal of Statistical Software, Wikipedia (accessed June 2026)
- Manusights pre-submission reviews of Journal of Statistical Software-targeted manuscripts (2025 cohort)
Frequently asked questions
Open with a short letter to the section editor summarizing what you changed in the software and the paper. Then answer every comment in order under Reviewer 1 and Reviewer 2, quote the reviewer text in full, state the exact code, documentation, or text change you made, and give the page and line number in the revised manuscript plus the file and function in the revised package.
Yes. Two reviewers selected by the section editor review both the software and the manuscript, verify that the package installs and runs, and confirm that every figure, table, and number is exactly reproducible from your replication script. The guideline is that anyone with sufficient technical background should be able to install the software and run the replication material within about one hour on a regular PC. A rebuttal that explains a bug in prose without fixing it in the package fails on re-review.
Long. SciRev author reports put the first review round at roughly 9 months and total handling near 15 months, driven by a volunteer board and a growing submission queue. A clean revision with a complete point-to-point reply and a working replication script is the single biggest thing you control to avoid an extra round on top of that wait.
Answering a code or reproducibility comment with prose instead of a code change. When a reviewer reports that the package failed to install, that an example threw an error, or that a result did not reproduce, the only adequate response is a fixed package and an updated replication script, with the change cited by file and function, not a paragraph explaining why it happened. Adding a new feature the reviewer did not ask for instead of the requested unit tests or documentation is the second most common.
Yes. An invitation to revise is not an acceptance, and acceptance stays conditional on delivering a manuscript and package that satisfy the JSS style and reproducibility requirements. The revised manuscript, code, and your point-to-point reply go back to the reviewers, who re-run the replication material. If the code still does not reproduce the results or the documentation still does not match the paper, the paper can be rejected after re-review.
Sources
- Information for Authors, Journal of Statistical Software (accessed June 2026)
Final step
Find out if this manuscript is ready to submit.
Run the Free Readiness Scan. See score, top issues, and journal-fit signals before you submit.
Anthropic Privacy Partner. Zero-retention manuscript processing.
Where to go next
Same journal, next question
Supporting reads
Conversion step
Find out if this manuscript is ready to submit.
Anthropic Privacy Partner. Zero-retention manuscript processing.