Structured peer review question banks
Many of our journals employ “structured peer review”, whereby you will receive a series of questions to make it easier for you to convey recommendations for improvement in a structured manner. Even if your journal does not make use of structured peer review, we strongly recommend considering these questions before starting to review a manuscript. Here follows a list of the standard question banks for different article types. When answering questions, we recommend that you number each suggestion so that the author(s) can more easily respond. If a question is not applicable or outside of your expertise, then please indicate this in your response. Please note that individual journals might employ journal-specific or additional questions to the below.
Core questions for full length research articles
Is the background and literature section up to date and appropriate for the topic?
Are the primary (and secondary) objectives clearly stated at the end of the introduction? Please provide suggestions to the author(s) on how to improve the clarity of the objectives of the study.
Are the study methods (including theory/applicability/modeling) reported in sufficient detail to allow for their replicability or reproducibility? Please provide suggestions to the author(s) on how to improve the replicability/reproducibility of their study.
Are statistical analyses, controls, sampling mechanism, and statistical reporting (e.g., P-values, CIs, effect sizes) appropriate and well described? Please clearly indicate if the manuscript requires additional peer review by a statistician. Kindly provide suggestions to the author(s) on how to improve the statistical analyses, controls, sampling mechanisms or statistical reporting.
Is the results presentation, including the number of tables and figures, appropriate to best present the study findings?
Are additional sub-analyses or statistical measures needed (e.g., reporting of CIs, effect sizes, sensitivity analyses)?
Is the interpretation of results and study conclusions supported by the data and the study design? Please provide suggestions (if needed) to the author(s) on how to improve, tone down or expand the study interpretations/conclusion.
Have the authors clearly emphasized the limitations of their study/theory/methods/argument? Please list the limitations that the author(s) need to add or emphasize.
Core questions for review articles
Do the authors explain the reason for writing a review article in this field? Please provide suggestions to the author(s) on how to better justify their reasons for writing.
Does the review article provide a good overview of the development of the field while providing insights into the field's future development? Please list the historical developments and possible future developments that the author(s) should add or emphasize more.
Do authors adequately discuss the most relevant and recent advances in the field? Please provide suggestions to the author(s) on how to improve their reference list to include the relevant topics and cover both historical and recent developments.
Is the review reported in sufficient detail to allow for its replicability and/or reproducibility (e.g., search strategies disclosed, inclusion criteria and risk of bias assessment for individual studies stated, summary methods specified)? Please provide suggestions to the author(s) on how to improve the replicability/reproducibility of their review.
Is the statistical method (e.g., meta-analysis, meta-regressions) and its reporting (e.g., P-values, 95% CIs) appropriate and well described? Please clearly indicate whether the review requires additional peer review by a statistician. Provide suggestions to the author(s) on how to improve the statistical analyses, controls, sampling mechanism, or statistical reporting.
Does the review structure, flow or writing need improving (e.g., the addition of subheadings, shortening of text, reorganization of sections or moving details from one section to another, following PRISMA guidelines)? Please provide suggestions to the author(s) on how to improve the review structure and flow.
Could the manuscript benefit from language editing?
Optional question bank for research articles
Have the authors emphasized the novelty and/or originality of the study? Please provide suggestions to the author(s) on how to better emphasize the novelty.
Is the work presented in this article of interest to the audience of the journal/topic (e.g., data science researchers)? Please provide suggestions to the author(s) on how to broaden the scope of their work.
Are these serious flaws that invalidate the study or make it unpublishable? Please provide suggestions to the author(s) on how to improve their study design and/or methodology.
Does the literature section need expansion or improvement (e.g., are any key citations missing or are citations excessive)? Please provide suggestions to the author(s) on how to provide a broad and balanced review of current research and place their work in the context.
Does the explanation of the generalizability of the findings and/or comparison with other studies need expanding? Please provide suggestions to the author(s) on how to improve the manuscript.
Does the title need improving? Please provide suggestions to the author(s) on how to improve the title.
Does the abstract capture the main research findings or need improving? Please provide suggestions to the author(s) on how to improve the abstract.
Does the highlights section capture the main research finding or need improving? Please provide suggestions to authors on how to better capture the main research findings.
If the article reports on code, software, algorithms, or raw data, have you reviewed the accuracy, validity, and FAIRness of these?
Optional question bank for review articles
Does the title need improving? Please provide suggestions to the author(s) on how to improve the title.
Does the abstract capture the main research findings or need improving? Please provide suggestions to the authors(s) on how to improve the abstract.
Does the highlights section capture the main research findings or need improving? Please provide suggestions to authors on how to better capture the main research findings.