You are here

Resources > Journal Authors/Editors/Reviewers > Journal Author Gateway > Manuscript Submission Guidelines

Registered Reports Author Submission Guidelines

 

What is a Registered Report

Registered Reports are manuscripts that are reviewed and approved “in principle” prior to data collection and/or analysis. 

This page summarizes information that specifically applies to Registered Reports. For general information regarding the Journal’s aims and scope, review process, data policy, and formal details regarding manuscript preparation, submission, and production, please read the Journal’s submission guidelines.

The Registered Reports model is an extension of a pre-registration study and refers to a type of research article. In the format, manuscript writing and review occurs in two stages. There are different forms of "Registered Reports", depending on what is being registered (e.g., a planned analysis on data that already exist vs. planned analysis on data that have yet to be gathered).  

Stage 1: Peer review of the study protocol, including introduction, methods and analysis plan, occurs before the research is conducted (Chambers, 2019). The decision to publish is made before the study is run and is based on the importance of the research question and rigour of the methods. Authors receive an in-principle acceptance (IPA), a commitment from the journal to publish the study irrespective of the results.   

Stage 2: Providing the researcher follows the Stage 1 protocol and obtains an IPA, the researcher can complete their study knowing that it will be published regardless of the results. Following completion of the study, authors will complete the manuscript, including Results and Discussion sections, and it will be sent for review. Reviewers will check whether the researcher has adhered to the Stage 1 plans, and whether the conclusions reflect the data.   

Stage 2 manuscripts will more closely resemble a regular research article format.

Reasons to do a Registered Report

As explained in detail in the FAQs, Registered Reports offer an elegant and straightforward way to increase the representativeness, trustworthiness, and robustness of the field’s findings. They also offer a number of advantages for authors, including in-depth conceptual and methodological feedback before the start of data collection and/or analysis (i.e., feedback that can indeed be considered without having to start anew) as well as a much faster and result-independent guarantee regarding the publication of one’s research.

Initial submissions of Registered Reports will be evaluated by the Editor/s, in the same way as other article formats. The Editor/s will decide whether to send the article to peer review (Stage 1). Following review, the article will then be either rejected, sent for revision, or accepted in principle for publication. Following in principle acceptance (IPA), the authors will then proceed to conduct the study, adhering exactly to the peer-reviewed procedures. When the study is complete the authors will submit (as a revision) their full manuscript for re-review (Stage 2). Pending quality checks, a sensible interpretation of the findings and a high-quality write up and formatting, the manuscript will be published regardless of the results.

 

Stage 1: Initial manuscript submission and review

Stage 1 submissions should include the manuscript (details below) and a brief cover letter.

The Stage 1 cover letter must include (submissions not including this information will be desk-rejected):

  • A brief scientific case for consideration in the case of novel studies. Authors who want to propose a replication study are encouraged to make a case for the scientific value of this replication (e.g., with regards to impact to the field or importance for increased precision of effect size). High-value replication studies are welcome and will be treated with equal priority to novel studies.
  • A statement confirming that all necessary support (e.g. funding, facilities) and approvals (e.g. ethics) are in place for the proposed research. We recommend having all ethical reviews and approvals in place so that studies can start immediately following in-principle acceptance. However, authors with alternative plans are encouraged to contact the Editor for advice.
  • An anticipated timeline for completing the study if the initial submission is accepted. Note that failure to submit the Stage 2 manuscript by the intended submission date will lead to a withdrawal of the manuscript unless an extension is agreed with the Journal office (see below for further details regarding the withdrawal process).
  • A statement confirming that the authors agree to share their anonymized raw data, digital study materials (including, for example, instructions, stimuli, variables, experiment code, coding and rating systems) and their analysis code, as appropriate. Please ensure you have considered the sharing of data in your ethics submissions and participant-facing documents.
  • A statement confirming that, following Stage 1 IPA, the authors agree to register their approved protocol on a recognised open repository such as Open Science Framework, Sage’s advance or some other recognised open repository, either publicly or under private embargo until submission of the Stage 2 manuscript. Accepted protocols can be quickly and easily registered using a tailored mechanism for Registered Reports on the Open Science Framework: https://osf.io/rr/
  • A statement confirming that if the authors withdraw their paper following IPA, they agree to publish a short summary of the pre-registered study on the open repository page under a headline Withdrawn Registrations.

Manuscript preparation guidelines – Stage 1

Initial Stage 1 submissions should include the following sections:

Introduction:

  • A review of the relevant theoretical and empirical literature that motivates the research question and a full delineation and description of the hypotheses (where appropriate). Please note that following the IPA, the Introduction section cannot be altered apart from correction of typographic errors and altering of tense from future to past (see below).

Methods:

  • Full description of proposed sample characteristics, including criteria for subject inclusion and exclusion, and detailed description of procedures for defining outliers. Procedures for objectively defining exclusion criteria due to technical errors or for any other reasons must be documented, including details of how and under what conditions subjects would be replaced.
  • A description of study procedures in sufficient detail to allow another researcher to repeat the methodology exactly, without requiring further information. These procedures must be adhered to exactly in the subsequent execution of the study or any Stage 2 manuscript may be rejected. If any deviations need to be made to the pre-registered methods, consult the Editor in advance. Please note that reviewers at Stage 1 will be asked to specifically consider whether the stated study procedures contain sufficient detail to prevent undisclosed procedural flexibility (data transformations, aggregations, inclusion of co-variates, etc).
  • Proposed analysis pipeline, including all pre-processing steps from raw data onwards, and a precise description of all planned analyses, including appropriate correction for multiple comparisons. Any covariates or regressors must be stated. Proposed analyses involving covariates must be reported with and without the covariate(s) included. Where analysis decisions are contingent on the outcome of prior analyses, these contingencies must be specified and adhered to. Only pre-planned analyses can be reported in the main Results section of Stage 2 submissions. However, unplanned post hoc analyses will be admissible in a separate section of the Results (see below).
  • Interpretative plan, including specification of which outcomes will be interpreted as support or disconfirmation of the proposed hypotheses, for each of the proposed analyses. In each case, authors should include a statement of what result would be taken as consistent with the prediction, what result would be taken as disconfirmation, and what result (if any) would be taken as inconclusive.
  • Since publication bias overinflates published estimates of effect size, power analysis should be based on the lowest available or meaningful estimate of the effect size. In the case of highly uncertain effect sizes, a variable sample size and interim data analysis will be permissible but with inspection points stated in advance, appropriate Type I error correction employed, and a final stopping rule for data collection outlined.
  • Full descriptions must be provided of any outcome-neutral criteria that are required for successful testing of the stated hypotheses. Such ‘reality checks’ might include the absence of floor or ceiling effects, or positive controls. Please note that reviewers will be asked to judge whether the manuscript includes sufficient specification of reality checks.
  • Timeline for completion of the study and proposed resubmission date if registration review is successful. Extensions to this deadline can be negotiated with the Editor.
  • Any description of prospective methods or analysis plans should be written in future tense.

Pilot Data

  • Optional pilot data can be included to establish reality checks, effect size estimations, feasibility, or proof of principle. Any pilot experiments will be published with the final version of the manuscript and will be clearly distinguished from data obtained for the main experiment(s).

Secondary Registrations

The journal welcomes submissions proposing secondary analyses of existing data sets, provided authors can supply sufficient evidence (e.g. self-certification; letter from independent gatekeeper) to confirm that they have had no prior access to the data in question nor to summary reports of the data through descriptive or inferential statistics or narrative descriptions of the data, in talks, papers, or personal communication with others). For advice on the eligibility of specific scenarios, authors are welcome to contact the Editor.

What reviewers look for in a Stage 1 submission

Stage 1 submissions that are judged by the editorial board to be of sufficient quality and rigor will be sent for peer review. In considering papers at the registration stage, reviewers will be asked to assess:

  • What is the main question being addressed in the study and how important is it to the field? 
  • Is everything clearly and transparently presented so it is clear what will be done?  
  • What are the key independent and dependent variable(s), and how they will be measured? 
  • What are the authors’ hypotheses, if applicable? 
  • Whether the data can test the authors’ proposed hypotheses  
  • Is there a coherent connection between theory/hypotheses and methods and analyses? If not, how can they be strengthened? 
  • How many and which conditions will participants/samples be assigned to? 
  • How many observations will be collected and what rule will be used to terminate data collection? 
  • What are the study inclusion criteria? 
  • What are the data exclusion criteria? 
  • What positive controls or quality checks are in place to provide a fair test of the stated hypotheses? 
  • Which analyses will be used to examine the main question/hypothesis? 
  • For empirical registered reports, where possible, authors should submit the code or syntax they plan to run on the data. That way, data-analytical steps can be better traced by reviewers, and errors can still be spotted. 
  • Are the authors proposing to collect new data or analyse existing data? This distinction is important for transparency.  

Following Stage 1 peer review, manuscripts will be either rejected outright, offered the opportunity to be revised, or accepted. Manuscripts that pass peer review will be issued an IPA, indicating that the article will be published pending successful completion of the study according to the exact methods and analytic procedures outlined, as well as there is a defensible and evidence-bound interpretation of the results, writing is coherent and clear, and the manuscript is formatted according to the journal’s author guidelines.

Please note that any deviation from the stated study procedures, regardless of how minor it may seem to the authors, could lead to rejection of the manuscript. In cases where the pre-registered protocol is altered after IPA due to unforeseen circumstances (e.g. change of equipment or unanticipated technical error), the authors must consult the Editor immediately for advice, and prior to the completion of data collection. Minor changes to the protocol may be permitted according to editorial discretion. In such cases, the IPA would be preserved and the deviation reported in the Stage 2 submission. If the authors wish to alter the study procedures more substantially following an IPA but still wish to publish their article as a Registered Report then the manuscript must be withdrawn and resubmitted as a new Stage 1 submission. Note that registered analyses must be undertaken, but additional unregistered analyses can also be included in a final manuscript (see below).

Manuscript preparation guidelines – Stage 2

Once the study is complete, authors prepare and resubmit their manuscript for full review, with the following additions:

Cover letter: the Stage 2 cover letter must include:

  • a link in the method section to the public archive containing anonymized study data, digital materials, and/or statistical code. 
  • a link in the method to the approved Stage 1 protocol on the OSF or other recognized repository.
  • for primary Registered Reports, no data for any pre-registered study (other than pilot data included at Stage 1) was collected prior to the date of the IPA.
  • for secondary Registered Reports, authors should confirm that no data (other than pilot data included at Stage 1) was subjected to the pre-registered analyses prior to the IPA, and that authors had no prior access to the data in question nor to summary reports of the data through descriptive or inferential statistics or narrative descriptions of the data, in talks, papers, or personal communication with others.
  • Submission of anonymized raw data, digital study materials, and laboratory log
  • Anonymized raw data and digital study materials must be made freely available in a public repository with a link provided within the Stage 2 manuscript. Authors are free to use any repository that renders data and materials freely and publicly accessible and provides a digital object identifier (DOI) to ensure that the data remain persistent, unique and citable.
  • Where authors, ethics committee, regulatory bodies, or others have ethical concerns about making data open, we expect authors to do what is within their ability to make as much of their data as open as possible. Please provide justification for any data that cannot be shared. No data shared should identify research participants.
  • Data files should be appropriately time stamped to show that data were collected after the IPA and not before. Other than pre-registered and approved pilot data, no data acquired prior to the date of the IPA is admissible in the Stage 2 submission. Raw data must be accompanied by guidance notes, where required, to assist other scientists in replicating the analysis pipeline. Authors are required to upload any relevant analysis scripts and other experimental materials that would assist in replication (e.g. stimuli & presentation code).
  • Any supplementary figures, tables, or other text (such as supplementary methods) can either be included as standard Supplemental Material (see Journal’s author guidelines for more details) that accompanies the paper, or they can be archived together with the data. Please note that the raw data itself should be archived (see above) rather than submitted to the journal as Supplemental Material.
  • Where appropriate, a basic laboratory log must also be provided outlining the range of dates during which data collection took place. This log should be uploaded to the same public archive as the data and materials.

Introduction and Methods

  • Apart from minor stylistic revisions, the Introduction cannot be altered from the approved Stage 1 submission, and the stated hypotheses cannot be amended or appended. At Stage 2, any description of the rationale or proposed methodology that was written in future tense within the Stage 1 manuscript should be changed to past tense. Any textual changes to the Introduction or Methods must be clearly marked in the Stage 2 submission. Depending on the timeframe of data collection, new relevant literature may have appeared between Stage 1 and Stage 2. Any such literature should be covered in the Discussion.

Results & Discussion

  • These will be similar to standard original research reports but with added requirements. The outcome of all registered analyses must be reported in the manuscript, except in rare instances where a registered and approved analysis is subsequently shown to be logically flawed or unfounded. In such cases, the authors, reviewers, and editor must agree that a collective error of judgment was made and that the analysis is inappropriate. In such cases the analysis would still be mentioned in the Methods but omitted with justification from the Results.
  • It is reasonable that authors may wish to include additional analyses that were not included in the registered submission. For instance, a new analytic approach might become available between the IPA and full review, or a particularly interesting and unexpected finding may emerge. Such analyses are admissible but must be clearly justified in the text, appropriately caveated, and reported in a separate section of the Results titled “Exploratory analyses”. Authors should be careful not to base their conclusions entirely on the outcome of statistically significant post hoc analyses.

The resubmission will ideally be considered by the same reviewers as in the Stage 1 submission but could also be assessed by fresh reviewers. In considering papers at Stage 2, reviewers will be asked to decide:

  • Whether the introduction, rationale and stated hypotheses are the same as the approved Stage 1 submission (required) 
  • Whether the authors adhered precisely to the registered experimental procedures 
  • Where appropriate, floor and ceiling effects, positive controls or quality checks, etc, should be reported. Failure to pass these conditions may lead to manuscript rejection.  
  • If there were many and/or serious deviations from the pre-registration
  • How the findings impact the overall paper and conclusions
  • Where applicable, whether any unregistered exploratory statistical analyses are justified, methodologically sound, and informative.  
  • Whether the authors’ conclusions are justified given the data. Please note that editorial decisions will not be based on the perceived importance, novelty, or conclusiveness of the results.  
  • If the paper written in a coherent and sound manner so as to clearly and efficiently communicate the research procedures and insights.  

Crucially, reviewers will be informed that editorial decisions will not be based on the perceived importance, novelty, or conclusiveness of the results. Thus, while reviewers are free to enter such comments on the record, they will not form a valid basis for editorial decisions.

Manuscript Withdrawal and Withdrawn Registrations

It is possible, although very rare, that authors with an IPA may wish to withdraw their manuscript during or following the research. Possible reasons could include (but are not limited to) a major technical error, inability to complete the study due to unforeseen circumstances, or the authors declining to revise the Stage 2 manuscript as requested by the Editor. In all such cases, manuscripts can of course be withdrawn at the authors’ discretion. However, the Journal will publicly record each withdrawal by marking the Stage 1 recommendation as "withdrawn" and including a statement in the Stage 1 recommendation explaining the key facts and the reason for the withdrawal. If the recommended Stage 1 protocol was registered under a private embargo, then this will also be released and the protocol made public. 

Withdrawn manuscripts will be hosted on the open repository where the study was pre-registered.

Partial withdrawals are generally not possible; i.e. with a regular RR, authors cannot publish part of a registered plan by selectively withdrawing one of the planned studies. Such cases must lead to withdrawal of the entire paper.

Should Stage 1 IPA be forthcoming, authors will be asked to provide the Journal with an estimated submission date for the completed Stage 2 manuscript. This deadline can be readily altered in consultation with the Editor (e.g. in case of delays requiring additional time to complete the research). However, in the event that the authors (a) fail to submit the Stage 2 manuscript within the designated number of months of the mutually agreed deadline, while also (b) becoming non-responsive during this period to enquiries, then the manuscript will be considered by the Journal to be withdrawn.

Tips for Avoiding Desk Rejection at Stage 1

Many Registered Report submissions are desk rejected at Stage 1, prior to in-depth review, for failing to sufficiently meet the Stage 1 editorial criteria. In many such cases, authors are invited to resubmit once specific shortcomings are addressed, although major problems can lead to outright rejection. To help minimize the chances of authors’ submissions being desk rejected, we list below the top ten reasons why Stage 1 submissions are rejected prior to review.

1. Cover letter doesn’t make necessary statements concerning ethics, data archiving, and so forth (see above).

2. The protocol contains insufficient methodological detail to enable replication and prevent researcher degrees of freedom. One commonly neglected area is the criteria for excluding data, both at the level of animals/participants and at the level of data within animals/participants. In the interests of clarity, we recommend listing these criteria systematically rather than presenting them in prose.

3. Lack of correspondence between the scientific hypotheses and the pre-registered statistical tests. This is a common problem and severe cases are likely to be desk rejected outright. To maximize clarity of correspondence between predictions and analyses, authors are encouraged to number their hypotheses in the Introduction and then number the proposed analyses in the Methods to make clear which analysis tests which prediction. Ensure also that power analysis, where applicable, is based on the actual test procedures that will be employed to test those hypotheses; e.g. don’t propose a power analysis based on an ANOVA but then suggest a linear mixed effects model to test the hypothesis.

4. Power analysis, where applicable, fails to reach the minimum level stated in journal policy.

5. Power analysis is over-optimistic (e.g. based on previous literature but not taking into account publication bias) or insufficiently justified (e.g. based on a single point estimate from a pilot experiment or previous study). Proposals should be powered to detect the smallest effect that is plausible and of theoretical value. Pilot data can help inform this estimate but is unlikely to form an acceptable basis, alone, for choosing the target effect size.

6. Intention to infer support for the null hypothesis from statistically non-significant results, without proposing use of Bayes factors or frequentist equivalence testing.

7. Inclusion of exploratory analyses in the analysis plan. Manuscripts proposing exploratory analyses will usually be desk rejected until such analyses are removed because inclusion of exploratory “plans” at Stage 1 blurs the line between confirmatory and exploratory outcomes at Stage 2. Instead, such analyses can be included at Stage 2 and need not be pre-registered. Under some circumstances, exploratory analyses could be discussed at Stage 1 where they are necessary to justify study variables or procedures that are included in the design exclusively for exploratory analysis.

8. Failure to clearly distinguish work that has already been done from work that is planned. Where a proposal contains a mixture of pilot work that has already been undertaken and a proposal for work not yet undertaken, authors should use the past tense for pilot work but the future tense for the proposed work. At Stage 2, all descriptions shift to past tense.

9. Lack of pre-specified positive controls or other quality checks, or an appropriate justification for their absence (See Stage 1 criterion 5). We recognise that positive controls are not possible with all study designs, in which case authors should discuss why they are not included.

10. Where applicable, lack of power analysis within proposed positive controls that depend on hypothesis testing.

References

Chambers, C. 2019. What's next for registered reports?. Nature 573(7773), pp. 187-189. (10.1038/d41586-019-02674-6) 

Note

Our thanks to Dr Cathy Manning, Registered Reports Editor at Autism, Professor Chris Chambers at Cardiff University, and the Editors, past and present, of the European Journal of Personality for their input on these guidelines.