Guidance: Rigor and Reproducibility in Grant Applications

The NIH is committed to promoting rigorous and transparent research in all areas of science supported by a variety of grant programs. Grant application instructions and the criteria by which reviewers are asked to evaluate the scientific merit of the application include components that address reproducibility, rigor, and transparency. Learn how to address rigor and reproducibility in your grant application and discover what reviewers are looking for as they evaluate the application for scientific merit.

Overview of Guidelines for Rigor in Your Application

Infographics on new grant guidelines, including information on the four key areas of rigor, and details the updates to research strategy guidance, authentication, and new reviewer guidelines.

infographic thumbnail image fpr rigor infographic
Infographic courtesy of Ms. Nichole Swan, Dr. Shana Spindler, and Dr. Yvette Pittman of the
Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD).
 
infographic thumbnail image for rigor and reproducibility chart
Enhancing Reproducibility in NIH Applications: Resource Chart
Infographic courtesy of OER
   



Research Grants and Mentored Career Development Awards

Beginning with applications due on January 25, 2019 the application instructions and review criteria will be clarified to replace the term “scientific rigor” with the term "rigor of the prior research". Applicants will also be instructed to describe plans to address any weaknesses in the rigor of prior research within the Research Strategy. For additional details, see NOT-OD-18-228 and NOT-OD-18-229.

NIH research grant and career development award application instructions
and review language focus on four key areas:


1. The rigor of the prior research
  • A careful assessment of the rigor of the prior research that serves as the key support for a proposed project helps to identify weakness or gaps in a line of research. NIH expects applicants to describe the general strengths and weaknesses in the rigor of the prior research (both published and unpublished) that serves as the key support for the proposed project. It is expected that this consideration includes attention to the rigor of the previous experimental designs, as well as the incorporation of relevant biological variables and authentication of key resources. Applicants are expected to include plans to address any weaknesses or gaps identified.
  • See related FAQs    Blog Post
2. Rigorous experimental design for robust and unbiased results
  • Scientific rigor is the strict application of the scientific method to ensure robust and unbiased experimental design, methodology, analysis, interpretation and reporting of results. NIH expects full transparency in proposing and reporting experimental details so that reviewers may assess the proposed research and others may reproduce and extend the findings.
     
  • See related FAQs    Blog Post
3. Consideration of relevant biological variables
  • Biological variables, such as sex, age, weight, and underlying health conditions, are often critical factors affecting health or disease. In particular, sex is a biological variable that is frequently ignored in animal study designs and analyses, leading to an incomplete understanding of potential sex-based differences in basic biological function, disease processes and treatment response.
  • NIH expects that sex as a biological variable will be factored into research designs, analyses, and reporting in vertebrate animal and human studies. Strong justification from the scientific literature, preliminary data or other relevant considerations must be provided for applications proposing to study only one sex.
     
  • See related FAQs   Blog Posts   Reviewer Guidance   Article Link to Non-U.S. Government Site - Click for Disclaimer 
4. Authentication of key biological and/or chemical resources
  • Key biological and/or chemical resources include, but are not limited to, cell lines, specialty chemicals, antibodies and other biologics. Key biological and/or chemical resources may or may not be generated with NIH funds and:
    • may differ from laboratory to laboratory or over time;
    • may have qualities and/or qualifications that could influence the research data;
    • are integral to the proposed research.
  • The quality of resources used to conduct research is critical to the ability to reproduce the results. Each investigator will have to determine which resources used in their research fit these criteria and are therefore key to the proposed research.
     
  • See related FAQs   Blog Post
     

 

Institutional Training Grants, Institutional Career Development, and Individual Fellowships

The NIH plans to require formal instruction in rigorous experimental design and transparency to enhance reproducibility for institutional training, institutional career development, and individual fellowship applications no sooner than 2017. See NOT-OD-16-034

When implemented, applications will be expected to provide the following:

Institutional training grant applications will be required to include within the training program plan a summary of the instruction planned for all predoctoral and postdoctoral trainees to ensure the knowledge and skills required to design and conduct rigorous, well-controlled experiments that consider all relevant biological variables, use authenticated biological and chemical resources, and apply appropriate statistical tests for data analyses.  In addition, a separate attachment will be required to describe in more detail the instructional content and curricular content. 

Institutional career development applications (K12/KL2) will be required to include within the career development program plan a summary of the instruction planned for all scholars to ensure the knowledge and skills required to design and conduct rigorous, well-controlled experiments that consider all relevant biological variables, use authenticated biological and chemical resources, and apply appropriate statistical tests for data analyses.  In addition, a separate attachment will be required to describe in more detail the instructional content and curricular content. 

Individual fellowship applications will be required to summarize in the research strategy section plans to ensure rigorous, well-controlled experiments that consider all relevant biological variables, use authenticated biological and chemical resources, and apply appropriate statistical tests for data analyses.  In addition more detailed description of instruction in rigorous experimental design to ensure reproducibility will be required in the section on Institutional Environment and Commitment to Training.