Referral & Review

Streamlined Review Procedures Used In CSR

Background: Although a "triage" procedure has been used for a number of years at the NIH for the review of applications submitted in response to certain Requests for Applications (RFAs), the Center for Scientific Review (CSR) only recently began to explore the utility of such a process for the review of investigator-initiated grant applications. Basically, reviewers are asked to categorize applications as either in the upper half or lower half in quality. Applications in the upper half are given full discussion at the study section meeting and receive a priority score, and are routinely taken to advisory council for second-level review. All other applications are not discussed nor scored at the study section meetings. After a series of pilot studies and beginning with the February 1995 round of study section meetings, all CSR regular study sections have incorporated a streamlined review process as part of their peer review procedures. In 1997, this process was extended to the review of SBIR and STTR applications.

Benefits of the Streamlined Review Process: The procedures used in CSR have been designed to enhance the review process in several substantial ways:

  • Only those applications judged highly meritorious are discussed at the study section meeting. Since it is intended that only approximately one-half of the applications will be placed in this category, there is ample time for in-depth discussion of each of these applications.
  • In many instances, the process results in significantly shorter study section meetings and some savings in costs.
  • Assigned reviewers are encouraged to focus written critiques primarily on major strengths/weaknesses, issues and concerns. (This provides applicants with a better understanding of the major concerns they must address if preparing a revised application.)
  • As part of the streamlined review process, the summary statements consist primarily of reviewers' prepared critiques, essentially unaltered. Production of summary statements in this fashion reduces the work of SRAs and GTAs after the study section meeting, allowing prompt feedback to applicants.

Streamlined Review Procedures: To carry out this process most effectively, the upper half of applications is tentatively identified prior to the study section meeting. "Upper half" simply means the approximate upper half, in quality, of applications assigned to the study section. More precisely, these are the applications which reviewers believe represent qualitatively the upper half of research customarily reviewed, round to round, in their study section. The streamlined review procedure is as follows:

  • By a predetermined date prior to the study section meeting, assigned reviewers/readers are asked to identify to the Scientific Review Administrator (SRA) of the study section any of their assigned applications that they believe do not fall within the upper half.
  • A few days prior to the study section meeting, all study section members receive a list from the SRA of those applications proposed by at least two assigned reviewers/readers to be excluded from the upper half, so that all members might give additional attention to these.
  • At the beginning of the meeting, the list is read aloud for final concurrence by the entire study section. Non-concurrence by only one member (whether regular or temporary) is sufficient to bring an application to full review at the meeting. Occasionally, it may also happen that study section members will unanimously agree, either at the outset of the meeting or later, during discussion of applications, to designate additional applications as not requiring full discussion and scoring.
  • For applications not in the upper half, only reviewers' critiques, essentially unaltered, are incorporated into the summary statement and provided to the investigator, along with a paragraph briefly describing the review process and the resultant summary statement.
  • For those applications that are considered to be in the upper half and therefore scored, reviewers are asked to modify their critiques appropriately during the meeting, removing, for example, criticisms that are negated through discussion among reviewers. Otherwise, the reviewers' critiques are included in the summary statement, essentially unaltered by the SRA.
  • Additionally, the SRA prepares a "Resume and Summary of Discussion" section that conveys the highlights (i.e., major strengths and weaknesses identified) of the discussion at the study section meeting leading to the final rating.
  • The full range of priority scores from 100 to 500 is not used for the applications in the upper half. Logically, if precisely half of the applications are unscored, when scoring an application, members should assign a score of approximately 300 for an application of "average" quality, and distribute scores for applications in the upper half across the range from 100 to 300. However, if more than 50% of the applications are designated in the "upper half," scores beyond that median may be assigned. In addition, reviewers are free to "vote their conscience." That is, if a reviewer maintains that an application is not in the upper half, despite full discussion and general consensus, he/she is free to provide the priority rating he/she believes is appropriate.

Two Additional Notes:

  • All competing R01 applications reviewed by a study section, whether scored or not, are considered in the base for calculating percentiles. Thus there is no mathematical advantage to scored applications if the "upper half" of applications constitutes more than 50% of the total.
  • R13, R18, F06, and F32/33 applications are not currently subjected to a streamlined review process in CSR, although a modified summary statement format may be employed.

[Referral & Review]

[DRG Home
Page]