|
 Peer
Review Policy |
Date of Issue: September 27, 2002
CIO Contact: Office of the Director, Associate Director for Science
Material Superseded: None
Sections:
Exhibit:
Table Shells for Reporting Data to Monitor Implementation
| I.
PURPOSE |
This policy
provides guidance on the implementation of expanded and consistent external
merit (peer) review (hereafter, peer review) of extramural research and
intramural research at CDC. All subsequent references to CDC also include
the Agency for Toxic Substances and Disease Registry.
Return
to top
| II.
BACKGROUND |
The concept
of peer review is strongly accepted by the scientific community and provides
confidence to management, academic and other partners, various branches
of government, and the public that funds appropriated for research will
support the most meritorious research ideas and projects. Since 1994 the
Office of Management and Budget has expected Federal Research and Development
Agencies to enhance the utilization of merit review with peer review for
competitive selection of research projects. More recently, in January
2002, OMB issued “Guidelines for Ensuring and Maximizing the Quality,
Objectivity, Utility, and Integrity of Information Disseminated by Federal
Agencies” (See Reference 1) that regard technical information that
has been subjected to formal, independent, external peer review as presumptively
objective. The HHS and CDC are subject to these guidelines, which mandate
not only peer review but also other quality control processes to ensure
utility and integrity of disseminated information, as of October 1, 2002.
This proposed policy will make CDC practices more consistent with other
HHS OPDIVS such as AHRQ and NIH, and strengthen CDC’s visibility
and credibility as a research-focused public health agency.
Return
to top
| III.
DEFINITIONS |
A.
Peer Review
Peer review is a process that includes independent assessment of research
by panels of experts who provide written assurance that their reviews
are free of real or perceived conflicts of interest. Typically, for extramural
research, two sequential levels of review include a first level review
by panels of experts for the purpose of evaluating the scientific and
technical merit of the applications. A second level of review involves
a separate senior advisory panel whose purpose is to factor in the preliminary
recommendations (merit evaluations and rankings) from the first level
review, with program, policy, and funding level considerations.
B.
Research
Decisions whether an extramural project is research or non-research (such
as surveillance, building infrastructure, evaluation, or public health
practice activities) should be based on guidance in the CDC Human Subjects
Research document “Guidelines for Defining Public Health Research
and Public Health Non-Research” (See Reference 2) available at (http://intranet.cdc.gov/od/ads/opspoll1.htm).
In this document, the determination is made by examining the intent of
the project: the intent of research is to generate generalizable knowledge
to improve public health practice. Consistent with existing policy, the
CIO Human Subjects Contact is responsible for determining if the project
is research. Consultation with the CIO Associate Director for Science
(ADS) is encouraged when questions arise (See Reference 2).
Decisions
on whether a contract is research should be based on guidance in the CDC
Project Officers’ Training Manual (Research and Development version
(See Reference 3) and the NIH Manual Chapter 6315-1- Initiation, Review,
and Evaluation, and Award of R&D Contract Projects (See Reference
4). Research contracts are referred to by the term Research and Development
(R&D), which includes research, development, demonstration and support
activities. Research contracts do not include acquisitions to provide
general supplies and services, construction, and information technology
resources. They also do not include the conduct of program evaluations,
public or technical information services or clearinghouses, scientific
conference or logistics support, or other services unrelated to performing
or supporting R&D.
C.
Extramural Research
Research activities funded through a grant or cooperative agreement instrument
(assistance relationship).
D.
Intramural Research
Research supported by CDC and conducted by CDC staff (employees, contractors,
visiting scientists, fellows, and students) in its own facilities or its
components (research programs and research studies) and scientific investigations
and developmental efforts performed by other organizations through contracts.
Research programs are typically the mission related research agenda or
portfolio for the CIO. Research studies include projects undertaken by
CDC scientists that involve research findings intended for dissemination
and that are not funded through assistance (grant or cooperative agreement)
or acquisition (contract).
Return
to top
| IV.
POLICY |
A.
Extramural Research
All
extramural research awarded or conducted by CDC on or after October 1,
2005 (FY2006) will be subject to peer review, except in emergency situations.
In such situations, the CIO Director can explain the emergency and request,
with justification, an exclusion from this policy which can be granted
by the Deputy Director for Science and Public Health or his/her designee.
This
policy applies to:
- Research
funded by grants or cooperative agreements;
- Research
as a component of a non-research announcement;
- If any
research project is a part of the announcement, all components of the
proposal must be peer reviewed;
- Institutional
awards to research centers to support centralized resources and facilities
shared by investigators conducting research.
Timeline
for Implementation
Peer review of extramural research projects will be phased in over three
years by proportion of total number of new announcements (which includes
competing continuations) for awards by a CIO. The denominator includes
announcements any component of which is research.
Targets
for the proportion of total research announcements subject to peer review
in each CIO:
- 1 of
3 (or 33%) for FY2003 announcements;
- 2 of
3 (or 66%) for FY2004 announcements;
- 9 of
10 (or 90%) for FY2005 announcements;
- All FY2006
announcements.
B.
Intramural Research
Intramural
Research Programs
By
October 1, 2005 (FY2006), all CIOs will initiate peer review of intramural
research programs by Federal Advisory Committee Act (FACA) chartered advisory
committee, board of scientific counselors, or special emphasis panel.
This review will take place at least once every five years.
Timeline
for implementation: CIOs are to implement the first five-year cycle of
review starting October 1, 2002, with review of all intramural research
programs at least once by October 1, 2007.
Return
to top
Intramural
Research Studies
By October 1, 2005 CIOs will use external experts to review major studies
for scientific and technical quality at inception, and will review all
studies for scientific and technical quality at least once every five
years. CIOs will also review research results from studies prior to dissemination.
Major studies
are defined as those with large budget or staff commitments, or projects
likely to produce findings of high importance or interest. For the ongoing
review of studies every five years, studies may either be reviewed singly,
or as part of a portfolio.
The peer
review prior to dissemination may be conducted using the existing mechanisms
many journals use to peer review manuscripts or may be performed separately
by the CIOs using external experts or internal experts not affiliated
with the program and without conflict of interest.
Timeline
for implementation: Peer Review at inception of a major study and of study
results prior to dissemination will begin October 1, 2002, with full compliance
by October 1, 2005. The review of all studies every five years will begin
October 1, 2002, with all studies having undergone review by October 1,
2007.
C.
Research Contracts
All
research contracts with direct costs of $100,000 or greater, awarded on
or after October 1, 2005 (FY2006) will be evaluated by peer review, except
in emergency situations. In such situations, the CIO Director can explain
the emergency and request, with justification, an exclusion from this
policy which can be granted by the Deputy Director for Science and Public
Health or his/her designee.
Timeline
for Implementation
Peer review of R&D contracts will be phased in over three years by
proportion of total number of new RFP solicitations over $100,000 in direct
costs by a CIO. The denominator includes solicitations any component of
which is research.
Targets
for the proportion of total research announcements subject to peer review
in each CIO:
- 1 of
3 (or 33%) for FY2003 announcements;
- 2 of
3 (or 66%) for FY2004 announcements;
- 9 of
10 (or 90%) for FY2005 announcements;
- All FY2006
announcements.
Return
to top
| V.
RESPONSIBILITIES |
A.
Centers Institutes, and Offices
CIO
Directors are responsible to the Director, CDC, for the implementation
of peer review of research originating in their CIO. Directors are responsible
for monitoring the status of implementation and provide reports as requested
to the CDC Associate Director for Science. The recommended table shells
for reporting extramural grants and cooperative agreement activity and
intramural contract activity are provided in Exhibit 1.
B.
CDC Associate Director for Science
The
CDC ADS is responsible for providing overall guidance and oversight as
needed to CIOs to implement this policy. The ADS and the Excellence in
Science Committee will monitor implementation of peer review during phase
in. Questions and requests can be directed to Dr. Mary Lerchen at mlerchen@cdc.gov
or at 770 488 2443.
Return
to top
| VI.
PROCEDURES |
A.
Extramural Research
Procedures for the conduct of peer review of extramural research can be
found in the Peer Review Manual (See Reference 5).
B. Intramural Research
Intramural
Research Programs
The scientific and technical quality of intramural research programs will
be reviewed and monitored by advisory committees, boards of scientific
advisors, or special emphasis panels. The advisory committees and boards
of scientific advisors may elect to utilize workgroups or subcommittees
to assist in the review.
Intramural
Research Studies
Subject
matter experts should review major research studies at inception of the
study and prior to dissemination of results of studies. CIOs may conduct
these reviews by using subcommittees of existing FACA committees, or by
using external experts recruited on an ad hoc basis. If reviewers are
recruited on an ad hoc basis separate from a FACA committee, reviewers
must provide individual, independent comments, and consensus decisions
must be avoided. (See Reference 7, the Federal Advisory Committee Act.)
Peer review
of research studies at inception can be accomplished through a number
of mechanisms. The following are provided as examples of ways that peer
review might be conducted by a CIO:
- A subcommittee
of the CIO's advisory committee (augmented with subject matter experts
as needed) is formed to review the existing research portfolio of the
CIO or a Division. The review consists of a one or two-day site visit
meeting with oral presentations by scientists. After the meeting, the
subcommittee drafts a report, which is submitted to the full FACA committee
for approval.
- Three
independent subject matter experts are recruited to review a single
study or a portfolio of studies. After review of detailed research proposals,
written comments are prepared independently by each reviewer and then
e-mailed back to CIO management.
- A panel
of independent experts is recruited to review a new research study or
a portfolio of studies planned by a CIO. The reviewers are provided
with a written research proposal of the project or projects, and then
meet with the researchers and CIO management at an onsite meeting. After
the meeting, each reviewer prepares independent written comments, which
are sent to the CIO management.
Research
results from intramural projects must be reviewed by external experts
prior to dissemination outside of CDC. For many scientific manuscripts,
this review process will be conducted as part of submission to peer reviewed
journals. For results published in CDC publications or non-peer reviewed
journals, the review will be conducted by the CIOs using external experts
or internal experts not affiliated with the program and without conflict
of interest.
Contracts
The contracting process, including selection criteria and review, are
regulated by the Federal Acquisition Regulations (6). Contracts are evaluated
in a two-step process. The first is review by a Technical Evaluation Panel
(TEP) of experts organized according to scientific disciplines or specialty
research area for the purpose of evaluating the performance of the offeror.
Consistent with other OPDIVs in HHS, such as the Agency for Healthcare
Quality and the National Institutes of Health, CDC TEPs will include a
maximum of 25% government employees as members (external experts make
up 75% of membership). TEPs with external membership are FACA chartered
committees (usually special emphasis panels). The second level of review
is conducted by the Contracting Officer, the Project Officer, and the
TEP, if needed, to determine the competitive range and negotiation with
offers of best and final offers. The final source selection is typically
made by the CIO Director.
Contracts
that are not R&D, but generate Task Orders (TO) that are R&D,
are subject to peer review. This policy shall apply only to contracts
with direct costs exceeding $100,000, including multiple TOs.
Task
Orders
Task
orders (TO) are used to authorize worked required under a contract. Because
the Federal Acquisition Regulations do not regulate selection criteria
and review of TOs, CDC can determine procedures, provided each awardee
is given a fair opportunity to be considered for each TO. We recommend
that TOs be reviewed and selected using existing PGO procedures (peer
review is not required).
Return
to top
| VI.
REFERENCES |
Return
to top
|
