 
Working
Group members
Stephanie Pfirman, Chairperson and
1999
OAC
Chairperson
OAC members
Mary Albert,
2000 OAC
Chairperson
John Carlstrom
John Priscu
NSF staff
members
Maryellen Cameron
Julie Palais
Jane Dionne
|
|
Working
Group on Implementation of Review Criterion #2
|
|
|
February 7, 2001
|
|
|
|
|
|
|
|
|
Introduction1
|
|
|
|
|
|
The National Science Board approved revised criteria for
evaluating National Science Foundation proposals in March of 1997. These
criteria, which are designed to be relevant across NSF’s many different
programs, were implemented on October 1, 1997.
The two new merit review criteria are listed below. Following
each criterion are issues reviewers may wish to consider. Not all suggestions
apply to all proposals.
Criterion 1: What is the intellectual merit of
the proposed activity?
- How important is the proposed activity to advancing
knowledge and understanding within its own field or across different
fields?
- How well qualified is the proposer to conduct the project?
- To what extent does the proposed activity suggest and
explore creative and original concepts?
- How well conceived and organized is the proposed activity?
- Is there sufficient access to resources?
Criterion 2: What are the
broader impacts of the proposed activity?
- How well does the activity advance discovery and understanding
while promoting teaching, training and learning?
- How well does the proposed activity broaden the participation
of underrepresented groups (e.g., gender, ethnicity, disability, geographic,
etc.)?
- To what extent will it enhance the infrastructure for
research and education, such as facilities, instrumentation, networks
and partnerships?
- Will the results be disseminated broadly to enhance
scientific and technological understanding?
- What may be the benefits of the proposed activity to
society?
Recent GPRA and COV discussions by the OPP advisory committee
indicated a need to raise awareness about, and address the importance
of, these criteria at the proposal writing, review, and project evaluation
stages if PIs were to meet NSF’s strategic goals. In particular, the 2000
OPP COV included in its report the statement "To help clarify the
intent of [review] criterion 2 and to raise awareness of PIs about its
importance, NSF should provide examples of suitable criterion 2 activities
and results...."
|
|
|
|
|
|
|
|
|
|
|
|
Criterion #2 Background
and Representative Activities
|
|
|
|
|
|
|
|
|
|
|
|
The report that follows focuses exclusively on Review
Criterion #2 and is an initial step in addressing some of the concerns
raised in discussions of the OPP advisory committee. It provides specific
examples of activities appropriate for review under Criterion #2. The
examples are not meant to be exhaustive or directive; the goal in providing
them is to stimulate creativity on the part of the community.
|
|
|
|
|
|
|
|
Advance
Discovery and Understanding While Promoting Teaching, Training, and Learning
|
|
|
|
|
|
|
|
Background:
Integration of research and education is one of "three core strategies
that guide [NSF] in establishing priorities, identifying opportunities,
and designing new programs and activities ... . Effective integration
of research and education at all levels infuses learning with the excitement
of discovery and assures that the findings and methods of research are
quickly and effectively communicated in a broader context and to a larger
audience" (NSF GPRA Strategic Plan 2001 - 2006)
|
|
|
|
|
|
|
|
Examples of Suggested
Activities:
|
|
|
|
- Integrate research activities into teaching of graduate
students, undergraduate science majors, and general education students
at all levels.
- Integrate research activities into guest lectures,
case studies, and problem sets.
- Develop research-based educational materials and contribute
to databases (e.g. K-16 digital library) useful in teaching.
- Partner with educators to research effective means
of incorporating research into learning and education.
- Establish special mentoring programs for undergraduates,
graduate students, and technicians conducting research.
- Involve graduate and post-doctoral researchers in undergraduate
teaching activities.
- Develop links to special programs such as REU or TEA.
REU is particularly relevant, providing a platform where promising undergraduates
can participate directly in cutting edge research projects. The TEA
program allows teachers to experience field research and relate these
experiences to students and other teachers through the web, through
their own lectures and through guest lectures by the sponsoring PI.
The Schoolyard project allows investigators involved with the LTER program
to work one-on-one with students and teachers at local schools.
- Give science presentations to the broader community/promote
life-long learning: e.g. at museums, and libraries, on radio shows and
with other venues that reach broad audiences.
- Develop, adopt, adapt and implement effective models
and pedagogic approaches.
|
|
|
|
|
|
|
|
Broaden
Participation of Under-represented Groups
|
|
|
|
|
|
|
|
Background:
One of NSF’s five-year strategies is to "broaden participation and
enhance diversity in NSF programs. At present, several groups, including
underrepresented minorities, women, certain types of institutions, and
some geographic areas, perceive barriers to their full participation in
the science and engineering enterprise. NSF is committed to leading the
way to an enterprise that fully captures the strength of America’s diversity."
(NSF GPRA Strategic Plan 2001-2006)
|
|
|
|
|
|
|
|
Examples of Suggested
Activities:
|
|
|
|
- Partner with members of under-represented groups at
the researcher’s home institution in research, education and outreach
activities.
- Partner with members of under-represented groups from
other institutions — for example, those serving minority groups
or women.
- Make campus visits and presentations at colleges and
universities that serve underrepresented groups.
- Establish collaborations with underserved groups, institutions
and geographic regions in order to encourage new entrants into student
and proposal applicant pools.
- Establish research and education partnerships with
faculty and students at RUI and EPSCoR institutions.
- Develop partnerships with community colleges, which
serve approximately half of all U.S. undergraduates and close to half
of every minority and ethnic group.
- Mentor early-career scientists who are submitting NSF
proposals for the first time.
- Document the impact of research in terms of relevance
to under-represented groups.
- Participate in developing strategies that involve new
approaches, such as use of information technology and connectivity,
to engage underserved individuals, groups, and communities in science
and engineering.
- Participate in conferences, workshops and field activities
where diversity is a priority.
|
|
|
|
|
|
|
|
Enhance Infrastructure for Research and Education
|
|
|
|
|
|
|
|
Background:
The NSF Act of 1950 authorizes and directs the Foundation "to foster
and support the development and use of computer and other scientific and
engineering methods and technologies, primarily for research and education
in the sciences and engineering... NSF investments provide state-of-the-art
tools for research and education, such as instrumentation and equipment,
multi-user facilities, ... telescopes, research vessels and aircraft,
... Internet-based and distributed user facilities, ..., research networks,
digital libraries and large databases." (NSF GPRA Strategic Plan
2001-2006).
|
|
|
|
|
|
|
|
Examples of Suggested Activities:
|
|
|
|
- Stimulate and support the development and dissemination
of next-generation instrumentation, multi-user facilities, and
other shared research and education platforms.
- Maintain, operate and modernize shared research and
education tools such as science and technology centers, engineering
research centers, and other facilities that house telescopes, mass spectrometers,
electron microprobes, ion probes, accelerators, etc.
- Upgrade the computation and computing infrastructure,
including advanced computing resources.
- Develop/upgrade new types of tools, such as large databases,
networks and associated systems, and digital libraries; contribute to
development of new types of facilities such as environmental observatories
and collaboratories.
- Develop activities which ensure that multi-user facilities
are not only sites of technological breakthroughs in instrumentation
but also sites of research and mentoring for many science and engineering
students.
- Identify and establish collaborations between disciplines
and institutions, among the U.S. academe, industry and government, and
with international partners in order to enable optimal development of
large infrastructure projects.
|
|
|
|
|
|
|
|
Broad Dissemination
to Enhance Scientific and Technological Understanding
|
|
|
|
|
|
|
|
Background:
"NSF advocates and encourages open scientific communication. It expects
significant findings from supported research and educational activities
to be promptly submitted for publication... . It expects PIs to share
with other researchers, at no more than incremental cost and within a
reasonable time, the data, samples, physical collections and other supporting
materials created or gathered in the course of the work. It also encourages
grantees to share software and inventions . . . and otherwise to make
the innovations ... widely useful and usable." (GPG; NSF 00-2)
|
|
|
|
|
|
|
|
Examples
of Suggested Activities:
|
|
|
|
- Publish in the peer-reviewed scientific literature
in a timely manner.
- Make available data in a timely manner by means of
databases, or other venues as CD-ROMS and provide data and other information
for digital libraries.
- Publish in diverse media (e.g. technical and non-literature,
websites, CD-ROMS, presskits) to reach broad audiences.
- Identify venues for presenting research results in
easy-to-use formats that are useful to members of Congress, policy-makers,
and broader audiences.
- Participate in multi- and interdisciplinary conferences,
workshops, and research activities.
- Integrate research with education activities in order
to communicate in a broader context.
|
|
|
|
|
|
|
|
Benefits
to Society
|
|
|
|
|
|
|
|
Background:
NSF is committed to fostering connections between discoveries and their
use in service to society. The knowledge provided by NSF funded research
projects provides a rich foundation for its broad and useful application.
For example, projects may contribute to understanding the environment,
commercial technology, public policy, health or safety and other aspects
of the public welfare.
|
|
|
|
|
|
|
|
Examples of Suggested Activities:
|
|
|
|
- Demonstrate the linkage between discovery and societal
benefit by providing specific examples and explanations regarding the
potential application of research results.
- Partner with other academic scientists or staff at
federal agencies to integrate research into broader programs and activities
of national interest.
- Partner with the private sector (e.g., consulting firms)
on both technological and scientific projects to develop products that
will benefit society.
- Analyze, interpret, and synthesize research results
in formats understandable and useful for non-scientists.
- Provide information for policy formulation by Federal,
State or local agencies.
|
|
|
|
|
|
|
|
1Paragraphs
1 and 2 are taken directly from the NSF Grant Proposal Guide (NSF 00-2).
|
