Return to OPP home page

Return to NSF home page

Office of Polar Programs

Polar Sciences

Response to the Recommendations of the FY 2000 Committee of Visitors

(July 25-27, 2000)

All recommendations and suggestions made in the report are included.  Comments are assigned to one of three categories entitled "immediate recommendations," "recommendations for the next Committee of Visitors (COV)", and "general comments."  Responses are in italics.

Immediate concerns and recommendations:

1.  To help clarify the intent of [review] criterion 2 and to raise awareness of PI's about its importance, NSF should provide examples of suitable criterion 2 activities and results in progress and final reports. (p. 3)

As noted by the COV, "the NSF review criteria were in transition during the period covered by this COV review." Review of the oldest proposals in this COV utilized the four older NSF review criteria — especially the one forcused on intrinsic merit of the research. Review of the more recent proposals in the COV utilized the two "newly developed" criteria — intellectual merit and broader impacts.

We are aware that members of our community have questions about certain elements of the second criterion. However,  there is some concern that adding examples that are too specific may restrict flexibility or limit creative approaches and  interpretations.

This is an important NSF-wide concern that must be addressed both within and outside of polar programs. We will ensure that the comments and suggestions of this COV are forwarded to the appropriate NSF office.

2.  NSF/OPP should highlight both [review] criteria and have them be explicit parts of the form that needs to be filled out by PI's, just as prior support is part of the form.  This could be done easily with Fastlane submissions.  Reviewers would be more likely to comment on both criteria if they were addressed independently in the proposal. (p. 3)

This is a good suggestion, and one that has been discussed informally in internal NSF meetings. There areat least two aspects of this issue: (1) proposers addressing both criteria, and (2) reviewer response to both criteria. Again, we will ensure that COV comments are forwarded to the appropriate office.

3.  Where appropriate, OPP program managers and educational liaisons should help PI's link up with program managers in EHR to assist them in developing meaningful education efforts. (p. 4)

OPP is interested in facilitating the establishment of linkages with program managers in EHR. Some assistance is already provided, but perhaps a greater variety and/or more effective strategies to enhance the linkages could be developed. For example, OPP could add the names of OPP liaisons and EHR program managers to our web site.

4.  OPP should continue to request PI's to better highlight and document educational activities. (p. 4)

OPP will discuss and implement new strategies to encourage PIs to document more fully their educational activities in annual and final project reports.  Possible strategies include distribution of a Dear Colleague letter, providing special guidance or examples on the OPP web page, and using the portion of the new Fastlane project reports system that allows program managers to "reject" project reports that do not address educational activities sufficiently. (Follow-up to such a decision involves contacting the PI and requesting a revised report.) Program managers could be encouraged to follow this procedure on a routine basis.

5.  The COV recommends that NSF collect more complete information from reviewers regarding underrepresented groups. (p. 5)

This recommendation is very difficult to address because of privacy issues.

6.  The COV recommends establishing a conduit for promoting bi-polar research, especially in those areas where such an approach would lead to unique insights or capitalize on the resources of OPP. Early efforts in this regard have been fruitful and should be expanded beyond instrumentation to intellectual activity. (p. 6)

OPP agrees that a bipolar approach is useful in some, but not all, of the research areas encompassed by OPP programs.  The recent solicitation for Polar Instrumentation proposals was fruitful.  Prior to this, significant partnering (both between scientific communities studying the two poles and between program managers) has successfully occurred in upper atmosphere physics, glacial ice-drilling and paleoclimate reconstructions, and meteorological and oceanographic studies.  Most successful split-funded proposals (those funded by more than one program) result from an active cooperation between Program Managers of different programs within each section and between different programs within OPP.  We believe that this avenue of split-funding is fully available within OPP for those PIs interested in bipolar studies.

OPP could reinforce its interest and willingness to support scientifically appropriate bipolar studies by further developing the wording in announcements and solicitations.

7.  OPP needs to better document examples of the transfer of Arctic/Antarctic knowledge to the users of this knowledge in the private economy and government.  If little transfer is taking place, then better efforts must be initiated, including scientist/end-user partnerships.  NSF-STC programs may have well-developed examples and vehicles that OPP can draw upon.  ARCUS may be an appropriate forum to lead these efforts for Arctic knowledge.  It is likely that a different type of forum may be needed for this transfer for Antarctic research results. (p. 10)

Knowledge transfer to non-polar scientists needs to be enhanced.  For example, a recent development has been the linking of the SEARCH initiative with the WCRP CLIVAR program. (p. 10)

Publication in open scientific literature, in popular literature, and in various public media are traditional means for transferring scientific knowledge among various constituencies. Other activities central to knowledge transfer include direct one-on-one exchanges between PI and user groups, movement of personnel such as students or post-doctorals from universities to other organizations or vice-versa, preparation of documents such as the Congressionally mandated annual GPRA reports, involvement in research activities that have a policy component (such as ACIA), and participation in policy-making bodies and broad-based fora such as the Arctic Council and COMNAPS. Other approaches, such as ONR's creation and distribution of an annual CD-ROM series, should be discussed and considered. 

OPP has successfully and effectively utilized the traditional approaches mentioned above in the transfer of information to other communities, especially to other science-based groups. For example, on a government-wide basis, the Interagency Arctic Research Policy Committee (IARPC) disseminates Arctic research results to other scientists, agencies and to policy makers, via the journalArctic Research of the U.S.Our involvement with  NSF contractors such as ARCUS in helping to distribute information about discoveries has also been extremely successful - note for example,  ARCUS's  publication Witness the Arctic, which is published about twice each year. The number and breadth of press releases on polar activities is also noteworthy. In overview, however, most of these documents address science initiatives and directions, and are not focused specifically on transferring results from NSF polar research into the broader science or policy community.

And finally, a central component in all aspects of knowledge transfer is the role of NSF PIs and their associates, including students, in society's broader context. It is possible that they are indeed involved in some of  the alternative approaches noted above, and that NSF has not captured that information in its entirety. This again could be addressed informally in conversations with PIs or in a Dear Colleague letter that requests full information in annual and final project reports. It is clear that OPP can play a pivotal role in information transfer by providing complete, up-to-date information, whatever the mechanism, about ongoing research to both the public and the scientific community. 

8.  OPP should develop a comprehensive policy, based upon input from the OAC, on the digital archival of appropriate science results for both Arctic and Antarctic programs. (p. 10)

OPP has developed a data policy statement (posted on the OPP web site at OPP Guidelnes and Award Conditions for Scientific Data) to facilitate full and open access to data and materials for polar research from projects uspported by OPP. Those products which are appropriate for submission to a national data center or OPP specified repository should be promptly submitted within a reasonable amount of time.

 Additionally, each program within the Arctic Section has additional requirements on data archiving, with the most detailed being that of the Arctic System Science Program.  That program recently performed a review of data archiving activities and capabilities and found significant complaints by active scientists on the ability to submit, store, find, and acquire data from several of the national data repositories.  Within the ability of that program, changes in the management strategy are being evaluated and implemented, but the difficulties extend beyond the direct jurisdiction of the Office.  In the other Arctic Section programs, substantive issues result in different norms on data archiving and sharing; thus, the needs of each section define different approaches to this goal.

Interpretation of this suggestion in a broader context has far-reaching implications.  Development of digital archives for scientific results that range from reports and publications to raw or processed data to audio and video records touch upon issues of interoperablity concepts, standards, and intellectual property rights. NSF is currently defining its interests in developing digital libraries and other large distributed digital repositories. Such activities are expected to compete with others proposed by NSF science communities for resources.  

9.  The Polar Information Program is presently serving only the Antarctic section.  OPP should apply sustainable resources to both the Arctic and Antarctic sections for outreach to the media. (p. 10)

Outreach to the media (i.e. press releases, statements, etc.) for all Foundation programs and offices is coordinated and handled through NSF's Office of Legislative and Public Affairs (OLPA). Assistance in developing materials for these OLPA releases is commonly provided by NSF PIs and Program Managers, by OPP contractors (e.g., Raytheon), or by organizations tasked through cooperative agreements (e.g., ARCUS). The OPP/OLPA partnership has been extremely effective in promoting NSF-funded discoveries.

Specific requests (e.g., for web site references, publications, information on NSF and OPP programs, teachers activities) from external communities, which may or may not include members of the media, are largely handled for the entire office by the Information Reference Assistant. These requests cover general polar, Arctic, and Antarctic activities, and may total a thousand or more during any given year. OPP does not have a Polar Information Program per se. It did have a small Polar Information section prior to 1995, when the office was reorganized. There is currently an Antarctic Information Program Manager and a Writer/Editor, who increasingly are serving the needs of the entire office.

The NSF and OPP web sites also provide information to the external community. Maintenance of the OPP site is shared between NSF contractors (e.g., Andrulis) and OPP staff.  The site continues to evolve, and further development is needed to achieve balance among the various components. Staff resources to accomplish this task may need to be further examined or re-allocated.

Visual aids developed and utilized in outreach activities conducted by NSF staff are prepared in a variety of ways — sometimes by the staff member herself/himself, sometimes with the assistance of other NSF or OPP staff, and at times by contractors and science management offices.  The balance among these activities should be examined and adjusted, as necessary.

10.  OPP should encourage and consolidate explicit documentation of project impacts and outcomes, including, for example, feedback from users of educational materials such as compact discs, follow-ups with teachers who have received training, and numbers of schools that have adopted modules.

This type of information will be captured in a more regular manner in reports required by the Government Performance and Results Act.

11.  OPP [should] foster publicity for educational projects at meetings of professional groups such as the National Science Teachers Association (NSTA), National Association of Geoscience Teachers (NAGT), American Geophysical Union (AGU) and Geological Society of America (GSA).  We note that participants in the TEA program are already encouraged and supported to attend NSTA meetings.

Educational projects and educational activities are currently publicized at meetings of various professional groups, typically through formal presentations and/or by means of NSF booths rented at the meetings. The extent of this publicity varies, with the  topical emphasis of the various meetings dependent to a certain extent on time/space constraints imposed by the professional societies. Interactions between science communities and society personnel could be especially pivotal in effecting changes or modifying the balance of activities.

Additionally, OPP could consider developing a bi-polar display on educational activities for these meetings.

12.  It is recommended that NSF improve its networked data handling programs to allow NSF staff — particularly those without specific expertise in programming — to readily access reliable data on proposals and awards.  One problem that should be addressed is how to identify a research project that addresses a number of NSF goals.  For example, within one OPP project the REU supplement provided a mechanism to include underrepresented groups in research. (p. 17) 

Addressing this issue is related to development of NSF, not OPP, data systems. We will ensure that these comments are forwarded to the appropriate office within NSF.

13.  ...it is important that workload distributions across the directorate be evaluated on an ongoing basis.  It is also important to strike a healthy balance between permanent and temporary positions.  On this note, the COV recommends that the number of temporary positions in the Arctic division not be decreased.  (p. 18)

The Office Director utilizes a wide range of input in managing workload distributions in the office.  OPP values the fresh perspective that rotaters provide, and we hope to retain the current IPA level in the office in support of existing programs and new opportunities.

14.  The COV was not charged with evaluating the Antarctic logistics program and so information was limited.  However the budget to personnel ratio in the Arctic versus Antarctic suggests that there might be cause for reconsideration of logistics staffing.  (p. 18)

Recommendations for the next COV:

15.  There are insufficient data to evaluate the goal "Improved achievement in mathematics and science skills needed by all Americans". The COV could not readily locate data on actual effectiveness and widespread adoption of materials that have been produced and activities that have been undertaken.  We are aware that cooperative programs between EHR and OPP aimed at large audiences such as TEA and the Glacier programs are undertaking extensive evaluation and publicity efforts.  We recommend that the outcome of such joint efforts be summarized by the cognizant OPP program manager for the benefit of future COV's. (p. 14)

OPP will provide information, as required in future COV guidelines, on effectiveness and evaluation of projects that it funds. It will work with EHR and other offices, as appropriate, to develop similar information on programs that are jointly funded.

16.  There appear to be numerous cross-directorate, cross-NSF activities undertaken by the program managers.  It would be helpful to future COV's, if information on this is presented in some consolidated form. (p. 19)

Information on the full spectrum of NSF activities is available on the Foundation web site.  OPP will provide summary information on its involvement in these activities for future COVs.

17.  The COV notes that it would be helpful for future COVs to be afforded a comprehensive overview of long-term planning efforts across the directorate. (p. 19)

OPP will provide an overview of its planning efforts, especially as they relate to future activities, response to emerging scientific opportunities, and balance among new and existing  projects and programs.

18.  ... it is recommended that future COV's examine this [IARC] jacket in the context of the cooperative agreement and comment upon the success of IARC's programs. (p. 19)

OPP will include the IARC jacket among those reviewed in the next COV.

19.  Before the format for the next COV is decided, these alternatives [for methods of selecting what jackets are reviewed] should be carefully considered and weighed in view of the goals of the COV and of the time limitation of the COV review. (p. 19)

OPP will consult with the Chair of the COV well in advance of the meeting to discuss options.

20.  This present COV membership contained so many persons that had submitted proposals to the Long Term Observatory competition that no review of the jackets could be carried out by the committee as a whole.  It would be appropriate for the next COV to carry out such a review. (p. 19)

Representative jackets from this LTO competition were reviewed by a COV member who had no conflicts-of-interest.  That examination identified no issues requiring further review.

General Comments:

21.  Despite deliberate efforts by OPP, the participation of underrepresented groups is small; this is typical of the whole of NSF. (p. 2)

This is both an OPP and a NSF issue. Advisory and internal NSF groups are currently examining strategies to address underrepresentation.

22.  . . . program managers also need to regularly attend scientific meetings and visit field sites and institutions as a necessary component of the review processes.  It is important for OPP to prioritize and financially support these activities. (p. 2)

NSF's appropriation provides separate funds for program activities (awards to universities and IPA costs) and what is termed S&E, or Salaries and Expenses.  As an office in the Office of the Director (O/D), OPP receives its S&E allocation from the O/D.  S&E is used for staff travel, computers, supplies, training, and any other general administrative expense.  We try to balance the needs in all of these areas for all staff.  For example, it has become increasingly important to maintain computer equipment of sufficient capability that staff can efficiently use NSF standardized systems and software.  Even with those competing needs, in FY 2000, OPP allocated 55% of its available S&E to travel.  This is a higher percentage of funds spent on travel than all but one other Directorate at NSF.  We will continue to look at staff needs in all areas, and will be particularly sensitive to the need for staff to attend scientific meetings and make site visits.

23.  Managers at all levels should ensure that the infrastructure is in place to maintain dwell times of less than 6 months. (p. 2)

OPP will continue to try to provide human and physical infrastructure that enables OPP to achieve dwell times of six months or less.

24.  Proposal evaluations are consistent with the priorities and criteria stated in the program's solicitations, announcements, and guidelines. Because of this it is important that these documents clearly highlight OPP and NSF priorities and evaluation criteria.  (p. 3)

OPP will ensure that reviewer letters and  announcements/solicitations  contain consistent statements regarding  review criteria and priorities.

25.  The COV thinks that instructions to PI's should make clear how the review process will proceed and how the outcome was determined.  (p. 4)

The new Grant Policy Guide explains in many sections that proposals will be rated on two standard merit review criteria and possibly on special criteria. The results of the review process are conveyed to proposers by program managers and through receipt of anonymous copies of individual reviews.

26.  ...it would be more efficient and effective for NSF to provide adequate funding in the first place.  To accomplish this, the average size and duration of grants would have to be significantly increased and the number of OPP awards decreased assuming no increase in OPP's budget.  Clearly this trade-off requires careful thought on the part of OPP.  (p. 5)

OPP agrees that proposal size and duration should increase, to the extent justified by the proposed project. An ideal scenario would be based on an OPP budget that allows increases in grant duration and amount and increases in the number of awards.

27.  The ARCSS model should be considered for adoption by the Antarctic Section.  Having said that, it is noted that within the Antarctic Section there have been successful multidisciplinary programs in Oceanography, Glaciology, and Atmospheric Sciences.  (p. 6)

Conceptually this suggestion is interesting. However, at what point should the concretion terminate — with an "Earth System Science" (ESS) program that is not constrained by arbitrary political and geographic boundaries and thus includes both polar regions.

28.  OPP program managers may want to encourage appropriate polar research groups to consider developing STC proposals for the next round of solicitations. (p. 16)

Such encouragement may help broaden the pool of applicants, but the STC program is a well-established program that is already widely known among NSF science communities.

29.  The COV suggests that it may be desirable to evaluate Science Management Offices' performance at appropriate intervals (project/theme dependent but at least every three years), particularly in the cases where SMO support is being renewed beyond the length of time for which it was initiated.  (p. 18)

If the [ARCUS] renewal proposal is successful, the COV strongly supports continuing review of the annual operating budget by the program officer, and review of performance every three years by external reviewers. (p. 17)

Science Management Offices (including that of ARCUS) are utilized in the Arctic Section. They are primarily used by the Arctic System Science Program and the Long Term Observatory Program, but they can fulfill Arctic Section tasks as in the case of ARCUS. They are funded after a proposal is submitted and successfully reviewed by qualified members of the scientific community. To establish a new SMO, the proposal submitted to NSF must address the overall need for the SMO, the duties to be performed, the products that will result from the SMO's operation, and the costs for operating it. To renew an existing SMO, past performance is a critical part of the review.  As with the typical NSF proposal review process, the comments by reviewers are shared with the PIs of the SMO proposal. For successful proposals, during the final negotiation and subsequently, the Program Managers interact directly with the P.I.(s) to make sure the goals are being fulfilled.The grant period is commonly three years, but this varies depending on the need. The program manager who handles the ARCUS award carefully reviews the annual operating budget.

30.  ... the ANS portfolio is extraordinarily diverse and could benefit from broadened managerial expertise. (p. 18)

This statement is interpreted as a recommendation to add a third program manager to the Arctic Natural Sciences Program. The challenge of the multidisciplinary ANS is in balancing the disciplines it supports. This is currently handled successfully with two program managers who communicate well with each other and with program managers in other directorates. Adding a third program managercould  increase the complexity of communication and tend to "carve" ANS into three separate programs and/or budgets. Adding another program manager probably would not be as desirable as hiring a science assistant.