Since assessment techniques are in relatively early stages of development, and since agency-specific or program-specific techniques will be needed, the paper does not seek to establish particular performance measures or methods. Nor does it seek to be a handbook of "how-to" instructions.¹ Rather, it aims to provide a set of principles and other information for use by science agencies in designing and testing a range of methods appropriate to their particular goals and programs.

The Federal science and technology system consists of a diverse array of science programs dispersed among and within the Federal agencies. These programs encompass the full spectrum of science, mathematics, and engineering disciplines and a growing set of interdisciplinary activities. They include basic research, applied research, and developmental activities. Some individual programs include the entire range from basic research to development; others focus primarily on fundamental science. Some are directed to agency missions such as national defense; others are oriented to expanding the general stock of scientific knowledge. All share a commitment to the conduct of world-class science. All are engaged in an interactive and evolutionary process that builds incrementally on the cumulative results of prior efforts. Their eventual impacts may require long time lags before they are realized.

Agency goals for fundamental science are derived from the Federal government's ultimate purpose which is to contribute to the over-arching national goals of improved health and environment, prosperity, national security, and quality of life. Fundamental science is a necessary contributor to these goals.

To promote scientific contributions to over-arching national goals, the Administration's science policy statement, Science in the National Interest (1994), has established a critical enabling goal for fundamental science. This enabling or intermediate goal is to

• Maintain leadership across the frontiers of scientific knowledge
  

That is, from an NSTC or national perspective, the goal is to maintain United States' leadership. For an individual agency, the goal is to conduct world-class science and to contribute to United States' leadership.

Also in support of the over-arching national goals are four additional enabling goals defined by the Administration in Science in the National Interest. These are to:

• Enhance connections between fundamental research and national goals
  

• Stimulate partnerships that promote investments in fundamental science and engineering and effective use of physical, human, and financial resources
  

• Produce the finest scientists and engineers for the twenty-first century
  

• Raise the scientific and technological literacy of all Americans
  

All agencies which support the Federal portfolio of fundamental science contribute to one or more of these intermediate goals. The health of the science enterprise is in turn enabled by a strong infrastructure--human resources and facilities.

The way in which an agency contributes to enabling our over-arching national goals will depend on the specifics of its strategic plan, as developed in consultation with Congress, stakeholders, and the Office of Management and Budget. The specific goals defined in the agency's strategic plan will yield further criteria for performance assessment, in addition to the world-class quality standard.

Science in the National Interest points out that the principal sponsors and beneficiaries of our scientific enterprise are the American people. Their continued support, rooted in the recognition of science as the foundation of a modern knowledge-based society, is essential. Maintaining public support for fundamental science depends on clear communication of its varied, inter-connected, and complex results to the Executive Branch, Congress, other policy makers, and the public. Scientists and decision-makers from colleges and universities, industry, and government increasingly recognize the importance and value of such communication. They sense opportunities for building public understanding and sharing the excitement of science. Moreover, effective communication of progress and findings to stakeholders through constructive use of electronic and printed media can help diffuse program results and produce insightful feedback for program managers.

Managers of science and technology programs in the public and private sectors are devising techniques for assessing the outcomes of their research programs. All continue to grapple with methodological challenges. Methods are well developed for manufacturing production which is composed of repetitive, well-defined processes that yield tangible products having benefits that accrue in the short run. In comparison to assessment methods used for manufacturing production, assessment techniques for fundamental science are much less advanced.

Some may interpret GPRA as suggesting that the simple assessment model appropriate to well-defined manufacturing processes should be applied to government programs as well. However, the framers of GPRA recognized the limitations of the simple manufacturing template and wisely built flexibility into the GPRA structure in order to facilitate development of concepts appropriate to more complex situations. GPRA's flexibility provides science agencies with latitude to develop methods appropriate to fundamental science and the often serendipitous routes through which it contributes to national well being.

The challenge is to devise assessment strategies appropriate to the creative processes of science and innovation and their dynamic interactions. There is always the possibility that unexpected opportunities may appear, that important findings may be ignored until information from other sources puts them in a new light, or that the time lag is so long that the significance of the original finding is lost. For example, Alan Turing's "machine" and related advances laid the groundwork for today's massive advances in computational capabilities, but the technical advances were not anticipated or even considered feasible until the subsequent independent development of the semi-conductor.

The significant consideration in assessing fundamental research programs is that these programs are aimed at the intermediate or enabling goal of conducting world-class science in order to advance the frontiers of knowledge. These programs are not intended to yield tangible applications, such as a new silicon chip or next generation computer, directly or in short periods of time. Rather, they are intended to yield fundamental knowledge which provides the basis for a range of eventual applications--some anticipated and some not.

The essential purpose of fundamental scientific cutting-edge research is to advance knowledge. Regardless of whether information of potential relevance to particular applications is sought at the time the research is initiated, the insights produced by the research enlarge the knowledge base on which future scientific and technological advances can draw. For example, studies of quantum mechanics in the 1920s were considered to be "pure esoterica" by many at the time--few people understood the theory. However, in the succeeding fifty years, results of this work in combination with findings and applications from other fields produced transistors, lasers, and electronic devices used today in a wide array of activities, including information processing, communications, and video imagery.

A project or program that is aimed at a particular purpose can, in addition, reveal insights beyond the area originally intended. Eventual applications may appear after long lags during which subsequent studies can build knowledge in a slow process of discovery and consolidation. For example, studies of the plant pathogen Agrobacterium tumefaciens that were initiated to gain clues about tumor formation in humans revealed that the pathogen causes the disease "crown gall" in plants by inserting its own genes into the host's genome. This discovery provided the basis for the new field of genetic engineering for crop and garden plants.

A project or program that is aimed at a particular purpose but does not produce the insights originally sought may, nevertheless, provide other information which by unexpected and circuitous routes supports further scientific or technological advance. For example, at Columbia University in the 1950s, Charles Townes undertook postdoctoral research aimed at eventual development of a supplementary means for defining and identifying materials properties using microwave spectroscopy. After unintended turns over several years and across several continents and diverse fields of science, his work failed to meet his original objective. However, the outcome was the maser, forerunner of the laser, a basic building block for many current telecommunications, data handling, medical, and industrial technologies.

When fundamental research does produce the information originally sought, the new insight is an intermediate or enabling achievement. The new knowledge will contribute to over-arching national goals--usually, only through complex causal chains that require inputs from other activities and institutions to yield practical applications. Moreover, long periods of time may generally be required before tangible improvements in overall well being become evident. For example, biomedical research has contributed mightily to our understanding of how to reduce infant morbidity and mortality. But medical research alone is not sufficient to produce a reduction in infant mortality. Many additional activities are required, including transfer of research results into medical practice, development and diffusion of any necessary medical devices or products, and education of parents. Other vital factors are postponement of pregnancy until physical maturity; late child-bearing; a social culture which will accommodate new medical procedures or health guidelines; and, probably most important, adequate financial resources to assure that all mothers and infants have access to appropriate medical care and food.

Each agency must consider its role in contributing to leadership across the frontiers of scientific knowledge in order to enable the achievement of over-arching national goals. Each agency must be able to assess its contribution to world-class science, even though it may be only one of several agencies and institutions building knowledge in a particular area, and even though investment in science alone does not assure transfer of research results to practice.

Any scientific organization (whether a government agency or a private sector organization) invests in a "portfolio" of projects in order to pursue a number of avenues of possible enlightenment. Science proceeds through a slow process of accretion of results. Major breakthroughs do not necessarily occur on a regular basis, and an essential element of scientific research is the replication of earlier findings in order to confirm or generalize them. Moreover new research findings are significant not just because they yield a new technical application, but simply because they help to add to the precious stock of scientific knowledge available for future human use.

¹Two groups which facilitate informal sharing of "how-to" information are the Research Round Table, co-initiated by Mike English of the Department of Health and Human Services and David Rust of the Agricultural Research Service, U.S. Department of Agriculture; and the Federal Research Assessment Network, initiated by Susan Cozzens of the National Science Foundation.