survey methodology

Survey Methodology:
Survey of Federal Funds for Research and Development

Overview top

a. Purpose

The Survey of Federal Funds for Research and Development (Federal Funds Survey) is the primary source of information about Federal funding for R&D in the United States. It is used by policymakers in the executive and legislative branches of the Federal Government in determining policies, laws, and regulations affecting science and by those who follow science trends in every sector of the economy: university administrators and professors, economic and political analysts, R&D managers inside and outside the Government, the science press, and leading members of the science community in the United States and foreign countries. The results of the survey are also used for budget purposes for four Federal programs: Federal Laboratory Consortium for Technology Transfer (FLC), Small Business Innovation Research (SBIR), Small Business Technology Transfer (STTR), and Experimental Program to Stimulate Competitive Research (EPSCoR).

b. Respondents

The survey is completed by Federal agencies and subagencies.

c. Key variables

Note that variables in this survey use definitions comparable to those used by the Office of Management and Budget and the related Survey of Federal Science and Engineering Support to Universities, Colleges, and Nonprofit Institutions (Federal S&E Support Survey).

2. Survey Design top

a. Target population and sample frame

The population consists of all Federal agencies that conduct research and development programs. In the survey cycle for data collection on fiscal years (FYs) 2002 through 2004, 30 Federal agencies reported R&D; data. Since multiple subdivisions of an agency were in some cases requested to complete the survey, there were a total of 74 individual respondents. The sample frame is obtained from information in the Presidential budget submitted to the Congress.

b. Sample design

All elements in the population are included.

c. Data collection techniques

Data collection on FYs 1998 through 2000 was performed by QRC Division of Macro International Inc. under contract with the NSF's Division of Science Resources Statistics (SRS). Initial contact to the agencies was by phone to verify name, address, and phone number. Prior to 1998, the mailing consisted of a cover letter, instructions, a listing of the agency's responses to the related Federal S&E Support Survey, the agency's submitted information from the preceding survey, and a computer diskette with a data entry program called FEDFUNDS. Receipt of the mailed package was verified by phone. Data were collected using FEDFUNDS (developed for the survey in 1990), which calculated mathematical error checks on the data entered by the respondent. The contractor provided agencies with help in answering questions about using FEDFUNDS as needed. Manual checking techniques were used to supplement automated techniques.

Beginning in 1999, web-based data collection (FEDWEB) has been used for the Federal Funds Survey and is reducing the annual reporting burden to both the responding agencies and NSF. The reporting burden hours include time to read instructions and introductory letters; compile data (which usually requires the most time and may include staff participation in various parts of the agency); and data entry onto the system.

Data are collected for three fiscal years-the year preceding data collection, the year of data collection, and the year subsequent to data collection. Collection starts in February and ends in May. A few agencies, however, request time extensions to June. In prior years collection took longer to complete-it began in March and finished in October. More contact with respondents through workshops and improvements in data collection helped to move up the survey completion cycle.

d. Estimation techniques

Since this survey is a census of all Federal R&D-sponsoring agencies except the Central Intelligence Agency and the survey has a response rate of 100 percent and no known item nonresponse, there is no weighting of responses or imputation used.

e. Mode

FEDWEB, a web-based data collection system, is used for the Federal Funds Survey. FEDWEB is part of SRS's effort to improve survey reporting and cut data collection and processing costs by offering respondents direct online reporting and editing. The goal is to offer an easier, more cost-effective way both for respondents to complete the survey each year and for the contractor to collect and process the data.

All data collection efforts, data imports, data editing, and trend checking are accomplished using FEDWEB. The FEDWEB system consists of a data collection component that allows survey respondents to enter their data online and a monitoring component that allows the contractor to monitor support requests, data entry, and data issues. FEDWEB's two components are password-protected, so that only authorized users can access them.

3. Survey Quality Measures top

a. Sampling variability

Since all eligible agencies are included, there is no sampling error.

b. Coverage

Since identification of relevant Federal agencies is straightforward, coverage is assumed to be complete except for the Central Intelligence Agency.

c. Nonresponse

(1) Unit nonresponse - Response rate for this survey has always been 100 percent, indicating that there is no nonresponse bias in this survey.

(2) Item nonresponse - Since agencies are encouraged to estimate information when actual data are unavailable, there is no known item nonresponse.

d. Measurement

Some measurement problems are known to exist in the data. More specifically, some agencies are not able to report the full costs of research and development. For example, the Department of Defense (DOD) does not include headquarters costs of planning and administering R&D programs (estimated at a fraction of 1 percent of its total) because DOD has stated that identification of the amounts is impracticable.

R&D plant data are also underreported to some extent because of the difficulty encountered by some agencies, particularly DOD and the National Aeronautics and Space Administration (NASA), in identifying and reporting these data. DOD's respondents report obligations for the R&D plant funded under the agency's construction appropriation, but they are able to identify only a small portion of the R&D plant support that is within R&D contracts funded from DOD's appropriation for research, development, testing, and evaluation. Similarly, NASA respondents cannot separately identify the portions of industrial R&D contracts that apply to R&D plant; R&D plant data are subsumed in the R&D data covering industrial performance. NASA R&D plant data for other performing sectors are reported separately.

Beginning in FY 2000, NASA reclassified space station as a physical asset and space station research as equipment, and it transferred funding for the program from R&D to R&D plant. Also, the National Institutes of Health (NIH) classified all of its development activities as research beginning in FY 2000.

4. Trend Data top

The information included in this survey has been quite stable since FY 1973, when Federal obligations for research to universities and colleges by agency and detailed S&E field were added to the survey. Many of the other variables are available from the early 1950's on. However, in studying trends, analysts are encouraged to obtain up-to-date data from the SRS web site, because agencies reclassify their responses for prior years as additional budget data become available.

5. Availability of Data top

a. Publications

The data from this survey are published annually in Detailed Statistical Tables in the series Federal Funds for Research and Development. Historical information is also included in three annual historical volumes: Detailed Historical Tables; Federal Obligations for Research by Agency and Detailed Field of Science and Engineering; and Federal Obligations for Research to Universities and Colleges by Agency and Detailed Field of Science and Engineering. Information from this survey is also included in Science and Engineering Indicators and in National Patterns of R&D Resources.

b. Electronic access

Data from this survey are available on the SRS Web site.

c. Contact for more information

Additional information about this survey can be obtained by contacting:

Ronald L. Meeks
Senior Survey Statistician
Research and Development Statistics Program
Division of Science Resources Statistics
National Science Foundation
4201 Wilson Boulevard, Suite 965
Arlington, VA 22230
(703) 292-7787
via e-mail at rmeeks@nsf.gov
Last Modified: Nov 02, 2004 Comments to srsweb@nsf.gov