Survey Methodology


National Survey of Academic Research Instruments and Instrumentation Needs

Overview
Survey Design
Trend Data
Availability of data

1. Overview
a. Purpose
The Survey of Academic Research Instruments and Instrumentation Needs is a congressionally mandated survey that serves as the primary source of information on the need, stock, cost, and utilization of research and development equipment within academia in the United States. It is used by Congress and Federal agencies in planning programs for funding academic instrumentation.

b. Respondents
The survey is completed in part by departmental chairpersons at selected institutions of higher education and in part by individual investigators within the departments.

c. Key variables

2. Survey design
a. Target population and sample frame
The 1994 institutional population consisted of the 214 colleges and universities (other than medical schools, military schools, and academic
FFRDCs) with nonmedical R&D expenditures of at least $3 million in any year plus the 104 medical schools with NIH extramural funding of $3 million or more. The instrument population consisted of instrument systems originally costing $20,000 or more within one of the following fields: agricultural, biological, computer, or environmental sciences, chemistry, physics/astronomy, and engineering.

b. Sample design

A three-stage sample design was used. The first stage consisted of sampling institutions, the second stage consisted of sampling research units (departments and research facilities) within those institutions, and the third consisted of sampling research instruments costing $20,000 or more within the sampled departments and facilities. The institutional sample size in 1992-1994 consisted of 55 institutions in the nonmedical sample and 24 medical schools. These institutions were originally selected in 1986-87 using stratified random sampling with probability approximately proportionate to size. The measure of size used for the nonmedical sample was R&D expenditures in S&E in FY 1984. For the medical sample, the measure of size was FY 1982 awards from NIH.

In the 1994 survey, the sampled institutions contained a total of 1,541 research units identified as having at least one instrument costing $20,000 or more in 1993. A total of 988 of these units were sampled. Sampling at the departmental level was done only in the S&E fields with large numbers of research units (engineering and the agricultural, biological, and environmental sciences). Eligible units in other science fields were sampled with certainty. A total of 34,508 instruments were identified as being within scope in the sampled units. A sample of 8,784 instruments was selected from those in-scope instruments identified. This total includes 66 "supersystems," i.e., large specialized research units built around a single integrated instrument system that cannot be disaggregated in a meaningful way (such as an observatory or a central computer center). All identified supersystems were included in the survey.

c. Data collection techniques
The 1992-94 survey was conducted by Quantum ResearchCorporation under contract to SRS. Several different survey instruments were utilized. A Department/Facility Questionnaire was used to obtain information from heads of departments and research facilities. This questionnaire elicited information about the instrumentation expenditures in the unit and opinions on the adequacy of research instrumentation in the unit as a whole. An Instrument Data Sheet was used to obtain information about instruments costing at least $20,000 from principal investigators responsible for them. Two different Supersystem Data Sheets were used for collecting information about supersystems. One questionnaire was used for central computer facilities and the second for other supersystems.

Data were collected in two phases. During phase one (collected in 1992), only the Department/Facility Questionnaire was used to collect data at the unit level. In phase two, information was collected for both units and instruments, using all questionnaires.

Data collection was coordinated by institutional coordinators appointed by the presidents or chancellors of the institutions.

Except for the 1992 report, information about current equipment needs and priorities is obtained with reference to the year of the survey and information about dollar amounts is for the year preceding the survey. All information for the 1992 survey is reported with 1992 as the reference year.

d. Estimation techniques
The nonmedical institutions were originally sampled in 1986, using 1984 expenditure information. The medical sample was selected in 1983, using 1982 expenditure information. In order to reflect changes in expenditures by institutions in the intervening years, the 1992-1994 institutional sample was restratified and sampling weights were adjusted accordingly. Departments/facilities and instruments were weighted by the inverse of the probability of selection in the 1992-1994 samples. All three stages were adjusted for nonresponse. At the institutional level, adjustment was done within strata. At the department/facility level weighting was done within cells, defined by type of institution (medical versus nonmedical), field of science and engineering (six categories), size of the department (large versus small) and type of control (private versus public). When cell size was below 25, adjacent cells were combined to reach the minimum size of 25. A total of 17 cells were used in 1992 for academic departments and 7 were used for other facilities.

e. Possible sources of error
(1) Sampling - The coefficients of variation for overall estimates (not broken out by field of S&E) of key variables are in the 2 to 6 percent range.

(2) Coverage - Coverage problems could arise through incorrect classification of colleges and universities as being out-of-scope. Institutions may also incorrectly classify departments/facilities, instruments, or supersystems. The degree of such coverage error is not known.

(3) Unit nonresponse - Of the 79 sampled colleges, universities, and medical schools in 1992, 97 percent were able to provide at least partial data. Usable questionnaire responses were received for 84.4 percent of sampled departments/facilities. In order to minimize the impact of this source of error, results are adjusted for nonresponse through the use of statistical weighting techniques. These unit nonresponse rates are within the normally accepted ranges.

(4) Item nonresponse - Item response rates ranged from 95.0 percent to 100.0 percent in the 1992 survey. Data for institutions that partially responded were imputed for most questions using hot deck procedures.

(5) Measurement - The subjective nature of many of the variables in this survey (e.g., adequacy of research equipment and instrumentation needs) indicates that measurement error is likely to be relatively high. Information on more objective measures (e.g., maintenance/repair costs) is probably less subject to measurement error. However, there have not been empirical studies of measurement error for this survey.

3. Trend data
This survey was first conducted in 1983-84. Surveys were originally conducted on a triennial basis; beginning in 1992 they were conducted biennially. Changes in coverage may impact some of the trends. Using a constant cut-off of $3 million in R&D expenditures as a minimum for inclusion would result in more institutions being defined as part of the sample frame if the only change over time was inflation. This factor would contribute to a slight upward bias in trend estimates for the total values of the variables over time. On the other hand, the dollar value on instruments, as a minimum for inclusion, was increased between 1989 and 1992 (from $10,000 to $20,000). This would lead to a decrease in the total dollar estimates of the stock of instruments.

4. Availability of data
a. Publications
The data from this survey are published in several different series of Detailed Statistical Tables. The most recent report is
Academic Research Instruments: Expenditures 1993, Needs 1994 (NSF 96-324). Reports are also published on academic equipment in more narrowly defined areas, e.g., Characteristics of Science and Engineering Instrumentation in Academic Settings: 1993 (NSF 98-311).

b. Electronic access

Due to confidentiality concerns, data from this survey are not available on CASPAR or on public use files.

c. Contact for more information
Additional information about this survey can be obtained by contacting:

Carolyn Arena
Senior Program Analyst
Science and Engineering Education and Human Resources Program
Division of Science Resources Studies, Room 965
National Science Foundation
4201 Wilson Boulevard
Arlington, VA 22230
Phone: (703) 306-1774
Internet: carena@nsf.gov

[SRS Homepage]
Last Modified: Friday, 28-Aug-98 13:28:07 Comments to srsweb@nsf.gov