The U.S. "industrial competitiveness crisis" of the 1980s spawned a number of experiments in civilian technology policy. Among these was the Cooperative Research and Development Agreement (CRADA), an instrument that was created in the Technology Transfer Act of 1986. A CRADA is a contractual arrangement between a federal laboratory and participating firm that enables the laboratories to conduct joint R&D projects with private firms. Federal agencies are prohibited from providing direct funding to the industrial participants in CRADAs, but federal funds can be used to support the overhead and other expenses of the government research facilities participating in CRADAs. Under the terms of a CRADA, the private-firm partner can be assigned the rights to any intellectual property resulting from the joint work, while the federal government retains a nonexclusive license to the intellectual property.
The CRADA represents an interesting initiative in the restructuring of the U.S. industrial R&D system and U.S. technology policy that accelerated in the aftermath of the Cold War. At present, however, relatively little information on CRADAs is collected or disseminated by the National Science Foundation. As a result, we lack sufficient information to address the following basic questions: What are the characteristics of firms and laboratories that participate in CRADAs? What are the results of CRADAs and how do they affect the innovative performance of the private-firm or federal-laboratory participants over the long term? How do the size of the federal investment in support for CRADAs and the results of CRADAs compare with other federal programs for supporting industrial innovation?
This short paper reviews the limited data on CRADAs that are published by the National Science Foundation (mainly in the biennial Science and Engineering Indicators volume), in an effort to shed some light on a few of these questions. But as I have noted, addressing even this basic list of questions requires more information than currently is available from NSF and non-NSF sources. Nevertheless, several scholars have conducted interesting empirical analyses of the effects of CRADAs on industrial innovation, and their findings suggest that there is considerable value in collecting and disseminating additional information on these federal policy instruments. Immediately below, I review the existing evidence on the number and characteristics of CRADAs formed during the 1990s, drawing on NSF and non-NSF data. I follow this discussion with a survey of the limited literature on the effects and effectiveness of CRADAs. The final section of the paper outlines additional CRADA-related data that NSF should consider collecting and disseminating as part of its activities on the description and analysis of the U.S. R&D system.
Although CRADAs were created by the Federal Technology Transfer Act of 1986, government-owned, contractor-operated federal laboratories (GOCOs), such as those in the Department of Energy laboratory system, were allowed to conduct CRADAs with private firms only with the passage of amendments to the Act in 1989.[1] Federal agencies and research laboratories have signed hundreds of CRADAs each year since the late 1980s. The data in Table 1, drawn from the forthcoming Science and Engineering Indicators volume, report the number of "active CRADAs," based on agency reports that are tabulated by the Department of Commerce's Office of Technology Policy (U.S. Department of Commerce, 2000), for each year during 1987-98.
The number of active CRADAs grew sharply after 1990 (the first full year during which GOCO laboratories could negotiate such agreements), more than doubling by 1992 and more than doubling again by 1994. The number of active CRADAs peaked in 1996, and declined during 1996-98. Unfortunately, these NSF data do not report either "births" or "deaths" of CRADAs, and therefore say little about the reasons for these trends in the number of active CRADAs. CRADA activity is dominated by the Departments of Energy and Defense, which in some years accounted for more than two-thirds of all active CRADAs. Other federal agencies with large numbers of CRADAs during this period include the Department of Agriculture and the Commerce Department, both of which also maintain large laboratory networks. These data suggest that a great deal of government-industry collaboration was carried out through CRADAs during the 1990s, but they provide no information on the scale of industrial or federal investments in the R&D carried out within these CRADAs. Nor do the NSF data establish any clear links between the number of CRADAs reported and the other data on federal agencies' technology transfer activities (invention disclosures, patents, licenses, licensing income) reported in this section of the Indicators volume.
| Agency | 1987 | 1988 | 1989 | 1990 | 1991 | 1992 | 1993 | 1994 | 1995 | 1996 | 1997 | 1998 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| USDA | 9 | 51 | 98 | 128 | 177 | 172 | 172 | 208 | 229 | 244 | 273 | 288 |
| Commerce | 9 | 44 | 82 | 115 | 177 | 292 | 368 | 407 | 406 | 377 | 337 | |
| Defense | 3 | 10 | 36 | 113 | 193 | 277 | 365 | 563 | 845 | 1086 | 1360 | 1424 |
| DOE | 1 | 43 | 250 | 582 | 1094 | 1392 | 1677 | 963 | 868 | |||
| EPA | 2 | 11 | 31 | 30 | 28 | 35 | 30 | 35 | 34 | 37 | ||
| HHS | 22 | 28 | 89 | 110 | 144 | 146 | 149 | 147 | 152 | 158 | 161 | 163 |
| Interior | 1 | 12 | 11 | 1 | 3 | 9 | 15 | 22 | 23 | 30 | ||
| Transportation | 1 | 9 | 17 | 30 | 38 | 37 | 43 | 36 | 39 | |||
| All agencies[2] | 34 | 98 | 271 | 460 | 731 | 1078 | 1628 | 2471 | 3121 | 3688 | 3239 | 3201 |
Indeed, the statement in the forthcoming Indicators discussion that the "invention disclosures" data are entirely attributable to CRADAs[3] appears to be incorrect. The data reported by NSF on invention disclosures reveal that nearly 3000 such disclosures were made by the Department of Energy, most of whose laboratories are GOCOs, during FY 1987-89. Since the DOE's GOCO laboratories were unable to execute CRADAs during this period, the average annual flow of more than 900 invention disclosures during FY 1987-89 presumably reflected other inventive and technology transfer activities.
Since federal agencies have several alternative vehicles for patenting and licensing their technologies in addition to CRADAs, there appears to be no basis for the claim in the discussion of these data in the NSF Indicators volume that these invention disclosures are all associated with CRADAs. The 1999 report of the U.S. Department of Energy's "Technology Transfer Working Group" (U.S. Department of Energy, 1999), for example, identifies seven "Technology Partnership Mechanisms" other than CRADAs or programs targeted specifically at small firms.[4]
Significantly, the Commerce Department report from which the NSF Indicators data are drawn makes no claims about the linkages between CRADAs and any of these outcomes. Moreover, the report notes that some federal agencies generate significant numbers of patents outside of CRADAs: "...NIH reports that intellectual property is being generated at approximately the same level in CRADA research as in non-CRADA research—about 15 percent of projects." (p. 19). Commerce Department staff responsible for assembling these agency-level data also assert that a large fraction of reported invention disclosures, perhaps as much as 50%, derive from activities unrelated to CRADAs (Paugh, 2000). Unfortunately, the Commerce Department data provide no information on the linkage between CRADAs and these other measures of the outcomes of federal agencies' technology transfer and collaborative activities. Better data on the different federal technology transfer policies that produced the disclosures, patents, licenses, and licensing revenues tabulated in the NSF and Commerce Department reports are badly needed.
Some additional information on trends in CRADA formation is available in the NSF's Science and Engineering Indicators: 1998, which tabulates the number of new CRADAs signed by major federal agencies during 1992-95 (see Table 2). These data on CRADA formation, which do not appear to be replicated in the 2000 Indicators volume, reveal rapid growth, from roughly 500 new agreements in 1992 to more than 1100 in 1994, followed by a slight decline in 1995. The 1994-95 decline in new CRADAs spans all of the federal agencies in table 2, with the exception of the Defense Department, where the number of new CRADAs continued to grow during the 1994-95 period. The trendline for "new CRADAs" roughly parallels that for "active CRADAs" in Figure 1 during the brief period covered by these data, turning downward two years before the "active CRADAs" trendline does.
| Agency | 1992 | 1993 | 1994 | 1995 |
|---|---|---|---|---|
| USDA | 41 | 103 | 72 | 54 |
| Commerce | 86 | 147 | 97 | 82 |
| Defense | 131 | 201 | 298 | 371 |
| DOE | 160 | 367 | 564 | 462 |
| EPA | 20 | 5 | 10 | 8 |
| HHS | 53 | 25 | 36 | 22 |
| Interior | 3 | 15 | 39 | 4 |
| Transportation | 8 | 14 | 14 | 0 |
| TOTAL | 502 | 877 | 1130 | 1003 |
The NSF 1998 Indicators volume contains a modest amount of information on the structure of the CRADAs contained in table 2. According to NSF (National Science Board, 1999, p. 4-32), nearly three-quarters of the new CRADAs executed in 1995 were single-firm CRADAs, involving collaboration between a federal agency and an individual industrial firm. "Consortia and nongovernmental organizations," which are not defined by NSF, accounted for 87 of the 1995 new CRADAs. Universities accounted for 86 new 1995 CRADAs and state and local governments executed 10 new CRADAs in 1995. The 1003 new 1995 CRADAs involved 688 private-sector organizations, 124 of which executed two or more CRADAs during 1995. Similar information on the structure of the CRADAs reported in table 1 would be very valuable.
Very little published data provides additional detail on CRADA activities by agency. A recent paper by Guston (1998) on CRADAs in the National Institutes of Health, the largest single source of federal funding of nondefense R&D, provides some information on NIH CRADAs during the 1990s. Guston's data on trends in the execution of NIH CRADAs (Figure 2) display trends that contrast somewhat with those in Figure 1 for overall CRADA formation. Rather than a rapid increase throughout the 1990s, Guston's data depict rapid growth in CRADAs during the late 1980s, followed by a period of stability in the rate of new CRADA formation during the FY 1990-1995 period.[5] The sharp increase in executed CRADAs during FY 1995-96, according to Guston, reflects the execution of a large number of "material transfer agreement CRADAs" (MTA CRADAs) during this period. MTA CRADAs are initiated by NIH researchers to gain access to research materials from non-NIH researchers in situations where a simple materials transfer agreement will not suffice.
According to Guston, almost one-half of the 352 CRADAs executed by the NIH during the FY 1985-96 period were still active as of FY 1997. NIH CRADA execution is dominated by the National Cancer Institute, which accounted for 106 (30.6%) of all CRADAs formed during this period. The NCI is followed by the National Institute of Allergy and Infectious Diseases (NIAID), which accounts for 56, almost 16% of total executed CRADAs. The Institute of Diabetes and Digestive and Kidney Diseases (8.8%), the Institute of Mental Health (7.4%) and the Institute of Neurological Disorders and Stroke (7.9%) are the next most active NIH entities in CRADA formation. The five most active components of the NIH complex account for more than 70% of CRADA executions during this period.
The NSF data and the data reported in Guston (1998) provide no information on the size of federal expenditures in support of CRADA-related research. Additional information from internal agency sources on expenditures on CRADAs by DOE, the federal agency responsible for the majority of CRADAs, are available for FY 1993-99. As was noted above, most of the CRADAs signed by DOE during the early 1990s included cost-sharing provisions that provided laboratory support (either funding or "in-kind" contributions) for up to 50% of total project costs. Internal DOE data indicate that federal funding for the federal portion of CRADA expenses rose from $176 million in fiscal 1993 to more than $346 million in fiscal year 1995, declining to an estimated level of $94.5 million in fiscal year 1999 (Table 3).
DOE expenditures on CRADAs have amounted to more than $1.4 billion during the 1990s in constant (1999) dollars. This amount represents a small share of DOE's cumulative R&D budget during this period (also in constant dollars) of almost $50 billion, but its CRADA program is among the larger DOE initiatives dedicated to the support of civilian "pre-commercial" technology development. These data indicate that the expenditures by DOE on support for CRADAs are roughly comparable to the budget of another federal technology policy initiative of the late 1980s and early 1990s, the Advanced Technology Program (ATP). The ATP's budget has been cut sharply in recent years, but at its peak in fiscal year 1995, total budget authority for the program amounted to $341 million, below the nearly $350 million peak budget for DOE support of CRADAs.[6] Cumulative appropriated budget authority for ATP during fiscal years 1994-97 amounted to $986 million in current-year dollars (Hill, 1998), less than DOE expenditures (in current-year dollars, slightly more than $1 billion) in support of CRADAs during the same period.
| Fiscal Year | Current ($000) | Constant ($000) |
|---|---|---|
| 1993 | 175,799 | 194,747 |
| 1994 | 259,598 | 281,686 |
| 1995 | 346,437 | 368,065 |
| 1996 | 251,414 | 241,003 |
| 1997 | 148,700 | 152,591 |
| 1998 | 110,239 | 111,804 |
| 1999 (est.) | 94,156 | 94,516 |
| TOTAL | 1,386,703 | 1,444,411 |
In other words, the annual expenditures by a single federal agency on support for CRADAs (albeit an agency that accounts for the largest share of overall federal CRADAs) appear to exceed the annual budget for one of the "flagship" civilian technology programs of the Clinton Administration. Total annual federal expenditures on support for CRADAs are likely to be substantial, although data on this point are not available. Despite their large number and the (apparently) substantial expenditure on public funds on CRADAs, virtually nothing is known about their structure or composition, in contrast to ATP.[8]
This brief summary of available data on CRADAs suggests that they may have accounted for significant public expenditures during the 1990s in support of the federal portion of the joint research activities at their center. Yet no annual data are available on total federal expenditures on CRADAs. The data on CRADAs reported by the National Science Foundation omit some detail (e.g., the breakdown by agency of the structure of CRADAs executed in 1995) that would be very useful to scholars, analysts, and policymakers. CRADAs are only one of several mechanisms for federal agencies to manage collaboration with and technology transfer to industrial firms; but virtually no data are reported by NSF on these other mechanisms. Indeed, the Commerce Department's reports on federal technology transfer activities, which form the basis for the NSF Indicators data on CRADAs, do not link the reported "outcomes" of these activities to the different channels through which federal agencies are authorized to conduct them.
The available data reveal almost nothing about the effects of CRADAs on the performance of participating laboratories or industrial firms. The "performance measures" collected by both the DOE and NSF report outcomes that may reflect the use of any of several technology transfer mechanisms, rather than CRADAs alone. The next section summarizes the limited evidence on the effects of CRADAs on industrial innovation, drawing on several recent studies that employ diverse methodologies.
The paucity of public data on CRADAs may be partly responsible for the small amount of empirical analysis of their effects or effectiveness. As I noted earlier, the data now being reported by NSF on CRADAs provide no information on CRADA outcomes, nor do they distinguish between CRADAs and other federal policy vehicles for the support of collaboration, patenting or licensing. The small number of studies that have been undertaken rely on data collected (or selected) by researchers, and the conclusions of these studies accordingly may not apply to all CRADAs, firms, or federal laboratories. Nevertheless, both quantitative and case study evidence reveal some interesting findings concerning the effects of CRADAs on firm performance and the conditions that appear to support effective management of CRADAs. But these studies reveal little about the long-term effects of CRADA participation on the federal laboratories.
Two recent quantitative studies that shed some light on the role of CRADAs in industrial innovation are those by Adams et al. (2000) and by Jaffe and Lerner (1999). The study by Adams et al. examines the effects of CRADAs on industrial firms' innovative performance, and finds these effects to be significant. These conclusions must be interpreted with considerable caution, however, because of the characteristics of the data used by Adams and colleagues. Indeed, these data themselves illustrate both the potential of better public data on CRADAs to support research on industrial innovation and the inadequacy of the available data for these purposes.
The Adams et al. (2000) study was part of a broader project that sought to understand the effects of firms' "external linkages" on their innovative performance. The particular project reported in the research team's 2000 paper focused on the effects on firms of relationships with federal laboratories. Because of the inadequacy of publicly available data, Adams and colleagues conducted two surveys—one focusing on the activities of the industrial R&D laboratories of U.S. firms, and the other focusing on federal laboratories' investments in specific research fields.
Adams' survey of industrial firms was confined to relatively large firms, since he required that all respondents be publicly traded (in order to obtain additional firm-specific data on sales, etc. that are included in Compustat). In addition, his measures of industrial labs' "external linkages" with federal laboratories relied on laboratory directors' assessment of the importance of various mechanisms for knowledge transfer and interaction, including CRADAs. Finally, of course, Adams' data on industrial firms are confined to those firms with dedicated industrial research laboratories, and thus exclude many small or new firms. Several of the firms whose CRADAs were included in the Ham and Mowery (1998) study (see below) were firms that are excluded from the Adams et al. study.
The Adams et al. measure of CRADAs does not capture the existence or nonexistence of a CRADA between a given industrial and federal laboratory; nor does it measure the number of CRADAs that a given industrial laboratory maintains at any point in time. Instead, Adams' data provide information on the characteristics and inventive performance of industrial laboratories that report that CRADAs are "important" or "unimportant." Adams' data measure the influence of CRADAs deemed by laboratory managers to be important, without controlling for the number, the cost, or the failure rate of such undertakings.
These caveats notwithstanding, Adams et al. (2000) find that industrial laboratories that rank CRADAs as "important" vehicles for technology transfer display significantly higher rates of patenting, the primary measure of laboratory performance used in the study.[9] This "CRADA effect," which is much stronger among laboratories affiliated with firms that are not government contractors, outweighs the influence of virtually all other measures of federal laboratories' linkages with or influence on the industrial laboratories in the Adams et al. study. Adams and colleagues also find that industrial laboratories indicating that CRADAs are important also spend more on R&D in federal laboratories; they spend more on in-house R&D; and they have larger in-house R&D budgets funded from federal sources. In addition to these strong firm-level effects, Adams et al. also find considerable differences among the measured influence on industrial innovation of R&D investments by the laboratories of various federal agencies—the "knowledge stocks" resulting from the R&D investments of DOE and DOD laboratories negatively influence industrial laboratories' patenting. This result may reflect the tendency for DOE and DOD laboratories to work with government contractors that patent a smaller share of the results of their R&D.
The Adams et al. (2000) study is virtually the only systematic, rigorous study of the influence of CRADAs on innovative performance within a large sample of firms. Both firm-level and agency-level effects also appear to be important influences on the relationship between industrial laboratories' external linkages and their innovative performance (as measured, imperfectly, by total patenting). Although the study's findings must be qualified by a recognition of the unrepresentative nature of the sample of firms and the characteristics of its measure of "CRADA influence," its findings are intriguing and important. The results of the study underscore the need for the collection and dissemination by federal agencies of more reliable aggregate data on CRADAs.
The second paper does not examine the effects of CRADAs per se on industrial innovation. Instead, Jaffe and Lerner set out to analyze the effects of the much broader array of post-Cold War federal "technology transfer" initiatives (including CRADAs) on the 23 largest DOE laboratories. Their analysis compares the characteristics of DOE laboratories' patenting before and after 1987, a year chosen to mark the inception of the "technology transfer" initiatives. In addition, they examine laboratory-specific influences on the number of CRADAs executed by each during 1991-97 (omitting 1996). They find considerable laboratory-specific differences. Laboratories operated by universities appear to patent more heavily, as do laboratories that shift to narrow their distribution of patents among technology classes. Laboratories that focus more heavily on "basic science" patent less, controlling for size and other characteristics. Patent output also is significantly higher during the post-1987 period, with no measured decline in the significance (measured in terms of citations) of laboratory patents relative to patents from other sources within the same technology class.
Similarly significant laboratory-specific differences appear in the analysis of the determinants of the number of CRADAs executed by laboratories, than as the beginning of the periods. Defense-oriented laboratories execute fewer CRADAs during this period, as do laboratories with a greater "basic science" focus. A recent change in contractor (at GOCO laboratories) also is associated with a higher number of CRADAs.
Taken together, these quantitative studies suggest the importance of disaggregating the CRADA activities of different federal agencies and laboratories. The Jaffe-Lerner study suggests that at least some of these laboratory-specific differences may be explained by competition and organizational focus, results that have interesting implications for the management of the enormous network of federal laboratories. The interesting results of the Adams et al. study also suggest that CRADAs may influence innovative performance, although more information on the number and cost of CRADAs is needed to place this finding in context. Both of these quantitative studies' coverage of the CRADA activities of the overall federal laboratory establishment is incomplete, meaning that care must be exercised in interpreting their results. The inability of either study to develop a more comprehensive dataset on CRADAs reflects the sorry state of the publicly available data, even as these studies' findings indicate the high payoff to improving the quality of these data.
As part of a larger comparative study of the U.S. "R&D system," Bozeman and Crow (1998) compiled data on the benefits of R&D collaboration (including technology transfer) from a sample of large federal laboratories and industrial-firm collaborators. The sampling strategy employed by these researchers made no attempt to compile a "representative" sample of federal laboratories, and (like other authors) Bozeman and Crow appear to have more extensive data on relatively large federal laboratories. Because their study focused primarily on the pre-1993 period, Bozeman and Crow have limited information on CRADAs (which became important technology-transfer vehicles for federal GOCO laboratories only after 1989), and their observations on CRADAs cover a period during which many federal laboratories were still learning to employ this instrument. Nevertheless, at least two interesting findings emerge from this study.
Consistent with this paper's earlier discussion of alternatives to CRADAs, Bozeman and Crow find that industrial firms rely on a number of non-CRADA mechanisms to interact with federal laboratories. The authors examined more than 330 "interactions" between industrial laboratories and 27 large federal laboratories; 70% of these interactions involved DOE laboratories. According to the authors:
The authors' data suggest that even during the period of extensive use of CRADAs, these vehicles still accounted for only slightly more than one-half of the "interactions" between industrial firms and large federal laboratories. The second interesting finding in Bozeman and Crow (1998) draws on the same study of "interactions" between federal and industrial laboratories:
The authors speculate that this surprising finding may reflect political pressure on the DOE laboratories to increase the number of CRADAs in the early 1990s, resulting in a large number of nonviable projects. The case studies discussed immediately below suggest that internal DOE financial incentives also may have contributed to the execution of a number of CRADAs during this period that had significant operational problems.
Ham and Mowery (1998) conducted a series of case studies of individual CRADAs undertaken by Lawrence Livermore National Laboratory (LLNL), a DOE nuclear weapons laboratory operated by the University of California, in 1994-95. The case studies covered five single-firm CRADAs ranging in size from less than $1 million to more than $20 million, and involved extensive interviews with project personnel from the laboratory and from the participating industrial firms. It is important to note that these case studies covered CRADAs that were executed in the early phases of DOE's CRADA program; a number of policies have been instituted subsequently that seek to address some of the problems highlighted in these cases. Nonetheless, these case studies illustrate the high variance in CRADA outcomes, as well as the importance of project- and firm-specific factors in affecting these outcomes. A subsequent study by Linden et al. (1999) examined the development of a very large CRADA, dealing with "extreme ultraviolet" (EUV) photolithography technologies for semiconductor manufacture. This multi-laboratory, multifirm CRADA, which was initiated in 1997, is structured quite differently from those examined in Ham and Mowery (1998), and its operations suggest some other important characteristics of the CRADA mechanism.
The legislative and administrative requirements of CRADAs produced long delays in the negotiation and approval by DOE of many of the projects examined in Ham and Mowery (1998). Such delays imposed serious handicaps on the projects that were exacerbated by the inability of most LLNL research teams to begin work before a project's final approval. Since most of the CRADAs had insufficient budgetary flexibility to allow for a gradual "ramp-up" of project activities, informal agreement between the firm and laboratory researchers on the specific goals of a project was in some cases followed by months of inactivity. During these prolonged lapses in communication between laboratory and firm personnel, project goals often changed considerably.
Another budget-related problem emerged during the final stages of several projects in the Ham and Mowery study. Laboratory researchers, rather than gradually phasing out their involvement with firm personnel and assisting with technical issues associated with the transition from laboratory prototype to high-volume manufacturing, frequently had to terminate their participation as soon as the project's budget was exhausted, often as soon as a prototype had been demonstrated. Translation by private firms of a prototype into a commercially desirable (and manufacturable) product, however, requires extensive, technically demanding work. In several of our cases, laboratory personnel continued to work informally with the firm after the end of CRADA-related funding, but they did so without budgetary support.
Other budgetary policies made it difficult for firms pursing smaller projects to obtain timely access to the laboratory's expertise and resources once the project was under way. Project managers from LLNL noted that inflexible internal budgetary allocation policies and the high unit costs of the laboratory's R&D (especially when overhead charges are included) prevented senior engineers and managers from committing significant portions of their time to small-budget projects. Budgetary constraints also limited the ability of LLNL managers to respond to changes in the technical or budgetary scope of the project.
Commercialization of the results of a technology codevelopment project requires considerable technical sophistication and managerial competence within the private-firm partner, which may be particularly scarce in smaller firms. The demands for such expertise extend to the formulation of projects—in one of our cases, technical and managerial weaknesses within the small-firm partner produced an unrealistic set of project goals that impeded the execution of the CRADA and contributed to a commercially unsuccessful product. R&D consortia in other high-technology industries, such as SEMATECH, have also found that many of the small firms with whom they collaborate need more than technology—in addition to technological collaboration or assistance, the management, marketing, and manufacturing skills of these enterprises must be improved (Grindley, et al., 1994). This task requires a more ambitious and multidisciplinary effort than most DOE laboratories can support within a CRADA.
The findings of these case studies suggest the importance of developing clear criteria for selecting projects and (at least as important) industrial clients for CRADAs. These studies also underscore the need for both federal laboratories and their erstwhile industrial collaborators to consider an array of non-CRADA alternatives in structuring a joint R&D project. The political salience of CRADAs in the early 1990s and the associated efforts to expand the number of DOE CRADAs rapidly, paradoxically may have undercut the effectiveness of these vehicles. Finally, the financial incentives that led many DOE laboratory managers to aggressively promote CRADAs to prospective industrial partners may have reduced the effectiveness of these vehicles for collaboration and technology transfer. All of these factors underscore the importance of examining a much broader array of channels and vehicles for federal technology transfer in any assessment of the level or effectiveness of these activities.
Although the number of new CRADAs executed by the Department of Energy has declined, a large DOE CRADA with the "Extreme Ultraviolet Limited Liability Company" (EUV LLC), was begun in 1997. The EUV LLC is a consortium owned by three U.S. semiconductor manufacturers, Intel Corporation, Motorola, and AMD. The EUV LLC has established a CRADA with three of the largest DOE laboratories—Sandia, Lawrence Livermore, and Lawrence Berkeley National Laboratories—to develop technologies for "next-generation lithography" (NGL) in the semiconductor manufacturing industry. The EUV initiative relies on both government-industry and intra-industry collaboration to develop a technically effective and commercially feasible next-generation lithography technology. How if at all do the structure and management of the EUV CRADA address the problems of CRADA management and collaboration discussed earlier?
Perhaps the most important difference between the EUV CRADA and those examined in Ham and Mowery (1998) is the relationship between the private firms and the participating DOE laboratories. Rather than a jointly funded undertaking, in which both laboratory and firm participants shared management responsibility, the EUV CRADA is one in which the private firms provide all of the operating budget, as well as contributing a number of pieces of costly laboratory equipment. The participating DOE laboratories provide unique facilities and research skills, but they do so in a capacity that resembles that of a research contractor, rather than a collaborator with significant control over the agenda or budget.
The structure of the EUV CRADA avoids an important pitfall in the previous DOE laboratory CRADAs that relied in part on funding from the DOE headquarters. The availability of matching funding from DOE created strong incentives for laboratory personnel to market the research facilities and capabilities of DOE laboratories. In some cases these marketing efforts created unrealistic expectations among private-firm participants about the size, cost, and likely time horizon of technical and commercial benefits from the CRADA. The availability of public subsidies for these CRADAs encouraged laboratory personnel to pursue activities that were too distant from the historical strengths and capabilities of laboratories, especially those specializing in nuclear weapons design.
In the case of the EUV CRADA, however, the private firms that are providing the operating budget are unambiguously in charge of the research agenda. Moreover, their continued financial support depends on the ability of laboratory personnel to address the challenges of the collaborative research agenda, reducing some of the incentive conflicts associated with previous CRADAs that drew in part on DOE funds. Rather than competing among themselves for private-firm partners in order to obtain additional funds from DOE headquarters, these three large DOE laboratories are motivated to collaborate by the structure of the EUV LLC CRADA. Indeed, one participant interviewed for this study characterized the level of cooperation among the labs as unusually high, and argued that this effective collaboration was attributable in part to the unusual nature (and large size) of this CRADA.
Private financing also may allow for greater flexibility in CRADA administration, enabling the project to "ramp down" more gradually in the transition from laboratory to prototype development and production of the equipment under development. At the same time, however, the scope of the technological challenges posed by this CRADA is such that participation yields benefits for the DOE laboratory personnel. The ambitious technological goals of this CRADA contrast with those of the CRADAs studied in Ham and Mowery (1998)—in several of those cases, the availability of DOE funding led laboratory researchers to pursue short-term "job-shop" projects with limited technical benefits for the laboratories' missions. More technically challenging CRADAs address another long-standing problem for the DOE laboratories, especially those focusing primarily on national security missions—maintaining the technical expertise of laboratory researchers.
Another important contrast with many other CRADAs is the sheer scale of the EUV CRADA, whose 3-year budget is $250 million. Given the relatively high operating costs of the DOE laboratories when indirect charges are included, private firms in small-budget CRADAs often had problems in gaining or maintaining access to senior laboratory staff researchers. The scale of this CRADA, however, means that even senior laboratory managers and researchers are more likely to become involved. In addition, the size and technical capabilities of the participating firms dwarf those of the private-sector participants in many of the CRADAs examined in Ham and Mowery (1998). As a result, the firms leading the EUV CRADA are better able to sustain the considerable investments in supporting this collaboration. These include frequent face-to-face meetings, temporary assignment of firm personnel to work in the DOE laboratories, and the necessary in-house investments in related R&D that improve these firms' ability to evaluate, absorb and exploit the results of the project.
Although these characteristics of the EUV CRADA appear to represent improvements over the structure of previous DOE CRADAs, significant political and management challenges remain. Perhaps the greatest challenge is that of "handing off" the results of this CRADA to enterprises capable of developing commercial versions of the tools embodying these results. The LLC members are semiconductor manufacturers who have expressed no interest in entering the production of manufacturing equipment (indeed, any such entrant would face serious competitive challenges in selling equipment to semiconductor manufacturers who are competitors).
The role of leading semiconductor manufacturers in supporting the development of the EUV technology and the two-year period of exclusive access by LLC members to commercial versions of the EUV tool could impede commercialization of a next-generation lithography technology. The EUV LLC partners, rather than their competitors, will have early access to commercial versions of the EUV equipment. Competitors of the EUV LLC partners may elect not to adopt the technology, choosing instead to pursue alternative technologies with no delays in early access to commercial tools. Such reluctance could fragment the market for EUV or other lithography technologies, making it difficult for equipment manufacturers to recover their development investments.[11]
Other difficult issues affecting the commercialization of an EUV prototype are associated with the management of the intellectual property produced by the LLC and with the need to assemble a substantial portfolio of patents to develop the EUV technology. LLC members require access to the intellectual property of nonmember firms to develop this technology. LLNL, Lucent, and Ultratech are among the organizations that control EUV-related intellectual property, and commercial development of the technology requires that they license their patents to the EUV LLC. At least one of these owners of EUV-related intellectual property (Lucent) is a manufacturer of semiconductor devices who competes with the LLC member firms. This fact may reduce Lucent's willingness to license a group of competitors that will in turn profit from the application of Lucent's patents to EUV. Moreover, since Lucent is committed to the development of a substitute for EUV (SCALPEL), facilitating the development of EUV could undercut the returns to its SCALPEL-related intellectual property, further reducing its incentives to license.
Still another uncertainty affecting the development of EUV concerns the ability of U.S. equipment firms to support the investments in product development, high-quality manufacture, and product support that are necessary to commercialize this technology. In particular, product-support issues (maintenance, troubleshooting, spare parts) have proven critical to the success and failure of semiconductor equipment producers in the past[12] and remain a serious issue at many U.S. equipment suppliers. The uncertain commercialization capabilities of many equipment producers (particularly U.S.-based equipment firms) argue in favor of widespread dissemination of EUV-related intellectual property, in order that as many equipment firms as possible have an opportunity to undertake the costly investments. But the policies of the EUV LLC member firms emphasize profitable (and therefore, restricted) licensing, which may limit entry by prospective producers of the equipment and could impede the commercial development of EUV. Moreover, potential licensing of foreign equipment producers has sparked political controversy and remains contentious.[13]
The EUV CRADA illustrates the importance of another critical gap in the data collected by the federal government on CRADAs. Information on the financing and the structure of these undertakings is currently unavailable in any centralized tabulation; yet this brief descriptive case study suggests that both the financing and the structure of CRADAs influences their effectiveness.
There are three basic deficiencies in the current reporting by NSF of data on federal Cooperative Research and Development Agreements:
As I noted earlier, the current discussion in Science and Engineering Indicators: 2000 by NSF of the data compiled by the Commerce Department on CRADAs and technology transfer "outcomes" incorrectly attributes all of the invention disclosures reported by federal agencies during fiscal years 1987-98 to CRADAs. Instead, a substantial fraction, perhaps as much as 50%, of the disclosures reported by each agency result from other activities, including intramural research or other types of collaborative R&D with industry.
The data reported by NSF suffer from two other broad defects: like other "evaluations" of federal agencies' performance in technology transfer, they place too much emphasis on CRADAs, to the exclusion of other instruments; and the data on CRADAs omit a great of important information. Although NSF cannot be faulted for failing to report data that it does not have, it seems clear that the emphasis within the Indicators discussion on CRADAs is misplaced. NSF, along with the Department of Commerce, should urge other federal agencies to report technology transfer activities in areas other than CRADAs, including the number of "work-for-others," "user facility agreements," "personnel exchange," and other technology-transfer activities.
Rather than just the total number of licenses, the number of licenses associated with each of these vehicles, as well as licenses from other sources, should also be reported. Similarly, the sources of "invention disclosures" should be reported in the Commerce Department and NSF data. The data currently reported by the Department of Commerce and the National Science Foundation on "technology transfer outcomes" provide very little information on the importance or performance of the numerous different federal policies to encourage technology transfer that have been instituted since 1980. The bias in the NSF and Commerce Department reporting reflects the inexplicable bias in overall federal technology transfer policy that emphasizes CRADAs above alternative instruments for technology transfer, even in situations where CRADAs may be less desirable than these alternatives. But this bias makes no more sense in data reporting than it does in overall policy.
The data collected by the Commerce Department and reported by the NSF on CRADAs also are deficient in a number of dimensions. Among the most glaring omissions is the failure to report agency-level expenditures on support of CRADA-related research and other activities. Expenditures for this purpose by the Department of Energy, the agency accounting for the largest number of CRADAs, were substantial during the FYs 1993-99 period, exceeding federal spending on the Advanced Technology Program. The DOE appears to have provided more extensive financial support for CRADAs, and its expenditures on this activity may not be representative of other agencies' spending. But CRADAs are not a "free good"—their execution requires federal resources. The lack of data on agency expenditures on CRADAs distorts evaluation of these instruments of federal policy.
Other data on CRADAs that should be collected by the Commerce Department on a regular basis include information on the number of new CRADAs executed each year, by agency, and the number of CRADAs terminated each year, by agency, along with some basic information on the reasons for termination (e.g., "project was completed," "failure to realize objectives," "goals of partners changed"). Data on the financial structure of the CRADAs (i.e., the size and share of any federal funding for agency R&D expenses), by agency and year, also should be collected. Finally, the data on CRADAs collected by the Commerce Department and reported by the National Science Foundation lack any detail on the characteristics of the industrial partners or other participants in CRADAs. Additional detail on the number of entities, whether they are industrial firms, universities, or other types of organizations, and additional detail on the characteristics of the participating firms (size, primary industry) would be very valuable. Disclosure of some of this information, especially that on the characteristics of participating firms, may be restricted. But the information on the number and type of CRADA participants, by year and sponsoring agency, are not subject to disclosure restrictions and should be made available to the public.
Cooperative Research and Development Agreements have been the focus of a great deal of political "hype" and (perhaps not coincidentally) very little systematic evaluation. The limited scholarly research on CRADAs suggests that these instruments may be associated with superior inventive performance, and that the performance of CRADAs depends on a number of factors that are specific to individual projects, such as the project budget, characteristics of the industrial participants, and sponsoring agency. Other evidence suggests that CRADAs may have absorbed substantial federal spending during the 1990s. But the data reported by the NSF and the Commerce Department do not permit a more systematic assessment of CRADAs. Worse yet, these data perpetuate a form of evaluative myopia, in which agency-level technology transfer performance is judged solely in terms of the number of CRADAs that they report, ignoring the numerous alternative and important tools available to agencies to support collaborative R&D and technology transfer activities.
If federal policymakers wish to do more than issue platitudinous endorsements of "public-private collaboration" in R&D, they might consider trying to understand the shape and contours of the landscape. Without better data on both CRADAs and agency-level R&D collaboration and technology transfer, such understanding will remain elusive.
[1] The Federal Technology Transfer Act of 1986 and its 1989 amendment extended the legislative framework developed in the Stevenson-Wydler and Bayh-Dole Acts of 1980, which authorized recipients of federal research to gain title to the intellectual property created in such projects.
[2] "All agencies" includes several not listed in the table, including the VA.
[3] "Invention Disclosures arising out of CRADAs increased rapidly at first, rising from 2,662 in 1987 to 4,213 in 1991..." (National Science Board, forthcoming).
[4] The eight mechanisms are "Cooperative Agreements," "Cost-Shared Contracts/Subcontracts," "Licensing," "Personnel Exchange Programs," "R&D Consortia," "User Facility Agreements," and "Work-for-Others" (U.S. Department of Energy, 1999, Appendix A, p. 1).
[5] The "fair pricing" debates of the early 1990s may have depressed the rate of CRADA formation at the National Institutes of Health. NIH CRADAs originally included a clause allowing NIH to require that a licensee submit confidential documentation "showing a reasonable relationship between the pricing of a Licensed Product, the public investment in that product, and the health and safety needs of the public" (quoted in Guston, 1998, p. 236). The clause led to public controversies over the pricing of the AIDS drugs AZT and ddl, but it was eliminated in the spring of 1995. Although we lack sufficient post-1995 data on CRADA formations to determine whether the elimination of this pricing clause has been followed by a significant upsurge in CRADAs, the data in table 1 do not indicate a dramatic surge in overall HHS CRADAs during FYs 1996-98.
[6] The Small Business Innovation Research (SBIR) program, which mandates a 2.5% "set-aside" of large federal agencies' R&D budgets for small firms, is estimated by Walsten (1998) to have spent more than $1 billion in FY 1997, well above either ATP or the DOE expenditures on CRADAs.
[7] These data from DOE, which are denoted as "Annual Federal Funding for CRADAs," indicate substantially higher federal expenditures on support for the public laboratories' participation in CRADAs than the figures cited in Guston (1998, footnote 39). Guston provides no source for his data, which report the costs of the federal and non-federal contributions to DOE CRADAs during FYs 1993-96.
[8] Data reported in Hall et al. (2000) indicate that since its first funding awards in 1991, ATP has provided financial support for 352 projects, 234 of which are single-firm and 118 of which involve multiple non-federal participants. The mean total project budget (federal plus private funds) for ATP projects was roughly $6.6 million, although project budgets ranged from $490,000 to nearly $63 million. Mean project budgets for single-firm projects are $3.24 million, roughly one-quarter the size of those for "joint venture" projects ($13.24 million). The majority of ATP projects are in the information and computer technology field (29%), followed by biotechnology (19%) and materials (16%). Very little data are available as yet on the outcomes of ATP projects, reflecting the fact that the program is relatively new. Interestingly, and in some contrast to the situation with CRADAs, ATP has sponsored a number of evaluation studies.
[9] Adams and colleagues do not adjust patents for "quality" based on citations, possibly because of the relatively recent time period covered by the study.
[10] This section draws on Ham and Mowery (1998) and on Linden et al. (1999).
[11] It is interesting to note in this connection that the SEMATECH consortium ultimately abandoned a similar policy of exclusive access (involving a two-year period of exclusivity) by member firms to tools incorporating SEMATECH-funded improvements because of opposition by equipment firms participating in the consortium, who argued that the more restrictive policy limited the market for their tools (Grindley et al., 1994).
[12] See Grindley et al., 1994 or Randazzese 1996 for a discussion of the case of GCA, formerly a leading U.S. semiconductor equipment supplier. GCA's attempts to commercialize its state-of-the-art optical lithography tool, despite assistance from SEMATECH, ultimately failed, partly because of the firm's poor reputation for product quality and field support.
[13] Discussions among the LLC member firms, DOE and foreign firms over their participation in the EUV CRADA attracted Congressional criticism in October 1997. Although the language of the EUV CRADA was not revised in response to the Congressional criticism, at least one leading foreign lithographic equipment supplier, Nikon, elected not to participate in the CRADA as a result of the controversy (see Holstein, 1988).
|