1. Charge to the NIH Director's Panel on Clinical Research
In the spring of 1995, Dr. Harold Varmus, the Director of the National Institutes of Health (NIH), convened the NIH Director's Panel on Clinical Research (CRP), composed of physicians from academia and industry, "to review the status of clinical research in the United States and to make recommendations to the Advisory Committee to the Director, NIH about how to ensure its effective continuance." Topics for the Panel to consider included, but were not limited to:
Dr. Varmus challenged the Panel with three important questions: Who is going to do clinical research and how are these individuals to be properly trained? Where will clinical research be conducted? How will clinical research be funded?
2. The Panel's Definition of Clinical Research
Early in its deliberations the Panel realized the importance of reaching a consensus on a working definition of "clinical research" for all its tasks, particularly for tracking and monitoring funding activities. The Panel's three-part definition is:
(a) Patient-oriented research. Research conducted with human subjects (or on material of human origin such as tissues, specimens and cognitive phenomena) for which an investigator (or colleague) directly interacts with human subjects. This area of research includes:
- Mechanisms of human disease
- Therapeutic interventions
- Clinical trials.
- Development of new technologies
(b) Epidemiologic and behavioral studies
(c) Outcomes research and health services research.
Excluded from this definition are in vitro studies that utilize human tissues but do not deal directly with patients. In other words, clinical or patient-oriented research is research in which it is necessary to know the identity of the patients from whom the cells or tissues under study are derived. Therefore funding for research that is concerned with examining patients has been the area of concern for the Panel and its subcommittees, not funding for basic studies. However, mixed grant applications, as in program projects in which one element of the proposal is an animal model and one is a human model, were classified as clinical research for the purposes of the Panel's activities.
The Panel recognizes that there is no definition of clinical research upon which there is total agreement. The Panel's definition has been criticized for being too inclusive; some commentators prefer a narrower definition confined to patient-oriented research in which the physician and patient interact directly. Nevertheless, the definition above is the one upon which the Panel has based its recommendations. Furthermore, the Panel recommends that henceforth, the NIH use this definition as its standard for all analyses.
3. Panel Subcommittees -- Composition and Charges
At its first meeting, the Panel divided into four subcommittees, each with defined charges. They are:
Members: Dr. Ezra Davidson, Dr. William Peck
Charge: Examine the mechanisms by which NIH supports
clinical research including clinical trials; determine the
amount of funding NIH dedicates specifically to
clinical research, how it distributes those funds and
the decision-making processes underlying the
assignment of funds.
Members: Dr. William Peck, Dr. Mary Lake Polan, Dr. Samuel Thier
Consultants: Dr. William Crowley, Dr. Lee Goldman,
Dr. Louis Kunkel
Charge: Examine the question of who is going to do
clinical research in the future, how such
individuals will be trained and how they will be
supported. After reviewing various existing academic training programs, including the core curriculum at the Clinical Center, make recommendations for improvement and innovations. Evaluate current and
future job opportunities in patient-oriented research.
Members: Dr. Claude Bennett, Dr. David Nathan, Dr. Philip Pizzo
Consultants: Dr. Arthur Asbury, Dr. Janet Schlechte,
Dr. Gordon Williams
Charge: Review the operations and approaches
to research of the GCRCs and the NIH
Clinical Center and recommend ways to
enhance the efficiency and improve communications
and collaborations between the Clinical Center and
the extramural community. Examine the question of
where clinical research will be done in the future.
Members: Dr. Haile Debas, Dr. David Nathan,
Dr. Philip Pizzo, Dr. Herbert Pardes,
Dr. Leon Rosenberg
Charge: Consider novel as well as existing non-
NIH mechanisms for the funding of clinical
research by obtaining information directly
from the pharmaceutical industry, insurance
companies, private foundations, for-profit
health-care providers, etc. Develop ways,
for example, through the scientific and lay press
and other media, to inform the public about
the importance of and need for clinical research.
Dr. Lawrence Shulman, formerly Director of the National Institute of Arthritis, Musculoskeletal and Skin Diseases, and currently the NIH Director's Emissary to the Extramural Community for Clinical Sciences, served as consultant and adviser to the Panel. NIH staff who provided assistance to the Panel and its subcommittees were:
The Panel wishes to express its gratitude to Dr. Harold Varmus for his confidence and for the unfailing generosity of the NIH staff as it pursued its labors. The Panel is particularly grateful to Drs. Wendy Baldwin and Belinda Seto for their efforts in providing extensive NIH data, and was greatly assisted by Mrs. Janet Smith and Dr. Richard Wyatt, without whose competence and unswerving effort this report could not have been developed.
In addition to numerous working sessions, the Panel held public meetings at the National Institutes of Health on July 7, 1995, October 31, 1995, May 16, 1996, November 5, 1996 and November 7, 1997. The report of the May 16, 1996 meeting was widely circulated to the biomedical community for comment, and an interim report with preliminary recommendations was provided to the Advisory Committee to the NIH Director (ACD) in December 1996. Copies of reports of all the meetings as well as the December 1996 interim report to the ACD(1) are available upon request to NIH. Not all the detailed information provided in the foregoing reports is reproduced in this report, which summarizes the work of the Panel and provides its final recommendations to NIH.
The recommendations that follow supersede those listed in the December 1996 interim report, and are grouped into four broad areas in priority order of importance. The background and rationale for these recommendations are discussed in greater detail in Section III of this report.
Recommendation #8. The NIH should continue to improve the
quality of clinical research and strengthen research management
in the Warren Grant Magnuson Clinical Center (CC) and extend
the availability of its resources and expertise, as well as those of
the Institutes and Centers (ICs), to extramural investigators.
Recommendation #10. The NIH should expand efforts to
educate the public about the crucial importance of clinical
research for the future health of the nation.
The NIH Director's Panel on Clinical Research was commissioned in the spring of 1995 by Dr. Harold Varmus, the Director of NIH, because the perception of crisis in clinical research that had simmered for decades had intensified by a funding shortage induced by managed care and new restrictions on the Federal budget. James Wyngaarden's now classic paper published in 1979(2) brought the issue into bas-relief, while Edward Ahrens' 1992 treatise(3) defined the details of the problem. Since then, there have been many other thoughtful commentaries that have reemphasized the Wyngaarden and Ahrens analyses(4-13). It is generally feared that clinical investigators are not sufficiently renewing themselves and are therefore an endangered species facing extinction. Some of the most important challenges are:
Dr. Varmus challenged the Panel with three important questions: Who is going to do clinical research and how are these individuals to be properly trained? Where will clinical research be conducted? How will clinical research be funded? He asked the Panel to consider several topics including but not limited to:
At first glance, it seems impossible to believe that a crisis in clinical research is at hand because a career in clinical research today would appear more rewarding than ever. Advances in molecular medicine are providing enormous scientific opportunities. Never before has the bench/bedside interface been more exciting and productive. Never before have clinical trials been more promising as new products of the genetic revolution flow from pharmaceutical and biotechnical companies. The era of managed care, while challenging in the extreme, has also opened opportunities for outcomes analyses and epidemiology that would never have occurred in the absence of a demand for a more quantitative approach to the results of medical care. Yet this Panel has gathered data that show that the ratio of M.D. to Ph.D. applicants for NIH support has progressively fallen over the past thirty years even though success rates for the two types of applicants are similar. Importantly, the absolute number of M.D. applicants has fallen further in the past three years. Furthermore, M.D.s who fail to achieve fundable priority scores from study sections following their initial applications are less likely to reapply than Ph.Ds. This represents a dispirited attitude among M.D. faculty members that bodes ill for the future of academic medicine and the public's health. The sense of excitement, opportunity and determination that should permeate the field is compromised by financial and career anxieties.
This Panel was organized to advise the NIH Director at a time when the Federal budget was under enormous pressure and the NIH was struggling to maintain, let alone increase, its share of that budget. This circumstance severely limited the Panel's options. It would have been impossible to suggest a decrease in the basic biomedical science budget of the NIH to support clinical research because new ideas in clinical research demand a vibrant and productive basic science program. In fact, biomedical research is a hugely successful aspect of American society and doing more of it, including both basic and clinical research, is both a rational goal and expected by the American public.
The Panel therefore decided to examine the present contribution of NIH to clinical research, hoping to find areas that could be improved and would strengthen the field. It was determined to find some useful solutions to the problem because NIH exists to improve the future health of the American public. Academic physicians trained to perform clinical research play a special and indeed vital role in that mission. Such physicians understand the emotional and physical limitations of patients, can select appropriate patients for study with sympathy and gentleness born from that understanding, teach students the art and science of clinical research and understand the impact of research procedures and clinical trials on individual patients. Though Ph.D.s can and certainly do take the lead in many types of clinical research, skilled research-oriented physicians are a vital resource whose role in the NIH mission is absolutely essential. Indeed, it is no exaggeration to state that without such physician-scientists, the mission of the NIH will fail.
Even to suggest that the mission of the NIH could fail is an unacceptable notion. The future health of all Americans and of the world's population depends on the continued success of the NIH. In the immediate post-World War II period, research facilities on the NIH campus and in the academic health centers were, in many ways, primitive. Yet, inspired by the remarkable advances that had led to victory, encouraged by developments such as the discovery of antibiotics and steroid hormones, thrilled by the incredible gain in knowledge of intracellular metabolism and ion transport, physician-scientists, now called clinical investigators, began to apply basic science to the bedside in a remarkable fashion.
They were a remarkable breed, who seemed to know everything about clinical medicine and applied the biochemistry and physiology of their time to their patients who stayed for weeks on research wards without charge because hospital day charges were so low. In such an environment, the medical leaders of the time carried out exciting experiments and made the annual meeting of the American Society for Clinical Investigation (ASCI) and like societies a thrilling experience.
In the 1950s and 1960s, promising and already established clinical scientists were offered the opportunity to join the expanded U.S. Public Health Service to staff the newly-opened Building 10, now the Warren Grant Magnuson Clinical Center. A gleaming new hospital in 1956, it had comfortable patient rooms with excellent adjacent laboratories. There, physicians learned the new biology of the time and learned as well to apply it in patients. Marvelous clinical investigators were developed from the ranks of those early clinical associates. Some remained on the NIH staff and developed illustrious careers. Many others flocked back to the medical schools throughout the land and formed the academic faculties that developed training programs in clinical research.
Now, a mere thirty years later, disappointment and worry have emerged. Indeed, Joseph Goldstein has predicted that before the end of the first decade of the next century, the ASCI meeting will be defunct(10). Clinical investigators are searching for the reasons for their decline.
There are some obvious reasons. First, scientific research and clinical medicine have become more specialized, complex and demanding so that the combined teacher-clinician-scientist with broad interests is harder to train. Even teaching is more time consuming. Students have become deeply in debt as average medical school tuitions have risen 30-fold over the past 40 years. The rapid growth of Ph.D. applicants to the NIH extramural grant program has forced down the success rate of all applicants to the point of anxiety, and many M.D.s tend to give up early because they have massive debts. In addition, it is increasingly expensive to make a significant contribution to biomedical sciences because the technology required to do so has become very sophisticated and it is difficult for many clinical research training programs to keep up with that sophistication. Finally, the managed care revolution has made the multiple aspects of clinical investigation far more difficult. The reasons are too numerous to detail here, but one of the most serious consequences has been the rise of financial constraints leading to increased demand on the time that clinical investigators must devote to delivering health services rather than to research.
In 1991, the Institute of Medicine (IOM) turned an inquiring eye on the NIH and concluded that NIH policies are themselves partially responsible for the problem(14). The IOM reported that only ten percent of the NIH extramural budget is devoted to clinical research as its committee defined it. If that were true, clinical research would surely disappear. The IOM report, delivered in 1994, and a sense that the Division of Research Grants (now the Center for Scientific Review) study sections actually discriminate against clinical research applications(15), created a feeling of emergency. Dr. Varmus called the Clinical Research Panel together to examine the total problem, with particular emphasis on the NIH contribution.
In many ways it seems paradoxical to question the NIH commitment to clinical research. NIH exists to provide better care for patients in the future. Its Congressional support depends on the fulfillment of that mission, and that mission represents the central core of basic and clinical biomedical science. A magnificent plan for a new research hospital, named after Senator Mark O. Hatfield, is under way to provide state-of-the-art clinical research facilities on the NIH campus. It is impossible for NIH to turn its back on clinical research. On the other hand, NIH has never been alone in the struggle to improve medical care and conquer terrible chronic diseases. Long ago, James Shannon recognized that the conquest of cancer, heart disease, blood, kidney, neurological and psychiatric disorders and infectious diseases would require a strong partnership between the NIH, the academic health centers, the research universities and the pharmaceutical and biotechnical industries. Today the pharmaceutical industry sponsors, in aggregate, the majority of our nation's clinical research. In recent years, the payment system for medical care has changed so radically that a fifth and very heterogeneous partner has entered the scene. That partner -- the public and private insurance industry -- has a significant impact upon clinical research. The role of managed care in the contribution to the problems facing clinical investigators and the national clinical research effort must also be thoroughly understood.
The Panel has attempted to weave its way through all of the complexities of the present dilemma and to provide a set of useful and realizable recommendations. Its studies of the problem have led to an important conclusion. NIH is making a substantial contribution to the extramural clinical research effort; about two-thirds through its Center for Scientific Review study sections and about one-third through peer review mechanisms operated by the individual Institutes. The percentage of extramural dollars and the percentage of extramural grants devoted to clinical research as defined by the Panel is reasonable. To address some of the problems, the Panel has developed ten recommendations that should lead to improvements in the training, morale and therefore the self-renewal of clinical investigators, and in the granting mechanisms.
Finally, the medical research community must not allow itself to fall into a trap of limited expectations. There are those in Congress who are favorably considering a substantial increase in the NIH budget. This augurs particularly well for clinical research, and the Panel applauds the Congressional effort.
This report is therefore presented in order to promote the enhancement of the careers of clinically-oriented physician-scientists. Such enhancement depends on the development of programs that will stimulate a flow of successful applications from such physicians for clinical research support from the NIH, private foundations, industry and private and public health insurance carriers.
III. DISCUSSION OF THE PANEL RECOMMENDATIONS
(A) Recommendations Concerning Data on and Review of Clinical Research
Recommendation #1. The NIH should continue to monitor and track the percentage of NIH resources devoted to clinical research, as defined by this Panel, and report these results annually to the Advisory Committee to the NIH Director. The Panel views the current fraction of the extramural portion of the NIH budget devoted to clinical research as reasonable at the present time, although it should be subject to ongoing review and analysis. In addition, however, based on the proliferation of rich opportunities in medical research, the Panel strongly endorses efforts to increase the NIH budget as a whole and believes that such overall increases should at least include proportional increases for clinical research.
Clinical research funding at NIH is distributed within and between
the broad categories of basic, applied and developmental research, and the funding mechanisms can be divided into three categories: investigator-initiated awards, contracts, and cooperative agreements. When the Subcommittee on NIH Mechanisms for Funding Patient-oriented Research began work, the amount of funding allocated specifically to clinical research within these categories was not clear, nor was the distribution and impact of the funding known. At that time the Subcommittee knew that a total of approximately $4.7 billion was spent in FY 1995 on "human subjects research", i.e., research requiring review by an Institutional Review Board (IRB), a category that encompasses studies that do not, in fact, fall under the definition of clinical research used by the Panel (see page 1, above). The Subcommittee also knew that the funding of clinical trials, a category of research that accounts for only a portion of clinical research as defined by the Panel, amounted to approximately $1.2 billion in FY 1995. The Subcommittee therefore realized that it was essential to determine the amount between $4.7 and $1.2 billion that could be more accurately described as funding for other forms of clinical research.
Using the Panel's definition, an indexing staff in the NIH Office of Extramural Research reviewed and indexed all competing (primarily type 1 and type 2, but also including a small number of type 3, 4, and 9) awards beginning in FY 1996. The competing awards represent a cohort that is new or beginning. Thus, this was a prospective approach to an assessment of NIH research awards. Existing projects (type 5) were not indexed for clinical research. Since the duration of research projects averages approximately four years, many projects in the NIH portfolio are not included in these data. For example, in FY 1996, there were a total of 26,316 research project grants; only 6,839 (26%) were competing.
The clinical research indexing data (including clinical trials as a subset) for FY 1996 were collected for competing awards, and are summarized in Tables 1 and 2. Note from Table 1 that 27% of such awards and 38% of associated dollars supported clinical research projects. Table 2 shows the subtotal spent on clinical research for six selected major types of NIH competing grant applications funded in FY 1996, in comparison with the inclusive total for all other funding mechanisms.
Table 1. Clinical research indexing data, including clinical trials as a subset.
The types of grants are:
U01 - a cooperative agreement between NIH and a group of investigators, often a clinical trial
N01 - a research and development contract to evaluate a product or device
P01 - a standard program project grant
SBIR - Small Business Innovation Research Grant (R43, R44)
STTR - Small Business Technology Transfer Grant (R41, R42)
Centers - entity with shared resources and facilities for categorical research by a number of investigators from different disciplines.
*This figure includes funding for 518 clinical trials in FY 1996 at a total of $313.4 million.
Table 3 shows the percentage of successful M.D.s and Ph.D.s receiving competing awards. It is noteworthy that only 36% of the principal investigators of clinical research awards have an M.D. degree. This mirrors the low rate of applications from M.D.s in general. A further concern is the observation that between 1994 and 1996, first-time M.D. applications fell by 30% compared with 6% for first-time Ph.D. applications and 16% for first-time M.D./Ph.D. applications.
Table 3. FY 1996 NIH competing clinical research awards by degree of the principal investigator*
It is important to note (see Table 4) that review of applications for research funding takes place both in the Center for Scientific Review (CSR) and in the different Institutes and Centers (ICs). In FY 1996, 66% of the successful competing awards in clinical research shown in Table 2 were reviewed by the CSR, and 34% by the ICs, generally, but not always, according to the award mechanism to be used. For example, R01 and R29 applications are reviewed predominantly in CSR; fellowship applications are spread across CSR and the ICs, while program projects and centers are generally reviewed in the ICs. It is not the case, however, that the ICs review only solicited applications and the CSR only unsolicited applications. The Panel learned that the NIH Peer Review Oversight Group is addressing the question of when it is better to have reviews performed in CSR and when in the ICs, based not on whether applications are responsive to Requests for Applications, Program Announcements or are investigator-initiated, but on the science, including how fields of science develop in the future and who can provide the best possible review for the science.
Table 4. Site of review of NIH competing R&D; awards, FY 1996
The Panel recognizes that clinical trials are a very important component of clinical research and cannot be curtailed. These trials receive multiple levels of review by the ICs, by Institutional Review Boards, Data and Safety Monitoring Boards, etc. The NIH has a unique opportunity and responsibility to fund those clinical trials that are not, or will not, be funded by other sources, such as the pharmaceutical industry.
The discrepancy between the conclusions of the Institute of Medicine (IOM)(14) and the Panel about the percentage of NIH grants devoted to clinical research (10% in FY 1991 according to the IOM, versus 27% in FY 1996 according to the Panel) requires explanation. In 1991, the IOM analyzed a 10% sample (or 446 grants) of R01s, that had the human subject box checked on the application form, and that were reviewed by the Center for Scientific Review (then the Division of Research Grants). Applications reviewed by the ICs themselves were not included. In contrast, the NIH study initiated by the Panel included new and competing awards from all ICs, and all mechanisms (6,839 successful applications). Other factors contributing to the differences include: (a) the Panel's broader definition of clinical research, which includes epidemiology (excluded by the IOM). The IOM's focus was on "patient-oriented clinical research, defined as that which requires 'hands-on' participation with a human subject as opposed to the entire spectrum of clinical research"; (b) the fact that the Panel counted an entire study as clinical research, even if it had only one clinical research component; (c) the IOM's review did not cover a full budget year; (d) the absence from the IOM review of the National Institute of Mental Health, the National Institute on Alcohol Abuse and Alcoholism, and the National Institute on Drug Abuse, which were not officially part of the NIH until 1992, as well as the National Institute of Dental Research, the National Center for Nursing Research and the National Institute of Environmental Health Sciences; and (e) the fact that the Panel data were obtained by carefully trained and supervised scorers. The IOM acknowledged some of the pitfalls in its analysis,(14, page 95) including the absence of applications reviewed by panels convened by the ICs rather than the CSR.
Based on the data for this report generated by the Office of Extramural Research, which Panel members themselves have verified, the Subcommittee and the Panel believe that the present balance of funding between basic and clinical research as a whole is appropriate and that the proportion of funding between basic and clinical research should remain about the same, with careful oversight based on the quality of clinical research applications and the dictates of the scientific environment. Efforts to increase the NIH budget as a whole should be vigorously pursued.
Recommendation #2.
The NIH must ensure fair and effective reviews of extramural grant applications for support of clinical research: panels that review clinical research (a) must include experienced clinical investigators and (b) at least 30-50% of the applications reviewed by these panels must be for clinical research.
Although NIH budgets have increased in past years, the fraction of grant requests for basic and clinical research grants that are funded is not as high as it was twenty years ago. There is a widespread belief that funding for clinical research, in contrast to basic biomedical science, has not been awarded equitably during this period. This belief enhances the perception within academic medicine that M.D.s in general, and clinical investigators in particular, are disadvantaged in the competition for NIH grants.
In 1994, Dr. Gordon Williams chaired a study group that analyzed the review of patient-oriented grant applications by the Division of Research Grants, now the Center for Scientific Review (CSR)(15). The Panel Subcommittee on GCRCs and the Clinical Center carefully reviewed this group's report and endorsed its conclusions, namely that:
(B) Recommendations Concerning Training and Support for Clinical Investigators
The Panel Subcommittee on Training/Job Opportunities for Patient-Oriented Research (POR) recognized that other professionals besides physicians play a vital role in clinical research, but decided to focus its review on physician-scientists because they provide an important impetus for such research; because they are uniquely trained to care for all aspects of human disease, and because a continuing shortage of well-trained physician-scientists committed to careers in clinical research will have serious consequences for the future health of the country. Some commentators suggested that the Panel concentrated too narrowly on physicians at the expense of other health professionals, but the Panel believes that its conclusions and recommendations can be applied broadly to other health professionals as well.
The Subcommittee identified four main problems impeding the recruitment of physicians into clinical research.
The Subcommittee also evaluated selected successful clinical research training programs in research institutions throughout the country and found a wide variety of approaches to training, from traditional postdoctoral apprenticeships to more formal programs that incorporate course work on such subjects as molecular biology, gene therapy, cellular physiology, biostatistics, epidemiology, trial design, and other topics not usually covered in traditional medical education. Several well organized programs exist for training Ph.D. scientists in human biology and clinical investigation, such as the one instituted for a limited time by the Markey Charitable Trust.
The Subcommittee's initial recommendations to the Panel were in four general areas: (a) current training programs, including programs for training in and early recruitment into patient-oriented research; (b) mid-career support;
(c) partnerships in training, and (d) debt forgiveness. The Subcommittee has not made any projections about the optimum numbers of clinical researchers who should be trained in future. It is extremely difficult to make reliable projections of this kind.
Recommendation #3. The NIH should initiate training programs that will enhance the attractiveness of careers in clinical research to medical students.
This broad recommendation includes the four following subparts:
The biomedical community's response to the Subcommittee's preliminary recommendations for improved medical student training was favorable. Most commentators supported local rather than centralized programs to train students and suggested that the GCRCs and/or AHCs should take the lead in training initiatives, perhaps by means of federal/industry consortia, with emphasis on early recruitment into patient-oriented research, or by developing advanced degree programs.
Recommendation #4. The NIH should improve the quality of training for clinical researchers by requiring grantee organizations to provide formal training experiences in clinical research and careful mentoring by experienced clinical investigators.
The need for rigorous standards of training was stressed, if necessary with strict criteria for a smaller number of physicians who will obtain stable grant funding in the future. It is essential for NIH to monitor its clinical specialty training grants for evidence of quality in didactic and practical experience in clinical research and for careful mentoring. Serious consideration should be given to consolidation of current subspecialty programs into institutional clinical training grants encompassing research training across disciplines and research departments.
Recommendation #5. The NIH should initiate substantial new support mechanisms for young and mid-term clinical investigators, if possible in collaboration with the private sector.
The NIH should explore the possibility of partnerships in support and training, for example, with and between academic centers and hospitals and pharmaceutical companies; and consider supporting additional programs to provide clinically-oriented training for Ph.D.s who do not have M.D.s.
(B) NIH Support Mechanisms for Young Clinical Investigators.
The NIH currently offers a daunting array of awards to young physicians or dentists considering a career in clinical research and these awards are not currently effective overall in training them for clinical research. The most significant awards from a physician's point of view are the Mentored Clinical Scientist Development Award (K08), and the Mentored Clinical Scientist Development Program Award (K12). The K08, which now incorporates the K11 (Physician Scientist Award) provides for three to five years of non-renewable support for 75% of time spent in clinical research, including a basic science component. Stipend is approximately $50,000 with $20,000 funding for research support. The K12 provides five years of funding to an institution to support trainees spending 75% of their time in research in two phases; the first in didactic training in basic sciences and the second in mentored research. R01 applications are encouraged and the stipends for trainees are the same as for the K08s, approximately $50,000 with $20,000 in research support.
A recent analysis done by Dr. Lawrence Shulman disclosed that 929 individuals had K08 awards, 224 had K11 awards (Physician Scientist Award, now incorporated into the K08 award), and 39 institutions had K12 awards. In FY 1996, these awards cost a total of approximately $110 million. Overall, only about one in four of the individual awardees (K08 recipients) is doing clinical research, though the number varies considerably among the Institutes. Some smaller Institutes tend to support mainly clinical research awards, while most of the larger Institutes support on the average only one in six.
Although it appears that, over time, the majority of the K08 awards have been redirected from their original goals, this happened gradually and unintentionally because, during the 1970s and 1980s, in Gordon Gill's words, "many physician scientists had been seduced by the power of molecular biology and had abandoned patient-oriented research"(4) or, according to Joseph Goldstein, "the movement of M.D.s. toward basic research [has] created a vacuum in clinical research, often filled by M.D.s who lack research skills."(10)
The objectives of new or modified programs to generate a cadre of superb clinical or physician scientists are to provide young clinical professionals with the methodologic, quantitative, creative, critical and intellectual skills to enable them to perform first-class research; to recruit and start them as early as possible; to offer programs with great promise and likelihood of success; to combine in one award in-depth formal didactic training, including core and elective curricula tailored for their specific scientific interests, and mentored clinical research projects(s), and to provide adequate stipends and resources.
Implementation of the following two modified awards, as proposed by
Dr. Shulman, under the general heading of Clinical Scientist Development Awards is recommended by the Panel.
The current career development awards are not producing well-trained physician scientists who will spend a career in clinical research. The awards proposed above complement one another very effectively and would provide a new, direct approach to meeting this need. The institutional K12C would include the several broad interests of the academic health centers. It would invite administrative leadership at the universities; foster rigorous competition both within and between institutions in the selection of the didactic programs, the mentors and the trainees. Productive programs of high quality and new centers of excellence for clinical research would emerge at the earliest possible time.
In order to stabilize clinical research laboratories, prevent interruption of trainee mentoring and enhance the prestige of clinical investigators, the Subcommittee recommended establishment of mid-career salary awards and other special awards for clinical investigators, financed by NIH, pharmaceutical companies, private foundations, etc., to relieve clinical investigators from clinical duties. Such awards might also favorably affect tenure prospects for clinical investigators. The total amount of salary suggested would be limited to $125,000 per annum, hopefully with additional funds from receiving institutions.
Recommendation #6. A loan repayment program for clinical investigators should be instituted.
The Subcommittee recommended development of debt forgiveness programs by the government and by the private sector for M.D.s who complete a minimal time as full-time clinical investigators. Initially, the program could be limited to 100 M.D.s a year, at no more than $66,000 per individual for a total approximate cost of $6.5 million per year.
Representatives of some philanthropic foundations consulted by the Panel did not agree that clinical researchers should be singled out for debt-forgiveness, and others cautioned that it might create a "special class" of investigators. They suggested that if such a program were initiated, it should be designed specifically to attract potential clinical investigators rather than clinical fellows and junior faculty looking for a way to pay off medical school debt, and/or it should be matched to medical students who take a year off for intensive research training.
The Panel nonetheless strongly believes that, given the enormous debt incurred by many medical students nowadays, loan repayment programs would tip the balance for those who have a research bent but who may otherwise forego a research career in order to earn money quickly to pay off debt. Specific details of the amount of debt-forgiveness, time period covered and clinical research expectations remain to be negotiated.
Recent legislative proposals include provision for loan repayment for medical school costs for physicians committed to research careers. The Panel strongly endorses such legislative remedies in support of its recommendations.
(C) Recommendations Concerning the General Clinical Research
Centers and the NIH Clinical Center
General Clinical Research Centers (GCRCs)
Recommendation #7. The scope of the GCRCs should be broadened to enhance their leadership role in clinical research and research training, and NIH should significantly increase its financial support of these centers.
The GCRC Program is part of the clinical research component of the National Center for Research Resources (NCRR), and its main function is to provide an infrastructure for interdisciplinary, primarily clinical, research funded by NIH. In FY 1997 the 74 Centers at 64 institutions in 31 states support approximately 6,000 research projects and nearly 8,000 researchers at a cost of approximately $154 million, a relatively small proportion of the NIH budget. The GCRCs host investigators who are funded by other components of the NIH as well as by other Federal, state and local agencies, and by private institutions and organizations. They also serve as training centers for young researchers through programs such as the Clinical Associate Physician Program (CAP) and the Minority Clinical Associate Physician Program (MCAP). The GCRC program has been highly successful and has contributed significantly to clinical research. It provides one of the few government mechanisms that allows for quick turn-around of small-scale pilot clinical studies. Some of the GCRCs have also been effective in shifting clinical research from an inpatient to an outpatient mode and have highly developed expertise in clinical design, statistics and collection of patient data and samples.
In its review of the GCRCs, the Subcommittee on GCRCs and the NIH Clinical Center consulted current and former GCRC Directors, the report analyzing the review of patient-oriented research grant applications(15) and the evaluation report of the GCRCs done in 1993 (17). Subcommittee members also participated in the GCRC Directors' annual meeting in March, 1996, and the NCRR Council meeting in the spring of 1997. They also sent out a questionnaire to the GCRC Program Directors, requesting their assessment of the major impediments to conducting research at their Centers; asking whether their institutions had a Clinical Trials Unit (CTU) that supports or is limited to industry-initiated research, and asking whether they favored merger or regionalization of some GCRC functions.
Most responses to the questionnaire focused on funding issues, and problems of institutional support, especially limited flexibility in the use of GCRC funds. The survey also showed that 27% of respondents had a CTU or office of clinical research at their institutions providing alternate pathways for clinical research, often driven by the interface with industry, and another 23% had plans to develop CTUs. Fifty percent of the Program Directors favored merger or regionalization of core laboratories of GCRCs.
The Subcommittee made detailed recommendations, listed from (a) to (f) below, on various aspects of the GCRCs, but, at the same time, recognized that there is considerable variability in the modus operandi and scope of the different GCRCs and their relationship to their home institutions.
Clinical Center
Recommendation #8. The NIH should continue to improve the quality of clinical research and strengthen research management in the Warren Grant Magnuson Clinical Center (CC) and extend the availability of its resources and expertise as well as those of the Institutes and Centers (ICs) to extramural investigators.
The CC is currently undergoing significant changes in governance and management as recommended in a 1996 report(18) entitled "Revitalizing the NIH Clinical Center for Tomorrow's Challenges", which was being developed when the Subcommittee began work. The Subcommittee has since undertaken its own preliminary review of the CC, with greater emphasis on scientific policy, the quality of clinical research and how collaborations with the extramural community can best be implemented.
The Panel endorses funding and building the new Clinical Research Center (CRC), which they believe will enhance clinical research in the nation as a whole, not merely in the intramural research program, and supports more interactions between the Clinical Center and the GCRCs. However, opportunities for meaningful interactions between the Clinical Center and the extramural community, although potentially useful and important, may in fact, be limited. The Panel also questions whether the Clinical Center can attract the
patient populations necessarcy to conduct some types of clinical research without greater interaction with extramural investigators.
The Panel also noted that approximately 30% of the intramural research budget of the NIH is spent on clinical research and has been kept informed of the recommendations to the NIH Director of an internal committee on clinical research chaired by Dr. Stephen Straus(16) . The Panel endorses the recommendations made by the Straus committee and welcomes their implementation. However, it has reservations about NIH's ability to recruit and retain first-class clinicians to conduct protocols in the new CRC and believes that special efforts must be made to ensure that the new CRC will be capable of conducting cutting-edge protocols and be a resource and national model for clinical research for the Nation.
(D) Recommendations Concerning Partnerships
Recommendation #9. The NIH should sustain a productive dialogue on enhancing clinical research with its partners: the academic health centers, the foundations, and the pharmaceutical and managed care industries.
The Subcommittee on Funding Sources and Public Information attempted to obtain more detailed information regarding the exact amount spent on clinical research, as defined by the Panel, from the various public and private funders of clinical research. However, it found that data, except from the pharmaceutical industry, are difficult to quantify.
Research and development investment by research-based pharmaceutical companies is projected to reach $18.9 billion in 1997, an 11.5 percent increase over 1996 and an increase of more than double since 1990(19) These expenditures include $15.1 billion spent within the United States by both U.S.-owned and foreign-owned firms, 29.9 percent of which (or approximately $4.5 billion) is devoted to Phase I, II and III clinical trials. This does not account for other clinical research which falls under the Panel's definition. An additional 5 percent of research and development is allocated to Phase IV clinical trials, which occur after the product has been approved by the Food and Drug Administration. This amount makes the pharmaceutical industry the largest funder in the aggregate of clinical research in the United States.
It is the Subcommittee's impression that AHCs receive approximately 60% of their human research funding from NIH and that the percentage of NIH's contribution to clinical research as a whole, although considerable, as demonstrated by the Subcommittee on NIH Mechanisms for Funding Patient-Oriented Research, may be smaller relative to the other large contributors than was originally thought. There is no doubt that, with steady-state budgets for NIH and other Federal agencies that support biomedical research, it is increasingly important to utilize the Federal dollars that are available by using them effectively in partnership with the private sector.
In spite of the difficulty of estimating percentages of financial support from the various partners, the Subcommittee and the Panel Chair succeeded in organizing several valuable meetings with representatives of organizations such as the American Association of Medical Colleges, the American Medical Association, the American Association of Health Plans (the umbrella organization for 1,200 managed care organizations) several philanthropic foundations and large pharmaceutical companies, as well as many different individuals, including members of Congress. These meetings have led to important interactions between the Director of NIH and these organizations.
Several organizations commended the Panel's outreach efforts, and there is general agreement that funding partnerships between Federal agencies and the private sector are essential to sustain the clinical research enterprise. It was emphasized that clinical studies, together with strong basic science, offer the best hope for reducing the cost of medical care in the United States and that all who benefit from clinical research need to share the cost. Legislation may be necessary to ensure that managed care and other insurers contribute to the cost of clinical research; such legislation should also guarantee insurance coverage for patients participating in research. It was also suggested that careful review and support from health related industries other than NIH should be provided for the very high cost of clinical trials, which may contain elements of non-experimental health care. NIH could provide a valuable service if it were to provide review mechanisms for other funders.
Recommendation #10. The NIH should expand efforts to educate the public about the crucial importance of clinical research for the future health of the nation.
Rarely a day goes by without media attention to reports of recent scientific findings related to health. The frequency, scope and popularity of these reports demonstrate a keen public interest in medical research, although it is not clear that the public fully understands the ways in which the financial support for this research is changing as health care costs increase and the implementation of managed care widens. It is therefore incumbent upon research professionals to keep the public informed. Although NIH, as a Federal agency, is not in a position to lobby for additional funding, it can inform the public about research results in various ways. NIH's research partners can also publicize the benefits of research, as for instance, the Pharmaceutical Research and Manufacturers Association (PhRMA) does in its frequent media advertising. Other organizations boosting research include Research!America, the Association of American Medical Colleges, etc. The American Medical Association's Council of Scientific Affairs is a strong advocate for research and for transmitting news of research advances to the private physician, and there are many other entities actively engaged in publicizing the benefits and need for medical research. Nevertheless, efforts to keep the public informed cannot be relaxed, and the public and policy makers, including Congress, state and local governments and communities must be made constantly aware of the health and economic benefits of research as a public good.
This Panel has attempted to respond to its charge as efficiently as possible, and it is confident that its recommendations will, when fully implemented, prove useful to the field. Indeed, the Panel is gratified to observe that some of its recommendations are already in the process of implementation and more will be very shortly. [Implementation of the Panel's recommendations will be the subject of a separate report from NIH.] The Panel is also aware of the complexities of this long-standing problem. Several issues remain untouched by its inquiries and the Panel or a subset of it needs more time to address them. These issues include:
(1) The role of the medical schools and the teaching hospitals (AHCs)
The Panel refocused on the dominant responsibility that AHCs have for conducting translational research. It recognized that translational research is the "seed corn" for future progress in clinical medicine. Support for it originates mainly from two sources, NIH awards and institutional revenues primarily derived from the general funds of medical schools. While the Panel believes that much of the responsibility for such research is indeed properly assigned to the AHCs, their financial capacity is under severe stress because of changes in the financing and delivery of health care, as well as Federal budgetary practices. The fundamental mission of the AHCs -- to advance health care through research, teaching, clinical care and community service -- is costly. During the past 35 years, AHCs have come to rely heavily on patient care revenues to fill the gap in meeting their responsibility for research and teaching. In 1994, such clinical revenues amounted to more than $8 billion, $2.5 billion of which were used to subsidize the academic mission; $800 million of that sum was devoted to research. This significant revenue stream is increasingly jeopardized by the new health care market, thus endangering the clinical research enterprise of the AHCs.
The Panel was encouraged that the Secretary of DHHS recently commissioned a report on the Future of the Academic Health Centers to address the health and future of these institutions, and it eagerly awaits the release of the findings.
Finally, the Panel believes it is timely and important to consider specific new mechanisms of support or support from multiple sources for clinical research by which funds can be targeted for programs including training, faculty development and support, and creating and sustaining the inpatient and ambulatory environments. Such support should complement anticipated increased funding of research project grants and might be accomplished through a mechanism such as the Biomedical Research Support Grant. This type of investment would help to ensure the Nation's world leadership in medical research and in the translation of that research into improved health and well-being. The Panel considers this an area of unfinished business that must be addressed expeditiously.
(2) The role of study sections and the NIH Clinical Center.
While the Panel has made some valuable forays into these issues, the work is not completed.
(B) The Clinical Center. Planning for the new Clinical Research Center is taking place at a time of rapidly changing conditions for clinical research. Thus, plans made in 1995 may not be appropriate for the year 2000.
This Panel has had neither the time nor the expertise to review the plans and operations of the Clinical Center in its present or future forms. It urges that there be continuing review of the management of the Clinical Center by the Board of Governors and also of the clinical research programs of the individual Institutes. Some Institutes have already undertaken detailed review of their clinical research programs. The Panel urges that reviews be conducted by other Institutes of both intramural and extramural clinical investigations.
The issues which might be considered in these reviews include:
The Panel believes that this area requires further study.
(3) Recruitment of minorities into clinical research
Clinical investigation faces a growing problem in regard to the recruitment of minority groups into clinical trials and clinical outcome studies. On the one hand, it is clear that the findings in one ethnic group may or may not be universally applicable; sickle cell disease, hypertension, breast cancer, prostatic hyperplasia, cystic fibrosis and colon cancer are examples of diseases which exhibit significant variation in incidence among different ethnic groups. The extent of these diverse ethnic differences will become more evident with progress of molecular biology and the Human Genome Project. Interracial and cultural diversity are increasing in this country and will make this issue more acute in the future.
On the other hand, it is difficult to recruit many minority groups into clinical trials, and it is established that recruitment of such members can best be done by research teams that include minority members. Different racial and ethnic groups require different approaches and educational techniques to overcome these barriers to recruitment in clinical trials, approaches best undertaken with participation of minority scientists themselves. Recent decreases in minority recruitment into medical schools will make this situation even worse, and, if they continue, could constitute a major threat to the health of the Nation.
The Panel believes that this issue is of great importance and wishes to devote a considerable fraction of any future effort to this vital recruitment need.
As indicated in Recommendation 9, the Panel has initiated interactions between the Director and these important allies in clinical research. However, the dialogue has only just begun and the Panel strongly recommends its continuation.
V. CONCLUSION
The NIH Director's Panel on Clinical Research has worked for over two years to produce this report. Its efforts began because of great concern in the academic medical community that clinical researchers are a dying breed poorly supported by the NIH. To its surprise, the Panel found that NIH devotes over a third of its budget for new and competing awards into clinical research as the Panel defines it. However, only thirty-six percent of the principal investigators on clinical research awards are M.D. or M.D./Ph.D. scientists, and the Panel believes that the physician clinical investigator is a vital contributor to the Nation's clinical research effort. The Panel's data further demonstrate that this low physician funding is due primarily to the fact that physicians do not apply in sufficient numbers for NIH awards. The Panel's ten recommendations are therefore built around an effort to enhance the frequency and quality of such applications.
Hence, the Panel has focused its efforts on training, from medical student experiences through fellowships, and requests the NIH to develop young clinical investigator awards and to support mid-career clinical investigators. The Panel encourages strengthening the vital GCRC Program and urges NIH to be certain that the Warren Grant Magnuson Clinical Center and the new Mark O. Hatfield Clinical Research Center operate at the highest possible level of quality. Additionally, the Panel requests a loan repayment program for young physician-scientists committed to careers in clinical research. Finally, the Panel calls on NIH's partners in clinical research, the Academic Health Centers, the private foundations, the pharmaceutical and biotechnical industries and the health insurance companies, to examine their own clinical research portfolios and join with NIH in an effort to improve the likelihood that young physicians will adopt a clinical research career.
The Panel is pleased with the initial response of the NIH management to its recommendations. The NIH has already acted on some of these and implementation of others is on the way. The Panel believes that all of its recommendations should be established as NIH policy within a year from the submission of this report. One of the Panel's recommendations, a loan repayment program for physician-scientists committed to clinical research, will require an act of Congress. Therefore its implementation may take longer.
The Panel has been unable to deal with every issue with which it has been faced and some of these issues are already described in this report. The Panel recommends examination in greater depth of the needs of the AHCs for further Federal support at a time of serious fiscal constraint; the roles of the study sections and the Clinical Center, and the recruitment of minorities into careers in clinical research. Therefore the Panel believes that it should continue to exist in some form, not only to monitor implementation of existing recommendations, but also to pursue this important unfinished business.
1. Interim Report of the NIH Director's Panel on Clinical Research (CRP),
December, 1996. (Available upon request by telephone to 301-402-3444)
2. Wyngaarden, J. B. 1979. The Clinical Investigator as an Endangered Species.
New England Journal of Medicine, 301, 1254-1259.
3. Ahrens, E. H. 1992. The Crisis in Clinical Research: Overcoming Institutional
Obstacles. New York: Oxford University Press.
4. Gill, G.N. 1984. The End of the Physician Scientist. Am. Scholar 53, 353-368
5. Goldstein, J. J. 1986. On the origin and prevention of PAIDS (Paralyzed Academic Investigator's Disease Syndrome). Journal of Clinical Investigation, 78, 848.
6. Evans, R.N. et al. 1993. Physician Researchers: An Endangered Species?
In Swartz, H.M. and Gottheil, D.L. The Education of Physician Scholars.
Betz Publishing Co., Rockville, Maryland.
7. Culliton, B.J. 1995. Clinical investigation: An endangered science.
Nature Medicine, 1:281.
8. Blumenthal, D. and Meyer, G.S. 1996. Academic Health Centers in a Changing Environment. Health Affairs, 15:200-215.
9. Shulman, L.E. 1996. Clinical Research 1996: Stirrings from the Academic
Health Centers. Academic Medicine 71:362-363, 398.
10. Goldstein, J.L. and Brown, M.S. 1997. The Clinical Investigator, Bewitched,
Bothered and Bewildered-But Still Beloved. Journal of Clinical Investigation,
99:2803-2812.
11. Schrier, R.W. 1997. Ensuring the Survival of the Clinician-Scientist. Academic
Medicine, 72:589-594.
12. Thompson, J. N. and Moskowitz, J. 1997. Preventing the Extinction of the Clinical Research Ecosystem. JAMA 278, 241.
13. Shine, K.I. 1997. Some Imperatives for Clinical Research. JAMA 278:245-246.
14. Kelly, W. N. and Randolph, M. A., eds. 1994. Careers in Clinical Research:
Obstacles and Opportunities. National Academy Press, Washington, DC
15. Williams, G. H., Chair. 1994. "An Analysis of the Review of Patient-Oriented
Research (POR) Grant Applications by the DRG, NIH Clinical Research Study
Group.
16. Straus, Stephen, et al., 1997. Report of the NIH Committee on the Recruitment and Career Development of Clinical Investigators, in press.
17. Report of the Panel to Formulate Recommendations for the GCRC Programs,
National Center for Research Resources, Based on a Program Evaluation,
August 17-18, 1993.
18. Smits, H. L., Chair. 1996. "Opportunity: Revitalizing the NIH Clinical Center for
Tomorrow's Challenges." A Report for the Secretary from the Department of
Health and Human Services' REGO Options Team on the National Institutes
of Health Warren Grant Magnuson Clinical Center.
19. Pharmaceutical Research and Manufacturers of America (PhRMA) 1997
Industry Profile, Chapter 2.
Chair: Dr. Judith Swain
Chair: Dr. Jean Wilson
Chair: Dr. Guy McKhann
Chair: Dr. William Friedewald
Dr. Wendy Baldwin, Deputy Director for Extramural Research (DDER)
Dr. Elvera Ehrenfeld, Director, Center for Scientific Review
Dr. John Gallin, Director, Warren Grant Magnuson Clinical Center and Associate
Director for Clinical Research
Dr. Michael Gottesman, Deputy Director for Intramural Research
Dr. William Harlan, Associate Director, Office of Disease Prevention,
Office of the Director
Dr. Ruth Kirschstein, Deputy Director, NIH
Dr. Walter Schaffer, Director, Research Training and Special Programs Office,
Office of Extramural Research
Dr. Belinda Seto, Senior Adviser to the DDER
Dr. Lana Skirboll, Director, Office of Science Policy and Legislation, Office
of the Director
Dr. Judith Vaitukaitis, Director, National Center for Research Resources
Dr. Richard Wyatt, Executive Secretary for the Clinical Research Panel
Mrs. Janet Smith, Staff Assistant for the Clinical Research Panel
Recommendation #1. The NIH should continue to monitor and track the percentage of NIH resources devoted to clinical research, as defined by this Panel, and report these results annually to the Advisory Committee to the NIH Director. The Panel views the current fraction of the extramural portion of the NIH budget devoted to clinical research as reasonable at the present time, although it should be subject to ongoing review and analysis. In addition, however, based on the proliferation of rich opportunities in medical research, the Panel strongly endorses efforts to increase the NIH budget as a whole and believes that such overall increases should at least include proportional increases for clinical research.
Recommendation #2. The NIH must ensure fair and effective reviews of extramural grant applications for support of clinical research: panels that review clinical research (a) must include experienced clinical investigators and (b) at least 30-50% of the applications reviewed by these panels must be for clinical research.
Recommendation #3. The NIH should initiate training programs that will enhance the attractiveness of careers in clinical research to medical students.
Recommendation #4. The NIH should improve the quality of training for clinical researchers by requiring grantee organizations to provide formal training experiences in clinical research and careful mentoring by experienced clinical investigators.
Recommendation #5. The NIH should initiate substantial new support mechanisms for young and mid-term clinical investigators, if possible in collaboration with the private sector.
Recommendation #6. A loan repayment program for clinical investigators should be instituted.
Recommendation #7. The scope of the GCRCs should
be broadened to enhance their leadership role in clinical
research and research training and NIH should significantly
increase its financial support of these centers.
Recommendation #9. The NIH should sustain a productive
dialogue on enhancing clinical research with its partners:
the academic health centers, private foundations, and the
pharmaceutical and managed health care industries.
*From the NIH Information for Management Planning, Analysis, and Coordination database (IMPAC)
# of Projects Total dollars
Type 1, 2 Awards* 10,493 $2,361,434,220
Human Subjects* 3,665 $1,139,061,717
Clinical Research 2,795 (27%) $905,852,246 (38%)
Clinical Trials** 518 $313,435,513
**Subset of clinical research cited above
Selected Type Total Number Total Dollars (Millions) CR Number CR Dollars (Millions) %CR Grants %CR Dollars
R01 4,932 1,195.4 1,298 383.6 26.3 32.1
U01 111 102.2 99 98.9 89.2 96.7
N01 408 237.4 94 54.6 23.0 23.0
P01 165 163.1 76 80.0 46.1 49.1
SBIR/STTR 803 125.2 181 29.2 22.5 23.3
Centers 200 227.0 111 137.1 55.5 60.4
Other 3,874 311.1 936 122.4 24.2 39.3
TOTAL 10,493 2,361.4 2,795 905.8* 27.0 38.0
R01 - traditional, discrete investigator-initiated grant
*Represents the population that received the total of $905,852,246 (38%) and
2,795 awards (27%)
Degree Percent Total
M.D./M.D.-Ph.D. Ph.D. Other, e.g. D.D.S., D.D.M.,
D.Sc., etc.
Scientific Review (%)
Other Centers (%)
(A) Partnerships in Training.
(A) Study sections. The committee applauds the actions of
Dr. Ehrenfeld, the Director of the Center for Scientific Review, to address this issue. The findings of the Williams' committee(15 ) have been substantiated, that is, that clinical research proposals reviewed by study sections that review a lower percentage of clinical research grants (less than 30%) do not fare as well as non-clinical proposals. Dr. Ehrenfeld and her colleagues are taking active measures to evaluate this problem, to develop corrective measures, and to try experimental approaches. The progress and implementation of these approaches should be reported periodically to the Advisory Committee to the NIH Director.