<
 
 
 
 
×
>
hide
You are viewing a Web site, archived on 02:26:16 Nov 18, 2004. It is now a Federal record managed by the National Archives and Records Administration.
External links, forms, and search boxes may not function within this collection.
FDA Logo U.S. Food and Drug AdministrationOffice of Women's Health  U.S. Department of Health and Human Services

FDA Home Page | Search FDA Site | FDA A-Z Index | Contact FDA

rule

Response to Congressional Request on
FY 2002 Funding for Gender-based Research

Report Language -- Conference Action (107-76)

"The conference agreement provides an increase of $500,000 for the Office of Women's Health, instead of $700,000 as proposed by the House. The Senate language did not provide an increase. The conferees are concerned that the FDA has paid insufficient attention to gender-based research. The conferees direct that the agency develop an agency-wide database focused on women's health activities to include demographic data on clinical trials. The conferees require a report to the Committees by June 3, 2002, which should include an update on the current pilot program and a capability assessment of the agency's ability to review clinical trial databases, coordinate data collection, and identify areas in which gaps exist."

Development of an Agency-wide Demographic Information and Data Repository: Improving Information Management Capabilities

[This document is also available in PDF format (3 MB)]

 

TABLE OF CONTENTS

Executive Summary

I. Charge to Office of Women's Health

II. Response to COngressional Charge: Development of an Electronic Demographic Information and DataRepository (DIDR)

III. Current capabilities and future directions

IV. The Role of the Office of Women's Health

V. Looking to the Future: Conceptual Description of the Demographic Information and Data Repository System

VI. Ongoing Activities

Appendices

  1. Legislative Charge to the Office of Women's Health
  2. Center-Based Capability Assessment Overview
  3. Patient Profile Viewer Pilot Project Description
  4. Reviewer Templates Pilot Project Description
  5. Reviewer Template Warehouse Systems
  6. Clinical Study Data Warehouse Systems
  7. Labeling Data Warehouse Systems
  8. Initiatives to Improve Health and Health Care

Footnotes

References


Executive Summary

Information about the level of participation of women in clinical trials and the presence or absence of differences in sex/gender response to medical products is necessary to address questions of equity and access to clinical studies. Additionally, this information is required to ensure that potential differences in safety and efficacy of a product have been evaluated in submissions and reviews. However, solely tracking participation in clinical trials does not fully address concerns expressed by Congress about the Food and Drug Administration's (FDA's or "the agency's") lack of attention to women's health and women's health research. To better address women's and minority and sub-population health issues, the agency must develop processes to conduct more efficient and effective risk-benefit assessments and improve risk management practices. To conduct these assessments accurately and meaningfully, the FDA must change the way it collects, uses, and stores information. The use of electronically submitted data has facilitated the review process but the full range of benefits offered by "intelligent" computer systems has yet to be realized.

The current processes for managing information within the FDA were developed individually in response to various organizational needs, rather than within a comprehensive quality management framework. Although current systems are successful, they do not utilize information or technology to its fullest potential for the degree of investment. Mindful of its public health mission, FDA is currently engaged in a number of efforts directed at improving its business practices and strengthening its ability to manage information associated with the review and approval of regulated products. However, the development of a standardized, well-designed information management system is still needed.

This report responds to the Congressional mandate to develop an FDA-wide database, describes the FDA-wide Demographic Information and Data Repository (DIDR), and outlines key elements needed to provide the foundation for its development.

I. Charge to the Office of Women's Health

In the FY2002 Conference Committee report Conference Action and Public Law 107-76, Congress stated "...The conferees are concerned that the FDA has paid insufficient attention to gender-based research. The conferees direct that the FDA develop an agency-wide database focused on women's health activities to include demographic data on clinical trials. The conferees require a report to the Committees by June 3, 2002, which should include an update on the current pilot program and a capability assessment of the FDA's ability to review clinical trial databases, coordinate data collection, and identify areas in which gaps exist" (for more detailed Congressional appropriations language see Appendix I).

II. Response to Congressional Charge - Development of an Agency-wide Demographic Database: the Demographic Information and Data Repository (DIDR)

Tracking the participation of women in clinical trials and monitoring sex/gender analyses is necessary to address questions of equity or access to studies. Additionally, collecting and monitoring this information ensures that potential differences in response have been evaluated and reviewed. Other demographic variables such as race/ethnicity, age, geographic origin/location of the study are important variables to monitor and evaluate as part of this effort. However, solely tracking these elements does not fully address concerns expressed by Congress about the FDA's lack of attention to women's health research nor the health issues of other sub-populations as required by regulation.1 In a report on Women's Health,2 the General Accounting Office (GAO) recommended that the FDA develop management tools to ensure that the collection, presentation, and analyses of data related to sex differences are addressed and monitored.

Differences between men and women in the safety and efficacy of drugs and biologics have been of concern to the FDA since the early 1980's.i Scientific studies have brought greater understanding to the biological basis for sex-based differences, particularly differences in response to medical products.ii For example, in a recent review of the biological differences between men and women, differences in drug response were explained by sex-based differences in the number and distribution of kappa receptors, the site of the drug's action.3 Differences in drug response are not limited to sex-based differences but also encompass differences in other sub-groups of the population. Race/ethnicity, age, hepatic and renal impairment can alter responses to medical products. Differences in drug metabolism between racial groupsiii have been identified that can result in differences in product safety and efficacy.4,5,6 Age-related differences in drug safety and efficacy have also been described and differences in efficacy due to interactions between concomitant therapies are of concern.7,8 These examples are only a few of the differences in drug safety and efficacy between sub-groups that are described in the scientific literature. The FDA has been interested in improving characterizations of potential differences in safety and efficacy among subsets of the population to enhance the risk management of medical9 and food related safety issues.

Definitions for the Demographic Information and Data Repository

Repository-- the area holding "warehouse" systems, the automated document management system and table of contents or index.

"Warehouse System"-- the area contained within the repository dedicated to a topic area, such as clinical studies, labels, reviewer templates. This is the location from which data are transferred and processed. Each area contains the submission data, reviewer tools and interface programming to facilitate that component of the review.

Document Management System-- the system that controls the flow of information ensuring the secure and efficient access to and transfer of information.

Portal/Index-- the interface with users allowing rapid identification and accurate searching capabilities.

Scope of Report

This report establishes the scope of the information system, describes a conceptual framework for the agency-wide Demographic Information and Data Repository (DIDR) and lays out components of the DIDR that are currently under development. Also addressed are specific elements in the Congressional charge to the FDA Office of Women's Health (OWH), the role of OWH, a description of on-going activities toward the development of an integrated agency-wide DIDR and a description of the gaps limiting FDAs ability to achieve a fully operational system. Additional information is appended and referenced in this report to provide more detailed information to readers about selected topic areas.

Once developed, the DIDR would permit more efficiency and flexibility in tracking sub-populations participating in studies. It would also enhance the FDA's capabilities to use the information for assessing health issues by improving FDA's ability to process, review, archive and analyze information that is currently contained in submissions and reviews. Development of the DIDR would help ensure sub-population health issues, including women's health issues, are adequately addressed.

Improved Information Management

Improving risk management approaches, as proposed by a 1999 FDA Task Force on Risk Management10, can be achieved through better use of information submitted in applications. Specific recommendations of the Task Force included:

  1. integration of existing systems so analytical tools, data entry, and editing may be uniformly applied;
  2. making all information readily available to every reviewer; and
  3. development of new methodological tools for inference from available datasets.

These recommendations address the need for better management of information within the review environment. Other risk management improvements can be realized through improvements in the delivery of services, such as changes in the way labeling information is provided to citizens, healthcare providers and pharmacists. The development of the DIDR will leverage the informational and intellectual assets of the FDA to modernize the drug review process and achieve improvements in regulatory objectives. These improvements will enhance the effectiveness and efficiency of the organization, promote the delivery of services to citizens and improve public health.

The increasing need to use informatics in the transmission of health information has focused considerable attention upon the development of interoperableiv systems within health care. To obtain usable information, the development of standards is necessary to achieve seamless systems capable of exchanging information across different computer systems without additional transformation steps. FDA receives large volumes of information in applications for review and evaluation. This information is currently submitted in a form that does not support its easy transfer between industry, the public and government. This means that the FDA must perform additional steps in the conduct of reviews of submissions, making the review process less efficient and delaying the transmission of information to those who need it. To promote effective interoperability between systems, information must flow with minimal transformation at each exchange.

Much of the groundwork for the development of interoperable systems has been established within the FDA through the development of requirements and capabilities for electronic submissions, electronic signatures and advances in information technology. Additional steps are needed to integrate technical capabilities with regulatory processes to achieve a fully functional electronic review environment. At the most basic level, implementing these changes requires evaluation of process, integration of processes with technology, and the development of standards for structure, content, language, and code sets to construct fully interoperable systems for information exchange. The time for the conduct of a review has shortened considerably, while at the same time reviews have increased in complexity. The quantity and flow of regulatory and scientific information has also increased substantially. Improvements in the management of information and changes in processes are imperative to enhance the efficiency of review processes, improve the FDA's risk-benefit assessments, and provide the timely delivery of risk management information to citizens, industry and other government agencies.

Improving management tools to allow better use of information involves developing better data handling systems and the adoption of information management practices. An effective strategy would provide the FDA flexibility it needs in managing the large amounts of complex information submitted for review and analysis. The information would be held in data "warehouse systems". The software application systems would be organized into specific topic areas containing structured and unstructured data. This will allow the FDA to better manage the information and data it currently generates and receives. Advantages in the design of a data warehouse over that of a database are their larger capacity, flexibility (such as easy integration of additional data or new data elements) and design features enabling more robust data analysis. For complex information management software applications, data warehouses are more appropriate systems for processing and archiving large amounts of data, tools and documents.

To manage the vast quantities of information contained in submissions, critical technical design issues arise in the effective management of the data in computer systems. These design issues dictate the establishment of data element standards, structured formats for unstructured content, and the establishment of hierarchical relationships between data elements to establish the foundation for an "intelligent" system that can receive, process and query complex information. Standards that provide structure for data elements utilizing standard terminology and organization of information will support the development of the repository and software application systems.

Standards for data elements will harmonize when possible with international standards, where they exist, and will be established through public processes with the participation of industry, the healthcare community, and vendors. Standards development supporting the business processes is anticipated to result in substantial cost savings for industry and government and result in improvements in communicating health information to consumers and the healthcare industry. Built upon standards for structure, terminology and content, the DIDR represents the conceptual information storage and processing area from which the electronic review environment will be developed.

[Figure 1. Conceptual organization of information stored in a Demographic Information and Data Repository: enhancing the reviewer's electronic environment--GRAPHIC OMITTED--text for graphic]

III. Current Capabilities and Future Directions

"...The conferees require a report to the Committees by June 3, 2002, which should include an update on the current pilot program and a capability assessment of the FDA's ability to review clinical trial databases, coordinate data collection, and identify areas in which gaps exist."

FDA's Ability to Review Clinical Study Databases and Labeling

Making changes in the way data are submitted to the FDA will enhance the utility of study data and label information for operational and informational needs. Establishing standards will enable research for women's, minority, and other demographic sub-group health issues within the FDA. In addition, standardizing the format and structure of certain submissions to the FDA will provide a mechanism for more timely updates of publicly available information such as labeling changes and warnings and allow more efficient reporting of adverse events from the medical community.

Although the FDA does not require data to be submitted electronically, the majority of clinical trial study data are voluntarily submitted using electronic file formats recommended in FDA guidance. Although advances in computer technology provide adequate capabilities to ensure the secure housing, receipt and reporting of large amounts of data, limitations are imposed upon the review by the lack of standardization and uniform structure. For example, data for clinical studies are submitted to the FDA using inconsistent structure and terminology (see Appendix VI). Labeling information is currently submitted on paper or in an electronic representation of the paper document, such as a "pdf" format (see Appendix VII). Due to this lack of uniformity, these files must be archived as discrete entities.

Data archived as discrete entities cannot be efficiently searched or analyzed across studies or applications unless the files are re-structured and standardized. Restructuring involves writing programs to coordinate the variables of interest for each study or label element included in the assessment. For example, to obtain statistics on the level of participation of women in a group of clinical trials, data manipulations of each of the files must coordinate the demographic variables before statistical programs can be used. Additional extensive data manipulations are necessary to evaluate more complicated research questions directed at women's and minority health issues. Without established standards, a structured data warehouse cannot be developed to effectively manage information in a useful manner. Currently, methods for coordinating and collecting data and the ability to conduct searches or pose complex queries of study data or labeling do not exist.

FDA's Ability to Coordinate Data Collection

Data that share uniform standards for format, structure, and content offer many benefits that optimize the use of computer technology in the management of information. Apart from improvements in the utility and access to information, other benefits include enhanced interoperability between computer systems, more consistency in the use of terminology and vocabulary, and the ability to develop software "reviewer tools" to facilitate components of reviews and increase consistency and efficiency of the review.

As mentioned above, data are submitted to the FDA in the formats recommended in guidance. Current guidance recommends submission of clinical study data in SAS transport files and provides instructions outlining the organization of components of the submission but not the data accompanying the application. At this time there are no Center- or agency-wide databases that can continuously collect and monitor sub-population data. The data are currently not submitted in a way that would enable collection of the information into a database or warehouse system to facilitate the use of the information to address sub-population and other health issues.

The need to develop interoperable systems in healthcare has been driven by dependence upon the need for computer technology for communications and the constraints imposed by that technology. Interoperability between systems hinges upon the use of standards to define the file structure, content and terminology of the communications. Standards for the collection of coordinated electronic health information have recently been recommended by the Office of Management and Budget (OMB), Congress, the Department of Health and Human Services (HHS) and the National Committee for Vital and Health Statistics (NCVHS) (see Appendix VIII). HHS has established a framework for establishing, adopting, modifying, and maintaining electronic standards for medical terminology and code sets for health services transactions under regulations for the Health Information Portability and Accountability Act (HIPAA). The FDA is partnering with the OMB directed HHS Consolidated Health Informatics e-Government initiative in the development of a national government standard for vocabulary to be used in healthcare. The FDA has worked with Health Level Seven (HL7), an American National Standards Institute (ANSI) accredited organization, to establish a technical committee where consensus standards for the collection of clinical study data and labeling may be developed. The FDA is currently drafting regulations requiring the submission of standardized electronic clinical study data and has proposed regulations requiring the electronic submission of labeling information.

Developing standardized templates for reporting the content of labeling to the FDA will provide the information in a consistent format and structure so the information can be used more effectively within and outside the FDA. Standardization of reviews through use of "Reviewer Templates" will facilitate the collection of reviews into a database/warehouse system, allowing improvements in search and report capabilities for reviews. Upon adoption of standards and consistent review processes, industry and the FDA will realize many benefits from its enhanced information management capabilities. Industry will realize substantial savings in the collection and management of study data and in communications during the drug development process. The FDA will realize enhancements in the efficiency and effectiveness of its review and risk management programs, enhancing the safe use of regulated products for consumers.

Gaps

Currently, the FDA cannot easily track the level of participation or the presentation of analyses for women or other demographic variables, such as race, age or geographic location. The FDA does not have the capability to monitor sub-population clinical data for sex or other demographic differences without the expenditure of considerable time and resources. Even when surveys are conducted, they often result in the collection of very limited amounts of information. The DIDR will enable enhanced business processes, specifically the review environment, and provide management tools for operational quality assurance such as those recommended in the 2001 GAO Report on Women's Health. The DIDR will consist of electronic structured and standardized information and reviewer tools to enhance the review environment (Figure 1). This will support better access to data and information, improve risk-benefit assessments in the FDA and enhance risk management practices for drugs, biologics, foods and devices.

Evaluation of current processes and the careful design and planning of changes to the review environment are vital to the development of the DIDR. The system design will be developed with the participation of individuals working within the review environment and IT staff. Input from staff who are familiar with review processes will be important throughout all phases of development of the repository, including but not limited to: standards development, warehouse/database development, design of reviewer interfaces, piloting and implementation. This activity will require considerable investment at the reviewer level of the FDA. The effectiveness of the proposed changes will hinge upon this participation.

To effectively re-engineer business processes, reduce redundancies and improve the business-IT interface, the development of the various warehouse systems and integration between the Repository and the review environment will be complex, requiring coordination within and between Centers. Although difficulties may be minimized through adequate planning, piloting, and a well constructed implementation plan, the development of the Repository will rely upon the availability of resources to develop, coordinate and integrate its various components. FDA has many competing priorities that demand financial and human resources. These proposed priorities will need to be balanced with FDA's other priorities to move these initiatives forward. Development and integration would take several years and include the following elements: the development of standard file formats, file structure, content, organization, terminology and code sets, planning the architecture and hardware requirements, software development and testing, development of automated file management systems, and the integration of these elements into the FDA's review processes.

IV. The Role of the Office of Women's Health

In establishing the FDA OWH within the Office of the Commissioner in 1994, the mandate stated the Office "...monitors inclusion of women in clinical trials and the implementation of guidelines concerning the representation of women in clinical trials and the completion of gender analysis."

In working toward fulfilling its mandate, FDA OWH has observed the difficulties that arise in assessing differences in sub-population representation and response throughout the agency. Surveys sponsored by the FDA OWH and the Office of Special Health Issues indicate that, when sub-population data are tracked and evaluated in submissions and reviews, the FDA's ability to collect and report this information is extremely limited because it is time and resource intensive.11,12,13 Efforts are being made in each Center to address sub-population health differences. Until recently, these disparate efforts have not been coordinated throughout the agency to develop a uniform strategy for analysis and monitoring of sub-population differences.

Constraints forced by the need for interoperability between computer systems for the most effective use of technology in the review environment are imposing requirements for standardization. The OWH is in the unique position of having reviewed and evaluated practices across Centers within the FDA. In working toward the development of the DIDR, the OWH is collaborating across the agency on a solution that is consistently applied, is flexible to accommodate the different needs of the Centers and diversity of products, and is responsive to new and emerging sub-population and demographic issues. The OWH considers this approach to be the most effective long-term solution to monitoring gender, minority and other sub-population health issues for the agency.

V. Looking to the Future: Conceptual Organization of Information in the Demographic Information and Data Repository System

Overview

Technological advances have dramatically changed the FDA's ability to receive and store information, however, less has been accomplished in terms of effectively managing the information and data in ways that can fully utilize technology to enhance risk-benefit decisions to protect and improve the public health. To improve the FDA's information management capabilities, business practices must be more fully integrated with advances in computer technology. To achieve this goal, information needed in the review environment must be redefined so it can be processed, reviewed and archived. To more effectively and efficiently manage information, the data must be exchangable between computer systems, minimizing the need for additional data verification, manipulation or modification steps. In response to reports issued by the NCVHS, several government and private sector entities have initiated standards development to promote better communication between systems (see Appendix VIII). Conclusions of OMB e-Government initiatives indicate the lack of interoperability of computer systems between industry and government results in less efficient and effective delivery of services.

Scope

In developing the DIDR, the OWH is working toward the development of an agency-wide information management system with interfaces to enhance the integration of information technology with review processes. The agency-wide DIDR would house information, data, and software to support a fully electronic review environment. Elements of the review environment include submissions, review documents, and software tools developed to facilitate the review process.

Conceptual Framework

The DIDR would contain an index to locate the data and information held within the system and an automated document management system to facilitate access, secure flow and control of data and information within the review environment. The DIDR would contain data to serve operational quality assurance and informational needs, adopting information management strategies commonly utilized in business to improve data handling and use. It would process data organized into topic-oriented data warehouses,v holding standardized data and software tools to query, analyze and present information. Warehouse systems would contain the topic-oriented data and information, stored in a useful format to enable complex searches and analyses of the data, and reviewer tools to facilitate components of the review process (see Figure 1).

Components

Development of the DIDR would begin with elements that can provide the most pertinent information to women's, minorities and sub-group population health issues. Although all areas of the review environment contribute to evaluations of safety and efficacy and will be developed in the future, the DIDR warehouse systems targeted for early development include; Electronic Clinical Study Data Warehouse Systems (Appendix VI), Electronic Labeling Warehouse Systems (Appendix VII) and Reviewer Template Database Systems (Appendix V). To construct data warehouses the data must be defined on several levels. The data must have a consistent, structure and format and use standardized terminology and code sets. Standardization of structure, terminology and code sets, is necessary to develop a structured repository where information and data can be effectively and efficiently processed, reviewed and archived.

Standardized Electronic Clinical Study Data Warehouse System

To adopt consensus standards, as they are developed by HL7 (see Appendix VIII), FDA has drafted a proposal to amend the regulations governing the format in which clinical study data are submitted for new product applications. Benefits of standardized electronically submitted clinical study data include:

Standardized Electronic Labeling Warehouse System

The FDA has proposed to amend current regulations for product labeling. Under the proposed change, applicants would be required to submit the content of labeling in electronic form. Improving the consistency of formatting in these submissions will allow better integration of technology into the review environment. Effective use of current technology will provide the following advantages:

Reviewer Template Warehouse System

Under the Good Review Practices Initiatives the Center for Drug Evaluation and Research (CDER) is in the process of implementing standardized review templates for eight pre-marketing review disciplines (including: Biostatistics, Chemistry, Clinical, Clinical Pharmacology/Biopharmaceutics, Microbiology, Pharmacology/Toxicology Manufacturing Microbiology and Bioequivalence). These electronic "templates" organize components of the review into a uniform presentation for each discipline. This allows more consistent presentation of reviews and provides the opportunity to develop improved quality assurance capabilities. The standardization offered by Reviewer Templates will facilitate archiving the information from reviews into a structured warehouse system. Structured archives will facilitate efficient and accurate computer searches across applications by specific element or section of the template. For example, special populations sections for specific classes of products may be queried to improve the understanding of similarities and differences in drug response or effect in sub-populations (see Appendix V).

VI. On-going Activities

Several activities have been initiated toward developing a DIDR. These activities are represented here:

Center-Based Capability Assessment Overview

Table 1. Center-Based Capability Assessment: Summary of On-going Activities in the Development of the Demographic Information and Data Repository (DIDR)

Center/
Office
Clinical Study Data
Warehouse System1
Labeling
Warehouse System2
Reviewer Template
Warehouse System3
E Format Review
Tools
Database
Standards
E Format Review
Tools
Database
Standards
E Format Review
Tools
Database
Standards
CDER B C B B   A C B A
CBER B A B B   A A    
CFSAN A A A n/a n/a n/a A    
OC-WH agency-level standards development, coordination and support

1) Appendix VI; 2) Appendix VII; 3) Appendix V

Key:
A = planning; B = initiated; C = piloting; D = fully completed; n/a = not applicable; blank = no activity


APPENDIX I: Legislative Charge to the Office of Women's Health

House Report 107-116 and H.R. 2330

Office of Women's Health - The Committee is concerned that insufficient attention has been paid to gender-based research by the FDA. Since the Office of Women's Health was established in 1994, its budget and its functions have been stagnant in spite of greatly increased needs. Last year, GAO reported a serious disproportionate impact on women of drugs withdrawn from the market for safety reasons. To address this issue, the Committee directs that FDA develop an agency-wide database focused on women's health activities, and that FDA commence a capability assessment for each Center and the Office of the Commissioner to review currently available critical clinical trail databases, coordinate data collection and identify areas in which data gaps exist. The Committee directs FDA to provide an additional $700,000 to the Office of Women's Health for this effort, from within sums provided for all programs, and to provide the Committee with the capability assessment report and detailed plans for the database by January 31, 2002.

Senate Report 107-41 and S. 1191

Office of Women's Health - The Committee is concerned that the FDA has paid insufficient attention to gender-based research. Last year, GAO reported a serious disproportionate impact on women of drugs withdrawn from the market for safety reasons. To address this issue, the Committee directs FDA to continue piloting the drug application database system that collects demographic information for specific New Drug Applications (NDAs) in the Center for Drug Evaluation and Research (CDER). Additionally, the FDA should study the possibility of developing an agency-wide system by commencing a capability assessment for each Center and the Office of the Commissioner to review currently available critical clinical trial databases, coordinate data collection and identify areas in which data gaps exist. The Committee directs FDA to provide the Committee with the assessment report of the agency-wide system and the status of the pilot program within CDER by June 3, 2002.

Conference Action and Public Law 107-76

[Gender-based Research]...The conference agreement provides an increase of $500,000 for the Office of Women's Health, instead of $700,000 as proposed by the House. The Senate language did not provide an increase. The conferees are concerned that the FDA has paid insufficient attention to gender-based research. The conferees direct that the agency develop an agency-wide database focused on women's health activities to include demographic data on clinical trials. The conferees require a report to the Committees by June 3, 2002, which should include an update on the current pilot program and a capability assessment of the agency's ability to review clinical trial databases, coordinate data collection, and identify areas in which gaps exist.

APPENDIX II: Center-Based Capability Assessment Overview

Center-Based Capability Assessment Overview

Table 1. Center-Based Capability Assessment: Summary of On-going Activities in the Development of the Demographic Information and Data Repository (DIDR)

Center/
Office
Clinical Study Data
Warehouse System1
Labeling
Warehouse System2
Reviewer Template
Warehouse System3
E Format Review
Tools
Database
Standards
E Format Review
Tools
Database
Standards
E Format Review
Tools
Database
Standards
CDER B C B B   A C B A
CBER B A B B   A A    
CFSAN A A A n/a n/a n/a A    
OC-WH agency-level standards development, coordination and support

1) Appendix VI; 2) Appendix VII; 3) Appendix V

Key:
A = planning; B = initiated; C = piloting; D = fully completed; n/a = not applicable; blank = no activity

 

Center for Drug Evaluation and Research (CDER) Overview

Developing a Conceptual Framework for Knowledge Management
CDER protects the health of Americans by helping to assure that all prescription and over-the-counter drugs are safe and effective for their intended use. CDER evaluates all new drugs before they are marketed, and provides oversight for more than 10,000 drugs currently available to help ensure that they meet the standards for safety, efficacy, and manufacture contained in the Food, Drug and Cosmetic Act. In 2001, CDER approved 66 new drugs, 24 of which are new molecular entities, or active ingredients never before marketed in the United States.

CDER also plays a critical role in providing information to health professionals and consumers about appropriate and safe use of approved drugs. Recent drug approvals represent important advances for children, women, elderly persons, and patients with heart disease and cancer.

In addition to the review of new drug products, CDER priorities include:

Mindful of its public health mission, CDER is currently engaged in a number of broad-based efforts directed at improving its business practices and strengthening its ability to manage the information and knowledge associated with the review and approval of prescription drugs. The introduction of the Electronic Signatures regulation (21 CFR 11) in 1997, allows CDER to accept submission of electronic records in lieu of paper for review and archive. This regulatory innovation has provided the impetus to realistically plan for the introduction of a large variety of technology-based improvements in CDER's scientific and regulatory processes, including the development of infrastructure and tools to more effectively use:

The successful application of effective new technologies and the development of new information resources depends on efforts to:

Given the current regulatory information infrastructure, CDER cannot be as effective and efficient as it would like in utilizing the data that describes the results of clinical trials to answer important scientific and societal questions. For example, a report prepared by OWH (Patterson, 2001) illustrates the difficulties in constructing a comprehensive, accurate representation of the demographic characteristics of subjects included in reported clinical trials. There is no single source of information to answer relatively simple questions regarding the participation of women and minorities in clinical trials. Researchers must laboriously construct tables abstracted from incomplete, non-standard paper reports and reviews. Current versions of labels, the final product of the review and the primary source of information about the drug at the time of approval, are not readily available for systematic review.

The science also suffers. In cases when electronic data are available to answer important research questions, it is difficult for researchers to manage the data for systematic cross-study and cross-product analyses. The content, shape, structure and quality of sponsor-submitted clinical trials data are highly variable and, based on traditional review of NDAs, primarily product or study specific. General guidance describing submission media and format for submission of electronic data has been available since 1999 as described in the "Regulatory Submissions in Electronic Format; General Considerations and Regulatory Submissions in Electronic Format; New Drug Applications" guidance document. While guidance documents provide formatting recommendations, sponsors make a wide variety of individual decisions in organizing and formatting datasets for agency submission. Thus, these data cannot be used for automated, in-depth analyses nor are they useful in establishing a clinical trials data warehouse. With current regulatory and resource constraints, it is not possible for CDER to effectively use the data generated by industry or the knowledge reflected in the reviews. However, a number of steps have been undertaken by CDER to turn the vision of a more technology-based decision capability into reality.

Standards, Information Warehouses, Review Tools and Regulation
Standards hold the key to much of what can be accomplished to better utilize data and associated knowledge regarding pharmaceuticals. With detailed, uniform standards, it is possible to establish data/information warehouses for clinical trials containing demographic information. The data warehouses will serve as the foundation for review tools and cross-product analyses. With standardized labeling terms and structures, it is possible to build a useful repository of up-to-date labeling information for prescribers, consumers, and reviewers. By designing and applying structured report templates, CDER and the public can quickly access publicly releasable information presented in regulatory reviews

Center for Biologics Evaluation and Research (CBER) Overview

CBER regulates biological products. CBER is responsible for ensuring:

Biologics, in contrast to drugs that are chemically synthesized, are derived from living sources (such as humans, animals, and microorganisms). Biological products often represent the cutting-edge of biomedical research and, in time, may offer the most effective means to treat a variety of medical illnesses and conditions that presently have no available treatments. CBER's review of new biological products, and review of approved products for new indications, requires evaluation of scientific and clinical data submitted by sponsors to determine whether the product meets CBER's standards for safety, potency, and efficacy. After a thorough assessment of the data, CBER makes a decision based on the risk-benefit evaluation for the intended population and the product's intended use. This review requires knowledge of new developments and concepts of basic research in the relevant biological disciplines.

Currently, CBER receives about 1000 original or supplemental product applications (BLAs). These numbers are expected to increase significantly as numerous products currently under development are approved. Some new areas of research, such as gene therapy, show potential for development for therapeutic indications. CBER's review of biological products requires evaluating scientific and clinical data submitted by manufacturers to determine whether the product meets CBER's standards for safety, potency, and efficacy. CBER is committed to a product approval process that maximizes benefits and minimizes risks to patients. A wide variety of changing technical and scientific issues related to safety, potency, and efficacy of novel biological products requires knowledge of new developments and concepts of basic research in the relevant biological disciplines. Because of the rapid advancements in both conventional and new biotechnologies, the scope of research is diverse and dynamic. CBER reviewers must keep pace with rapidly advancing scientific disciplines, manage regulatory change and simultaneously apply their expertise in the evaluation of the submission.

The exchange of paper submissions for electronic submissions has been identified as a priority. Although some benefits have been realized from this change, many additional benefits could be realized from the integration of review processes and enhancements in managing the information needed in the review. Much of the foundation for making the transition to electronic submissions capabilities has been established within CBER. To fully realize the potential benefits of currently available technology and design more relevant work environments, review processes and technology must be fully integrated to reduce and eliminate inefficiencies. This integration requires an additional level of implementation targeted at designing tools for specific disciplines and development of systems to hold the information. Clinical and statistical reviews are considered a high priority for implementing an electronic structure. In the interest of enhancing communications, a secure electronic mail system was developed at CBER to facilitate faster exchange of information with sponsors. The system plans to accommodate receipt of amending submissions to INDs and BLAs. Electronic enhancement of human and animal toxicological reviews will help to characterize a class of biological products and their impact on the human body. This type of electronic system could be searched for correlations that exist between a class of biological products and its observed side effects. Immunological effects may similarly be classified from bioassay data if the results from several studies are reported in an electronically mergable format.

CBER is committed to developing standardized electronic labeling, electronic clinical study data, reviewer templates, and tools that will facilitate reviews. This initiative affords CBER the opportunity to move the review environment forward, concurrently making improvements in review management and risk-benefit evaluations.

Center for Food Safety and Applied Nutrition (CFSAN) Overview

CFSAN is committed to the development of electronic capabilities that promote efficient reviews throughout the Center. CFSAN is currently planning, developing, evaluating, and updating its IT projects that affect its participation in FDA programs for implementing electronic government into the review environment. Currently, CFSAN plans to: 1) investigate the possibility of developing regulations to require regulatory submissions to be in standardized electronic file format for selected Program Offices; 2) leverage current electronic capabilities; 3) develop review templates; and 4) leverage resources to accomplish mission-related projects.

Although short-term costs associated with the implementation of e-government improvements are expected, CFSAN anticipates an overall decrease in costs due to increases in efficiency. Upon implementation of standardized electronic capabilities, additional benefits will be better decision-making and improvements in communication between government and citizens, industry and health care providers.

When study data are standardized, tools will be developed to better integrate technology into the review process, improving the efficiency, accuracy, and consistency of reviews. Additionally, reviewers would be able to use the data to build a knowledge base for better risk-benefit decisions when data may be limited. Electronic capabilities added to the review process may be able to identify trends for different demographic, safety, or medical characteristics for a particular effect of an ingredient or class of ingredients and improve assessments of safety and therefore have a positive impact on the public health and welfare.

Center for Devices and Radiological Health (CDRH) Overview

The quantity of information submitted to The Center for Devices and Radiological Health (CDRH) in each submission is much less than that received by other Centers. Few advantages are likely to be realized from standardization of data as diverse as that submitted to CDRH. When a public health issues arises, CDRH will continue to conduct reviews of the available and relevant data by extracting and evaluating the submitted information for the product or product class of concern.

APPENDIX III: Patient Profile Viewer Pilot Project Description14

Guidances published by CDER offer recommendations on how to organize datasets and how to provide descriptive information on the datasets and the data variables (metadata) for clinical study data. For NDAs, submission of Case Report Tabulations (CRTs) for each patient is recommended.

CRTs are files containing clinical study data from earliest pharmacological studies through studies collecting the final safety data. The CRTs contain the data, including demographics, on each patient in the study, including those who did not complete the study and the reasons for withdrawal. The CRTs are analyzed to verify the results of the analysis conducted by the sponsor. The verified results of the analyses are used to support evaluations of the risks and benefits and to support the decision to approve a product for marketing. FDA has worked with CDISC on the development of recommendations for a standard presentation of the most common CRT datasets and metadata.

In January 2002, CDER enlisted eight sponsors to volunteer to participate in a pilot project involving the testing of the Patient Profile Viewer (PPV). The PPV is computer software that allows a reviewer to display data collected from the electronic CRTs. The purpose of the pilot project is to test the PPV module using standardized datasets and metadata and to obtain feedback from reviewers and pharmaceutical companies on the creation and use of standardized clinical data and metadata. The pilot project is part of an effort to improve the standards for submission of clinical data. We expect to recommend detailed clinical data and metadata standards for the submission of CRTs. During the pilot project, the FDA will make technical instructions available to the public to provide clinical data and metadata for testing. Participants in the pilot project will be asked to provide clinical trial datasets and metadata as described in the technical instructions and to provide technical feedback. The pilot project will test the PPV module and the preparation and use of the submitted data and metadata.

The time frame for the development and testing of the software is planned to take two years.

APPENDIX IV: Reviewer Templates

Description

Within CDER there are 15 clinical divisions, 6 review disciplines and multiple drug classes to review. When a new drug product is approved or an already approved product becomes available for a new use, the reviews conducted by CDER staff are posted publically on CDER's website. Due to the large and varied audience that references CDER's regulatory review documents, including consumers, pharmaceutical industry, clinicians, and health maintenance organizations, the Center embarked upon the development of standardized templates for all discipline reviews of NDAs.

Activity

Beginning March 30, 2001, all primary and clinical reviews of NDAs and efficacy supplements were prepared on CDER's Clinical Review Template as part of its Good Review Practice plan. The use of templates is CDER's effort to provide a broad audience with a consistent and uniform presentation of the reviewed data. The eight pre-marketing review disciplines for which standardized review templates have been developed include: Biostatistics, Chemistry, Clinical, Clinical Pharmacology/Biopharmaceutics, Microbiology, Pharmacology/Toxicology, Product Quality Microbiology, and Bioequivalence. CBER and CFSAN also plan to implement Reviewer Templates for selected review disciplines. These electronic "templates" organize the most important elements of a review into a uniform presentation of the reviewed data. Each primary review is expected to contain a brief, high-level, discipline-specific executive summary that seeks to capture the important scientific and regulatory issues related to the drug being reviewed.

Plan

This standardized approach allows for quality assurance functions that support future improvement and planning. The templates were designed by reviewers and piloted within the appropriate division. Since their initiation, the Templates have undergone several revisions to improve their use and functionality. Enhancements, apart from providing a standard organization, have included the addition of on-screen prompts and hyper-links to web-based guidances that provide instructions for the conduct of the review. Additional enhancements are planned to facilitate searches of the information recorded in the Templates. The implementation date for the final Clinical Reviewer Templates is October 1, 2002. The FDA plans to routinely access and update the Templates for usefulness and functionality. The standardization offered by Reviewer Templates will facilitate archiving the information from reviews into a structured system. Structured archives will facilitate efficient and accurate computer searches across applications by specific element or section of the template. For example, special populations sections for specific classes of products may be queried to improve the understanding of similarities and differences in drug disposition or effect in sub-populations.

APPENDIX V: Reviewer Template Warehouse Systems

Center for Biologics Evaluation and Research Reviewer Templates

Review Consistency and Access

Current Capabilities: Within CBER there are 22 review divisions and seven major review disciplines. Each reviewer conducts a review according to the guidelines for their specific discipline. Reviews within each discipline are individually prepared without uniform organization or format and, therefore, are of limited use in the construction of an automated data system. Redacted portions of reviews are provided to the public in the "Summary Basis for Approval." These summaries are available by written request and are posted individually on CBER's website.

Limitations: Although reviews are stored in single electronic files, access to specific elements of a review and the ability to extract information across reviews is not easily achieved. The review documents lack a common structure and organization, and quantitative and qualitative summary evaluations within a review are not consistent in their presentation and format. Without consistent formatting and organization, development of electronic tools to extract information and report on specific elements within or across reviews cannot be developed. Valuable information contained in product reviews is not readily available for postmarketing surveillance or other potential applications.

Gaps and Benefits of Proposed Innovations: A common structure and organization for review documents would allow the development of a repository capable of conducting complex and accurate searches of information and provide better access to specific elements from a single review or across reviews. This ability would enhance the utility of the information for its current uses and allow uses to expand into new areas. For example, to address a question about differences in sub-population response for a compound or class of compounds, the information in the data repository could be used to generate an up-to-date report by extracting pertinent information from reviews. This information would be helpful to verify adverse event signals that may appear during post-marketing. The inclusion of summary qualitative and quantitative assessments will increase the efficiency of searches for specific items of interest. These assessments may include a worksheet regarding the presentation of an analysis by gender, race/ethnicity and age included in the application. Additional electronic capabilities can be developed to prepare summary documents from reviews using tools designed to capture specific elements from structured review templates, further enhancing the efficiencies realized by these changes.

Summary

Review Templates designed to accommodate each review discipline would ensure consistency between reviewers and documents. Consistent organization and structure in a review document will enable the development of review tools such as on-screen prompts for specialized instructions or provide considerations for reviewers in relevant sections of the review that can improve the reviewer's access to guidance and enhance the quality of reviews. Also, standardized and formatted reviews organized into "Warehouse Systems" would allow reports to be extracted from specific review sections in response to queries or searches.

Proposed Implementation Steps

CBER staff are reviewing Center disciplines to determine the number of Review Templates to be developed. The content of the review for the relevant discipline must be evaluated to define and organize review components into a meaningful Template structure. The integration of reference materials, qualitative and quantitative assessments,vi and section tags must be electronically linked to the Template. After a pilot period, the Templates would become integrated into the review process. These Templates would be stored in a warehouse where specially tailored tools to query the data can be created.

Center for Drug Evaluation and Research Reviewer Templates

Review Consistency and Access

Current Capabilities: Within CDER there are 15 clinical divisions and six review disciplines. When a new drug product is approved or when an already approved product offers a new indication, the reviews conducted by CDER staff are redacted and posted on CDER's website. Due to the large and varied audience that references CDER's regulatory review documents, including consumers, pharmaceutical industry, clinicians, and health maintenance organizations, the Center embarked upon the development of standardized templates for all discipline reviews of NDAs. The use of templates is CDER's effort to provide a broad audience with a consistent and uniform presentation of the reviewed data. As of March 2001, these standardized review templates have been implemented in each discipline to define the most important areas of concern when reviewing scientific data for any new drug application. Each review is expected to contain a brief, high-level, discipline-specific Executive Summary that seeks to capture the important scientific and regulatory issues related to the drug. This provides for a clearer understanding of the drug approval process and serves to improve public health, the practice of medicine, and patient drug use.

Limitations: Reviewer Templates have been developed for the six disciplines (two additional templates are being initiated) within CDER. Templates with a common structure and organization and having integrated reviewer reference materials are currently being piloted and implemented. Summary evaluations, both quantitative and qualitative, of demographic variables are not currently included in a structured format in Reviewer Templates. Inclusion of these elements would enhance the ability to make administrative assessments of compliance with regulations and guidances that are currently resource intensive and, therefore, not conducted on a regular basis. Document warehousing and search capabilities have not been developed for Reviewer Templates.

Gaps and Benefits of Proposed Innovations: Reviewer Templates are developed and in the process of being implemented. The lack of summary assessments of demographic variables in the Templates impairs the efficiency of search functions. The development of search capabilities for Review Templates will enhance access and use of the information.

Summary

Reviewer Templates are well developed for six review disciplines. The pilot phase has been successful thus far and CDER anticipates an easy transition to fully templated reviews for all disciplines. Electronic bookmarking of sections within the Review Template and the addition of search and reporting capabilities will allow better use of the information.

Proposed Implementation Steps

CDER's reviewer templates are well developed and the integration of qualitative and quantitative assessmentsvii and electronic bookmarking are planned. Review tools, such as query and reporting capabilities, are in very early stages of planning.

Center for Food Safety and Applied Nutrition Reviewer Templates

Currently, CFSAN has not developed standards or formats for structured reviews of submissions. CFSAN is in the process of developing reviewer templates for pre-clinical (animal) toxicology studies and several other disciplines. In the future, CFSAN plans to participate in FDA working groups to develop submission and reviewer templates for clinical and other studies.

APPENDIX VI: Clinical Study Data Warehouse Systems

CBER/CDER - Electronic Standardized Clinical Data

Media

Current capabilities: FDA regulations describe the content and format requirements for submissions, including the requirements for submission of clinical study information. Currently, almost all study data dealing with effectiveness of the product and some safety data received by CBER and CDER are submitted electronically and are consistent with the format recommended in guidance issued by CDER/CBER15. However, it is not mandatory to submit data electronically and, as a result, a proportion of the data are received on paper or in non-standard electronic file formats. Clinical study data are complex, containing data from each adequate well controlled study, from data on the earliest clinical pharmacological studies to the large scale clinical trial studies. The tabulations are required to include data on each patient in the study, including those who did not complete the study and the reasons for withdrawal. Although much of the data are submitted consistent with guidance there is considerable diversity in file content and terminology. For example, each dataset may use different terminology to characterize the same variable. These data are analyzed by reviewers to verify the results of the analysis conducted by the applicant. The verified results of the analyses are used to support evaluations of the risks and benefits and support the decision to approve, or not approve, a product for marketing. When data are submitted on paper, the reviewer must enter the data manually into a computer system and, prior to the analyses, conduct manual checks for accuracy. When data are submitted using a non-standardized structure, reviewers must manage the data, taking additional programming steps to unify datasets for this process.

Limitations: The submission of electronic study data is voluntary. Submissions on paper or non-standard dataset structures does not allow development of a single archival system to hold study data in a manner that permits the uniform processing, storage or archiving of study data. When an electronic submission is received, the files are stored on a network server that allows read-only access to the reviewer. Paper submissions must be transcribed by hand into an electronic format, a task that is time-consuming and prone to human error. When the review is complete, the data are held in a media-specific document storage area and are not readily accessible for cross-reference purposes. While this information may be helpful in reviews and consultations, additional planning and programming steps are necessary to obtain the needed reference data. The lack of a requirement for electronic submission of clinical study data means that storage of the data must accommodate paper and electronic media and require maintenance of two separate file systems.

Gaps and Benefits of Proposed Innovations: The current form of data management is inefficient and does not maximize the use of the clinical information. The FDA is aware of the inefficiencies and limitations involved in submission of study data in a variety of media, using variable dataset structures and terminology, and has worked toward the development of a regulation requiring the electronic submission of study data in a standardized format. Electronic submission of study data in a consistent format is the initial step in the development of uniform systems for the processing and archiving of study data.

Integration of Technology and Process

Current Capabilities: Each year, FDA receives hundreds of original marketing applications and supplements. Upon receipt of a submission the reviewer will verify that the content of the files is complete and accurate. Verification of the content of the application is a manual process and is time consuming and must occur prior to beginning the review process.

The applicant's statistical analyses are conducted in support of safety and efficacy. Verifying the results provided by the applicant, when programs are not provided by the applicant, is primarily a manual, time consuming process. Additional time is required to identify errors that may arise. Electronic submission recommendations have utilized many of the business processes that were developed for paper-based systems with the addition of some computer-based enhancements. Enhancements such as computer analyses of the data have improved the reviewer's capability and efficiency but provide limited support for other processes in the review. For example, checking data content, verification of results, the conduct of standard analyses for sub-group differences, among others, may be facilitated through the development of electronic review tools.

Limitations: At this time, the completeness of submitted study data must be evaluated by an individual reviewer before a review can be conducted. This verification step is time consuming, requiring re-checks of the data. The current data storage system severely limits the FDA from utilizing computer technology to assist in the assessment of the completeness of clinical study data to its fullest potential. When data content is not standardized, review tools developed for one system cannot be reused in another. This lack of systems integration limits the efficiency of the review and access to the information. Until study data are standardized, tools can not be developed that would better integrate technology into the review process to improve the efficiency and consistency of reviews. Once the review is complete, paper documents are forwarded to a document holding room and electronic files remain in an electronic storage area. These information storage areas are not easily accessible and do not provide a "lessons learned" framework. Reviewers in other disciplines or in other review divisions do not have the advantage of broadening their knowledge base when unique situations are successfully resolved. In addition, data cannot be easily processed, archived or reviewed to identify pertinent public health issues, such as clinically relevant differences in response among certain populations.

Gaps and Benefits of Proposed Innovations: The implementation of electronic tools into the processing of applications and conduct of reviews has been impeded by a lack of defined standards for file structure and content. A standardized format for submitting clinical study data is essential in effectively managing clinical study information. When study data become standardized, tools can be developed that would better integrate technology into the review process, improving the efficiency, accuracy, and consistency of reviews.

Structure/Terminology, Storage and Access

Current Capabilities: Clinical study data are complex and are currently reported using non-standardized structure, terminology and code sets. Study data are stored in files that are available upon request, requiring additional planning steps and time to obtain the data files. Additionally, once available, data files may require additional processing and conversion to format the information into a usable framework.

Limitations: Clinical studies are costly to industry and the current system of information management does not lend itself to the efficient manipulation of clinical study data even when the data are submitted in the recommended file format. Valuable information contained in clinical study data, such as observed effects of a class of drugs in a particular population, are maintained within the FDA but are inaccessible across reviews. The lack of the ability to readily access and fully utilize study data impedes the development of other knowledge-based uses of this information that may have a significant impact on public health.

Gaps and Benefits of Proposed Innovations: Standardization of file structure, terminology and code sets would help to create one variable name dictionary that can be utilized by the industry as well as the FDA. Standardization of healthcare vocabulary has been recognized as a gap in e-government in the OMB Federal Interagency Consolidated Health Informatics initiative (a Government to Business initiative). In HHS, standards for a health care vocabulary are under development for health information exchanges within government and between government and industry to improve the interoperability of systems. Standardization of data warehouses would permit the development of tools to automate the reviewer's analysis. Additionally, reviewers can use the data to build a knowledge-base for better risk-benefit decisions when data may be limited. Standardization will allow the development of interfaces tailored to the needs of a particular discipline or application. The data can be used to provide information for better risk-benefit decisions.

Summary

Development of standards for data submission will provide a basis for the development of an architecture for warehouse data elements. Enhanced data management will enable the development of secure systems, allow interoperability between systems and efficient process controls to ensure integrity of the system. Standardized data will facilitate the data analysis process and will lead to enhanced uses of the information for regulatory and public health purposes. The FDA can then provide accurate, detailed assessments of the demographics of participants in clinical trials, by product, compound class, indication and disease. The information will provide information needed for risk-benefit evaluations and, therefore, have a positive impact on the public health and welfare.

CFSAN -- Electronic Standardized Clinical Data

Media

Current capabilities: Regulatory submissions to CFSAN may contain study data, such as: human clinical studies, pre-clinical animal studies, various toxicological studies, nutritional studies, pharmacological studies, and microbiological studies. In particular, the Office of Food Additive Safety (OFAS) routinely receives study data. For example, human clinical studies often are involved with food and color additive petitions. Additionally, clinical and other studies may be involved with food contact notifications (FCN) and Biotechnology Consultations (BNFs) and Generally Recognized as Safe Notifications (GRNs). In some circumstances, the Office of Nutritional Products, Labeling and Dietary Supplements (ONPLDS) receives study data for review. Study data submitted to CFSAN are complex, containing case report data and safety data from clinical studies. These data may be submitted electronically or on paper. The petitioners are required to include data on each patient in the study, including those who did not complete the study because of an adverse event. Upon receipt of the submission, the study data are analyzed to verify the results of the petitioner's statistical or other analysis. The results of the verified analyses are used in the reviewer's evaluations of the risks and benefits associated with use of the product. The decision to approve a food or color additive, or food contact substance is based, in part, upon the data obtained from clinical studies and other studies conducted for assessments of safety.

Certain divisions within CFSAN do not receive electronic submissions but receive paper filings for regulatory submissions for Structure/Function claims, Notifications for New Dietary Ingredients, temporary marketing permits, and new infant formula products. CFSAN's Office of Food Additive Safety requires the submission of study data to support the safety review of food ingredients. Currently, it does not require submission of clinical data for all reviews. Although OFAS accepts data in electronic format, it does not require that study data be submitted electronically in any particular format, file structure, or using standardized terminology. However, it does provide guidance on how to submit data in order to allow for efficient processing and review of the data.

Limitations: Elective submission of study data on paper or in non-standard electronic formats means that a single file system will not hold study data in a manner that permits its uniform processing, storage or archiving. The lack of a requirement for electronic submission of study data means that paper files must be converted to electronic format before they are stored and reviewed, a task that is time-consuming and results in archives of files that contain information that can not be efficiently or effectively analyzed. Converting paper documents to an electronic format that can be easily searched and analyzed is costly and time consuming.

Gaps and Benefits of Proposed Innovations: Efficiencies in review processes from the application of state of the art advances in CFSAN's information technology (IT) capabilities can not be realized until study data or other review information are submitted using standard file structure, terminology and code sets. The FDA is aware of inefficiencies and limitations involved in submission of clinical study data in a variety of media using several file formats and has worked toward the development of regulations requiring (and guidance recommending) the electronic submission of study data in a standardized file format. Electronic submission of study data in a consistent file format is the initial step in the development of uniform systems for the processing, reviewing and archiving of study data.

Integration of Technology and Process

Current Capabilities: Each year, CFSAN receives hundreds of regulatory submissions, including Food Additive Petitions, Color Additive Petitions, Food Contact Substance Notifications, Generally Recognized as Safe Notifications, and Infant Formula Petitions, as well as hundreds of amendments and supplements to pending petitions and previously approved food products or ingredients. When study data are received in regulatory submissions, the contents of the files are checked to be certain they are complete and accurate. During the review, the study data are analyzed to verify the results of the statistical or other analysis conducted by the petitioner. Each dataset that is submitted has a unique file structure and terminology. This means that when checking for completeness and verifying the analysis, these checks are conducted for each submission and sometimes individually for studies within a submission. This process is time consuming because it relies on manual procedures that may be subject to human error. Additional time is required to identify the source of errors that may arise.

Limitations: See CBER/CDER Limitations above.

Gaps and Benefits of Proposed Innovations: See CBER/CDER Gaps above.

Structure/Terminology, Storage and Access

Current Capabilities: CFSAN study data are complex and are currently reported using non-standardized structure and terminology.

Limitations: Clinical studies are costly to industry and the current system of information management does not lend itself to the most efficient manipulation of clinical study data even when the data are submitted in the current recommended file format. Valuable information contained in clinical study data, such as observed adverse effects of a class of ingredients in a particular population, are maintained but are inaccessible to outside reviewers. The lack of the ability to readily access and fully utilize study data impedes the development of other knowledge-based uses of this information that may have a significant impact on public health.

Gaps and Benefits of Proposed Innovations: Standardization of file structure, terminology and code sets would help to create a set of standards for file content that can be utilized by the industry as well as the FDA. Standardization of electronic information management systems would permit the development of programs that would automate the reviewer's analysis in the verification of the petitioner's findings. Additionally, reviewers would be able to use the data to build a knowledge base for better risk-benefit decisions when data may be limited. Standardization would allow reviewers to easily combine the same data into a single system that could be provided to different reviewers using different interfaces customized to the needs of the particular discipline. Also, the data may be merged from several different submissions and thus enable integrated analyses where appropriate to enhance the use of information in better risk-benefit decisions. These integrated analyses may be able to identify trends for different demographic, safety or medical characteristics for a particular effect for an ingredient or a class of ingredients and improve assessments of safety.

Summary

Development of these standards for data submission will provide the basis from which a data repository may be developed. Enhancing data information management would enable the development of security and process controls to ensure the integrity of the system. It would make the data analysis process more efficient and would lend itself to a multitude of manipulations that could easily include summarization by demographic or medical characteristics that, in due time, would improve decision-making and therefore have a positive impact on the public health and welfare.

Proposed Implementation Steps

APPENDIX VII: Labeling Warehouse System

CBER/CDER Electronic Labeling

Media

Current Capabilities: The term "labeling" used in this discussion is the comprehensive prescribing information described in 21 CFR 201.56 and 201.57. It is sometimes referred to as the package insert. The regulations outlining the requirements for labeling list specific topics to be included in labeling. Currently, these elements may be submitted in paper or electronic formats.

Limitations: Submission of required labeling and changes in labeling in electronic format is currently voluntary. At this time, most labeling changes for biological products are being submitted electronically and less than half of the labeling changes for drug products are being submitted electronically. There is no mandate requiring all labeling to be submitted electronically. The lack of a requirement for the electronic submission of labeling means that the storage of these files must accommodate paper and electronic media and requires the maintenance of two separate file systems for labeling. After a review, paper files are returned to storage and are not readily accessible for reference purposes. At times, the labeling may only be available by making specific requests in advance to document holding rooms. This results in the need for additional planning steps that may result in delays in obtaining reference labeling needed for a review or consultation. Since the labeling can be provided in either paper or electronic format, searching the files for electronic labeling is inefficient as it does not provide a complete assessment of labeling changes.

Gaps and Benefits of Proposed Innovations: The agency, aware of the inefficiencies and limitations involved in the processing, archiving, review and access to paper labeling, has drafted a regulation proposing to require the electronic submission of labeling16. When labeling is submitted electronically the efficiency of the review process can be enhanced and access to the information can be improved, especially if additional steps are taken to standardize submissions.

Integration of Technology with Process

Current Capabilities: Each year, CBER and CDER receive more than 3,500 proposed labeling changes for approved products. As part of the review process, reviewers must conduct detailed, manual word-for-word comparisons of the proposed labeling with the last approved version of the labeling to verify that all labeling changes have been identified. In the case of generic products, similar word-for-word comparisons to the referenced listed, or innovator, drug labeling are conducted to verify that any differences have been correctly annotated and explained. A reviewer performs these assessments manually by comparing paper or electronic copies of each version of the labeling.

Limitations: Manual label reviews are resource intensive and have the potential to miss changes due to human error. Manual reviews do not employ currently available innovations in technology that can improve the efficiency and accuracy of the review process. Current electronic file formats are free text files and do not share a standard structure or terminology for the content of labeling. A change to even a single word in the labeling requires the submission of the entire labeling. This requires review of the entire label when conducting computer based comparisons between different versions of labeling. Even when labeling is submitted electronically, the current technology used in making electronic comparisons identifies some "false positive" changes which, while better than paper, is not the most efficient or accurate system. Identification of "false positive" changes in labeling requires reviewers to conduct additional, manual verification steps during the label review process.

Gaps and Benefits of Proposed Innovations: Standardization of the label structure, content and terminology would enable reviewers to limit reviews to specific sections of labeling where changes occur and would improve the accuracy of electronic comparisons of text. This could also facilitate the submission of labeling updates by pharmaceutical companies. Changes could be limited to specific sections of the label reducing the amount of information requiring review. Another advantage of standardization is realized when there are changes to the layout or presentation of the label content. A standardized file structure would permit more efficient use of computer technology to compare very different versions of labeling on a section by section basis. This would reduce the potential for "false positive" results, reducing the need for manual comparison by reviewers.

Structure/Terminology, Storage and Access

Current capabilities: The electronic files are free text files and do not share a standard structure, content or terminology in labeling. Electronic label files lacking this degree of standardization do not permit the implementation of innovations in technology that can improve the efficiency and accuracy of the review process and enhance storage and access to the information. To obtain summary information about the labeling for several labels or products, a manual review and extraction of the information must be manually conducted from filed labels. This is resource intensive and is rarely undertaken.

Limitations: Currently, guidance recommends an electronic file format for submission of labeling information that allows the agency to process, review and archive each individual label submission. Paper labeling is archived upon receipt and is generally available only during the review, after which it is archived again for future reference. Electronic labeling is archived with an application. At this time, labels are not required to be submitted using any specific file structure, standardized terminology or coding identifiers.

Gaps and Benefits of Proposed Innovations: There have been several identified gaps in the current system. First is the absence of a standardized format for labeling submissions that would allow extraction of specific sections of the labeling or the development of structured labeling information that would allow efficient searching and sorting of labeling and its content. Standardization of the format and structure would allow more efficient updates of labeling for industry and more efficient reviews of labels and label content. However, for optimal use of structured labeling information, standardized terminology and definitions would be required to improve the searching capabilities of the labeling and its content. The terminology that is currently in use is incomplete and inadequately defined and this lack of standard use of terminology has impeded the development of systems and processes that would enable the efficient archiving, processing and reviewing of labeling information. A proposed change to the labeling of drugs and biologics to include highlights addresses this functionality. The development of specifications to assist sponsors in submitting their labeling information in a standardized electronic format and utilizing standardized terminology would require the development of electronic systems for the submission and processing of labeling contents. Making labeling information available to the public in an electronic form that is updated as soon as changes are approved can improve the safety of prescription drug use. This can also improve the risk management of prescription drugs. Digital label information could be made available to the public at no-cost and tools may be developed that can assist health care providers in managing risks associated with drug use.17

Summary

Development of standards for labeling will provide the basis from which a labeling warehouse system may be developed. Enhancing label information management would enable the development of security and process controls to ensure the integrity of the system. Meeting these objectives will enhance patient safety through making medication information accessible to patients and their physicians and allow them to make better, more informed risk-benefit decisions.

Proposed Implementation Steps

Proposed Implementation Steps for creating an electronic labeling repository has been established. The plan consists of six levels, or "projects," as described below:

Project One - Medication Information Database (MedID) Medication
Reference Terminology Assessment
Project one will perform a medication reference terminology assessment for the medication information database. Terminology and coding requirements will be defined, existing systems will be identified and assessed, and a detailed report and plan with recommendations and options for implementation will be developed. *A detailed explanation of MedID content in the powerpoint presentation. The Comprehensive prescribing information will be defined into text fields.

Project one steps include:

Project Two - Computerized Labeling Process for NDA and BLA, Part 1
Project two will focus on the requirements and design of the Computerized Labeling Process for use within FDA for processing labels submitted via the new drug application/bilologics license application (NDA/BLA) process and dissemination of labeling information. The CLP will include functionality to search, query and compare label data.

Project Three - Computerized Labeling Process for NDA/BLA, Part 2
Project Three covers development and implementation, testing, and transition of the design defined in project two. It is a follow-on project and will begin upon the completion of Project Two.

Project Four - Computerized Labeling Process for ANDA
Project Four will enhance the CLP for use within FDA for processing labels submitted via the abbreviated new drug application (ANDA) process.

Project Five - Computerized Labeling Process for DRLS
Project Five will enhance the CLP for use within FDA for processing labels submitted to the Drug Listing and Registration System.

*The MedID Database

The contents of the Medication Information Database (MedID) would be comprehensive. MedID would contain drug coding identifiers unique to every drug. The drug coding identifiers would consist of an ingredient code, a clinical drug code, an imprint code, a product code, and a National Drug Code. The ingredient code is assigned to the product when an Investigational New Drug (IND) application is submitted for review. This will allow continued, consistent tracking of the product from earliest trials and will enable the agency to continue gathering data after the product has been marketed. MedID would also contain approved indication information in a standardized format. For example, the approved indication would consist of the name of the condition, the drug class, what the drug should treat (i.e., disease, condition, sign, symptom), the intent of the drug (i.e., treatment, prevention, mitigation, cure, diagnosis), the type of therapy (i.e., primary, adjunctive), modifications and effects in special population and any exclusions for use, dosing and monitoring. The dosing portion of the indication would similarly be in a standardized format consisting of initial dose, usual daily dose, a do not exceed dose, and dosage adjustments for special populations such as females. The active ingredient information would encompass the active ingredient name, the ingredient identification, the ingredient identification of the active moiety, the activity (i.e., serotonin re-uptake inhibitor, CYP3A4 inhibitor), and the clinical effects of the drug. This information would be stored electronically in format that can be used within the FDA and made available to the public (individual consumers, health care providers and software developers) to promote the safe use of drugs and biologics.

CFSAN -- Electronic Labeling

Currently, CFSAN is not accepting regulatory submissions having to do with labeling in electronic format. At present, there is no existing database of labels that previously were approved. This is because CFSAN does not require premarket approval of labeling in the context of foods or nutritional supplements.

APPENDIX VIII: Initiatives to Improve Health and Health Care

These groups represent a collection of initiatives with a single goal: to improve communication through improvements in the interoperability of health and health care systems. These groups have recognized current limitations in the application of technology to communication and have recommended the development of standardized terminology for health communication and health care transactions.

National Committee for Vital and Health Statistics (NCVHS)

The NCVHS serves as the public advisory body for the Secretary of Health and Human Services on national health information policy. The Committee envisions a National Health Information Infrastructure (NHII) comprised of the set of technologies, standards, applications, systems, values and laws that support all facets of individual health, health care, and public health. The goal of the NHII is to deliver information to individual, consumers, patients and professionals, where and when they need it so they can use this information to make informed decisions about health and health care. The NHII is not a centralized database storing information about individuals. Instead, the NHII offers a way to exchange health data in a secure environment to promote better access to healthcare information and better delivery of healthcare services. One area of healthcare that would realize substantial benefit from this information is in the reduction of preventable medical errors. These errors account for 98,000 deaths each year and are the fifth leading cause of death18. The Committee noted that better risk management of prescription drugs, especially at the patient-provider-patient-pharmacist levels, could be achieved with improved interoperability of systems and enhanced communications. To achieve its goals, the Committee recognized the need for standard vocabularies, system interoperability and security.

Government Initiatives

HHS HIPAA Initiative19

Congress, recognizing the need for electronic data interchange among partners in health services, enacted the Health Insurance Portability and Accountability Act (HIPAA) in 1996. The Act provided for the administrative simplification of transfers of health information through the establishment of standards and requirements for electronic transmission. Standards are established by accreditedviii Standards Setting Organizations (SSOs) and adopted where appropriate by the Secretary of Health and Human Services (HHS). HHS has adopted standards for medical data, diagnoses and medical procedures20. HIPAA medical data code sets that have been adopted to date include: (1) the International Classification of diseases, 9th Edition, Clinical Modification (ICD-9-CM), Volumes 1 and 2; (2) the International Classification of diseases, 9th Edition, Clinical Modification (ICD-9-CM), Volume 3, Procedures; and (3) The National Drug Code; (4) The Code on Dental Procedures and Nomenclature; (5) The combination of Health Care Financing Administration Common Procedure Coding System (HCPCS) and Current Procedural Terminology, 4th Edition (CPT-4)21. The utilization of these standards is anticipated to improve the efficiency and effectiveness of healthcare services and result in substantial reductions in healthcare costs for government and business.

Office of Management and Budget (OMB) E-Government Initiatives22

Expanding E-government is an integral part of the President's Management Agenda for making government more focused on citizens' needs and results. In 2001, an OMB E-Government Task Force found that the federal government could significantly improve customer service over the next 18 to 24 months and reduce costs by focusing on 23 government initiatives that integrate FDA operations and information technology (IT). The Task Force identified several problems. First to implement electronic capabilities, agencies usually automate existing processes, rather than creating more efficient and effective systems. In the transition, agencies typically invested in traditional, paper-based and electronic approaches resulting in system redundancies that add unnecessary burdens and costs to citizens, government and business. Additionally, due to budget processes and agency cultures, there is resistance to the development of interoperable systems that integrate work and the sharing of information systems within or between agencies. The Task Force Strategy for improving E-Government focused on four areas: (1) Individuals/Citizens: Government-to-Citizens to improve the quality and efficiency of service delivery; (2) Businesses: Government-to-Business to reduce redundant data collection; (3) Intergovernmental: Government-to-Government to enable sharing of data and integrate government operations in risk management; and (4) Intra-governmental: Internal Efficiency and Effectiveness that bring commercial best practices to government operations. The Task Force addressed Implementation of E-government practices and identified barriers to successful change. One of the initiatives targeted for initiation is the establishment of standards for vocabulary used in healthcare. The FDA is a partner in this initiative and intends to apply these standards to clinical study data and labeling. This initiative is anticipated to reduce government and private sector administrative costs and improve healthcare for the U.S. population.

Congressional Health Information Initiatives

Congressional initiatives have laid a foundation for the development of a health information infrastructure and have included the High Performance Computing Act (1991), the Next Generation Internet Act (1998) and the Networking and Information Technology Research and Development Act (2000) all directed at the healthcare sector. The President's Information Infrastructure Initiative (1993) focused on making information available in the home and workplace. Also, the Health Information and Applications Work Group focused on recommending improvements in public health applications and consumer communication for health information.

Congressional e-government principles for regulatory agencies23

In the E-government Act of 2001, Congress required government to utilize internet-based IT to enhance citizen access to government information and services and for other health related purposes. The Act required that agencies provide information on websites and accept submissions by electronic means. Further, the Act promoted interoperability of systems, integration of data elements across agencies and reduced the burden of duplicate data collection. The Demographic Information and Data Repository proposes to implement these objectives for the review environment, staging implementation to prioritize clinical study data, labeling, and reviewer templates in the short term and planning expansion into other areas of the review environment after implementation of these three basic components of the review environment.

Private sector initiatives

Clinical Data Interchange Consortium (CDISC)

CDISC is a non-profit organization composed of multi-disciplinary representatives to support the development of industry standards to support the electronic acquisition, exchange, submission and archiving of clinical trial data and meta-data from medical and biopharmaceutical product development24. CDISC working groups have developed "Submission Domain Models," a vendor-neutral clinical data interchange standard that includes data domain models to guide the organization, content and format of datasets25. The current domain models address dataset submissions for the 12 safety-related domains listed in FDA guidance documents. Future models will be provided for common analysis formats and describe other types of data such as protocol design, pharmacokinetics, pharmacodynamics, and efficacy data for certain therapeutic areas.

Health Level 7 (HL7), Incorporated

HL7 is a non-profit American National Standards Institute (ANSI) accredited Standards Development Organization that provides standards for the exchange, management and integration of data that support clinical patient care and the management, delivery and evaluation of healthcare services. HL7's goal is to create flexible, cost effective approaches, standards, guidelines, methodologies to enhance interoperability between healthcare information systems. The primary focus for HL7 is the development of shared, well defined, and unambiguous knowledge of the transferred clinical and administrative data. The HL7 structure, balloting procedures and open membership policies ensure that each of its membership constituencies (users, vendors and consultants) uniformly and equitably share in the development of standards.

CDISC and HL7 have signed an Associate Charter agreement that formalizes their relationship calls for the creation of a Clinical Trials Technical Committee within HL7 that will have representatives from the existing CDISC Working Teams and other interested stakeholders. The collaboration will focus on identifying the scope and range of data element requirements for clinical trials, identifying appropriate controlled vocabulary for encoding those data elements and identifying or defining messages and objects required to support the specific information exchange needs of clinical research. The groups will work to harmonize CDISC and HL7 approaches.


FOOTNOTES:

i FDA's first formal encouragement to analyze population subsets appeared in the 1985 version of Sec. 314.50, in which paragraph (d)(5)(v) (integrated summary of effectiveness) called for evidence to support modifications of dosage for specific subgroups, e.g., pediatrics, geriatrics, patients with renal failure.

ii For example, women are more sensitive to drugs that prolong the QT interval and are more likely than men to develop life threatening cardiac arrhythmias from the use of these drugs. [Ebert SN, Liu XK, Woosley RL. Female gender as a risk factor for drug-induced cardiac arrhythmias: evaluation of clinical and experimental evidence. J Women's Health. 1998. 7(5):547-57.]

iii For example, Whites are more likely than Asians and African Americans (5-10% vs. 1-3%) to have abnormally low levels of an important enzyme (CYP2D6) that metabolizes drugs belonging to a wide variety of therapeutic areas, such as antidepressants, antipsychotics, and beta-blockers. These differences may require different doses or dose intervals for affected sub-populations to ensure the safety of the products.

iv Interoperable means the seamless and uninterrupted electronic transfer of data and information between computer systems so that there is no need for additional steps to transform the information between transfer points

v Refers to standardized data stored in a globally accepted fashion with consistent naming conventions, measurements, encoding structures and physical attributes.

vi The reviewer's summary assessment of analyses presented for sub-population data (race/ethnicity, gender and age) and whether differences in efficacy or response were noted in the review.

vii The reviewer's summary assessment of analyses presented for sub-population data (race/ethnicity, gender and age) and whether differences in efficacy or response were noted in the review.

viii Organizations are accredited by the American National Standards Institute (ANSI)

REFERENCES:

1 Federal Register 63;28. Investigational New Drug Applications and New Drug Applications. February 11, 1998. pgs 6854-62. (Docket 95N-0010)

2 General Accounting Office. Women's Health: Women Sufficiently Represented in New Drug Testing, but FDA Oversight Needs Improvement. July 2001. GAO-01-754.

3 Wizemann, T., Pardue, M., (eds), Exploring the Biological Contributions to Human Health: Does Sex Matter? Committee on Understanding the Biology of Sex and Gender Differences, Board on Health Sciences Policy, Institute of Medicine. National Academy Press, Washington, D.C., 2001.

4 Xie HG, Kim RB, Wood AJ, Stein CM, Molecular basis of ethnic differences in drug disposition and response, Annu Rev Pharmacol Toxicol 2001; 41:815-50.

5 Furuta T, Ohashi K, Kamata T et al, Effect of genetic differences in omeprazole metabolism in cure rates for Helicobacter pylori infection and peptic ulcer. Ann Intern Med 1998; 129: 1027-30.

6 Kim RB, Leake BF, Choo EF, et al, Identification of functionally variant MDR1 alleles among European Americans and African Americans. Clin Pharmacol Ther 2001;70(2):189-99

7 Futterman, R, Fillit, H, and Roglieri, J., Use of Ineffective or Unsafe Medication among Members of a Medicare HMO compared to Individuals in a Medicare Fee-for-Service Program. Am J Manag Care 1997; 3(4):569-75.

8 Food and Drug Administration, Gastrointestinal Drugs Advisory Committee: Chemoprevention of Sporadic Colorectal Cancer (CRC). Transcript of public meeting, March 19, 2002. pgs 336-342.

9 Food and Drug Administration. Task Force on Risk Management. Managing the Risks From Medical Product Use; Creating a Risk Management Framework available at http://204.29.171.80/framer/navigation.asp?charset=utf&cc;=US&frameid;=1565&lc;=en-us&providerid;=113&realname;=FDA&uid;=1722314&url;=http%3A%2F%2Fwww.fda.gov

10 Food and Drug Administration, International Conference on Harmonisation; Guidance on Ethnic Factors in the Acceptability of Foreign Clinical Data; Availability. Federal Register 63;111, June 10, 1998. Docket No. 97D-0299.

11 Evelyn, B., Toigo, T., Banks, D., et al. Women's Participation in Clinical Trials and Gender-Related Labeling: A Review of New Molecular Entities Approved 1995-1999. available at: http://www/fda.gov/cder/reports/womens_health/women_clin_trials.htm

12 Gruber, M. Participation of Females in Clinical Trials and Gender Analyses of Data in Biologic Product Applications.

13 Patterson, M. FDA Monitors Participation by Women and Minorities in Clinical Trials; Report on Current Status. October 9, 2001.

14 Federal Register 66;237. Patient Profile Viewer; Notice of Pilot Project. December 10, 2001. Pgs 63712-63713.

15 Federal Register 64;18. Providing Regulatory Submissions in Electronic Format - General Considerations. January 1999. Pgs. 4433-4434 (Docket 98D0075)
16 Federal Register 67;86. Requirements for Submission of Labeling for Human Prescription Drugs and Biologics in Electronic Format. May 3, 2002. Pgs 22367-75 (Docket 00n-1652)

17 Federal Register 67;72. Risk Management of Prescription Drugs; Public Hearing. April 15, 2002. pgs.18230-2 (Docket 02N0115)
18 National Committee on Vital Health Statistics, NHII - Information for Health: A Strategy for Building the National Health Information Infrastructure. November 15, 2001. Available at http://www.ncvhs.hhs.gov/nhiilayo.pdf

19 104th Congress, Public Law 104-191. Health Insurance Portability and Accountability Act of 1996. Sec 1171-1177 available at http://aspe.hhs.gov/admnsimp/final/txfinal.pdf

20 Health Insurance Reform: Standards for Electronic Transactions. Federal Register 65;160, August 17, 2000. pgs 50312-71. HCFA-0149-F; RIN 0938-AI58 45CFR 160 & 162.

21 DHHS. Frequently Asked Questions About Code Set Standards Adopted Under HIPAA. Updated 9/28/2000. available at http://aspe.hhs.gov/admnsimp/faqcode.htm

22 Office of Management and Budget. Implementing the President's Management Agenda for E-Government: E-Government Strategy, Simplified Delivery of Services to Citizens. February 27, 2002 available at www.whitehouse.gov/omb/inforeg/egovstrategy.pdf

23 107th Congress, 1st Session. S. 803, E-Government Act of 2001. Sec. 201, 204, 206, 207 available at http://www.contracts.ogc.doc.gov/cld/ecomm/S803.html

24 Clinical Data Interchange Standards Consortium (CDISC). available at http://www.cdisc.org/

25 Clinical Data Interchange Standards Consortium (CDISC), CDISC Submissions Data Domain Models, version 2.0, available at http://www.cdisc.org/models/sds/v2.0/index.html

rule