ACCE Core Group

James E. Haddow, MD, Principal Investigator
Glenn E. Palomaki, BS
Linda A. Bradley, PhD
Carolyn Sue Richards, PhD
Marta Gwinn, MD, MPH
Muin J. Khoury, MD, PhD
Paula W. Yoon, ScD, MPH

Introduction to ACCE

ACCE, which takes its name from the four components of evaluation - analytic validity, clinical validity, clinical utility and ethical, legal and social implications - is a model process for evaluating data on emerging genetic tests.  The process includes collecting, evaluating, interpreting, and reporting data about DNA (and related) testing for disorders with a genetic component in a format that allows policy makers to have access to up-to-date and reliable information for decision making.  An important by-product of this process is the identification of gaps in knowledge.  The ACCE approach builds on a methodology originally described by Wald and Cuckle (1) and on terminology introduced by the Secretary's Advisory Committee on Genetic Testing (2).

Groups Involved in ACCE Project

The ACCE evaluation process requires that many different groups and individuals work together as illustrated in Figure 1.  The groups involved in developing the draft reports are displayed within the large circle, along with the groups that review and suggest modifications and/or extensions.  A representative list of reviewers is also displayed, to give a sense of the type of review and input that the project is seeking.  One important function is to keep policy making groups – listed in the upper left corner of the figure - informed of the ongoing process.

Figure 1.  A diagram summarizing the groups involved
in the ACCE Project

Components of ACCE

The ACCE wheel (Figure 2) shows the relation among each of the four components of evaluation and the elements of each component.  At the hub are the clinical disorder being evaluated and the setting in which testing is done (e.g., classic cystic fibrosis in the setting of prenatal screening).  The evaluation process begins only after the clinical disorder and setting have been clearly established.  Specific questions 1 through 7 in Table 1 help to define the disorder, the setting, and the type of testing.

Figure 2.  The ACCE evaluation process for genetic testing.

The analytic validity of a genetic test defines its ability to accurately and reliably measure the genotype of interest.  This aspect of evaluation focuses on the laboratory component.  The four specific elements of analytic validity include analytic sensitivity (or the analytic detection rate), analytic specificity, laboratory quality control, and assay robustness.  Analytic sensitivity defines how effectively the test identifies specific mutations that are present in a sample.  Analytic specificity defines how effectively the test correctly classifies samples that do not have specific mutations (although the term “mutation” is used here, the terms “polymorphism” or “variant” may be more appropriate for certain situations).  Quality control assesses the procedures for ensuring that results fall within specified limits. Robustness measures how resistant the assay is to changes in pre-analytic and analytic variables. Specific questions 8 through 17 in Table 1 help organize the information available to document analytic validity.

The clinical validity of a genetic test defines its ability to detect or predict the associated disorder (phenotype).  The four elements of analytic validity are all relevant to assessing clinical validity, along with six additional elements:  clinical sensitivity (or the clinical detection rate), clinical specificity, prevalence of the specific disorder, positive and negative predictive values, penetrance, and modifiers (gene or environmental). Penetrance defines the relation between genotype and phenotype and allows the frequency of the clinical expression of a genotype (expressivity) to be determined. Clinical sensitivity measures the proportion of individuals who have a well-defined clinical disorder (or who will get the disorder in the future) and whose test values are positive. Clinical specificity measures the proportion of individuals who do not have the well-defined clinical disorder and whose test results are negative. Prevalence measures the proportion of individuals in the selected setting who have, or who will develop, the phenotype. The positive and negative predictive values more meaningfully define the genetic test performance by taking into account clinical sensitivity, clinical specificity and prevalence. Specific questions 18 through 25 in Table 1 help organize the information available to document clinical validity.

The clinical utility of a genetic test defines the elements that need to be considered when evaluating the risks and benefits associated with its introduction into routine practice. The natural history of the specific disorder needs to be understood so that such considerations as optimal age for testing might be taken into account. Another factor to be considered is the availability and effectiveness of interventions aimed at avoiding adverse clinical consequences (if no interventions are available, for example, testing may not be warranted). Quality assurance assesses procedures in place for controlling pre-analytic, analytic, and post-analytic factors that could influence the risks and benefits of testing.   Pilot trials assess the performance of testing under real-world conditions.  Health risks define adverse consequences of testing or interventions in individuals with either positive or negative test results.  Economic evaluation helps define financial costs and benefits of testing.  Facilities assess the capacity of existing resources to manage all aspects of the service.  Education assesses the quality and availability of informational materials and expertise for all aspects of a screening service.  Monitoring and evaluation assess a program’s ability to maintain surveillance over its activities and make adjustments.  Specific questions 26 through 41 in Table 1 help organize the information available to document clinical utility.

Ethical, legal, and social implications surrounding a genetic test are represented in Figure 2 by a penetrating pie slice, implying that the safeguards and impediments should be considered in the context of the other components.Specific questions 42 through 44 in Table 1 help organize the information available to document ELSI issues.

Next Steps

The ACCE project is successfully meeting its mandate to develop a model system for assessing available information about specific genetic tests and test protocols.  The first disorder to undergo an ACCE review was cystic fibrosis in the prenatal setting.  That review is nearly complete.  Currently reviews are underway on genetic tests for hereditary hemochromatosis, venous thromboembolism, and breast cancer.  Experience and knowledge gained from these reviews will help the project achieve its long-term goal: developing a process that can be used by others to critically review genetic tests.

 
Table 1.  The ACCE Model’s List of Targeted Questions Aimed at a Comprehensive Review of Genetic Testing 

References

1.   Wald N, Cuckle H. Reporting the assessment of screening and diagnostic tests. Br J Obstet Gynaecol 1989 Apr;96(4):389-96.

2.   Department of Health and Human Services, Secretary’s Advisory Committee on Genetic Testing. Request for public comment on a proposed classification methodology for determining level of review for genetic tests. Federal Register 2000;65(236):76643-76645.

Link to the Public Health Perspective Page on Genetic Testing