Centers for Disease Control and Prevention
Centers for Disease Control and Prevention
Centers for Disease Control and Prevention CDC Home Search CDC CDC Health Topics A-Z    
Office of Genomics and Disease Prevention  
Office of Genomics and Disease Prevention

 

Draft Genetic Test Review

Cystic Fibrosis
Clinical Validity
Print Version


CLINICAL VALIDITY 

Question 18:  How often is the test positive when the disorder is present?
Question 19:  How often is the test negative when the disorder is not present?
Question 20:  Are there methods to resolve clinical false positive results in a timely manner?  

Question 21:  What is the prevalence of the disorder in this setting?  
Question 22:  Has the test been adequately validated on all populations to which it may be offered?  
Question 23:  What are the positive and negative predictive values?  
Question 24:  What are the genotype/phenotype relationships?
Question 25:  What are the genetic, environmental or other modifiers?


CLINICAL VALIDITY 

Question 18:  How often is the test positive when the disorder is present? 

Question 19:  How often is the test negative when the disorder is not present?

page 1 | page 2 | page 3 | page 4


Summary
  • Clinical sensitivity is dependent on the mutation panel used and the racial/ethnic makeup of the population to be tested.
  • Clinical sensitivity is not dependent on the screening model used.
  • Using the recommended panel of 25 mutations, the clinical sensitivity (proportion of carrier couples or affected fetuses correctly classified) is:  
    • 78 percent among non-Hispanic Caucasian couples
    • 52 percent among Hispanic Caucasian couples
    • 42 percent among African American couples
    • 88 percent among Ashkenazi Jewish couples
    • 24 percent among Asian American couples
  • Actual clinical sensitivity is likely to be slightly lower because analytic sensitivity is less than 100 percent (estimated in Question 9 to be 97.9 percent).
  • Based on proficiency testing results alone, an analytic false positive result will likely occur about 5 times in every 1000 tests.
  • The clinical specificity cannot be reliably estimated, because the impact of confirmatory testing is undocumented.
  • Assuming routine confirmatory testing of all positive test results, false positive couples results are likely to be corrected, but the extent of correction is unknown.
  • The ‘initial positive rate’ (proportion of couples/partners receiving positive test results) varies by model used for screening, as well as by factors relating to race/ethnicity and panel composition.  Using the recommended panel of 25 mutations among non-Hispanic Caucasian couples, the initial positive rate is:
    • 3.4 percent for the sequential model (percent of identifiable mutations in screened women)
    • 0.1 percent for the couple model (percent of identifiable mutations in both partners)
    • 6.7 percent for the concurrent model (percent of identifiable mutations in either of the screened partners)

Introduction
The definitions of clinical sensitivity (Question 18) and clinical specificity (Question 19) can be derived using a two-by-two contingency table for data from either case/control or cohort studies.  If the data are from a general population cohort, both positive predictive and negative predictive values (Question 23) can also be directly computed.  Table 3-1 shows the definitions for these four screening characteristics, assuming that a general population of pregnancies is being tested.  The rows are defined by the test results.  In this instance, the DNA screening test for cystic fibrosis is considered positive when both partners of a couple have an identifiable mutation, regardless of the screening model.  The first row includes all screen positive couples, and the second row includes all couples who are not screen positive.  The columns are defined by what the screening test aims to detect.  In this instance, it is to identify carrier couples who are at a 1 in 4 risk (1:3 odds) for having a child with cystic fibrosis.  The first column indicates couples who are both carriers of a cystic fibrosis mutation, and the second column indicates couples who are not.  

Table 3-1.  A Two-by-Two Contingency Table for Deriving the Four Major Clinical Performance Parameters

Both Partners are Cystic Fibrosis Carriers

·        Clinical sensitivity [ A / (A + C) ] is the proportion of couples in which both partners are cystic fibrosis carriers (A+C) and who are correctly identified as being positive (A) by the screening test. 

·        Clinical specificity [ D / (B + D) ] is the proportion of non-carrier couples (B+D) who are correctly identified as being negative (D) by the screening test. 

·        Positive predictive value [ A / (A + B) ] is the proportion of positive tests (A + B) that correctly identifies carrier couples (A). 

·        Negative predictive value [ D / (C + D) ] is the proportion of negative tests (C + D) that correctly identifies non carrier couples (D). 

Figure 1 shows an example of applying prenatal screening for cystic fibrosis to a hypothetical cohort of 1,000,000 couples.  In this example, the prevalence of cystic fibrosis is 1:2,500 (carrier rate 1/25), and the DNA test panel identifies 77 percent of the carrier couples (if 88 percent of mutations are detectable in each individual, then 88 percent squared, or 77 percent are detectable in the couple).  The analytic sensitivity is taken to be 97.9 percent (Question 9), and the analytic specificity (after confirmatory testing) is assumed to be, in this example, 99.99 percent (false positive rate of 1 per 10,000 individuals tested).  Among the population screened, there are 1,600 carrier couples (1,000,000 * (1/25)2).  77 percent of the 1,600 carrier couples are detectable (1,232), and 1,181 of these are detected (1,232*.9792).  Among the 998,400 non-carrier couples, 76,800 will include one carrier partner, and, in six of these couples, a false positive result will occur in the non-carrier partner (76,800*.88*.979*0.0001).  The numbers from Figure 3-1 can now be entered into a two-by-two table (Table 3-2) by substituting actual numbers into the format shown earlier in Table 3-1.  The clinical performance estimates can then be computed. 

Figure 3-1.  A Schematic Showing the Results of Prenatal Cystic Fibrosis Screening for ‘Carrier Couples’

Figure 1:  1,000,000 Pregnant Couples

Table 3-2.  A Two-by-Two Contingency Table for Deriving the Four Major Clinical Performance Parameters in a Hypothetical Population of 1,000,000 Couples

Table 3-2: Both Partners are Bystic Fibrosis Carriers 

Clinical sensitivity  = 

Clinical specificity  =

Positive predictive value  = 

Negative predictive value  = 

 

74 percent  (1,181/1,600 * 100)

99.9994 percent  (998,394/998,400 * 100)

99.5 percent  (1,181/1,187 * 100)       odds 196:1

99.96 percent  (998,394/998,813 * 100)         odds 2355:1

Impact of the screening model on these estimates
In Figure 3-1, there are a total of 76,800 couples with one partner a carrier, and all are considered as having positive test results.  Only the expanded one-step (concurrent) screening model will identify all of these carriers.  In that model, samples are obtained and tested from both partners.  The other two screening models (the two-step or sequential, and the one-step or couple) only identify half of the carrier/non-carrier couples, thereby reducing the number of clinical false positive results from six to three couples.  The following sections consider additional issues relating to clinical sensitivity and specificity.

Clinical sensitivity
Clinical sensitivity refers to the proportion of carrier couples (or affected fetuses) that can be detected by screening couples during pregnancy.  In contrast, analytic sensitivity describes how often the laboratory correctly identifies a mutation that is included in its panel.  Because of the large number of mutations responsible for cystic fibrosis and the limited number of mutations that can currently be economically included in a prenatal screening setting, not all carrier individuals will be identified.  Thus, clinical sensitivity can be relatively low, even when analytic sensitivity is relatively high.  Table 3-3 shows the clinical sensitivity as a function of the proportion of mutations detected (assuming that the analytic sensitivity is 100 percent).  For example, the table shows that, under the assumption of Hardy-Weinberg, it is necessary to identify 70 percent of the mutations in order to detect 49 percent of the carrier couples.  In this example, the clinical sensitivity is 49 percent.  

Table 3-3.  Clinical Sensitivity: Proportion of Carrier Couples (or Affected Fetuses) Identified as a Function of the Proportion of Cystic Fibrosis Mutations Detected by a Given Screening Panel

Proportion of

Mutations Detected (%)

Clinical

Sensitivity (%)

 

 

20

4

30

9

40

16

50

25

60

36

70

49

75

56

80

64

85

72

90

81

95

90

Individual mutation frequencies in the non-Hispanic Caucasian population
Because of the way that race/ethnicity is collected, it is not always possible to define race/ethnicity in consistent and highly stratified groupings.  The United States Government, for example, usually stratifies data into three racial categories (Caucasians, Blacks and Asians) and into Hispanic/non-Hispanic ethnicity.  Many other data sources are also stratified in this way (e.g., Cystic Fibrosis Foundation Patient Database).  The analyses presented here are, of necessity, stratified according to the methods used by those sources.  When possible, more finely stratified groups are also considered (e.g., Ashkenazi Jewish). 

In order to estimate the proportion of carrier couples (or affected fetuses) that can be identified for any given panel of mutations, it is necessary to obtain the mutation frequencies in an unbiased sampling of individuals clinically affected with cystic fibrosis.  Table 3-4 shows the frequencies of the 25 mutations in the panel recommended for prenatal screening.  The table includes two studies of non-Hispanic Caucasians with cystic fibrosis.  The mutations are listed in order of decreasing average frequency.  As an indicator of reliability of these mutation frequencies, bolded entries indicate that the mutation was tested by more than one-quarter of the laboratories (CF Consortium) and was observed more than 10 times (CF Consortium and CF Foundation).  

Table 3-4.  Mutation Frequencies for non-Hispanic Caucasians in the United States Within the Recommended 25 Mutation Panel

Table 3-4: Mutation Frequency (%)

1  Cystic Fibrosis Genetic Analysis Consortium (Kazazian, 1994), based on between 2,187 and 9,792 cystic fibrosis chromosomes (Appendix A)

2  A new analysis of the Cystic Fibrosis Foundation Patient Database, based on 3,938 chromosomes (Palomaki et al., 2002, – Appendix B)

Mutation frequencies derived from the Cystic Fibrosis Genetic Analysis Consortium Report  The summary estimate of 85.05 percent shown above from the Cystic Fibrosis Genetic Analysis Consortium data is somewhat higher than reported (Kazazian, 1994), because studies that reported mainly on Hispanic, Ashkenazi Jewish or African American affected individuals are removed from the present analysis.  Also, the denominators for each of the mutation frequencies are computed in the present analysis by dividing the number of observed chromosomes by the total number of chromosomes reported only for studies that actually tested for the given mutation.  The earlier report (Kazazian, 1994) divided the observed number of each mutation by the total number of chromosomes reported for all studies.  This oversight was mentioned in the Kazazian report and has since been corrected (Giorgi et al., 1997).  For the more common mutations, this second correction has little or no impact.  However, for the less common mutations, the corrected frequencies will be higher than originally listed.  For example, the mutation frequency for A455E would be 0.28 percent prior to correction, and 0.54 percent after.  The entire reanalysis is contained in Appendix A, Table 3-14.  Additional information is available on-line (www.genet.sickkids.on.ca/cftr/newfreq/All.html), but it is not clear whether blank entries on these newer tables indicate “tested for and none found” or “not tested” (Markiewicz, personal communication, 2001).  For this reason, they are not included.  The three mutations in the recommended panel (3120+1G>T, 2814delA and I148T) that were not part of the Consortium’s report have been arbitrarily assigned a frequency of 0.10 percent (italics).  

There are several limitations to using these data to estimate mutation frequencies in the general population in the United States.  For example, it is not possible to determine to what extent these studies included individuals who were Hispanic (or of other racial/ethnic groups).  Also, some of the data reported to the consortium have been collected in reference laboratories using mutation panels of 50 or more mutations.  It is possible that these expanded test panels were used selectively in cystic fibrosis patients with less common mutations that could not be identified by initial testing (e.g., the initial test may have only analyzed delF508).  A major contributor to the consortium reports that this bias exists in its data (Heim et al., 2001).  Such a bias will lead to under-estimation of the mutation frequency for delF508.  Another possible bias might be the over-representation of some racial/ethnic groups.  For example, if couples of Ashkenazi Jewish heritage were to participate more fully in testing programs than other non-Hispanic Caucasian couples, this would lead to an underestimation of the delF508 mutation frequency and an over-estimation of other mutations (e.g., W1282X).  The present analysis attempts to take this into account by removing any study from the analysis if its population is mainly of Ashkenazi Jewish heritage. 

Mutation frequencies derived from the Cystic Fibrosis Foundation Database  In an attempt to address the shortcomings of the Cystic Fibrosis Consortium data, we undertook a reanalysis of the Cystic Fibrosis Foundation Database.  This data source represents approximately 85 percent of all cystic fibrosis patients in the United States.  Previous analyses have been applied to the entire collection of genotypes in that database.  This approach has strength in numbers (over 29,000 chromosomes studied), but it is not possible to document which mutations have been tested for by the various contributing centers.  Thus, a small number of less common mutations might indicate a low mutation frequency, but it might also indicate that few patients had been tested for that mutation.  This issue is addressed here by focusing only on patients attending one of nine Therapeutic Development Network (TDN) Centers that offer expanded mutation panels to all patients.  Previous analyses of the Foundation’s data did not distinguish Hispanic from non-Hispanic Caucasians.  The present analysis does.  Of the 2,507 self-declared Caucasian individuals with cystic fibrosis attending a TDN center in 1999, 2,130 (85 percent) declared themselves to be non-Hispanic, 302 (12 percent) did not answer (or were not asked) the question, and the remaining 75 (3 percent) responded that they were of Hispanic heritage.  Of the 2,130 non-Hispanic Caucasians eligible for analysis, 1,969 (92.4 percent) had been genotyped.  The remaining 7.6 percent either refused genotyping, or the results were not yet available.  The Foundation's data do not allow separation of Ashkenazi Jewish individuals from non-Hispanic Caucasians.  

When comparing the mutation frequencies from the two sources shown in Table 3-4, the most striking difference is found in the estimate for delF508.  The estimate from the Consortium is nearly 7 percentage points lower than the corrected estimate based on the Foundation’s data.  Based on biases that are likely to be present in the Consortium data, the Foundation's estimate may be closer to the truth.  The total proportion of mutations identified from the Cystic Fibrosis Foundation Patient Database is about 6.5 percentage points higher than from the Consortium data.  This overall difference is mainly due to the variation in the delF508 mutation frequency.  Both estimates are higher than initially reported (Kazazian, 1994) and than generally quoted in the literature (Grody et al., 2001).  The following analyses use the averages of the mutation frequencies from the two studies (Table 3-4). 

Table 3-5 shows the cumulative percentage of detectable mutations and the carrier couple detection rate for 1, 5, 10, 15, 20 and 25 mutations.  Mutations are added in the order shown in Table 3-4 and are, therefore, only appropriate for non-Hispanic Caucasians.  Mutation frequencies in other racial/ethnic groups will be considered later in this section.  Figure 3-2 graphically displays the data shown in Table 3-5. 

Table 3-5.  A Comparison of Mutation Panel Size and Percentage of Carrier Couples Detected, Assuming an Analytic Sensitivity of 100 percent

 Number of Mutations in the Panel 1

 Cumulative Percentage of Detectable Mutations

Cumulative Percentage of Carrier Couples Detected (Clinical Sensitivity)

 

 

 

  1

72.4

52.4

  5

80.0

64.0

10

84.3

71.1

15

86.5

74.8

20

87.9

77.3

25

88.4

78.1

1  The order of added mutations is from Table 3-4

Figure 3-2:  Detection Rate (%) and Number of CF Mutations Figure 3-2.  The Cumulative Percentage of Carrier Couples Detected as a Function of the Number of Mutations in the Panel.  The figure is only appropriate for non-Hispanic Caucasians.  The dashed line indicates the clinical sensitivity (detection rate) assuming an analytic sensitivity of 100 percent.  The solid line assumes an analytic sensitivity of 97.9 percent.

Prenatal cystic fibrosis screening models and test failure rates  
The impact of a test failure (i.e., no useable result) on prenatal cystic fibrosis screening depends on the model used.  If a program utilizes buccal samples, the test failure rate might be, for example, 1 percent.  If that program uses a two-step model, a new sample must be requested from 1 of every 100 women initially tested.  Pilot trials have shown that not all individuals with sample failures will submit a second sample.  If a one-step model were used instead, the couple would be considered a test failure only when both samples failed or when the second partner’s sample failed when the partner had an identified mutation.  Thus, the one-step model would require a repeat sample only in about 1 in 1300 couples.  If the expanded one-step model were used, each partner would have a 1 percent chance of needing a repeat sample.  Thus, about 2 of every 100 couples would need one of the partners to submit a second sample.  Using blood samples would result in fewer test failures, but blood collection is associated with increased costs and may be less convenient.  A summary of published pilot trials contains related information concerning failure rates and compliance with screening protocols (Question 33). 

Genotype and phenotype of the fetus
The aim of prenatal screening for cystic fibrosis is two-tiered; first, to identify carrier couples and then, to offer these couples a diagnostic procedure (usually amniocentesis) and testing to identify cystic fibrosis in the fetus.  When both partners are confirmed as carriers, the risk of the fetus inheriting both mutations is 1 in 4 (odds of 1:3).  The relationship between genotype and phenotype will be described later in some detail (Question 24).  In general, 95 percent or more of fetuses with two of the mutations contained in the screening panel will have manifestations that include serious lung problems, indicating that the cystic fibrosis mutations are of high ‘penetrance’. 

Clinical specificity
The analysis in this section is restricted to screened couples who, because of their genetic makeup, cannot have a child with cystic fibrosis.  Rarely, these couples may be incorrectly classified as a carrier couple (i.e., a mutation is reported in both partners).  Clinical specificity is a measure of how often this occurs.  The following lists several types of errors that might lead to such misclassification:

pre-analytic errors

  • a sample mix-up prior to receipt by the laboratory

  • a degraded or mishandled sample

  • non-paternity

analytic errors

  • a sample mix-up after receipt by the laboratory

  • benign polymorphism mistaken for mutation

post-analytic errors

  • data entry error

  • incorrect laboratory interpretation of assay results

  • report mix-up at the laboratory or health provider site 

According to the analysis shown earlier (Question 9), the analytic specificity is 97.9 percent. (i.e., a mutation is falsely reported to be present, or the wrong mutation is reported in about 2 to 3 per hundred tests).  This rate is derived from external proficiency testing and, therefore, may not reflect the checks and balances routinely in place in the clinical laboratory that are designed to identify and correct analytic errors.  Most of the errors identified by proficiency testing consist of assigning an incorrect mutation.  Routine confirmatory procedures would likely identify and correct many of these errors (Question 14).  When “wrong mutation” errors are taken into account properly, analytic specificity is increased to 99.5 percent.  This analysis utilizes six years of proficiency testing data.  Only one false positive result has been reported from that source for the last four years.  Thus, an analytic false positive rate of 5 per 1000 cystic fibrosis mutation analyses appears reasonable.  If this is true, how often will a positive couple be falsely identified?  A false positive couple will occur most often in situations where one partner actually has an identified mutation.  About 1 in every 28 non-Hispanic Caucasian individuals (1/25 * 0.88) will be correctly identified as being a carrier.  For every thousand such couples, five of the partners might be expected to have a false positive test result.  Thus, a false positive couple is expected to occur in 5 of every 28,000 non-Hispanic Caucasian couples tested (1:5600), in comparison to 35 of every 28,000 non-Hispanic Caucasian couples who are truly positive (1:800).  Thus, without confirmatory testing, perhaps as many as 1 in every 7 positive couples might be incorrectly classified.  How many of these are likely to be correctly reclassified by confirmatory testing?  If all positive couples were to provide another sample and that sample were to be analyzed using a different methodology, perhaps all such errors would be resolved.  This may not be done, however, in all screening laboratories.  If such confirmatory testing is not done, these false positive couples are unlikely to be found as part of the routine diagnostic testing of the fetus, since it is expected that 3 of every 4 fetuses of true positive couples will not have two mutations present.   

Some information about the possible impact of confirmatory testing is available from the European external proficiency testing program (Cuppens and Cassiman, 1995).  In that program, the protocol included a follow-up request to laboratories with incorrect cystic fibrosis testing results.  They were asked to repeat the analysis. Under these unblinded conditions, three  laboratories repeated the analysis of a false positive result and in all three instances, the correct genotype was confirmed.  Several other false positive results were not repeated and/or not reported to the program. 

References 

Cuppens H, Cassiman JJ.  1995.  A quality control study of CFTR mutation screening in 40 different European laboratories.  Eur J Hum Genet  3:235-245.

Giorgi S, Tandoi C, Ciminelli BM, Modiano G.  1997.  A correction of the estimates of the least common cystic fibrosis (CF) mutations published by “The Cystic Fibrosis Genetic Analysis Consortium” in 1994.  Gene Geograph  11:57-59.

Grody WW, Desnick RJ.  2001.  Cystic fibrosis population carrier screening: Here at last – Are we ready?  Genet Med  3:87-90.

Heim R, Sugarman EA, Allitto BA.  2001.  Improved detection of cystic fibrosis mutations in the heterogeneous US population using an expanded, pan-ethnic mutation panel.  Genet Med 3:168-176.

Kazazian HH for the Cystic Fibrosis Genetic Analysis consortium.  1994.  Population variation of common cystic fibrosis mutations.  Hum Mutat  4:167-177.

Palomaki GE, Haddow JE, Bradley LA, FitzSimmons SC.  2002.  An updated assessment of cystic fibrosis mutation frequencies in non-Hispanic Caucasians.  Genet Med, 4:90-94.

Witt DR, Schaefer C, Hallam P, Wi S, Blumberg B, Fishbach A, Holtzman J, Kornfeld S, Lee R, Nemzer L, Palmer R.  1996.  Cystic fibrosis heterozygote screening in 5,161 pregnant women.  Am J Hum Genet  58:823-835.

 

Updated on August 13, 2004