Centers for Disease Control and Prevention
Centers for Disease Control and Prevention
Centers for Disease Control and Prevention CDC Home Search CDC CDC Health Topics A-Z    
Office of Genomics and Disease Prevention  
Office of Genomics and Disease Prevention

 

Draft Genetic Test Review

Hereditary Hemochromatosis
Analytic Validity
Print Version


ANALYTIC VALIDITY 

Question   8:  Is the test qualitative or quantitative?  
Question   9:  How often is a test positive when a mutation is present (analytic sensitivity)?  
Question 10:  How often is the test negative when a mutation is not present (analytic specificity)?
Question 11:  Is an internal QC program defined and externally monitored?
Question 12:  Have repeated measurements been made on specimens?
Question 13.  What is the within- and between-laboratory precision?
Question 14:  If appropriate, how is confirmatory testing performed?
Question 15:  What range of patient specimens has been tested?
Question 16:  How often does the test fail to give a useable result?
Question 17:  How similar are results obtained in multiple laboratories using the same, or different, technology?


ANALYTIC VALIDITY 

Question 8:  Is the test qualitative or quantitative? 

The DNA test associated with HHC is qualitative (i.e., a mutation is reported as present or absent).  Several mutations have been described, but when DNA analysis is proposed as a screening test for morbidity and mortality associated with iron overload in the setting of the general adult population, the only mutation of interest is C282Y.  The genotype of interest is homozygosity for the C282Y mutation. 


ANALYTIC VALIDITY 

Question 9:  How often is the test positive when a mutation is present (analytic sensitivity)?

Question 10:  How often is the test negative when a mutation is not present (analytic specificity)?   

Appendix 1 | Appendix 2

Summary
External proficiency testing schemes are the only major reliable source currently available for computing analytic sensitivity and specificity for HFE testing.  The following caveats should be kept in mind, however, when examining these estimates.  First, external proficiency testing schemes are designed to be educational.  For that reason, the types of challenges may not represent routine testing.  Also, laboratories from outside the U.S. are included, and both research and clinical laboratories participate.  In spite of these shortcomings, this source of data can be useful in establishing a baseline for laboratory performance.

Based on data from the American College of Medical Genetics and the College of American Pathologists (ACMG/CAP) Molecular Genetics Survey Set MGL from 1998 through 2002:

·        The overall error rate for C282Y genotyping (analyzed by chromosome) is 1.0% (95 percent CI 0.6 to 1.5%).

·        Analytic sensitivity for C282Y homozygotes is 98.4% (95 percent CI 95.9 to 99.5%).

·        Analytic specificity for other non-homozygous genotypes is 99.8% (95 percent CI 99.4 to 99.9%).

·        It is not possible to determine whether errors in the survey occurred in the pre-analytic, analytic or post-analytic phase of testing.

·        The analytic performance (sensitivity and specificity) for the C282Y mutation is expected be consistent, regardless of the race/ethnicity of the population being tested.  It is possible, however, that rare, unknown polymorphisms (that could cause false positive results) might vary by race/ethnicity

Although the H63D mutation is not considered part of the screening test, the analytic sensitivity and specificity are similar to those described for the C282Y mutation, serving as further documentation of laboratory performance.

Definitions
Analytic performance is summarized by the sensitivity and specificity of the detection system.  Generically, analytic sensitivity is defined the proportion of positive test results, when a detectable mutation is present (i.e., the test is designed to detect that mutation).  Analytic sensitivity is equivalent to the analytic detection rate.  Given that this report is focusing on DNA testing for morbidity and mortality due to iron overload in the setting of general adult population screening, only the C282Y mutation is of interest (Question 4, Question 18).  More specifically, the main interest in such a screening program would be to identify C282Y homozygotes.  Thus, analytic sensitivity will be defined in this document as the proportion of C282Y homozygotes correctly identified.

Generically, analytic specificity is the proportion of negative test results when no detectable mutation is present.  Analytic specificity can also be expressed in terms of the analytic false positive rate.  The false positive rate is the proportion of positive test results when no detectable mutations are present (1-analytic specificity).  In keeping with the specific definition of analytic sensitivity above, analytic specificity is defined in this document as the proportion of individuals that are not C282Y homozygotes who are correctly identified as not being homozygous for C282Y.  

Optimal source(s) of data
Few data sources exist for estimating analytic validity.  Published reports of method comparisons and screening experiences provide limited information on only a few testing methodologies.  The data are derived from a small number of laboratories, and the “true” genotypes of the tested samples are often undocumented (i.e., not confirmed by another methodology, laboratory consensus or sequencing).  External proficiency testing programs such as the ACMG/CAP Molecular Genetics Laboratory (MGL) Surveys provide a source of data that has several advantages.  The ACMG/CAP survey serves a large proportion of clinical testing laboratories in the U.S.  Data provided by these laboratories represent the range of methodologies presently being used.  In addition, the samples distributed for proficiency testing exercises have consensus genotypes.  However, basing analytic performance estimates on external proficiency testing also has drawbacks, including:

·        mixing of clinical and research laboratories and methodologies

·        relatively few challenges

·        reporting summary results in ways that do not allow a straightforward computation of analytic sensitivity and specificity

·        challenges that do not represent the ‘mix’ of genotypes expected in a screening program (e.g., too few wild challenges and too many homozygotes). 

Future analyses should be aimed at providing reliable, method-specific analytic performance estimates.  One approach for collecting such data might include the following steps:

·        An independent body (such as the College of American Pathologists, American College of Medical Genetics, Food and Drug Administration or the Coriell Institute of Medical Research, Camden, NJ) would develop a standard set of samples, most of which would be randomly selected from the general population.  Included in the standard set, however, would be additional, less common genotypes.

·        The sample set would then be available for method validation.  Correct genotypes would be arrived at by consensus, or, if disagreements emerged, by a reference method (e.g., sequencing).  The current validation practice of having a laboratory (or manufacturer) run a series of samples with unknown genotype is inadequate, since there is no ‘gold standard’ with which to compare.  For example, how would a laboratory running an unknown sample determine whether a positive finding is a true, or a false, positive? 

·        Ideally, this blinded sample set would be available to manufacturers as part of the pre-market approval process, with the understanding that multiple laboratories using these commercial reagents would be asked by the manufacturer to analyze portions of the sample set independently.  This initial assay validation process is distinct from assay control samples that are discussed later (Question 13). 

Appropriate sample size for determining analytic sensitivity and specificity has been discussed in detail in an earlier ACCE review (Prenatal Cystic Fibrosis Screening via Carrier Testing – Question 11 and 12).  In brief, a target sensitivity (or specificity) can be chosen, along with an acceptable lower limit (assumed to be the lower limit of the 95% confidence interval).  Given these targets, the number of necessary samples can be derived.  For example, if a laboratory chose a target specificity of 98% and wanted to rule out a specificity of 90%, it would need to correctly identify at least 49 of 50 known negative samples (estimated using the binomial distribution).  When the estimates approach 100% and relatively tight confidence intervals are sought, it may not be economically feasible for laboratories to individually collect and analyze their data.  However, this level of confidence could be attained by a consortium of laboratories using the same methodology, or by a manufacturer that forms a consortium of laboratories using its reagents.   All of these suggested analyses could be done using a 2x2 table, and all rates could be accompanied by 95% confidence intervals (CI).  

The ACMG/CAP external proficiency testing scheme
As part of ACMG/CAP external proficiency testing in the United States, purified DNA from established cell lines (derived from human cells with known mutations http://locus.umdnj.edu) is distributed to participating laboratories.  Many of these laboratories are likely to be providing clinical services, but reagent manufacturers and research laboratories also participate.  In late 2001, there were 90 participants reporting HFE results.  In general, there are three types of errors.  A false positive result occurs when the laboratory reports finding a mutation in the sample, when none is present.  A false negative result occurs when a laboratory reports no mutation, but a mutation for which it tests is, in fact, present in the sample.  A third type of error occurs when the laboratory accurately identifies that a mutation is present, but it is not the correct mutation.  Given the nature of this scheme, it has not been possible to determine the phase of testing in which the error has occurred (e.g., pre-analytic, analytic or post-analytic).  When considering the performance of identifying the C282Y mutation alone in the setting of general adult population screening, these errors need to be carefully redefined. 

§         A false negative result occurs when an individual who is homozygous for C282Y has a test result that is not homozygous for C282Y (i.e., wild/wild or C282Y/wild)

§         A false positive result occurs when an individual who is not homozygous for C282Y has a test result falsely indicating homozygosity for this mutation.

§         The third type of error, wrong mutation, is not considered in this analysis, since it is assumed that the test will only be directed at one mutation: C282Y. 

A separate analysis of analytic sensitivity and specificity for both the C282Y and H63D mutations performed by chromosome can be found in Appendix 1 at the end of this section.  A listing of other mutations in the HFE gene can be found in Appendix 2. 

Error rates for the ACMG/CAP external proficiency testing scheme  Table 2-1 shows how each of the two types of error are counted in the analyses of analytic performance.  Column 1 shows those individuals who are actually homozygous for C282Y.  The first entry in that column contains those receiving positive test results (i.e., true positives with a result of homozygous for C282Y).  Any other result in this column (rows 2 and 3) is considered a false negative.  Among true heterozygotes (Column 2), the finding of homozygosity for C282Y would be a false positive (first row).  Any other test result would be considered negative.  Column 3 shows the three possible test results in individuals with no C282Y mutations.  If the aim of testing is to correctly identify individuals who are homozygous for the C282Y mutation, the table could be collapsed according to the darkened lines. 

Table 2-1.  All Possible Combinations of Test Results with the Three Genotypes of Interest Assuming Testing is Limited to the C282Y Mutation

Table 2-1:  Test Result, Actual Genotype, Total

Table 2-2 shows the results of the ACMG/CAP MGL survey for HFE mutations in the format described in Table 2-1.  That survey did include several challenges of the H63D mutation.  For this analysis, the H63D mutations and corresponding results are ignored, but the sample is still included.  For example, a compound heterozygote challenge of C282Y/H63D is viewed as a C282Y heterozygote challenge.  Overall, 20 of the 2,043 sample challenges were incorrectly genotyped for C282Y, for an overall error rate of 1.0% (95 percent CI 0.6 to 1.5%).  As indicated earlier, the major goal of DNA screening for hemochromatosis is to correctly identify C282Y homozygotes.  The “collapsed” table shows that 98.4% of the homozygous genotypes were correctly identified (243/247, 95 percent CI 95.9 to 99.5%).  In addition all but four of the 1,796 negative (non-homozygous samples) were identified as non-homozygotes (99.78%).  The error rate did not change appreciably over time, as shown in the summary of challenges and errors displayed in Figure 2-1. 

Table 2-2.  HFE Mutation Testing:  Results of the ACMG/CAP Molecular Genetics Survey 
When the Analysis is Restricted to the C282Y Mutation

Table 2-2:  Test Result, Actual Genotype, Total

Figure 2-1.  Summary of Errors Reported in the ACMG/CAP MGL Survey, with Interpretation restricted to the C282Y mutation 

·        In 1998, three samples were distributed to 67 laboratories (201 laboratory challenges)

      No errors

      Error rate 0.0% (95 percent CI 0.0 to 1.8%)

·        In 1999, two samples were distributed to 78 laboratories (156 laboratory challenges)

      One laboratory identified no C282Y mutations in a C282Y heterozygous sample

      One laboratory identified a C282Y mutation in a sample with no mutation present

      Error rate 1.3% (95 percent CI 0.2 to 4.6%)

·        In 2000-A, three samples were distributed to 81 laboratories (243 laboratory challenges)

      One laboratory identified a heterozygote as being homozygous for C282Y

      Error rate 0.4% (95 percent CI 0.1 to 2.3%)

·        In 2000-B, three samples were distributed to 90 laboratories (270 laboratory challenges)

      One identified a heterozygote when no C282Y mutations were present

      Two laboratories incorrectly identified a homozygote as having no mutations

      Error rate 1.1% (95 percent CI 0.2 to 3.2%)

·        In 2001-A, three samples were distributed to 100 laboratories (300 laboratory challenges)

      One identified a heterozygote when no C282Y mutations were present

      One identified a homozygote for C282Y when no C282Y mutations were present

      One laboratory identified no C282Y mutations in a C282Y heterozygous sample

      Error rate 1.0% (95 percent CI 0.2 to 2.9%)

·        In 2001-B, three samples were distributed to 90 laboratories (270 laboratory challenges)

      Two laboratories incorrectly identified a homozygote as being heterozygous

      Error rate 0.7% (95 percent CI 0.1to 2.7%)

·        In 2002-A, three samples were distributed to 103 laboratories (309 laboratory challenges)

      In two instances, a homozygote was reported when no C282Y mutations were present

      One identified a heterozygote when no C282Y mutations were present

      Two laboratories reported no C282Y mutations in a heterozygote

      One laboratory reported homozygosity for an individual heterozygous for C282Y

      Error rate 1.9% (95 percent CI 0.7 To 4.2%)

·        In 2002-B, three samples were distributed to 98 laboratories (294 laboratory challenges)

      One laboratory identified a heterozygote when non C282Y mutations were present

      Three laboratories reported no C282Y mutations in a heterozygote

      Error rate 1.4% (95 percent CI 0.4 to 3.5%)

Analytic sensitivity identifying C282Y homozygotes  Only eight of the 20 errors identified in the proficiency testing samples influence the analytic sensitivity of identifying C282Y homozygotes (first column in Table 2-2).  Overall, the analytic sensitivity is 243/247, or 98.4% (95 percent CI 95.9 to 99.5%).  These confidence intervals could be considered pessimistic and optimistic extremes of the analytic sensitivity.  Because of the relatively few challenges (and observed false negatives), it is not possible to determine whether analytic sensitivity varied over the four years. 

Analytic specificity for identifying non-C282Y homozygotes  The analytic specificity (computed using the second and third columns in Table 2-2) is 1,191/1,193 or 99.8% (95 percent CI 99.4 to 99.9%).

Genotyping Errors and Method of Testing  According to the ACMG/CAP Participant Summary Reports, there was no correlation between genotyping error and the laboratory method.  Errors were made by laboratories using restriction digestion and ASO analysis.  The majority of laboratories (77% on the 2000-B survey) used PCR and restriction digestion.  Other methods of analysis included ASO (9%), ARMS (8%), Light-cycler (3%), DNA sequencing (2%), and other (1%).  In one survey (MGL 2001-A), seven errors were made that involved either C282Y or H63D.  The Participant Summary Report notes that seven different laboratories made these errors, and that six of those laboratories provided clinical test results (only one was a research laboratory).

Recognition of a potential source of method-specific error As part of the ACMG/CAP Survey Program, concern was raised regarding the protocol validation of new laboratories, inexperienced in HFE testing.  Laboratories using restriction analysis were encouraged to ensure that their assays contain internal controls to validate enzyme restriction.  One other potential source for error is the use of the Feder primers for C282Y analysis, due to the G5569A polymorphism in the reverse primer.  Laboratories were cautioned that they should use alternate primers that do not include this polymorphism and that decreased annealing temperatures of 50-55°C would decrease the stringency of the PCR reaction and thus control for non-amplification due to primer site polymorphisms.  In the MGL 2000-B ACMG/CAP Participant Summary Report, participants reported that only 38/84 laboratories (45%) used the C282Y Feder primers, while 58/82 laboratories (70%) still used the H63D Feder primers.  A more in-depth discussion of this topic follows in the next two paragraphs.

In 1999, Jeffrey et al. reported that a previously described polymorphism, 5569A (Totaro et al., 1997), was associated with misdiagnosis of 15 C282Y/5569A heterozygotes as C282Y homozygotes.  Because this single base pair polymorphism is located in the primer binding site for the C282Y wild type allele in exon 4, Jeffrey et al. theorized that the Feder reverse primer might fail to anneal and thus prevent amplification of the wild type allele.  Since only the mutant allele would then be amplified, this could result in the appearance of a C282Y homozygote, and a false positive result.  Subsequently, two other laboratories reported misclassification of C282Y heterozygotes as homozygotes (Gomez et al., 1999; Somerville et al., 1999).  Because this polymorphism is relatively common (allele frequencies as high as 13%), this report raised immediate concern about C282Y results in genotyping studies worldwide and led some laboratories to re-analyze previous results. 

The ACMG/CAP Molecular Genetics Survey quickly responded that 67 U.S. laboratories (many using the Feder primers) had correctly genotyped a C282Y heterozygote sample that also carried the 5569A polymorphism (Noll et al., 1999).  Thorstensen et al. (2000) also reported no errors in genotyping in 433 patients tested using the Feder primers.  These authors suggested that the difference in performance might relate to a change in a PCR reaction condition (i.e., primer annealing temperature), and that most laboratories had used conditions that did not affect result accuracy.  The European Haemochromatosis Consortium reported that some laboratories had replaced the Feder reverse primer to remove the possibility of misclassification, but that previous publications by member laboratories had not been compromised.  Therefore, it appears that prevalence estimates of the C282Y mutation are unlikely to have been overestimated.  However, clinical laboratories should avoid primers containing polymorphic sites in which primer binding could affect test outcome.

Other polymorphisms  The DNA testing utilized for screening is aimed at identifying a specific mutation (C282Y) that, when found in the homozygous state, can be the cause of primary iron overload.  The test is designed to identify this mutation in any DNA sample, regardless of the characteristics of the individual being tested (e.g., race or ethnicity).  Although the prevalence of iron overload and the mix of mutations responsible for the disorder may vary by race, the test should reliably identify the target mutation.  One exception to this might occur if the presence and/or frequency of unknown polymorphisms were found to vary by race/ethnicity (or some other factor).  In reality, however, it would be difficult for laboratories to thoroughly examine this possibility in all populations to which testing may be offered.

Gap in Knowledge: Allele frequency by race/ethnicity.  Variation in allele and polymorphism frequencies by race/ethnicity have been well described in the literature for some population groups, while others have much less information available.  Laboratories should make efforts to report HFE allele frequencies as well as polymorphisms that could interfere with the analysis. 

References

ACMG/CAP Molecular Genetics Survey Sets (1998, 1999, 2000, 2001, 2002)  College of American Pathologists, Northfield, IL.

Arya N, Chakrabrati S, Hegele RA, Adams PC.  1999.  HFE S65C variant is not associated with increased transferrin saturation in voluntary blood donors.  Blood Cells Mol Dis  25:354-357.

Barton JC, Acton RT.  2000.  Transferrin saturation phenotype and HFE genotype screening for hemochromatosis and primary iron overload: predictions from a model based on national, racial, and ethnic group composition in central Alabama.  Genet Test  4:199-206.

Barton JC, Rothenberg BE, Bertoli LF, Acton RT.  1999.  Diagnosis of hemochromatosis in family members of probands:  a comparison of phenotyping and HFE genotyping.  Genet Med  1:89-93.

Best LG, Harris PE, Spriggs EL.  2001.  Hemochromatosis mutations C282Y and H63D in ‘cis’ phase.  Clin Genet  60:68-72.

Beutler E, Gelbart T.  1997.  HLA-H mutations in the Ashkenazi Jewish population.  Blood Cells Mol Dis  23:95-98.

Beutler E, Griffin MJ, Gelbart T, West C.  2002.  A previously undescribed nonsense mutation of the HFE gene.  Clin Genet  61:40-42.

Beutler E, West C, Gelbart T.  1997.  HLA-H and associated proteins in patients with hemochromatosis.  Mol Med  3:397-402.

Bradbury R, Fagan E, Goodson S, Steer K, Payne SJ.  1999.  New mutations in the HFE gene for haemochromatosis.  J Med Genet  36:S96.

Cullen LM, Gao X, Easteal S, Jazwinska SC.  1998.  The hemochromatosis 84G-->A and 187C-->G mutations: prevalence in non-Caucasian populations.  Am J Hum Genet  62:1403-1407.

de Villiers JN, Hillermann R, Loubser L, Kotze MJ.  1999.  Spectrum of mutations in the HFE gene implicated in haemochromatosis and porphyria.  Hum Mol Genet  8:1517-1522, Erratum in: Hum Mol Genet  1999 Sep;8(9):1817.

Dequeker E, Cassiman J.  2000.  Genetic proficiency testing in diagnostic laboratories – quality control is the message.  Am J Hum Genet  67:A274.

Feder JN, Gnirke A, Thomas W, Tsuchihashi Z, Ruddy DA, Basava A, et al.  1996.  A novel MHC class I-like gene is mutated in patients with hereditary haemochromatosis.  Nat Genet  13:399-408.

Gomez PS, Parks S, Ries R, Tran TC, Gomez PF, Press RD.  1999.  Polymorphism in intron 4 of HFE does not compromise haemochromatosis mutation results.  Nat Genet  23:272.  

Henz S, Reichen J, Liechti-Gallati S.  1997.  HLA-H gene mutations and haemochromatosis:  the likely association of H63D with mild phenotype and the detection of S65C, a novel variant in exon 2.  [Abstract]  J Hepatol  26(Suppl):57.

Jeffrey GP, Chakrabarti S, Hegele RA, Adams PC.  1999.  Polymorphism in intro 4 of HFE may cause overestimation of C282Y homozygote prevalence in haemochromatosis.  Nat Genet 22:325-326.

Le Gac G, Mura C, Ferec C.  2001.  Complete scanning of the hereditary hemochromatosis gene (HFE) by use of denaturing HPLC.  Clin Chem  47:1633-1640.

Liechti-Gallati S, Varga D, Reichen J.  1999.  Screening for hemochromatosis in Switzerland: detection of a new pathogenic mutation and two additional variants in exon 2 of the HFE gene.  Eur J Hum Genet 1999:122.

Monaghan KG, Rybicki BA, Shurafa M, Feldman GL.  1998.  Mutation analysis of the HFE gene associated with hereditary hemochromatosis in African Americans. Am J Hematol  58:213-217.

Noll WW, Belloni DR, Stenzel TT, Grody WW.  1999.  Polymorphism in intron 4 of HFE does not compromise haemochromatosis mutation results.  Nat Genet  23:271-272.

Oberkanins C, Moritz A, de Villiers JN, Kotze MJ, Kury F.  2000.  A reverse-hybridization assay for the rapid and simultaneous detection of nine HFE gene mutations.  Genet Test  4:121-124.

Palomaki GE, Haddow JE, Bradley LA, Richards CS, Stenzel TT, Grody WW.  2003.  Estimated analytic validity of the HFE C282Y mutation testing in population screening:  The potential value of confirmatory testing.  Genet Med  submitted for publication.

Piperno A, Arosio C, Fossati L, Vigano M, Trombinin P, Vergani A, et al.  2000.  Two novel nonsense mutations of HFE gene in five unrelated Italian patients with hemochromatosis.  Gastroenterology  119:441-445.

Pointon JJ, Wallace D, Merryweather-Clarke AT, Robson KJ.  2000.  Uncommon mutations and polymorphisms in the hemochromatosis gene.  Genet Test  4:151-161.

Sohda T, Yanai J, Soejima H, Tamura K.  1999.  Frequencies in the Japanese population of HFE gene mutations. Biochem Genet  37:63-68.

Somerville M, Sprysak KA, Hicks M, Elyas BG, Vicen-Wyhony L.  1999.  An HFE intronic variant promotes misdiagnosis of hereditary hemochromatosis.  Am J Hum Genet  65:924-926.

Steinberg KK, Cogswell ME, Chang JC, Caudill SP, McQuillan GM, Bowman BA, et al.  2001.  Prevalence of C282Y and H63D mutations in the hemochromatosis (HFE) gene in the United States.  JAMA  285:2216-2122.

Thorstensen K, Kvitland M, Asberg A, Hveem K.  2000.  5569G/A polymorphism of the HFE gene: no implications for C282Y genotyping in a hemochromatosis screening study of 65,238 individuals.  Genet Test  4:147-149.

Wallace DF, Dooley JS, Walker AP.  1999.  A novel mutation of HFE explains the classical phenotype of genetic hemochromatosis in a C282Y heterozygote.  Gastroenterology  116:1409-1412.

Worwood M, Jackson HA, Feeney GP, Edwards C, Bowen DJ.  1999.  A single tube heteroduplex PCR for the common HFE genotypes.  Blood  94:A405.


 

Updated on August 13, 2004