NEWBORN SCREENING QUALITY ASSURANCE

The Newborn Screening Quality Assurance Program (NSQAP) at the CDC in Atlanta, Georgia, provides services for laboratories that use DBS specimens to perform newborn screening tests. The mission of this program is to improve interlaboratory comparability and to work toward interlaboratory standardization. Current participants include newborn screening laboratories, confirmatory testing laboratories that use DBS tests, diet-monitoring laboratories, and manufacturers of testing products. The NSQAP interacts with state-affiliated newborn screening programs and the manufacturing community to (1) maintain laboratory methods, (2) evaluate and distribute reference and QC materials, (3) evaluate QC systems available to user laboratories, (4) conduct training and/or provide consultative services, and (5) develop mechanisms for the voluntary evaluation of laboratory performance, the transfer of new and improved technology, and the evaluation of programs screening for hypothyroidism, phenylketonuria, other inborn metabolic disorders, and hemoglobinopathies. Figure 1 shows the number of program participants in the QC and PT program components for each disorder. The NSQAP provides QC materials, PT services, and technical support to 64 domestic screening laboratories, 20 manufacturers of diagnostic products, and 122 laboratories in 33 foreign countries.

QA programs enable screening laboratories to achieve high levels of technical proficiency and maintain continuity despite changes in commercial assay reagents while maintaining the high-volume specimen throughput that is required. Laboratories that misclassified a PT specimen are provided immediate notification and consultation to resolve the analytical problem. Through interactive efforts, NSQAP and screening laboratories continually strive to improve program services in order to better meet the growing and changing needs of newborn screening activities in the public health community and to help ensure equivalent high-quality testing by these programs nationwide. Besides the two DBS distribution components, the program contains a filter paper evaluation component for the QA of this product. Methods developed by the NSQAP are used to evaluate and compare different production lots of filter paper from commercial sources.

Although the DBSs were used successfully in semiquantitative assays (i.e., bacterial inhibition assays) for newborn screening, the DBS matrix encountered mixed reports of adequate performance when quantitative assays for congenital hypothyroidism were introduced. A significant problem was that the variance contributions of the filter-paper matrix were not realized fully, and consequently they were not minimized and contained for use in quantitative assays. Standardizing the filter-paper matrix for performance parameters has contributed substantially to our confidence in its reliability and to the acceptance of DBSs as a specimen matrix for a variety of new applications and technologies.

The collection of DBS specimens for newborn screening and other related applications requires special grades of commercially manufactured paper as a unique whole-blood collection matrix. In the United States, only papers approved by the Food and Drug Administration (FDA) are acceptable for blood collection for clinical tests. Critical to proper and effective use of this matrix is an ongoing assessment and evaluation of new production lots as they are manufactured, as well as the monitoring of problems identified with individual lots in use by state-affiliated newborn screening programs. Since the paper-punch size selected from a blood spot is used as a volumetric sample for quantitative analysis, a high degree of uniformity is essential.

The NSQAP and the two FDA-approved manufacturers have reached an agreement whereby the NSQAP is provided with statistically valid sample sets from each new production lot of filter paper for evaluation by standardized procedures⁸ before the new lot is released to the user community. Each year the NSQAP evaluates, with the extensive cooperation of manufacturers (Schleicher & Schuell, Inc., Keene, NH, and Whatman, Inc., Fairfield, NJ), new lots of paper approved by FDA for blood collection. Each manufacturer is responsible for establishing its own evaluation laboratory that uses the same procedures as the ones used by the NSQAP. After all parties conducting independent evaluations of the filter-paper lot agree that the lot is acceptable, the lot is released for distribution. The independent evaluations by the NSQAP are an impartial and voluntary service offered as a function of the QA program and do not constitute preferential endorsement of any product over other specimen-collection papers approved by FDA. The specific role of the NSQAP in this evaluation process includes (1) evaluating manufactured lots of filter-paper products for physical parameters, serum absorption volume, homogeneity, and absorption time; (2) monitoring the performance of new production lots relative to previous lots; (3) maintaining criteria for certification of paper lots used by screening laboratories; (4) maintaining a controlled historical storage center of papers previously approved for newborn screening; and (5) reporting the outcomes of these evaluations to the user community. Standardization is required both for within-lot and lot-to-lot performance of filter paper to help ensure reliable analytical transitions and comparability of calibrators, patient specimens, and QC materials. The volumetric aliquots of serum contained in a punch size from a DBS are affected by several variables, including the paper thickness, the volume of blood applied, the humidity, the printing process used to produce blood-collection forms, the handling during specimen collection, the blood absorption time, and the storage environment for the paper.^9,10

In the future, other commercial sources of FDA-approved filter paper for specimen collection may become available in the United States. Theoretically, if the performance criteria⁸ are met, the transition between commercial sources of paper should not be different from the transition between lots from the same source. Because screening programs have limited experience with the impact of multiple sources of filter paper on performance, they should sort the specimens by filter paper source for analysis and closely monitor performance parameters until they are confident in the new product. Soon the screening community could be coping with calibrators, patient specimens, and control materials on filter paper from different commercial sources. The screening programs must be cautious and include monitoring for each parameter in their QC operations.

Most of the specimen-collection facilities have selected DBS specimens collected by a heel stick as the method of choice for newborn screening. However, procedures are also available for applying blood collected in capillary tubes and by dorsal vein puncture onto the preprinted circles of filter paper. Although these methods are not the preferred choice, they are now considered to be rational alternatives to direct application from the newborn's heel-stick site.⁷ The DBS has the advantage of being simple and easy to handle as well as posing fewer biological hazards than liquid specimens.

A national standard has been developed for the uniform collection of quality DBS specimens from newborns.⁸ This standard specifically defines how to collect blood onto filter paper, specifications for the specimen matrix, specimen shipping requirements, and other parameters. Screening laboratories routinely monitor the quality of collected specimens against their established criteria of acceptance and rejection. The primary justification for refusing to analyze a specimen and declaring it unacceptable is that its analysis may yield unreliable, misleading, or clinically inaccurate values for a particular analyte. Since, by this definition, unacceptable specimens give no usable information, such specimens should not be analyzed; and those responsible for collecting the specimens should be informed so that an acceptable specimen can be obtained as soon as possible.⁸ The collection of unacceptable specimens delays the analysis and potentially the treatment of affected newborns.

The NSQAP prepares and distributes to laboratories worldwide more than 250,000 DBSs per year. The manufactured DBS materials must simulate, as closely as possible, the actual specimens for the assay systems. Dried-blood materials that are prepared for QC and PT are certified for homogeneity, analyte accuracy, stability, and suitability for all assays from different commercial sources. The NSQAP distributes DBS materials used in screening tests for congenital hypothyroidism, PKU, galactosemia, congenital adrenal hyperplasia, homocystinuria, maple syrup urine disease (MSUD), biotinidase deficiency, and hemoglobinopathies. These materials must be stable under the routine conditions of shipment, laboratory logistics, analysis, and storage, and must be homogeneous enough that their variance will not contribute significantly to the analytical variance of the methods and their detection outcomes. The concentration of analyte in the control materials must cover the range of normal and abnormal specimens tested by the screening process and include a low-level analyte specimen with which to monitor the analytical sensitivity of the methods.⁷

The NSQAP prepares "zero-base" whole-blood pools for DBS materials by gently mixing washed, outdated Red Cross-packed red blood cells with clean-filtered serum to yield a whole-blood pool with a hematocrit of 55%. Portions of the pool are enriched with the desired analyte at predetermined concentration levels. Each portion is then uniformly applied to approved filter-paper cards in 100-µL aliquots, air-dried horizontally overnight, packed in sealed bags with desiccant, and stored at -20 ^oC. Blood collected from any source, including cord blood, that is prepared for use in the production of QC and PT materials must be negative for HIV, hepatitis, and other infectious agents. Documentation of the source, composition, homogeneity, and stability of all DBS materials as well as the procedure(s) used to assign their expected or target values are available upon request from the NSQAP. Specimens from newly prepared production lots of DBS materials are checked for stability and tested to ensure their homogeneity and the accuracy of their assigned concentration values.

To ensure that laboratories receive representative sheets of the production batch, the NSQAP uses a random number table to select the set of DBS sheets for each laboratory. QC shipments are distributed semiannually and include the blood-spot sheets, instructions for storage and analysis, and data report forms. Data from five analytical runs of each lot and shipment are compiled in the midyear and annual summary reports that are distributed to each participant. The reported QC data are summarized and show the analyte by series of QC lots, the number of observations, mean values, and the standard deviations by kit or analytical method. In addition, NSQAP used a weighted linear regression analysis to examine the comparability by method of reported versus enriched concentrations. Results of the linear regression analyses (Y-intercept and slope) for all lots within an analyte set are summarized in the reports. The mean value and the within-laboratory and total standard deviations are calculated for each concentration within a QC lot for a specific analyte. The summarized QC data provide information about method-related differences in analytical recoveries and method-related biases. Because each QC lot series is prepared from a single batch of hematocrit-adjusted, nonenriched blood, the endogenous concentration of a given analyte is the same for all specimens in a lot series. The Y-intercept of the regression analysis for reported values provides a measure of the endogenous analyte concentration. For amino acid-enriched DBS materials, participants measure the endogenous concentration levels by analyzing the nonenriched QC lots (the base pool). The values for the Y-intercept and the base pool are similar for most methods. Ideally, the slope should be 1.0, and most slopes calculated for reported data are close to this value. Because the endogenous concentration is the same for all QC lots within a series, it should not affect the slope of the regression line among methods. Generally, slope values substantially different from 1.0 indicate that a method has an analytical bias. Figure 2 shows an example of mean values of reported results for phenylalanine measurements by different methods used for linear regression analysis. These data routinely compiled by NSQAP help participants understand the performance of different methods and to select appropriate test methods and kits in the future.

For the PT component, designing evaluation materials that fairly assess screening laboratory performance is difficult because of the variable cutoff values both within and among laboratories. The NSQAP analyzed this problem and concluded that the only equitable method by which all laboratories could be evaluated, is one that is based on the first decision level (cutoff) for separating test results that require follow-up testing from negative test results that do not. This initial decision (cutoff) value should be an important part of the routine reporting scheme for all participants even though it will differ among laboratories. All PT panels contain five blind-coded specimens, each consisting of 100-µL blood spots. Specimens in the PT panels either contain endogenous levels or are enriched with predetermined levels of thyroxine, thyroid-stimulating hormone, phenylalanine, total galactose, 17 "-hydroxyprogesterone, leucine, or methionine. Special separate panels for biotinidase deficiency and galactose-1-phosphate uridyltransferase deficiency are prepared with purchased blood from donors with enzyme deficiencies. The DBS specimens for sickle cell disorders and other hemoglobinopathies are prepared from residual cord blood provided to the QA program by the Alabama and Georgia newborn screening programs.

In quarterly reports the NSQAP summarizes the quantitative data and clinical assessments for all data received from PT program participants by the due date. Because some of the pools in a routine PT survey represent a unique donor specimen, differences in endogenous materials in the donor specimens are important parameters for assessing performance and method-related differences. Presumptive clinical classifications (based on quantitative data) of some specimens may differ by participant because of specific clinical-assessment practices. Only the qualitative assessments are reported for the sickle cell disorders and other hemoglobinopathies. Table 1 shows the results for phenotype and clinical assessment misclassifications reported for hemoglobinopathies in 1998.

For those participants who provide their cutoff values, the NSQAP applies these cutoffs in the final appraisal of the error judgment. The errors for qualitative assessments in the PT component are split into misclassifications and transcription errors. Transcription errors continue to be a major error component in the PT surveys. Transcription errors are monitored to provide an indication of attention to detail by laboratory personnel. In 1998 data, the number of false-positive misclassifications exceeded the number of transcription errors and false-negative misclassifications for most disorders. The NSQAP calculates the rates for false-positive classifications on the basis of the number of negative specimens analyzed; the false-negative classifications are based on the number of positive specimens analyzed. Screening programs are designed to avoid false-negative reports; this precautionary design, however, may be the cause of many of the false-positive classifications.¹¹ False-positive classifications are a cost-benefit issue and a credibility factor with follow-up programs.¹¹ False-positive rates should be monitored and kept as low as possible. Figure 3 shows the total number of classification and transcription errors made by domestic and foreign laboratories that screened for selected disorders in 1998. For most disorders, the number of errors was greatest for foreign laboratory participants.

The introduction of tandem mass spectrometry to newborn screening has greatly expanded the potential number of disorders detectable by routine newborn screening with the DBS specimen. Over 15 disorders could be added to the present screening profiles, some of which raise many issues about follow-up and treatment of affected newborns. The most important of these disorders appears to be medium-chain acyl-CoA dehydrogenase deficiency (MCAD), a defect in the oxidation of fatty acids. The reported prevalence of MCAD is 1 in 10,000.¹² However, no external quality assurance program is now available for the measurement of acylcarnitines by tandem mass spectrometry for detection of these disorders. Laboratories using tandem mass spectrometry will need to operate and document quality assurance efforts within their respective facilities according to the CLIA regulations for situations where no PT programs are available.¹³ A laboratory must have a system for verifying the accuracy and reliability of its test results at least twice a year. If a laboratory performs the same test on different instruments or methodologies, a mechanism must be established to evaluate and define the relationship between test results. The laboratory must have documentation of all quality assurance activities, including problems and corrective actions taken, and must maintain accurate record-keeping.¹³

DNA testing for cystic fibrosis (CF) is carried out by only a few state newborn screening programs, but there is a growing interest by other programs. Specimens used to monitor the quality of testing should closely simulate the actual routine clinical specimen. Historically, these materials have been prepared by analyte enrichment (or spiking of normal base biological matrices) or from materials obtained from donors afflicted with the disorder or genetic defect being tested for. Presently, two PT surveys (non-HCFA approved) operate with worldwide participation for QA of CF testing: the CAEN QA Survey (France) and the Human Genetics Society of Australasia's Newborn Screening Quality Assurance Program (New Zealand).¹⁴ The Australasian QA Program offers PT services for DNA testing for CF ()F508). However, the preparation method for DNA QA materials used in these programs does not appeal to some participants because detection depends on the compatibility of the primers used in the )F508 assay with the amplicons. Determining the number of DNA mutations screened for as part of CF detection and whether QA materials can effectively simulate these mutations are unanswered questions. Efforts are ongoing to improve these QA test materials so that they provide better method harmonization and better simulation of the CF mutations.

The development of simulated materials through specific DNA-probe spiking of specimens is problematic for assessing the equivalency of performance for all test systems. Materials that harmonize for all test systems are essential to performance assessments of DNA tests. This is a difficult challenge because of the underlying analytical variables; but until such PT materials with the required mutations are developed, meaningful performance data are difficult to gather for the CF test and other emerging DNA tests for newborn screening. One possible solution to this dilemma is the development of unique evaluation processes for DNA tests. Perhaps such a process could assume that a laboratory's performance on any one mutation test adequately measures its performance on all similar DNA testing performed during the PT assessment window. This and other possible options need study and development by experimental trials within the DNA-testing community.