FDA Home Page | CDRH Home Page | Search | CDRH A-Z Index | Contact CDRH

Document issued on: February 5, 2003

This document supersedes Guidance on Review Criteria for Assessment of Antimicrobial
Susceptibility Devices; Draft, March 8, 2000

U.S. Department of Health and Human Services Food and Drug Administration Center for Devices and Radiological Health Division of Microbiology Devices Office of In Vitro Diagnostic Device Evaluation and Safety

Preface

Public Comment:

Comments and suggestions may be submitted at any time for Agency consideration to Dockets Management Branch, Division of Management Systems and Policy, Office of Human Resources and Management Services, Food and Drug Administration, 5630 Fishers Lane, Room 1061, (HFA-305), Rockville, MD, 20852. When submitting comments, please refer to Docket No. 00D-0109. Comments may not be acted upon by the Agency until the document is next revised or updated.

For questions regarding the use or interpretation of this guidance contact Freddie Poole at (301) 594-2096 or by email FMP@cdrh.fda.gov.

Additional Copies:

Additional copies are available from the Internet at: http://www.fda.gov/cdrh/ode/631.pdf, or you may send a fax request to (301) 443-8818 to receive a hard copy of the document, or send an e-mail request to GWA@CDRH.FDA.GOV to request hard or electronic copy. Please use the document number (631) to identify the guidance you are requesting.

Table of Contents

I. Introduction

II. Background

III. The Least Burdensome Approach

IV. The Content and Format of an Abbreviated 510(k) Submission

V. Scope

VI. Device Description

VII. Risks to Health

VIII. Device History

IX. Study Design

A. Reference Method

X. Data Presentation

XI. Evaluating the Results of your Study

XII. Labeling

XIII. Removing Certain Labeling Limitations from Legally Marketed AST Devices

XIV. QSR Considerations

XV. Glossary

XVI. References

TABLE 1: Recommendations for Antimicrobial Susceptibility Devices

TABLE 2: Recommendations for the Removal of Limitations from Antimicrobial Susceptibility Devices

TABLE 3: Presentation of Summary Data for Both Challenge and Clinical Data

TABLE 4: Example of Reporting Format for Quality Control Data

TABLE 5: Sample Table Format for Device Performance

TABLE 5A: Sample Table Format for Device Performance, continued

TABLE 6: Presentation of Reproducibility Results by Organism

TABLE 6A: Presentation of Reproducibility Results by Organism and Site

TABLE 6B: Presentation of Reproducibility Results by Organism, Pooled Across Sites

TABLE 7: Report Format for Inoculum Density

TABLE 8: Number of VMJ Discrepancies as a Function of the Number of Resistant Organisms Tested

TABLE 9: Essential Agreement as Function of the Number of Evaluable Organisms Tested

Class II Special Controls Guidance
Document: Antimicrobial Susceptibility Test
(AST) Systems; Guidance for Industry and
FDA

I. Introduction

This guidance document was developed as a special control guidance to support the reclassification of the antimicrobial susceptibility test (AST) system, when the device is a system employing short-term incubation (less than 16 hours) from class III into class II (special controls). The device, as proposed, is intended to determine the in vitro susceptibility of bacterial pathogens from clinical specimens. This guidance is issued in conjunction with a Federal Register notice announcing the reclassification of the automated short-term incubation cycle AST system.

Following the effective date of this final reclassification rule, any firm submitting a 510(k) premarket notification for aan automated short-term incubation cycle AST system will need to address the issues covered in the special control guidance. However, the firm need only show that its device meets the recommendations of the guidance or in some other way provides equivalent assurances of safety and effectiveness.

II. Background

FDA believes that special controls, when combined with the general controls, will be sufficient to provide reasonable assurance of the safety and effectiveness of the automated short-term incubation cycle AST system. Thus, a manufacturer who intends to market a device of this generic type should (1) conform to the general controls of the Federal Food, Drug & Cosmetic Act (the Act), including the premarket notification requirements described in 21 CFR 807 Subpart E, (2) address the specific risks to health associated with the automated short-term incubation cycle AST system identified in this guidance and, (3) obtain a substantial equivalence determination from FDA prior to marketing the device (see also 21 CFR 807.85).

This special control guidance document identifies the classification regulations and product codes for the automated short-term incubation cycle AST system (Refer to Section V – Scope). In addition, other sections of this special control guidance document list the risks to health identified by FDA and describe measures that, if followed by manufacturers and combined with the general controls, will generally address the risks associated with these automated short-term incubation cycle AST systems and lead to a timely premarket notification [510(k)] review and clearance. This document supplements other FDA documents regarding the specific content requirements of a premarket notification submission. You should also refer to 21 CFR 807.87 and other FDA documents on this topic, such as the 510(k) Manual - Premarket Notification: 510(k) - Regulatory Requirements for Medical Devices,    http://www.fda.gov/cdrh/manual/510kprt1.html.

Under "The New 510(k) Paradigm - Alternate Approaches to Demonstrating Substantial Equivalence in Premarket Notifications; Final Guidance¹," a manufacturer may submit a Traditional 510(k) or has the option of submitting either an Abbreviated 510(k) or a Special 510(k). FDA believes an Abbreviated 510(k) provides the least burdensome means of demonstrating substantial equivalence for a new device, particularly once a special controls guidance document has been issued. Manufacturers considering modifications to their own cleared devices may lessen the regulatory burden by submitting a Special 510(k).

_____________________
¹ http://www.fda.gov/cdrh/ode/parad510.html

III. The Least Burdensome Approach

The issues identified in this guidance document represent those that we believe need to be addressed before your device can be marketed. In developing the guidance, we carefully considered the relevant statutory criteria for Agency decision-making. We also considered the burden that may be incurred in your attempt to comply with the guidance and address the issues we have identified. We believe that we have considered the least burdensome approach to resolving the issues presented in the guidance document. If, however, you believe there is a less burdensome way to address the issues, you should follow the procedures outlined in the A Suggested Approach to Resolving Least Burdensome Issues document. It is available on our Center web page at: 
http://www.fda.gov/cdrh/modact/leastburdensome.html

IV. The Content and Format of an Abbreviated 510(k) Submission

An Abbreviated 510(k) submission must include the required elements identified in 21 CFR 807.87, including the proposed labeling for the device sufficient to describe the device, its intended use, and the directions for its use. In an Abbreviated 510(k), FDA may consider the contents of a summary report to be appropriate supporting data within the meaning of 21 CFR 807.87(f) or (g); therefore, we recommend that you include a summary report. The report should describe how this special control guidance document was used during the device development and testing and should briefly describe the methods or tests used and a summary of the test data or description of the acceptance criteria applied to address the risks identified in this guidance document, as well as any additional risks specific to your device. This section suggests information to fulfill some of the requirements of 807.87 as well as some other items that we recommend you include in an Abbreviated 510(k).

Coversheet

The coversheet should prominently identify the submission as an Abbreviated 510(k) and cite the title of this class II special controls guidance document.

Proposed labeling

Proposed labeling should be sufficient to describe the device, its intended use, and the directions for its use. (Refer to Section XII for specific information that should be included in the labeling for devices of the types covered by this document.)

Summary report

We recommend that the summary report contain:

• Description of the device and its intended use. We recommend that the description include a complete discussion of the performance specifications and, when appropriate, detailed, labeled drawings of the device. (Refer to Section VI for specific information that we recommend you include in the device description for devices of the types covered by this guidance document.) You should also submit an "indications for use" enclosure.²
• Description of device design requirements.
• Identification of the Risk Analysis method(s) used to assess the risk profile in general as well as the specific device’s design and the results of this analysis. (Refer to Section VII for the risks to health generally associated with the use of this device that FDA has identified.)
• Discussion of the device characteristics that address the risks identified in this class II special controls guidance document, as well as any additional risks identified in your risk analysis.
• A brief description of the test method(s) you have used or intend to use to address each performance aspect identified in Sections IX, X, and XI of this class II special controls guidance document. If you follow a suggested test method, you may cite the method rather than describing it. If you modify a suggested test method, you may cite the method but should provide sufficient information to explain the nature of and reason for the modification. For each test, you may either (1) briefly present the data resulting from the test in clear and concise form, such as a table, or (2) describe the acceptance criteria that you will apply to your test results.³ (See also 21 CFR 820.30, Subpart C - Design Controls for the Quality System Regulation.)
• If any part of the device design or testing relies on a recognized standard, (1) a statement that testing will be conducted and meet specified acceptance criteria before the product is marketed, or (2) a declaration of conformity to the standard.⁴ Please note that testing must be completed before submitting a declaration of conformity to a recognized standard. (21 USC 514(c)(2)(B)). For more information refer to the FDA guidance, Use of Standards in Substantial Equivalence Determinations; Final Guidance for Industry and FDA, http://www.fda.gov/cdrh/ode/guidance/1131.html.

_____________________
² Refer to http://www.fda.gov/cdrh/ode/indicate.html for the recommended format.
³ If FDA makes a substantial equivalence determination based on acceptance criteria, the subject device should be tested and shown to meet these acceptance criteria before being introduced into interstate commerce. If the finished device does not meet the acceptance criteria and, thus, differs from the device described in the cleared 510(k), FDA recommends that submitters apply the same criteria used to assess modifications to legally marketed devices (21 CFR 807.81(a)(3)) to determine whether marketing of the finished device requires clearance of a new 510(k).  .
⁴ See Required Elements for a Declaration of Conformity to a Recognized Standard (Screening Checklist for All Premarket Notification [510(K)] Submissions), http://www.fda.gov/cdrh/ode/reqrecstand.html

If it is not clear how you have addressed the risks identified by FDA or additional risks identified through your risk analysis, we may request additional information about aspects of the device’s performance characteristics. We may also request additional information if we need it to assess the adequacy of your acceptance criteria. (Under 21 CFR 807.87(l), we may request any additional information that is necessary to reach a determination regarding substantial equivalence.)

As an alternative to submitting an Abbreviated 510(k), you can submit a Traditional 510(k) that provides all of the information and data required under 21 CFR 807.87 and described in this guidance. A Traditional 510(k) should include all of your methods, data, acceptance criteria, and conclusions. Manufacturers considering modifications to their own cleared devices should consider submitting Special 510(k)s.

The general discussion above applies to any device subject to a special controls guidance document. The following is a specific discussion of how you should apply this special controls guidance document to a premarket notification for an automated short-term incubation cycle AST system.

V. Scope

The scope of this document is limited to the following device as described in 21 CFR 866.1645 Fully automated short-term incubation cycle antimicrobial susceptibility system (product code: LON).

The classification identification below identifies the device as it existed at the time of reclassification:

An antimicrobial susceptibility test system is a device that incorporates concentrations of antimicrobial agents into a system for the purpose of determining in vitro susceptibility of bacterial pathogens isolated from clinical specimens. Test results obtained from short-term incubation (less than 16 hours) are used to determine the antimicrobial agent of choice to treat bacterial diseases.

This document does not apply to devices intended for testing anti-mycobacterial, anti-viral, or anti-fungal agents or devices intended for testing the susceptibility of fastidious organisms for which there is no NCCLS standard reference method for testing.

Devices classified in section 21 CFR 866.1640, Antimicrobial susceptibility test powder (product codes shown below) are not subject to this special control guidance. However, information in this document may be useful to manufacturers of these devices.

• LRG - instrument for auto reader & interpretation of overnight susceptibility systems
• JWY - manual antimicrobial susceptibility test systems
• LTT - panels, test, susceptibility, antimicrobial
• LTW - susceptibility test cards, antimicrobial

This document does not address antimicrobial disks for the disk diffusion method classified in section 21 CFR 866.1620. These devices are addressed in the guidance, "Review Criteria for Assessment of Antimicrobial Susceptibility Test Discs," http://www.fda.gov/cdrh/ode/testdisc.pdf.

VI. Device Description

You should identify your device by regulation and product code and a legally marketed predicate device.

In order to help FDA quickly view all the aspects of your device compared with the predicate, you should include a table that outlines the similarities and differences between the predicate and your device.

VII. Risks to Health

In the table below, FDA has identified the risk to health generally associated with the use of the automated short-term incubation cycle AST system addressed in this document. The measures recommended to mitigate the identified risk are given in this guidance document, as shown in the table below. You should also conduct a risk analysis, prior to submitting your premarket notification, to identify any other risks specific to your device. The premarket notification should describe the risk analysis method. If you elect to use an alternative approach to address the risk identified in this guidance document, or have identified risks additional to those in the guidance, you should provide sufficient detail to support the approach you have used to address that risk.

VIII. Device History

This guidance document ensures well-standardized, reliable, and reproducible performance evaluation for AST devices. Clinically, results from AST devices are useful for therapeutic guidance whenever the susceptibility of a bacterial pathogen may be unpredictable or when the infecting organism belongs to a species that may be resistant to antimicrobial agents of choice. Additionally, susceptibility testing is useful for monitoring development of new or emerging resistance to antimicrobial agents.

A determination of substantial equivalence to a legally marketed predicate device is based on intended use, design, energy used or delivered, materials, performance, safety, effectiveness, labeling, and other applicable characteristics. FDA believes performance of this type device is best established by comparison to the NCCLS standard reference methods (Ref. 1, 2) for each antimicrobial agent.

Laboratory procedures used for determining susceptibility of bacteria to antimicrobial agents have been developed and standardized over the past five decades. Historically, there have been two general procedures applied to susceptibility testing, i.e., dilution and diffusion. Other manual testing methods are based on modifications and refinements of older techniques such as gradient diffusion. Voluntary consensus standards on methodology and interpretive categories were implemented for susceptibility testing results that are antimicrobial agent, organism, or methodology dependent. NCCLS is the major organization in the United States that establishes voluntary standards and guidelines for standardizing and maintaining performance of laboratory susceptibility tests. A system has been established for continual assessment and upgrading of recommendations and addition of test criteria for new antimicrobial agents and older agents particularly when emerging resistance is recognized. A separate subcommittee was established in 1986 to standardize methods (Ref. 1, 2) for developing in vitro susceptibility testing criteria. These methods are also used by the pharmaceutical industry for developing new antimicrobial agents.

The NCCLS standard reference methods use 16-24 hours incubation for aerobic bacteria and 48 hours for anaerobic bacteria. Because shorter incubation times may provide clinical advantages, a number of manufacturers have developed automated procedures designed to generate results more rapidly, generally by the use of shortened incubation times (<16 hours). The results of reference overnight (16-24 hours of incubation) tests are accepted as standards for evaluating methods with a shortened incubation for the following reasons:

• All accepted reference and standard tests use 16 to 24-hour incubations for rapidly growing aerobic bacteria.
• The knowledge and experience for laboratory-clinical correlation has been based on 16 to 24-hour incubation tests.
• Where discrepancies have occurred, they have most often involved failure of shortened incubation procedures to detect bacterial resistance. (Ref. 4)

NCCLS has an Approved Standard M7 "Methods for Dilution Antimicrobial Susceptibility Tests for Bacteria that Grow Aerobically" (Ref. 1) that recommends a reference method for non-fastidious organisms. Other organisms that will not grow satisfactorily in (or on) unsupplemented Mueller-Hinton medium within 24 hours are considered fastidious organisms and may be included in NCCLS approved standards but generally with a different medium recommended for testing. If the FDA approved pharmaceutical antimicrobial agent package insert includes fastidious organisms (e.g., Streptococci, Haemophilus) with interpretive criteria and there is an NCCLS standard methodology, the recommendations for performance assessment are similar, but the numbers necessary for review may vary. See Table 1 for recommendations.

A susceptibility result may suggest that an uncomplicated bacterial infection can be effectively treated if AST device results indicate that the bacterial isolate is susceptible to the antimicrobial agent selected. The inability of a new device to produce a susceptible result for an organism that is susceptible to an antimicrobial agent by the reference method is considered a "major discrepancy." In this case, if a new device yields a resistant result for an organism, the antimicrobial agent may not be made available for treatment when in fact it could be an effective choice. Such major discrepancies can lead to utilization of broad-spectrum agents and needlessly accentuate the pressure for selection of resistant flora. Conversely, the inability to detect resistance is assessed by the "very major discrepancy rate," since therapy with that antimicrobial agent may lead to treatment failure, particularly for serious infections or altered host conditions. Accurate detection of resistance is important for clinical effectiveness and for monitoring emergence of resistance in the community.

Resistance to antimicrobial agents can generally be classified into four basic mechanisms:

• production of antimicrobial-inactivating enzymes
• substitution of antimicrobial-insensitive targets
• alteration in the target site
• decreased drug entry.

The time needed for expression of resistance varies with different combinations of antimicrobial agents and organisms that have different mechanisms of resistance. The delay of expression of resistance can range from one to many hours. Studies comparing results of shorter incubation test results with conventional 16 to 24 hour incubation methods have documented the difficulties of detecting delayed resistance expression. Manufacturers of devices with shortened incubation times have adopted a variety of strategies to bring these results as close to conformity as possible when compared with results using the NCCLS standard reference methods. Examples of these strategies include:

• the use of higher concentrations of bacteria in the inoculum
• adjusting media to optimize resistance detection
• the use of sophisticated optical scanning devices with computer assisted reading determinations.

Other devices can detect resistance by the presence or absence of a genotype associated with in vitro resistance.

IX. Study Design

Table 1 outlines, in tabular form, FDA’s recommendations for the number of sites, and type and numbers of organisms for testing.

Generally, FDA recommends that you establish the performance characteristics of your AST device by agreement with the NCCLS standard reference method for each antimicrobial agent and the organisms intended for testing. Because variations in test procedures can affect performance, we believe you should conduct agreement studies on all of the procedural options included in the directions for use section of the package insert. Such procedural options include, but are not limited to, inoculation preparation methods and reading of results, for example:

• growth inoculation preparation method
• direct colony suspension inoculation method
• visual reading
• automated readings.

You should also address all possible combinations of these procedural options.

You should have a testing protocol describing testing procedures for both the reference method and new device. The protocols should include the exact procedures to follow for the reference and new device.

We recommend that you include your testing protocol in your 510(k). The protocol should describe your study design and contain the type of quality control recommended and the procedures for the reference and test method. The procedures should include:

• method(s) of inoculation
• media used
• incubation conditions
• recommendations for the selection of organisms.

For a valid comparison, FDA believes you should not deviate from the NCCLS standard reference method procedure. As stated above, you should include testing procedures for the new device with all procedural options, or possible combinations of these options that are included in the instructions of the package insert. This is especially important for certain organism-antimicrobial agent combinations that are affected by variations in inoculum and have growth patterns that may be interpreted differently when read visually or automatically. Your study design should include options for different methods of inoculation or additional dilutions of the inoculum suspension for certain groups of organisms (e.g., Proteus sp.), if any options are in the instructions for use.

Where appropriate to the device design and instructions for use, performance data using alternate methods of reading and/or inoculation procedures should include test results for all challenge, quality control, and reproducibility studies.

A.Reference Method

The reference method plates should contain two-fold dilutions of the antimicrobial agent for which FDA clearance is sought. The selection of dilutions should include the FDA and NCCLS interpretive standards with one two-fold dilution above the resistant threshold and several below the susceptible threshold to provide a range for evaluating the results. For example, if interpretive standards are: 1, susceptible (S); 2, intermediate (I); 4, resistant (R), then the reference plate should include serial two-fold dilutions between 0.25  µg/mL and 8 µg/mL. Including one concentration above the resistant threshold provides data for essential agreement (EA) evaluations. We believe including concentrations more than two dilutions above the resistant concentration provides little evaluable data. Including dilutions below the susceptible category provides more results that are on-scale, and therefore, available for inclusion in the calculations of the EA of evaluable results. The table format samples, Table 5 and 5A, show our recommendations for determining graphically whether a result is evaluable or not.

The reference method performed at the clinical sites may produce errors when testing clinical organisms. Manufacturers may avoid this problem with the challenge and reproducibility assessments by comparing the new device results to a pre-determined expected value, instead of to the results of the reference method performed at the sites. If expected values are not used for the challenge organisms and there is a concern about the variability of the reference method, FDA suggests performing the initial reference test in triplicate for all clinical and challenge organisms. This will help reduce any potential bias. Special care should be taken in the preparation of all reference plates, since the reference result will be used in the final analysis.

B. New Device

Both the new device and the reference panel should include a sufficient number of serial two-fold dilutions around the susceptible and resistant thresholds. For a quantitative minimal inhibitory concentration (MIC) device, the concentrations tested should include at least five two-fold dilutions that surround the susceptible and resistant thresholds of the antimicrobial agent as described above.

C. Organism Selection

You should select organisms for the comparative study that represent the clinical indications of the antimicrobial agent and are within its spectrum of activity according to the Microbiology and Indication and Usage Sections of the FDA approved pharmaceutical antimicrobial agent package insert, and the most recent NCCLS M100 (Ref. 3) Informational Supplement. (See NCCLS M100 Table 1 "Suggested Groupings of U.S. FDA-Approved Antimicrobial Agents That Should be Considered for Routine Testing and Reporting of Non-fastidious Organisms by Clinical Microbiology Laboratories" and Table 1A for fastidious organism recommendations). You should include organisms for which clinical efficacy and in vitro activity have been demonstrated.

A 50% susceptible, 50% resistant distribution within species would be ideal, but such a distribution may be rare when sequential clinical isolates are tested. You should avoid using the same organism from multiple sources and repeat isolates obtained less than three days apart from the same patient. Organisms with known mechanisms of clinically significant resistance should be included in the comparison study as either fresh or selected stock and challenge organisms.

The following example may help to clarify the types of organisms that we recommend you test. If the antimicrobial agent has been shown to be active against Enterobacter cloacae, Klebsiella oxytoca, and Citrobacter spp., both in vitro and in clinical infections, then all Enterobacteriaceae routinely isolated would be relevant for testing.

We recommend you avoid testing organisms that are not included in the Microbiology and Indication and Usage Sections of the FDA approved pharmaceutical antimicrobial agent package insert, because we do not believe such testing provides useful information. If, for example, Pseudomonas spp. are not indicated, they should not be selected for testing.

There are situations where the spectrum of activity (i.e., resistance) of the antimicrobial agent for certain organisms has not been demonstrated in bacteriological or clinical studies. In this instance, the FDA approved pharmaceutical antimicrobial agent package insert has only a susceptible interpretation. Since only organisms in the susceptible category are available, your labeling should recommend that results other than susceptible be referred to a reference laboratory for further analysis. In the event that resistant strains become available, they can be evaluated later, after which you may need to submit a new 510(k).

FDA discourages testing rare organisms for which antimicrobial agents are approved for use, since sufficient data for the rare organism is usually difficult to acquire in a clinical setting. Refer to Tables 1 and 1A in the NCCLS Approved Standard (Ref. 3), for recommended organisms to include or exclude in an evaluation.

1. Fresh clinical organisms

You should include organisms isolated from routine cultures processed in the clinical laboratory study site in the 7 days preceding testing. You should also include all isolates in the appropriate testing group as indicated for testing with the antimicrobial agent in the test device. You should perform the reference method in parallel with the new device.

2. Clinical stock organisms

Generally, each site has its own collection of infrequently isolated or less common organisms. These are organisms saved because of their unique growth or resistance patterns. This selection may be used to enhance the clinical isolates, but should not comprise more than 50% of any group of organisms or the total number tested. You should include these in the study as necessary to incorporate a wider variety of genus and species and also to augment the number of resistant organisms tested.

3. Challenge organisms

You should select challenge organisms from the organisms listed in the Microbiology and Indication and Usage Sections of the FDA approved pharmaceutical antimicrobial agent package insert. The challenge organisms should fall within the spectrum of activity of the antimicrobial agent. The selection of these isolates should favor resistant strains and include organisms for which the antimicrobial agent’s MIC is on-scale. If interpretive criteria are 4 (S), 8 (I), > 8 (R), then we believe organisms with known results in all dilutions between 0.25 and 32 µg/mL are appropriate selections. These challenge organisms are meant to demonstrate whether a device can reliably detect intermediate and resistant organisms. These organisms may be available from the Centers for Disease Control and Prevention (CDC) or a reference laboratory that collects and characterizes strains based on their resistance patterns or particular uniqueness. You may add a selection of organisms that were not used in the developmental stages of the antimicrobial agent algorithm for susceptibility testing, if they are clinically indicated organisms for in vitro testing as stated in the FDA approved pharmaceutical antimicrobial agent package insert.

If the organisms have been characterized phenotypically with repeated NCCLS standard reference method testing, these consensus results can be used as the "expected results." If the "expected result" is not known, you should perform MIC testing using the reference method before using them in the evaluation. You may do so internally or at an outside site. Only the reference method results should be used to determine the expected results.

If the challenge organisms have known expected values, reproducibly obtained using the NCCLS standard reference method, the clinical site need only perform testing with the new device. This will reduce the burden at the clinical site. You should code the set with the expected results, mask it and send it to one site for performance testing on the new device. Alternatively, you could conduct performance testing of the challenge organisms on the new device and the reference method at the same time.

D. Quality Control

We recommend that you conduct the following quality control testing for both the reference method and the new device as well as with any procedural options given in the labeling of the new device:

• daily testing of all quality control organisms recommended by NCCLS and you
• periodic inoculum colony counts
• purity check of all organisms.

  1. Selection of Quality Control Organisms

  Please refer to the appropriate NCCLS Approved Standard (Ref. 1, 2, 3) for recommended methods and quality control organisms. The FDA approved pharmaceutical antimicrobial agent package insert will also provide the expected quality control range for each organism.

  The selection of antimicrobial concentrations should include a minimum of one two-fold dilution below the lowest dilution and one two-fold dilution above the highest dilution for the recommended quality control organism. For example, if the expected range is 1 - 4 µg/mL, the reference plate should include 0.5 - 8 µg/mL.

  In instances where the quality control organisms’ expected results are significantly above or below the interpretive categories, on-scale results may not be possible. In this case you should recommend organisms selected from internal testing of organisms with well-established expected values that have been used in product development or as part of release criteria for the antimicrobial agent in your device. You should have on-scale results for at least one quality control organism.

  If multiple quality control strains are used and one strain has results outside of the expected range on any given day, you should repeat the quality control strain with the out of range result. When the results are interpreted the next day and the repeat testing is within the expected quality control range, the study data from the previous test day is acceptable and may be included in the comparative summary. However, if the repeated quality control result is still outside the expected range, the data from the previous day’s testing is invalid.

  If multiple quality control strains are tested and there are results for more than one strain that are outside the expected results in the reference method on any test day, you should not include test data from that day. You should repeat quality control strains with the out or range results with the reference method. If quality control is still out of range, conduct an investigation to determine the cause of the aberrant result(s). Do not continue testing until the problem has been resolved.

  We believe that you should perform quality control testing with the selected organisms on the reference plate daily to ensure that the reference method and reference plates are in control for each day of comparative testing. We recommend that you perform quality control testing with these same organisms on the new device a sufficient number of times to demonstrate that the user will be able to achieve the same results in the recommended ranges. We recommend a minimum of 20 quality control test results per site.

  2. Inoculum density check

  The purpose of the inoculum density check is to ensure that the final test concentration of an organism will result in the concentration recommended in the reference method (broth dilution of approximately 5 x 10⁵ CFU/mL) and the new device. Some antimicrobial agents are affected by variance in the final inoculum and performance may be compromised. You should perform plate counts as recommended in the NCCLS M7 Approved Standard (Ref. 1) on all methods of inoculum preparation that are recommended in the package insert of the new device. Ideally, this should include all quality control isolates daily, isolates for reproducibility testing, and 10% of fresh isolates.

  In the broth dilution test, you should perform plate counts (colony count study) directly from the inoculated panel to ensure the time period from the initial inoculum adjustment to the final time of inoculation has not adversely affected the inoculum density. For a non-broth device, you should perform a colony count determination immediately before conducting the test.

  There may be alternative approaches for this type of quality control if the inoculum method uses a spectrophotometric device. This type of device can be validated separately. You should provide adequate information to demonstrate that the colony count study described above is not necessary. However, if a non-spoectrophotometric method is used, it is also the manufacturer’s responsibility to provide adequate information to demonstrate with a study as recommended above that the inoculum is reproducible and in the expected range. The study should demonstrate that the inoculum for the ATCC 25922 Escherichia coli is in the expected range of 3-7 x 10⁵ CFU/mL. The study should also demonstrate that the inoculum method for the new device provides the same range as the reference method of inoculation with all organism groups. The calculations may be different if the inoculum density for the new device is different from the recommended reference inoculum.

  3. Purity check

  The purity check is necessary for broth dilution procedures to detect mixed cultures that may cause aberrant results. As recommended in the NCCLS M7 (Ref. 1) Approved Standard, you should conduct these checks after inoculation of the new device or reference plate. You should perform purity check plates for all inocula used for the reference method and the new device.

E. Reproducibility

You should test a minimum of 25 selected organisms with known on-scale results for which the antimicrobial agent is indicated. These may be challenge organisms or other organisms with known results. We recommend that you code organisms and send the organisms to three sites for testing, one time at each site on the new device only. Since this study design will not produce variability within sites, you should provide internal summary data to demonstrate this.

An alternative reproducibility assessment may be performed using 10 selected organisms with known results on-scale. You should not use the quality control isolates if they are not on-scale. These 10 organisms should be tested at each site on three separate days in triplicate with a different inoculum prepared for each test (27 results per isolate). Using this study design, you should calculate reproducibility for within-site (intra-site), for each site, and between-sites (inter-site). See also the table format samples for presenting your reproducibility results, Tables 6A and 6B.

Personnel at each site in the study should perform the same reproducibility study for all inoculum preparation methods and/or reading options recommended in the package labeling. See also Table 1 which outlines in tabular form, FDA’s recommendations for reproducibility testing.

X. Data Presentation

We recommend that you provide summary data for comparative performance, reproducibility, and quality control.⁵ We have provided several table format samples at the end of this guidance as examples of how to present your results.

A. Comparative Performance Data

Tables 5 and 5A are table format samples that summarize performance using:

• Essential Agreement (EA)
• Category Agreement (CA)
• Essential Agreement of evaluable results
• Major Discrepancy (maj)
• Minor Discrepancy (min)
• Very Major Discrepancy (vmj)

The formulas for calculating the percent agreement and the discrepancies listed above are included in Table 5A and in the glossary section.

1. Clinical – fresh and stock

A list of organisms tested should be presented in chart format for each site, identifying the numbers that are stock and fresh for each genus or species. A line listing for all organisms with MIC and/or category result discrepancies between the reference method and the new device should be presented to include genus or species, site, reference method result, test result, type of error, method of inoculation and reading, if applicable. You should submit summary data for all organisms and all study sites combined using Table 5A. This table can also be used to summarize all organisms by site. You should also provide summary data by organism using Table 5.

2. Challenge organisms

You should present results from challenge organisms with the comparison to the expected value or reference result performed at the time of testing. The table format sample in Table 5A is suitable for this purpose. The formulas for calculating the percent agreement and discrepancies are included in Table 5A and in the glossary section.

You should also present all methods of inoculation or reading of results separately. You may wish to use the table format sample in Table 5A for this as well.

We recommend that you also present a line listing of all discrepancies, including the name of the organism, site, reference method result, test result, type of error, and method of inoculation and reading, where applicable.

3. Challenge plus clinical organisms

We recommend that you also present summary data for the challenge data combined with the clinical data. You may wish to use the table format sample Table 5A for this purpose. If there appears to be trending for a particular group of organisms (e.g., Staphylococcus spp., Pseudomonas spp.) such as that observed in the clinical data presented in Table 5 of this guidance, you may present these groups separately. See also Table 3 for an additional table format sample you may wish to use for presenting the EA, CA, discrepancies, and evaluable results in a concise manner for both challenge and clinical data. Table 3 will not demonstrate trending like Table 5, but it will provide an overview for ease in selecting the appropriate organisms for inclusion in the final analysis.

B. Quality Control

Table 4 gives an example of how you should present quality control strain results. We recommend a minimum of 20 test results per site for each method of inoculation and/or reading included in the package insert. You should present both initial and repeat quality control results with an explanation of the action taken for all out-of-range test results.

C. Reproducibility

You should present reproducibility data for all procedural options. If you used the 25-organism study design, you may wish to use the table format sample in Table 6 for this purpose. You should also provide a summary of the internal studies demonstrating the variability across repetitions of the same organism. If you used the 10-organism study design, you may wish to use the table format sample in Table 6A and 6B for this purpose.

With multiple procedural options, the presentation of data is the same for each option.

________________
⁵ In order to meet the requirements of section 21 CFR 807.87(i), you must include in your 510(k), a financial certification or disclosure statement or both, as required by 21 CFR Part 54. Please refer to Guidance for Industry: Financial Diclosure by Clinical Investigators, issued 03/20/2001, http://www.fda.gov/oc/guidance/financialdis.html.

XI. Evaluating the Results of your Study

The following are recommendations for evaluating the results of your study. The quality control and reproducibility results should also be considered when assessing comparative performance of the reference method and the new device.

A. Fresh, Stock, and Challenge Organisms

Using the table format sample given in Table 5 will help you and FDA visualize discrepancies and trending by organism. However you choose to present the results, you should include only those organisms that would be routinely tested for the antimicrobial agent. For example, you should not include Pseudomonas spp. results for antimicrobial agents that have indications only in the Enterobacteriaceae group, or for Enterococcus, if the indications only include Staphylococcus spp. An additional purpose for the recommendation to use these tables is to identify the on-scale or evaluable test results based on the interpretative criteria of the antimicrobial agent and the concentrations tested on both the reference method and new device.

You should pay particular attention to the organisms with clinical utility and within the spectrum of activity of the antimicrobial agent as shown in the Microbiology and Indication and Usage Sections of the FDA approved pharmaceutical antimicrobial agent package insert. If the EA and CA that you obtain for the organisms listed in the FDA approved antimicrobial agent labeling are below 90%, we recommend that you add a limitation statement to your labeling and consider conducting a future study to support acceptable performance.⁶ This kind of limitation statement is not necessary for organisms (genus or species) for which the antimicrobial agent has no clinical utility or is inactive and has not been approved for use by FDA (e.g., cefdinir with Enterococcus and Pseudomonas sp.). If these are tested on a panel of antimicrobial agents, NCCLS recommends not reporting results.

You should also evaluate the overall performance of the AST device in the clinical studies for the ability to detect resistance. The use of challenge and stock organisms may be of particular importance in selecting organisms with resistance.

Tables 8 and 9 show the discrepancy rate and minimum acceptable EA rate we recommend. Discrepancy rates will vary depending on the proximity of the MIC of the organisms tested to the interpretative categories. FDA considers the following to be acceptable performance for the clinical data for AST devices for all organisms appropriate for testing:

• Percent essential and category agreement > 89.9 %. A CA of < 90% may be acceptable under certain circumstances (e.g., very good EA of the evaluable test results with the majority of the discrepancies as minor discrepancies).
• A maj rate of 3% based on the number of susceptible organisms tested.
• A vmj rate based on the number of resistant organisms tested. Table 8 lists the numbers of very major discrepancies as a function of the total number of resistant organisms tested with proposed statistical criteria for acceptance that include an upper 95% confidence limit for the true vmj rate of 7.5% and the lower 95% confidence limit for the true vmj 1.5%.
• Growth failure rates in the system < 10% for any genus or species tested.

B. Quality Control

Test results on the new device for the recommended quality control isolates should be within the expected range 95% of the time. In rare events, the expected result with the new device may not agree with the NCCLS recommended ranges for an antimicrobial agent. In this case, you should submit additional data following NCCLS recommendations in M23 "Development of In vitro Susceptibility Testing Criteria and Quality Control Parameters." (Ref. 5). These data should demonstrate the reproducibility of the newly submitted range, plus supportive data showing that all parameters of the test method are in control. These data should include all quality control parameters (e.g., inoculum density check). You should include a statement in the product insert that alerts the user to your unique quality control range. Quality control results that are frequently out of the recommended range will also require a closer scrutiny of the other data to determine if there is a similar trend that might affect clinical results.

If one procedural option provides quality control results that are not accurate for any particular antibiotic while another procedural option produces accurate results (e.g., inoculum preparation, automated reading), you should include a limitation in the labeling stating that results should not be reported for that antibiotic when this particular procedural option is used.

C. Reproducibility

It is difficult for the FDA to determine substantial equivalence for a device if the results of the overall reproducibility study from all test sites for any antimicrobial agent show < 95% (+/- 1 dilution) agreement as compared to the mode. If there is a trending-bias or reproducibility problem with a different procedural option (e.g., inoculum preparation, automated reading), you should include a limitation statement similar to that in section XII,. Labeling, stating that users should not report the results. This type of limitation may apply if some procedural options (method of inoculum, reading method, etc.) were considered unacceptable while another was acceptable.

Observations of trending by a particular organism group or by a procedural option should be investigated further in the other study data to assess the impact on interpretations of patient results.

________________
⁶ We believe revising this limitation in a legally marketed AST device significantly affects the safety and effectiveness of the device. Therefore, you will need to submit the results of this future study to FDA as a new 510(k) to support modifying this limitation according to section 21 CFR 807.87(g).

XII. Labeling

The premarket notification should include labeling in sufficient detail to satisfy the requirements of 21 CFR 807.87(e). The following suggestions are aimed at assisting you in preparing labeling that satisfies the requirements of 21 CFR 807.87(e).⁷

The product insert should be flexible to accommodate additional antimicrobial agents. Charts should be used when possible to facilitate additions of future antimicrobial agents, limitations and performance characteristics.

A. Intended Use Statement

The Intended Use statement should indicate:

• whether the assay is quantitative (MIC) or qualitative (breakpoint devices)
• whether results may be read and reported manually
• which organism groups the device is indicated for testing
• any instrumentation the device may be used with, if applicable.

A typical example of an intended use statement is:

"ABC's system is intended for the in vitro qualitative or quantitative determination of antimicrobial susceptibility of rapidly growing aerobic non-fastidious Gram positive and Gram negative organisms on the ABC Instrument."

B. Reagents

You should list antimicrobial agents along with concentration ranges and abbreviations. You may include these in the reagent section of the labeling or on each package container if different for different devices. To prevent confusion between different drugs with similar generic or trade names, you should use abbreviations recommended by the pharmaceutical manufacturer.

C. Reporting of Results

You should provide the interpretive criteria users should use for each antimicrobial agent on the MIC or breakpoint device. You should present the NCCLS/FDA interpretive standards that you used in the evaluation. AST systems may be able to provide results for organisms that may not be appropriate for all of the antimicrobial agents provided on a test panel or system. Therefore we recommend you explain the clinical utility of your interpretive criteria in your labeling. For example:

"There are antimicrobial agents included in this [panel, device, or section] that have not been proven to be effective for treating infections for all organisms tested. Refer to the individual FDA approved pharmaceutical antimicrobial agent package insert for interpreting and reporting results of antimicrobial agents that have shown to be active against organism groups both in vitro and in clinical infections."

Alternatively, you could refer to the most recent NCCLS M100 Informational Supplement, Tables 1 and 1A. If manual readings are an option, provide a chart of what thresholds should be used for SIR (Susceptible, Intermediate, Resistant) interpretations or refer to the NCCLS M100 Tables 1 and 1A for appropriate organism reporting with interpretations in Table 2 of the same NCCLS document.

Results should not be reported in instances where performance has not been established either because there are no interpretive criteria for a particular organism group or insufficient numbers of organism groups have been tested. Where feasible, FDA suggests that suppression of results be software driven. Interpretations (i.e., SIR) should not be reported for these groups of organisms. If you report MIC results, they should carry a disclaimer that device performance or antimicrobial agent clinical effectiveness have not been established. MIC results for this type of organism may be useful for antibiogram patterns, but the practice of reporting results should be discouraged when the antimicrobial agent has not been proven to be effective for treating infections caused by these organisms and the performance on your device has not been established.

D. Performance Characteristics

You should describe the study design, stating the reference method used, number of sites, etc. You should list the percent EA and/or CA in table format with the NCCLS standard reference method for each antimicrobial agent from comparative performance evaluations. You should also include results of reproducibility studies in either a table format or a summary paragraph describing the type of study and a statement that all reproducibility results were acceptable at 95%.

E. Quality Control

You should list all recommended quality control strains whether NCCLS or other and the expected results when tested with each antimicrobial agent.

F. Limitations

You should list all applicable limitations. If the device has software-generated interpretations, these limitations should be incorporated into the software. The following are examples of some limitation statements that may apply to your device:

• You should recommend the use of an alternative method for testing prior to reporting of any results when the spectrum of activity for any antimicrobial agent includes organisms with either unacceptable very major discrepancy or major discrepancy rates.
• If you did not test sufficient resistant organisms with an approved indication for use for the antimicrobial agent, you should include a statement in the labeling similar to this:

  "The ability of the ABC system to detect resistance to [Antimicrobial agent] in [organism(s)] is unknown because resistant organisms were not available at the time of comparative testing."

  However, this limitation may not be necessary if a sufficient number of evaluable results close to the interpretative categories are available and the EA is adequate.

• If the reproducibility results for any antimicrobial agent using one procedural option are not reproducible while another option is reproducible, you should include a limitation against reporting results, for example:

  "The results of testing (antimicrobial agent) showed < 95% reproducibility when inoculum method [cite which inoculum method] is used. Results should not be reported."

  This applies if any recommended procedural option (method of inoculum, reading method, etc.) was unacceptable while another was acceptable.

• You should recommend an alternate method for any specific organism group that had a "no growth" rate >10%. You should recommend that users not test these organisms because the results might be misleading. If the device is software driven, the device should block the results from being reported.

______________
⁷ Although final labeling is not required for 510(k) clearance, final labeling must comply with the requirements of 21 CFR 809.10 before an in vitro diagnostic device is introduced into interstate commerce. In addition, final labeling for prescription medical devices must comply with 21 CFR 801.109. Labeling recommendations in this guidance are consistent with the requirements of part 801 and section 809.10.

XIII. Removing Certain Labeling Limitations from Legally Marketed AST Devices

The FDA will request appropriate data and information in a new 510(k) to support the removal of any limitation included in the labeling that was placed there during the original clinical studies. These types of data are described below and detailed in Table 2. You should refer to the submission in which the limitation statement was made. If you made changes to the device to alter the performance, your additional studies should include all organisms previously tested if these changes significantly affect safety and effectiveness.

A. Performance

If a limitation in the labeling is a result of performance characteristics that are based on EA and/or CA and you wish to modify the device in order to delete the limitation, you should perform a comparative clinical laboratory study. This study should follow the design for the comparative study described in this guidance. The organism mix should be concentrated around those groups that provided the original EA or CA results. You should include all groups that might be affected by modifications to the device. You may wish to report the results using the table format samples provided in this guidance.

B. Insufficient Resistant Strains

If a limitation in the labeling is a result of not testing sufficient resistant strains, you should perform a comparative study to demonstrate that the device can detect resistance in organisms that are included in the Microbiology and Indication and Usage Sections of the FDA approved pharmaceutical antimicrobial agent package insert. This testing should utilize reference and test devices similar to those from the original comparative study. A special challenge set containing the resistant isolates and some susceptible organisms may be substituted for fresh isolates. You may wish to report the results using the table format samples provided in this guidance.

C. Reproducibility

If the reproducibility was <95% for a particular procedural option, you should perform a study to verify that this method is now reproducible. This study should include the problematic organism(s) or procedural options (alternate methods of inoculation, alternate reading procedures, etc.) which originally did not demonstrate reproducible results. You should conduct either the 25-organism study, or the 10-organism study at three test sites as described in this guidance. The strains should include organisms for which the antimicrobial agent is intended for testing with known results near the interpretive criteria range. Include organisms you determined to be problematic in the original reproducibility study. You may wish to report the results using the table format samples in Tables 6, 6A, and 6B. You should also include the testing of all quality control organisms. Notable bias or poor reproducibility of an alternate method of inoculation or reading may indicate additional concerns with this particular procedure. Additional challenge data may be needed to resolve these concerns. If you determine that the inoculum was the problem, you should perform an evaluation of colony count data.

D. Quality Control

If alternate methods of inoculation or reading produced quality control values that did not match NCCLS acceptable ranges, you should test a minimum of 20 replicates per site with each quality control organism on the test device to verify that the quality control values are now within the acceptable NCCLS quality control range. Prepare each quality control organism from a different inoculum suspension. You may wish to report the results using the table format samples in Table 4. You should perform colony counts if you do not use a standardized inoculation method (e.g., photometric device). Colony counts should be done once on each day of testing using the NCCLS recommendations for sampling from the inoculated test device. If the device package insert recommends additional methods of inoculation and/or reading, you should test all options. If you elect to propose an alternative range, you should follow an NCCLS M23 (Ref. 5) study design. You should explain any affect on clinical isolate results.

XIV. QSR Considerations

Part of the QSR (Quality System Regulation, 21 CFR Part 820) is to ensure that the finished product will be safe and effective and perform as intended. There is a concern to the public health when AST devices can not reproducibly generate the same results within +/- one well. If the non-reproducible result is around the interpretative criteria, a vmj could occur. The patient report would recommend for treatment an antibiotic to which the organism is actually resistant, which could lead to treatment failure, particularly for serious infections or altered host conditions. Another possibility is a major error, which can lead to utilization of broad-spectrum agents and needlessly accentuate the pressure for selection of resistant flora. This may also result in further risk to patients because the selection of treatment antibiotic may now be an antibiotic that could be more toxic to the patient when a less toxic antibiotic is available.

You should consider the reproducibility and stability of components in the design of the device and in the development of release criteria. All aspects of the final product will have an effect on the performance, but the lot-to-lot reproducibility and stability studies of the antimicrobial agent should be for performance of the antimicrobial agent only. You should validate the other aspects by other means or at different times. Testing should be performed internally with design components similar to the clinical trial protocol. You should keep the data on file and available upon request.

Manufacturers are also expected to use surveillance data and customer complaints as part of design controls (21 CFR 820.100) Corrective and preventive action). The NCCLS includes surveillance data in performing continuous assessment of older antimicrobial agents, particularly when new mechanisms of resistance emerge. The manufacturer is responsible for keeping abreast of all information to better reevaluate the product to determine if the change in resistance patterns has affected performance and accuracy of the test.

Although not part of the class II special controls, FDA recommends that you consider the following in your approach to complying with QSRs. The method you use to validate reproducibility and stability should be able to detect a change in potency of at least 50% for each antibiotic. For example, if the organisms you select for such testing have stable on-scale MICs, then a sufficient number of replicates of these organisms could detect shifts in the mode of the test organism. Although FDA acknowledges that the methodology has a +/- one well variability, each antimicrobial agent should be evaluated by you for even slight trending to ensure the product will continue to be safe and effective and perform as intended.

A. Lot to Lot Reproducibility

Your study design should demonstrate that different lots of prepared antimicrobial agents in the final format (minimum 3 lots) will perform with the same accuracy. If all other device components have been previously evaluated, you need only include one lot of these components since this study design is to monitor the antimicrobial agent.

B. Stability

Your study design should verify the shelf life of the antimicrobial agents in their final format for all conditions that are recommended by your labeling. The temperatures at which the product is stored should include the extremes of the range recommended for storage. Include observations of slight trending in one direction over time as part of the evaluation. Your study design should verify the shelf life of the antimicrobial agents in their final format for all conditions that are recommended by your labeling. The temperatures at which the product is stored should include the extremes of the range recommended for storage. Include observations of slight trending in one direction over time as part of the evaluation.

XV. Glossary

TABLE 1: Recommendations for Antimicrobial Susceptibility Devices  ^a

_____________________
^a See Tables 5, 8, and 9 for statistical calculation examples and evaluable results.
^b For Fastidious organisms such as Streptococcus, Haemophilus, anaerobes, etc. that have an NCCLS approved standard methodology, FDA/NCCLS interpretive criteria and quality control recommendations, refer to NCCLS approved standard M100 (Ref. 3) Table 1A. The routine testing of rare isolates is not recommended.
^c Minimal data to establish performance should be presented for each procedural option of the method of inoculation (growth, direct colony suspension etc.), reading of results (visual vs. automated), or combinations of options.
^d Fresh clinical - an organism isolated from a clinical specimen which has been on an agar plate for less than 7 days and not frozen.
^e Stock organisms - any organism from a clinical specimen which has been isolated greater than 7 days prior to testing. Should not include organisms for which the antimicrobial agent is not intended. Selection should be supplemental based on the listing in the FDA approved pharmaceutical antimicrobial agent package insert and should not comprise more than 50% of the clinical isolates.
^f Challenge - CDC or reference laboratory source with known results (preferably on-scale) to be tested on the test system. Organisms that are intended for the testing with the antimicrobial agent as stated in the FDA approved pharmaceutical antimicrobial agent package insert (microbiology section), should be selected for testing on the test device.
^g All on-scale results.
^h Inoculum density check should be performed daily on the QC isolates, on reproducibility isolates, and 10% of fresh isolates. Alternate approaches may be substituted if the inoculum method uses a spectrophotometric device.

TABLE 2: Recommendations for the Removal of Limitations from Antimicrobial Susceptibility Devices ^a

__________________
a For statistically evaluable numbers see Tables 5, 8, and 9
b To be used to demonstrate that the QC ranges are now in the same ranges as the FDA/NCCLS
c One may be in-house
d A minimum of 75 organisms either resistant or clustered near the susceptible threshold.
e On-scale

Note:
1. If changes have been made to the device to alter the overall performance, the testing should include all organisms previously tested.
    Refer to Table 1.
2. Perform testing for all procedural options.

TABLE 3: Presentation of Summary Data for Both Challenge and Clinical Data

TABLE 4: Example of Reporting Format for Quality Control Data

Antimicrobial agent: ___________________________

• Performed daily with a minimum of 20 per site.
• List all reference and test results including out of range results that required repeat testing.
• To be used for all procedural options.

TABLE 5: Sample Table Format for Device Performance

Antimicrobic agent: Oxacillin								Organism: Staphylococcus aureus
Test Results	Reference Results							Evaluation
	0.25a	0.5	1	2 S	4 R	8	16	Overall EAb	199/208	95.7%
0.25	6a	a	1				a
0.5	102	100	21	2			a	EA based on evaluable resultsa b	152/161	94.4%
1	a	10	8	1			a
2 S	a	6	11	1			a	CA based on interpretationc		100%
4 R	a						a
8	a				a	a	31a
Evaluable Results		116	41	4
Antimicrobic agent: Oxacillin								Organism: Staphylococcus epidermidis
Test Results	Reference Results							Evaluation
	0.25 S	0.5 R	1	2	4	8	16	Overall EAb	108/116	93.1%
0.25 S	29a	a					a
0.5 R	a						a	EA based on evaluable results a b	3/11	27.3%
1	a						a
2	a		1	1		1	a	CA based on interpretation c		100%
4	a				1		a
8	a			7	12a	12a	52a
Evaluable Results			1	8	1	1
Antimicrobic agent: Oxacillin								Organism: other CNS
Test Results	Reference Results							Evaluation
	0.25 S	0.5 R	1	2	4	8	16	Overall EA b	67/70	95.7%
0.25 S	25a	1a vmj					1a
0.5 R	a	8	3				a	EA based on evaluable results a b	11/11	100%
1	a						1a
2	a						a	CA based on interpretation c	67/70	95.7%
4	a						a
8	1a (maj)				2a	3a	26a
Evaluable Results		8	3

see footnotes on Table 5A

Present all fresh and stock results for organisms in a genus/species in a separate diagonal. For example all Staphylococcus aureus results from all sites would appear in the first diagonal.

TABLE 5A: Sample Table Format for Device Performance, continued

Antimicrobic agent: Ciprofloxacin								Organism: All
Test Results	Reference Results							Evaluation
	0.12	0.25	0.5	1 S	2 I	4 R	8	Overall EA b		398/407	97.8%
0.12	259a	a					a
0.25	a	4	2				a	EA based on evaluable results a b		85/88	96.6%
0.5	6a	4	2				a
1 S	a		10	10	5	1	a	CA based on interpretation c		371/407	91.2%
2 I	a		2	9	10	11	a
4 R	a				8	10	a	CA	Minor d	35/407	8.6%
8	a					1a	53a		Major e	0	0%
Evaluable Results		8	16	19	23	22			Very major f	1/76	1.3%

________________________ a Results are not considered evaluable if they fall in this box

To be used for:

• All fresh and stock organisms for each site presented in one 5A diagonal.
• All fresh and stock organisms for all sites combined presented in one 5A diagonal.
• All challenge organisms presented in one 5A diagonal chart.
• All challenge organisms presented in one 5A diagonal chart for each method variation.
• All organisms (challenge, fresh, stock) combined and presented in one 5A diagonal chart. Do not combine different method variations.

TABLE 6: Presentation of Reproducibility Results ^a by Organism

^{______________________
a} Results in the table are occurrences of the difference in the number of wells between the test result and the test mode. The study is based on 25 on-scale organisms, tested at 3 sites.
^b Most frequent new test result. If there is no mode, the median should be used.
^c Total number of results that fall within 1 well (+/-1) of the mode result divided by total number of results. This should be calculated for the best and worst case if some of the values are off-scale while some are on-scale. If all three results are off-scale they should still be included in the calculation as part of the best-worst case calculations. For this study the denominator would always be 75 whether the results are off or on-scale.
^d Best case calculation for reproducibility assuming the off-scale result is within one well from the mode.
^e Worst case calculation for reproducibility assuming the off-scale result is greater than one well from the mode.

TABLE 6A: Presentation of Reproducibility Results ^a by Organism and Site

________________
^a Results in the table are occurrences of the difference in the number of wells between the test result and the test mode. The study is based on 10 on-scale organisms, tested in triplicate on 3 separate days at 3 sites.
^b Most frequent test result.
^c Total number of results that fall within 1 well (+/-1) of the mode result divided by total number of results. This should be calculated for the best and worst case if some of the values are off-scale while some are on-scale. If all results are off-scale they should still be included in the calculation. For this study the denominator would be ninety whether they are all on scale or not.
^d Best case calculation for reproducibility assuming the off-scale result is within one well from the mode.
^e Worst case calculation for reproducibility assuming the off-scale result is greater than one well from the mode.

TABLE 6B: Presentation of Reproducibility Results ^a by Organism, Pooled Across Sites

___________________
^aResults in the table are occurrences of the difference in the number of wells between the test result and the test mode. The study is based on 10 on-scale organisms, tested in triplicate on 3 separate days at 3 sites.
^b Most frequent test result.
^cTotal number of results that fall within 1 well (+/-1) of the mode result divided by Total number of results. This should be calculated for the best and worst case if some of the values are off-scale while some are on-scale. If all results are off-scale they should be included in the calculation with a denominator of 270 for all calculations.
^d Best case calculation for reproducibility assuming the off-scale result is within one well from the mode.
^e Worst case calculation for reproducibility assuming the off-scale result is greater than one well from the mode.

TABLE 7: Report Format for Inoculum Density

_________________
^a Data should be available upon request for "by site" evaluation, by organism, etc.
^b Inoculum density should be performed on all methods of inoculation unless a standardized method (photometric device) is used.

TABLE 8: Number of VMJ Discrepancies as a Function of the Number of Resistant Organisms Tested

___________________
^a Est. Rate = estimated vmj rate = number of vmj discrepancies divided by number of resistant organisms.
^b Exact confidence intervals based on the binomial distribution.

TABLE 9: Essential Agreement as Function of the Number of Evaluable Organisms Tested

___________________
^a Evaluable (e.g., on-scale) organisms are those that fall within the test range of the reference and have the opportunity for a result on the test method that could also be on-scale. Any reference result that falls in the < or > category is considered not evaluable.
^b Estimated Essential Agreement = percent agreement = number of evaluable test results that are equal to or with in one dilution of the expected result divided by number of organisms that are evaluable.
^c Exact confidence intervals based on the binomial distribution.

Updated 2/12/2003

CDRH Home Page | CDRH A-Z Index | Contact CDRH | Accessibility | Disclaimer
FDA Home Page | Search FDA Site | FDA A-Z Index | Contact FDA | HHS Home Page

Center for Devices and Radiological Health / CDRH