United States Water Quality Methods and Data Comparability Board:
Creating a framework for collaboration and comparability

C.A. Peters*, H.J. Brass**, and J. Diamond***

*U.S. Geological Survey, WRD, 8505 Research Way, Middleton, WI

** U.S. Environmental Protection Agency, OGWDW, 26 W. Martin Luther King Drive, Cincinnati, OH

*** Tetra Tech Inc., 10045 Red Run Blvd., Suite 110, Owings Mills, MD 21117

ABSTRACT

 

Each year, government agencies, industry, academic researchers, and a wide variety of private organizations in the United States devote enormous time and several billion dollars to the monitoring, protection, and restoration of water resources and watersheds. Critical differences in project design, methods, data analysis, and data management have often made it difficult for monitoring information to be shared by other potential data users. In the absence of a focused effort to ensure consistent and comparable data, and a means to determine the utility of a data set for a particular use, there is considerable redundancy and inefficiency in water assessments. The Methods and Data Comparability Board is a partnership of water quality experts from federal agencies, states, tribes, municipalities, industry, academia, and private organizations.  It is chartered under the National Water Quality Monitoring Council, whose mission is to coordinate, and provide guidance on, implementation of the voluntary, integrated, nationwide monitoring strategy developed by its predecessor - the Intergovernmental Task Force on Water Quality Monitoring (ITFM, 1995a).  The Board provides the framework and the forum for comparing, evaluating, and promoting monitoring approaches that can be implemented in all appropriate water quality monitoring programs.

 

INTRODUCTION

 

Each year, government agencies (local, state, tribal, and federal), industry, academic researchers, and a wide variety of private organizations in the United States devote enormous amounts of time and several billion dollars to the monitoring, protection, and restoration of water resources and watersheds.   This work includes:

 

·                    monitoring the status and trends in water quality

 

·                    identifying existing and emerging water-quality issues

 

·                    designing and implementing resource management programs

 

·                    determining compliance with regulatory programs

 

The information gathered through these activities is certainly useful to the data collectors themselves, however, critical differences in project design, methods, data analysis, and data management often make it difficult for monitoring information to be shared by other potential data users.  Accurate, cost-effective and efficient assessment of the nation’s water resources—within and among watersheds— requires that monitoring entities work collaboratively and strive for comparability in methods and data management.  The design and implementation of assessment and management programs should be a cooperative product of the various monitoring agencies and organizations active in any given watershed. Knowing whether the water quality monitoring methods used and the data they produce are comparable is essential to achieving the goals of the Clean Water Action Plan (USEPA/USDA, 1998).

 

METHODS AND DATA COMPARABILITY FRAMEWORK

 

Producing comparable data is dependent on the use of a coordinated monitoring framework that includes four primary elements: developing data and measurement quality objectives for each monitoring activity, determining the performance of sample collection protocols, determining the performance of field and laboratory methods, and utilizing a standard data reporting approach to provide methods information (figure 1). The National Method and Data Comparability Board (Board) activities described in this report have been developed to address specific method and data comparability issues for each of the primary elements of the overall monitoring process. The Board’s strategic philosophy is to complete ongoing products in the short term, while continuing to develop its longer-term goals and objectives. The short-term emphasis is to marshal Board resources to successfully complete projects, so that accomplishments and success stories can be demonstrated.  A longer-term view includes the development of tools for the monitoring community through the use of pilot tests and other studies.

 

DQO/MQO Development

 

Data Quality Objectives (DQOs) and Measurement Quality Objectives (MQOs) are or should be the foundation of all monitoring studies, as these define the questions needing answers and the data quality needed to answer those questions (USEPA 1994; ITFM 1995b; MDCB 1999a and b).  DQOs are statements that define the confidence required in conclusions drawn from data produced by a project (USEPA 1994).  The USEPA’s DQO process is a 7-step strategic planning approach that is used to define what, how, when, and where data are collected and analyzed to ensure that the type, quantity, and quality of environmental data used in decision making will be appropriate for the intended application (USEPA 1994).  For example, the USEPA’s Office of Ground Water and Drinking Water used the DQO process to help ensure that water quality measurement data and engineering information gathered under their Information Collection Rule (ICR) were adequate to support development of a series of drinking water regulations regarding surface water treatment requirements and disinfectant and disinfection byproduct controls (RTI, 1995). MQOs are statements that contain specific units of measure such as percent recovery, percent relative standard deviation, or detection level.  They should be thoroughly specified to allow specific comparisons of data to a MQO. 

 

Field Sampling and Monitoring

 

Monitoring data are obtained by field sampling, direct field measurements, and remote sensing. The Board recognizes that field method performance is an area in need of attention, as sampling-induced error or biases can be larger than those associated with laboratory analysis (ITFM 1995c). One of the key concepts inherent in the Board’s mission is the use of a performance-based system (PBMS) approach.  The Board defines a PBMS as a framework that permits the use of any appropriate sampling or analytical technology that demonstrates the ability to meet established performance criteria and complies with specified DQOs and MQOs of the project.

 

Laboratory Analysis

 

Most monitoring samples are currently analyzed using prescriptive methods.  The widespread use of prescriptive methods will undoubtedly continue in the future, however, a performance-based systems approach, if effectively designed and implemented, could provide the framework for ensuring better, faster, safer, or less expensive analytical methods that provide comparable data.  A PBMS approach recognizes the need for well-defined DQOs and MQOs, an adequate supply of reference materials, the need for validated or reference methods that meet the stated MQOs, and the need for adequate training. Additionally, to ensure comparable data of known quality, it is critical to standardize laboratory quality systems. Standardization of quality systems is particularly important for nation-wide monitoring programs in which data from a variety of organizations are used to make large-scale assessments. A key component for ensuring quality systems is third party accreditation of laboratories that provide environmental data.  Accreditation is defined as a formal recognition of competence that a laboratory can perform specific tests, or types of tests.

 

 

 

Reporting Data to Include Methods Information

 

The ability for a user to judge the comparability of monitoring data is dependent on the availability of information about which methods were used to obtain the data, information about the performance of those methods, and the inclusion of metadata and data quality documentation reported with the data.  The Board recognizes that better documentation of existing methods, and more complete and consistent metadata reporting by monitoring organizations, will improve our ability to define and use comparable data for a variety of purposes. 

 

METHODS AND DATA COMPARABILITY BOARD APPROACH

 

The Board includes 5 federal, 5 state or tribal, and 5 private sector delegates and an equal number of alternates from each of those sectors. The Board accomplishes its objectives through Workgroups that include Board delegates, alternates and other volunteer experts in the water-quality field. These Workgroups define and specify goals, benchmark associated efforts, coordinate with other groups working toward similar objectives, and develop guidance for the monitoring community.  Where appropriate and feasible, pilot studies and work agreements with the private sector and other collaborative efforts are encouraged to enhance communication and use of comparable monitoring methods. 

 

Seven Workgroups have been developed to focus on specific methods and data comparability issues. Some of the Workgroups have a more inherent product focus (NEMI, WQDEs, Outreach), others have both a product and a process focus (Accreditation, PBMS), while still others have a primarily discipline-based focus (Nutrients, Biology). The discipline focused Workgroups are also involved in product development, however, much of their effort will go towards providing information to the product and process focused Workgroups. The Outreach workgroup integrates across all work groups using a liaison member approach. In addition to these more formalized Workgroups the Board has also been available to review and participate in the development of other related technical activities in a more ad hoc fashion.

 

The Board meets quarterly, however, the Workgroups conduct much of their business between the Board meetings via conference calls and electronic mail. A steering committee meets via conference call between Board meetings to coordinate funding, conference calls, integration efforts, and to provide overall focus and direction to the Workgroups. The full Board meets via conference calls between Board meetings to provide progress reports on product task force efforts and to discuss operational issues.

 

Current Workgroups and their specific objectives are:

 

Performance Based Systems (PBMS)

 

Inform the Board and the National Water Quality Monitoring Council (Council) on technical matters pertaining to the implementation of a PBMS.  Define the dimensions of a PBMS for field chemical, microbiological, and biological protocols and laboratory analyses and prepare guidelines to implement PBMS in ambient and compliance monitoring. The workgroup provides peer review of and coordination with PBMS development efforts being undertaken by other organizations.

 

National Environmental Methods Index (NEMI)

 

Provide a web-based searchable compendium containing chemical, physical, radiochemical, microbiological and biological field and laboratory methods, including protocol summaries and information. It will allow the rapid communication and comparison of critical parameters of methods for use with methods selection and (or) methods modification and data comparability. 

 

Laboratory and Field Accreditation

 

Promote laboratory accreditation and develop a Board position on federal laboratory accreditation and pre-laboratory certification in order to establish a uniform national accreditation process including the use of performance-based systems (PBMS). Coordinate the accreditation related activities of the various Board workgroups and communicate those efforts to the corresponding workgroups in the National Environmental Laboratory Accreditation Conference (NELAC) in order to avoid duplication of effort. NELAC’s purpose is to establish and promote mutually acceptable performance standards for the operation of environmental laboratories.

 

Water Quality Data Elements (WQDE)

 

Develop and recommend a "core" set of data elements for reporting water quality monitoring results, to be voluntarily implemented, that would allow data to be compared regardless of, but recognizing, the purpose of the monitoring activity.

 

Biology Measurement

 

Identify, compile and develop a framework for characterizing and comparing water monitoring efforts that diagnose environmental conditions using either: a) whole organisms; b) biomolecular materials; c) populations in the field; or d) microbiology. Develop pilot tests of the PBMS position guidelines and information for the NEMI database.

 

 

 

Nutrient Measurement

 

Identify, compile, and develop a framework for characterizing and comparing nutrient methods and data. Develop nutrient pilot tests of the PBMS position guidelines and information for the NEMI database. Coordinate these efforts with the EPA regional nutrient criteria strategy.

 

Outreach and Publicity

 

Develop and implement means to inform and solicit input from the water resources community regarding the efforts of the Board. Employ various media, including: the internet, brochures, fact sheets, reports, posters, and conference presentations. Develop methods comparability sessions and workshops for the biannual National Water Quality Monitoring Conference. Develop a coordinated outreach approach for the Board Workgroups and with the Council.

 

 

COMPLETED BOARD EFFORTS

 

PBMS Workgroup

 

One of the key concepts inherent in the Board’s mission is the use of a performance-based system approach.  Key aspects of a PBMS were identified in an issue paper developed by the Board entitled “Towards a Definition of a Performance Based Approach to Laboratory Methods” (MDCB, 1999a). Important elements include: a) the need to establish concise measurement quality objectives (MQOs) and data quality objectives (DQOs) for each parameter reported; b) the need for demonstrated methods capable of meeting these MQOs or DQOs; c) the need for adequate reference materials to assist laboratories in demonstrating the appropriateness of a given method (prescriptive or performance-based); d) the need for laboratories to adequately document method performance, and e) the successful completion of a pilot program to demonstrate the advantages and viability of a performance-based approach.  The paper addresses the issues and concerns regarding the use of PBMS, and defines the dimensions of a PBMS – focusing on laboratory, but also addressing field aspects. It includes chemical, microbiological and biological protocols used in compliance and ambient monitoring programs. The Board believes that setting data-quality objectives and using PBMS to meet these objectives will promote and enhance innovative technologies without compromising data quality. Data users can determine data comparability and make their own decisions concerning the applicability of data to fit a particular need. Performance criteria, such as precision, bias, sensitivity, specificity, and detection limits, used in conjunction with reference methods, will be integral to the implementation of PBMS and will provide the user with "judgment" factors for decision-making. If implemented successfully, the use of PBMS is a move towards the production of valid, scientific procedures and evaluations, and away from a prescriptive regulatory mandate.

 

 

NEMI Workgroup

 

The National Environmental Methods Index (NEMI) will ensure that the consideration of analytic methods is a more active part of the planning and implementation of monitoring programs. NEMI will include data fields for comparison such as, relative cost, sample preparation, instrumentation required, method detection level, sampling information, sample preservation and storage conditions, bias, precision, and other QA/QC requirements. Typical users of NEMI are expected to include regulators, regulated parties, scientists, volunteer monitoring groups, and watershed planning organizations. The Workgroup developed a list of critical methods parameters that relate to analytes, instrumentation, matrices, interference, sampling, sample handling, and data quality. These critical parameters will be developed for a list of 100 methods that were chosen to represent a wide range of organizations as well as method type.

 

Nutrients Workgroup

 

The Board completed a study to determine if the information available on nutrient methods from four sources  (APHA 1995, ASTM 1989 and 1999, and USGS (Fishman), 1993) were sufficient to populate the NEMI data fields. Descriptive information for 25 desired NEMI fields was compiled from 26 nutrient methods from these 4 sources.  Findings included: Information is readily available for 15 of the fields, information is predominantly not available for 6 fields, and information availability for the other 4 fields is variable.

 

Outreach and Publicity Workgroup

 

The Board developed sessions and workshops at each of the first two National Water Quality Monitoring Conferences - Monitoring: Critical Foundations to Protect Our Waters (NWQMC, 1998) and Monitoring for the Millennium (NWQMC ,2000). Sessions included: QA/QC for monitoring programs; data comparability and collection methods; and inorganic, organic, and biological methods comparability. Workshops were held on PBMS and Water Quality Data Elements. Additionally, as a part of its outreach effort, the Board has developed and published two Fact Sheets, entitled: “The National Methods and Data Comparability Board: Collaboration and Comparability” (MDCB, 1998) and “Why is a National Environmental Methods Index Needed?” (MDCB, 2000a); established an internet site  (MDCB, 1999b); prepared a portable display that provides an overview of the Boards efforts (MDCB, 2000b); prepared articles for a variety of newsletters; and made presentations and prepared papers (Brass 1998, 1999, 2000) for numerous conferences and meetings.

 

The Board provided oversight and guidance for the development and implementation of methods comparability projects. These projects included: “A Comparison of Water-Quality Sample Collection Methods Used by the U.S. Geological Survey and the Wisconsin Department of Natural Resources” (Kammerer, 1998);  “A Reconnaissance for Sulfonylurea Herbicides in Waters of the Midwestern USA” (Battaglin, 1998a and 1998b, Scribner 1998); and a review of the USGS-EPA Drinking Water Initiative (Patterson, 1997).

 

CURRENT BOARD EFFORTS

 

PBMS Workgroup

 

One of the technical challenges in implementing a PBMS is defining a rigorous, yet cost-effective framework by which laboratories can document that method performance has been achieved.  To address this challenge, the MDCB has designed and is coordinating a pilot study that evaluates the method verification process, within a PBMS framework.  The pilot study examines a new method for analyzing carbonaceous oxygen demand (COD) that does not generate hazardous waste (mercury), evaluating with respect to the current prescriptive COD method.  The study is testing and evaluating an analytical and statistical protocol that will reasonably and efficiently demonstrate: (a) laboratory competence with the method; (b) that the new method performance is appropriate for the matrices of interest; and (c) that laboratory performance is maintained. The pilot makes use of two different approaches that have been advocated for PBMS: (1) a DQO approach that relies on data quality objectives as reference criteria upon which to judge method comparability and (2) a reference method approach in which method performance and comparability is judged relative to the performance of an already established and validated method (Eaton, 1999).

 

NEMI Workgroup

 

The database is being developed in three phases.  The first phase is expected to be completed by December 2000 and will include the 100 methods chosen to test the approach to developing a data dictionary, business rules, user requirement rules, and design using an ORACLE database structure. The second phase, expected to be completed July 2001, will incorporate reviewer comments of Phase 1, include about 250 additional methods, and will create the functional, web-enabled NEMI database. Phase 3 includes updating methods in the database and adding additional laboratory methods and field protocols on an ongoing basis.

 

Accreditation Workgroup

 

Many Federal agencies have laboratories that evaluate water quality. The samples are collected for a variety of reasons and are analyzed by a variety of methods. There is considerable variability in the quality control standards implemented by the laboratories, based on the data quality objectives. An issue paper is being developed to assist federal laboratories in making an informed decision regarding the relevance of national accreditation for various water testing objectives.  The paper summarizes the range of purposes for which various federal laboratories conduct water testing and delineates the analytical services in which these laboratories are engaged.  The paper reviews several key elements in considering which accreditation program best suits the accreditation needs of federal laboratories.  Key elements considered in this evaluation include national or international authority, reliance on generally accepted accreditation standards such as ISO 25 or ISO 17025, and state recognized reciprocity. The issue paper also explores current laboratory accreditation standards available for consideration as candidates for the implementation of a national accreditation program. The Workgroup is also providing a review of the proposed privatization of the USEPA performance testing program.

 

Biology Workgroup

 

Assessment and documentation of method performance characteristics is essential for appropriate application of environmental sampling and analysis methods, and interpretation of results.  One of these characteristics, method precision, is important for establishing and evaluating measurement quality objectives (MQOs).  Further, quantification of method precision is necessary to develop data quality objectives (DQOs) for program design and assessment.   Similarly, the sensitivity of a method provides an indication of the responsiveness of an indicator to the stressor or stressors of concern.  There are a variety of statistical methods that can be used to assess the precision and sensitivity of a method. The Board has developed a draft issue paper (MDCB, in press) describing procedures for documenting precision of field collection methods for stream benthic macroinvertebrates.  This paper specifies several ways in which the precision of a given field collection method can be determined and uses case study data derived from several areas of the United States.  Many of these field methods are currently used in the U.S., however, their performance is largely unknown or undocumented (Barbour, 1999).  The lack of known data quality from these field methods has hampered attempts to assess biological quality on a national scale (e.g., state 305(b) assessments; USEPA/USDA 1998). 

 

WQDE Workgroup

 

Core data element lists for reporting chemical and microbiological water quality monitoring results have been developed. The core data elements were developed to be compatible with the major existing water quality databases. The core lists are continuing to be reviewed by the water quality monitoring community. The lists will be discussed during regional “public meetings” that will be announced in the Federal Register.

 

Nutrients Workgroup

 

The Board is providing review of two activities: 1) an appendix on nutrient methods that is a part of the Regional Nutrient Criteria Strategy for Streams and Rivers (USEPA in review); 2) an intercomparison exercise for nutrients in seawater that is being conducted by The National Research Council (NRC) of Canada (NOAA 2000).

 

PLANNED FUTURE BOARD EFFORTS

 

PBMS Workgroup

 

The Board will develop a framework to characterize the performance and determine the comparability of field sampling methods. Pilot studies will be developed to examine field sampling methods for nutrients and stream benthic macroinvertebrates.  These pilots will represent collaborations among several organizations (federal, state, and private) and will rely on the DQO process to establish appropriate methods and measurement quality needed. The Board will collaborate with the Water Environment Research Foundation (WERF) and EPA to provide peer review on the development of DQOs and the selection of appropriate methods for validating laboratory effluent toxicity tests using field biological assessments.

 

The Board will develop consensus issue papers on other aspects related to PBMS implementation. These may include: status and needs for reference material availability, liability issues, a national framework for accrediting laboratories, training required to implement PBMS, and develop a “how to” guide for conducting pilot tests. To ensure comparable implementation of a PBMS, the Board will study the need for a national program to accredit field personnel who provide environmental data. 

 

NEMI Workgroup

 

The Board will work to develop a NEMI biological methods database that is searchable and provides summaries of performance characteristics for biological methods and that could be used to determine comparability of different biological methods. The biological methods database will include laboratory and field methods that span taxonomic levels. Types of methods currently being considered include: (1) cellular and molecular (e.g., immunoassays), (2) whole organism (e.g., Microtox, algae, invertebrates), (3) microbiological activity/count (e.g., fecal coliform, Cryptosproridia), and (4) field population/community sampling and assessment (e.g., benthic invertebrates).  Methods will be searchable by: type of method,source of method,type of ‘matrix’ (e.g., freshwater, saltwater, tissue),type of organism (and species if appropriate), and by type of quality control requirements.  The biological methods NEMI will be linked with other appropriate on-line databases provided by consensus organizations, and federal and state agencies to ensure that current method updates are included. The information obtained during the development of the NEMI biological methods database will be useful for developing the business rules and data dictionary needed to include chemical and physical sampling protocols in NEMI.

 

The Board will compile relevant information produced by several agencies to develop concrete, hands-on guidance on developing DQOs, translating those DQOs into MQOs, and choosing appropriate methods using real-world examples from the water quality monitoring field.  A “expert-system based” front-end user interface will be developed for NEMI to guide a user to the appropriate field protocol or analytical laboratory method to satisfy user defined DQOs and MQOs. The user interface will be developed so that it can be incorporated into any “on-line expert system” that the National Water Quality Monitoring Council may develop.

 

 

 

WQDE Workgroup

 

The Board will develop core data element lists for reporting monitoring data from biological, chemical, and physical field sampling, field measurements, and remote sensing.

 

REFERENCES

 

APHA. 1995. Standard Methods for the Examination of Water and Wastewater.  19th ed.  Washington, DC:  American Public Health Association. 

 

ASTM. 1999.  Annual Book of ASTM Standards. Volume 11.02 Water (II).  West Conshohocken, PA:  American Society for Testing and Materials. 

 

ASTM. 1989.  Annual Book of ASTM Standards. Volume 11.01 Water (I).  Philadelphia, PA:  American Society for Testing and Materials. 

 

Barbour, M.T., Gerritsen J. , Snyder B.D. , and Stribling J.B..  1999.  Rapid Bioassessment Protocols for Use in Streams and Wadeable Rivers: Periphyton, Benthic Macroinvertebrates and Fish.  Second Edition.  EPA/841-B-99-002.  U.S. EPA, Office of Water, Washington, D.C.

 

Battaglin, W.A., Furlong, E.T., Burkhardt, M., Peter, C.J., 1998a.  A Reconnaissance for New, Low-Application Rate Herbicides in Surface and Ground Water in the Midwestern United States. in Proceedings of the NWQMC National Conference. July 7 – 9, 1998, Reno, NV

 

Battaglin, W.A., Furlong, E.T., Peter, C.J.. 1998b. USGS Fact Sheet FS-046-98.

 

Brass, H.J, 1998. The Methods and Data Comparability Board: Collaboration and Comparability, in Proceeding of the American Water Works Association, Water Quality Technology Conference.

 

Brass, H.J., Ardourel, H., Diamond, J.M., Eaton, A., Keith, L.H., Peters, C.A., 2000. Activities of the Interagency Methods and Data Comparability Board, in Proceeding of the American Water Works Association, Water Quality Technology Conference.

 

Brass, H.J., Spooner, C., Klein, J.M., Shockey, M., Objectives and Activities of the National Water Quality Monitoring Council, 1999.  Presented at the 7^th International Conference of the Israel Society for Ecology and Environmental Quality Sciences.

 

Eaton, A., Diamond, J., Annis, C., Strong, A., McChesney, D., Brass, H., Shockey, M., 1999. Reservoir Dogs and Performance Based Systems for the Analysis of Environmental Contaminants, in Proceedings of the WQTC.

 

ELAB.  1998.  Recommendations for Implementation of PBMS.  Draft.  Environmental Laboratory Advisory Board, Washington, DC.

 

Fishman, M.J., ed.  1993.  Methods of Analysis by the U.S. Geological Survey National Water Quality Laboratory--Determination of Inorganic and Organic Constituents in Water and Fluvial Sediments.  Open-File Report 93-125.  Denver: U.S. Geological Survey.

 

ITFM.  1995a.  The Strategy for Improving Water Quality Monitoring in the U.S.  Report #OFR95-742, U.S. Geological Survey, Reston, VA.

 

ITFM.  1995b.  Performance-based approach to water quality monitoring.  In:  Strategy for Improving Water Quality Monitoring in the U.S., Appendix M, Report #OFR95-742, U.S. Geological Survey, Reston, VA.

 

ITFM.  1995c.  Performance-based approach to field water quality methods.  In:  Strategy for Improving Water Quality Monitoring in the U.S., Appendix N, Report #OFR95-742, U.S. Geological Survey, Reston, VA.

 

Kammerer, Phil A., Garn, Herbert S., Rasmussen, Paul W., Ball, Joseph R.. 1998. A Comparison of Water-Quality Sample Collection Methods Used by the U.S. Geological survey and the Wisconsin Department of Natural Resources. In Proceedings of the NWQMC National Monitoring Conference, July 7 – 9, 1998, Reno, NV.

 

MDCB. 1998. The National Methods and Data Comparability Board: Collaboration and Comparability. USGS/EPA Fact Sheet

 

MDCB.  1999a.  Towards a Definition of a Performance-Based Approach to Laboratory Methods, v5.2.  Methods and Data Comparability Board, in http://wwwdwimdn.er.usgs.gov/pmethods

 

MDCB. 1999b. Methods and Data Comparability Board Web site. http://wwwdwimdn.er.usgs.gov/pmethods

 

MDCB. 2000a. Why is a National Environmentla Methods Index Needed? USGS Fact Sheet.

 

MDCB. 2000b. MDCB Projects and Products. Poster and URL tear off sheet.

 

MDCB. In review. Approaches to Documenting Performance Characteristics of Biological Methods: The Precision of Field Sampling and Taxonomic Identification.

 

MDCB. In review. Study Plan for Oxyver ™ COD PBMS Pilot

 

NOAA. 2000. NOAA/NRC Intercomparison for Nutrients in Seawater. NOAA Technical Memorandum NOS NCCOS CCMA 143.

 

NWQMC. 1998. Proceedings of the NWCMQ National Monitoring Conference. July 7 – 9, 1998, Reno NV.

 

NWQMC. 2000. Proceedings of the NWQMC National Monitoring Conference. April 25 – 27, 2000. Austin, TX.

 

Patterson, G.. 1997. USGS Drinking Water Initiative. Fact Sheet FS-047-97.

 

Scribner, E.A.; Goolsby, D.A.; Thurman, E.M.; Battaglin, W.A.,  1998. U.S. Geol. Surv. OFR 98-181,

 

RTI (Research Triangle Institute).  1995.  Data Quality Objectives for the Information Collection Rule.  Submitted to USEPA, Contract 68D-40091.

 

USEPA.  1994.  Guidance for the Data Quality Objectives Process, Environmental Protection Agency, EPA QA/G-4.  Quality Assurance Division, NCERQA, Office of Research and Development.  Cincinnati, OH.

 

USEPA.  in review. National Nutrient Guidance Manual: Rivers and Streams

 

USEPA/USDA.  1998.  Clean Water Action Plan:  Restoring and Protecting America’s Waters.  US Environmental Protection Agency, National Center for Environmental Publications and Information, Cincinnati, OH.