DRAFT – DO NOT CITE
The DQO/MQO process for comparability in monitoring:
nitrate as an example
Katherine Alben, Jerry Diamond, Larry Keith, and Charlie Patton
Version 1.4, Feb 5, 2003
1. Introduction
Excess nutrients in surface and ground waters of the U.S. have been reported with increasing frequency by a number of organizations. State assessments of their waters (305[b] reports) indicate that elevated concentrations of nutrients, such as nitrates and ammonia, are among the top 5 causes of impairment to aquatic life and/or public health and recreation (USEPA, 1998; 2000a-d). The growing national concern with elevated nitrate concentrations in many surface waters, aquifers, and drinking water sources of the U.S. (e.g., USGS, 1999), for example, has lead to greater interest in nutrient monitoring and development of nutrient criteria for protection of aquatic life (USEPA, 1998; 2000a-d).
With increased nutrient monitoring and the desire to detect trends in nutrient impairment, both spatially and temporally, data quality and method performance issues are becoming more critical. Currently, multiple agencies use a variety of methods to monitor the same nutrient analyte. The performance of these methods, and the comparability of data generated by these methods, is not always clear, nor have these issues been dealt with by data users in general (ITFM, 1995a,b). Nutrient data (some of them relatively low in concentration), are starting to be scrutinized by the public and others, in anticipation of numeric water quality criteria being set. The quality of data generated by different methods, and documentation of method performance, are necessary to make informed interpretations of monitoring data.
The ITFM (1995a,b), National Methods and Data Comparability Board (MDCB), and the National Water Quality Monitoring Council (NWQMC), have indicated that the reliance on methods without appropriate method performance documentation, has had significant negative consequences in water quality monitoring (NWQMC, 2001). Consistent with the goals of the Clean Water Action Plan (USEPA/USDA, 1998), the MDCB under the NWQMC has endorsed the development and use of a performance-based system (PBS) as one of its top priorities (NWQMC, 2001) because this system promotes the documentation of known quality data. A PBS should enhance data comparability assessments among various methods or programs, and encourage implementation of better and more cost-effective methods (ITFM, 1995b; Parr, 2000).
Key aspects of performance-based systems include: a) establishing concise data qualityobjectives (DQOs) and measurement quality objectives (MQOs) for each parameter reported; b) demonstrated methods capable of meeting these DQOs and MQOs or regulatory limits; c) adequate reference materials to assist laboratories in demonstrating the appropriateness of a given method; d) adequate documentation of method performance, and e) successful pilot studies demonstrating the advantages and viability of a performance-based system (NWQMC, 2001). All of these aspects are relevant to nutrient methods and monitoring.
Data Quality Objectives (DQOs) are qualitative and quantitative statements derived from the DQO Process that clarify study objectives, define the appropriate type of data, and specify tolerable levels of potential decision errors that will be used as the basis for establishing the quality and quantity of data needed to support decisions (USEPA 1994, 2000d,e). These are mandated by EPA Order 5360.1 A2 and the applicable Federal regulations which establish a mandatory Quality System that applies to all EPA organizations and organizations funded by EPA (USEPA 1994, 2000e). Many other organizations routinely use the DQO process as well, in a variety of environmental programs (e.g., DOE: Grumbly 1994; APHA, 1998).
Measurement Quality Objectives (MQOs) on the other hand are project-specific analytical parameters that are derived from project-specific DQOs. MQOs define acceptance criteria for the data quality indicators that are important to the project, such as sensitivity (e.g., what detection or quantification limit is desired), selectivity (i.e, what analytes are to be targeted), analytical precision. They are derived by considering the quantity and quality of data needed to actually achieve the project goals (as expressed in the DQOs). In formal terms, the DQOs and MQOs specify project requirements to demonstrate precision, accuracy, representativeness, completeness, and comparability (APHA, 1998). As with DQOs, MQOs specify benchmarks for validating/verifying method performance, without prescribing the technology or procedures to be used in producing analytical data.
The DQO/MQO Process is a systematic, iterative, and planning process based on the scientific method (USEPA 1994, 2000e). It produces quantitative and/or qualitative statements (DQOs) that express the project-specific decision goals. The DQOs then are used to define MQOs and guide the design of sampling and analysis plans that will be able to cost-effectively produce the right kind of data. The DQO/MQO process identifies what the goals are and what the consequences may be if the decisions are made in error. Environmental program managers usually determine how certain (i.e., confident) they want to be before making decisions that will either impact, or be impacted by, environmental conditions (Crumbling 2001).
An important part of theDQO/MQOprocess is developing an understanding of how uncertainties can impact the decision-making process. In brief, DQOs and MQOs require that analytical results be substantiated by quality control measurements, which in turn can be used to calculate confidence limits about a reported mean value. As explained in the Appendix, classical statistical methods can then be used to formulate hypotheses and test the validity of data interpretations at a specified confidence level.
This paper illustrates the DQO/MQO and method selection process, using nitrate to provide a focused case study:
1) development of historical perspective: preliminary screening of site specific data for nitrate, using examples from the USGS National Water Information System (NWIS; www.waterdata.usgs.gov/nwis/) and classic statistical methods of interpretation
2) development of DQOs and MQOs: side-by-side comparisons of criteria for regulatory and ambient scenarios for monitoring of nitrate, as suggested by the historical data
3) method selection: appropriate choices for the compliance and ambient monitoring scenarios, using nitrate methods from the National Environmental Monitoring Index (NEMI; www.nemi.gov), an online compendium of analytical methods for water quality monitoring (Peters et al., 2000; Brass et al., 2000)
The primary goal is to understand how the DQO/MQO-method selection process leads to comparability of data and methods - within a specific program (eg for different sites, monitored at different times), and between programs (carried out by different institutions, acting independently or in collaboration). The case study exercise raises several issues for clarification of the DQO/MQO-method selection process, which are addressed in the final discussion. For nitrate as the analyte of interest, several suggestions are made for future pilot comparability studies, which are relevant to a monitoring program for development of nutrient criteria.
2. Sources of nitrate data, methods of analysis and data interpretation
This paper uses NWIS data for a subset of three surface water sites in New York State (cf Table 1). The sites were chosen primarily for having a large quantity of data reported over a 20-year period. NWIS contains a wealth of information for nitrate and nitrite, and other analytes of current interest in surface and ground waters. Similar data from USGS monitoring stations in virtually any of the 50 states would serve equally well.
The NWIS data were obtained by the USGS National Water Quality Monitoring Laboratory and the New York State section. USGS methods were used to determine dissolved and total (dissolved and particulate) species: Method I-2545-90 for dissolved nitrate and nitrite (USGS 1993b); Method I-2540-90 for dissolved nitrite (USGS 1993a); dissolved nitrate, by difference (USGS 1997); Method I-4540-90 for total nitrate and nitrite(USGS 1993b); Method I-4540-90 for total nitrite (USGS 1993a); Method ??? for total and dissolved nitrate (USGS 1997).
Basic statistical properties and Student’s t-test are used for interpretation, to find t-values and their probabilities in comparing: a) a single mean to a target value; b) two means, using unpaired data, with standard deviations that are i) unequal or ii) equal; c) two means using paired data. Equations are given in an Appendix, with definitions of basic statistical terms, including type I (false positives) and type II errors (false negatives) (Devore and Peck 2001; Evans et al 2000). The examples of statistical t-tests use sets of data with more than 30 entries each, to avoid the requirement for a normal distribution of values, which, in some cases, may require use of mathematically transformed data. Calculations have been carried out in Excel: results of pre-programmed statistical tests were cross-checked manually.
3. Historical data perspective: nitrate case study
Analyses of historical data are generally recommended for planning a monitoring project, to determine representative concentrations for target analytes, and potential differences between sites, and thereby develop useful and attainable DQOs and MQOs for further monitoring and assessment (US EPA, 2000e; APHA, 1998). Table 1 summarizes nitrate data for the three surface water sites in New York State that are of interest to compare. Average values reveal large differences among the three sites. Relative standard deviations indicate that variability of the data is also large: from 33 to 83 percent for nitrate; from 56 to 157 percent for nitrite. The results pertain to samples collected at different times from different sites: technically, the historical data are unpaired; statistically significant differences are expected for parameters where differences between mean values are relatively large.
Using data for total nitrate and nitrite concentration as an example (parameter 630), differences in mean values among the three sites can be evaluated for statistical significance, to test the following hypotheses (cf Appendix):
Hypothesis (null): results for total nitrate and nitrite at the three sites are equivalent, at a 95 percent level of confidence
Alternate hypothesis: results for total nitrate and nitrite at the three sites are different, at a 95 percent level of confidence
Results in Table 2 show that the
difference between sites in mean concentration of total nitrate, 
/
, ranges from 29 to 51 percent. Therefore, it is not surprising that these
differences are statistically significant, at a 95 and even 99 percent
confidence level (probability P = 0.00 < level of significance a = 0.025 and 0.005, confidence level 100(1-2a); cf Appendix).
The historical data in Table 1 raise other questions about differences in the analyte(s) being determined, which are relevant to DQOs, as well as method selection in a future monitoring project. For example, is there a need to specify nitrate as
a) dissolved OR total (dissolved and particulate) nitrate
b) nitrate (alone) OR nitrate AND (possibly including) nitrite
As before, hypotheses can be proposed to test for significant differences in mean values:
Hypothesis a (null):results for the total (dissolved and particulate) are not significantly different from results for the dissolved species at a 95 percent level of confidence
Hypothesis b (null):results for nitrate including nitrite are not significantly different from results for nitrate alone at a 95 percent level of confidence
Tables
3 and 4 give results of the statistical tests, arranged in order of increasing
concentration difference relative to the measurement of the total, 
/
a. In both cases, the concentration
differences being examined are generally small and, as expected, statistical
significance is not found (probability P = 0.10 to 0.50 > level of
significance a = 0.025, confidence level 100(1-2a); cf Appendix):
a)
dissolved vs total
nitrate (Table 3): values of 
/
a range from
4.5 to 31%, remaining below 12% for all cases where there is sufficient data
(degrees of freedom df = N – 1 ³ 31); both
determinations are equivalent, with 95% level of confidence
b)
nitrate vs nitrate
plus nitrite (Table 4): values of 
/
a range from
4.8 to 23%, remaining below 13% for all cases where there is sufficient data
(degrees of freedom df = N – 1 ³ 49); both determinations are equivalent, with 95%
level of confidence
With regard to a) in particular, it is perhaps not surprising to find that USGS has also concluded that the distinction between ‘dissolved’ and ‘total’ forms of nitrate is not significant (USGS 1992). Two results in Table 4 are reported, where there is paired data for nitrate (parameter 630) vs nitrate plus nitrite (parameter 620). Statistical properties of the paired data are generally similar to those of the unpaired data (Table 5). For the Mohawk River at Schenectady, it can be concluded that determinations of nitrate, with or without nitrite, give significantly different results, but only by 4.8% (probability P = 0.000 < level of significance a = 0.05, confidence level 100(1-a); cf Appendix): this difference is unlikely to be of practical significance to a monitoring program.
Inferences
for DQO Selection and Subsequent MonitoringIt is important to note that the data quantity
contributes greatly to the statistical significance of the differences in
concentrations, while data quality (precision) is limited for the datasets examined in this case study. Relative standard
deviations for concentrations of total nitrate at the three sites are moderate,
ranging from 33 to 53 percent, but the degrees of freedom (df = N –1) in these
calculations are very large, ranging from 122 to 181, deflating the variability present. For a subsequent
monitoring project, it might be of interest to propose a DQO for identifying
sites that have a nitrate concentration that is 30 percentgreater than the mean for all sites. For two parameters
with relative standard deviations (RSD) of 30 percent, it can be shown that for
a difference 
/
to be significant at a 95 percent confidence level, each
data set must only have more than 7 values (or more than 10 values, if one set
has a 50 percent RSD).
With regard to setting DQOs and method selection for subsequent monitoring in this program, the foregoing results indicate that it may not be necessary to differentiate dissolved from total nitrate, or to specify nitrate in the absence of nitrite. Historical data for any of the measurements were equivalent. Therefore costs could be saved in method selection for a subsequent monitoring project, by not having to filter or centrifuge samples to isolate dissolved nitrate, and by not having to determine nitrite separately from nitrate. Moreover, data quality for nitrate may be improved: by avoiding potential contamination and loss in precision from increased sample preparation; by not having to estimate nitrate by difference (using imprecise data for nitrite), if using a method for nitrate which allows interference from nitrite.
In addition to the straightforward comparisons of historical data for different sites and analytes, there are remaining questions about the data variability, which could be of interest in design of a future monitoring project. In Table 1, there was high variability in concentrations of dissolved nitrate for the Hudson River at Waterford, with relative standard deviations of 47 to 54 percent. At all three sites, there was high variability in concentrations of nitrite: relative standard deviations range from 56 to 102 percent for the dissolved species, or to 157 percent if suspended particulates are included. Plots of total nitrate concentrations in Figure 1 suggest that the variability is random over time. However, it is somewhat surprising to note that, as the mean concentration decreases among the three sites, there is a decrease in variability (less separation between lines marking two standard deviations above and below the mean, Figure 1); also, the distribution of values peaks more sharply (and in a more normal distribution) about the mean (Figure 2). A subsequent monitoring project might try to determine sources of variability, at specific sites and from specific methods of analysis.
4. DQOs and MQOs for a new monitoring project for nitrate:
requirements for data comparability
Given the historical data, how might DQOs and MQOs be developed, and a method of analysis selected, for further monitoring of nitrate? These issues are examined for two project scenarios: compliance monitoring and ambient monitoring of nitrate because these two types of monitoring reflect different regulatory interests and goals, and therefore, different potential monitoring objectives. Some researchers have postulated direct and indirect effects of nitrate on aquatic communities at concentrations as low as 2 mg/L (a target concentration reported by Ohio EPA to be associated with ecological effects; OEPA, 2001). Using this target as an example, an organization performing ambient monitoring might be interested in a range of nitrate concentrations ranging from 0.01 to > 2.0 mg/L, whereas a compliance monitoring program may be satisfied with determining whether a site or sample is > 1.5 mg/L. Following guidelines provided by EPA (USEPA, 2000e), hypotheses are proposed for the two scenarios:
compliance monitoring
Hypothesis: the nitrate concentration at a particular stream is [null: less than or equal] OR [alternate: greater than] 1.5 mg/L with a 95% level of certainty
ambient monitoring
Hypothesis a (sites): entry of a particular stream into the river increases the nitrate concentration by [null: less than or equal to] OR [alternate: greater than] 30 percent, with a 95% level of certainty, and
Hypothesis b (times): nitrate concentrations in consecutive quarters at a particular river location increase by [null: less than or equal to] OR [alternate: more than] 50 percent, with a 95% level of confidence
All of the hypotheses require comparisons of data: the bottom line of both the compliance and ambient monitoring project is that data can be compared, to make statistically valid interpretations.
For both scenarios, it is assumed that project objectives (POs) have been defined by decision makers (i.e., a regulatory agency, property owner, the organization funding the work, etc.). Table 6 gives hypothetical POs for the two scenarios, following guidelines for monitoring projects that have been suggested by AWWA, WEF and APHA (APHA 1998). Criteria which are anticipated in the setting of DQOs and MQOs are shown in italics. Clearly the two projects differ with respect to the decisions to be made and the required level of precision and accuracy at a particular concentration: the compliance monitoring scenario places emphasis on at the targeted nitrate concentration (1.5 mg/L) whereas the ambient monitoring scenario requires good precision and accuracy across an entire range of nitrate concentrations (0.010 to 2.00 mg/L).
Table 6 also gives criteria for the hypothetical DQOs (confidence level, representativeness, completeness) which establish a statistical context for interpreting the results of monitoring. These DQOs can be combined in a statement for each scenario:
compliance monitoring:
DQO Collect samples quarterly (i = 1 to 4) for one year from 50 specified stream locations, measure nitrate concentrations, and determine to a 95% degree of statistical certainty if the annual average nitrate concentration (Ni = 4) at each site exceeds 1.5 mg/L
ambient monitoring:
DQO Collect samples of river water monthly for one year (i = 1 to 12), below consecutive entry points for 15 first- and second-order streams: measure nitrate concentrations and determine to a 95% degree of statistical certainty, if
a) above and below the entry of any stream, there is a significant 30 percent increase in nitrate concentrations (average of Ni = 12),
and if b) between quarterly samples for any river location being monitored, there is a significant 50 percent change in average concentration nitrate concentrations (average of Ni = 3)
Decision-making in the ambient monitoring project requires a greater number of samples per site (spatial, temporal comparisons, between and within sites, respectively) than in the regulatory monitoring project (independent comparisons of each site to regulatory standard).
Hypothetical MQOs for both projects are shown in Table 7. Accuracy and precision are defined operationally, by project requirements that are set for analysis and data interpretation, and by the required demonstrations of proficiency and quality control. As noted from hypotheses for the two monitoring scenarios, there are important differences in the required precision and accuracy at a particular concentration: the compliance monitoring project places emphasis on at the targeted nitrate concentration (1.5 mg/L) whereas the ambient monitoring scenario requires good precision and accuracy across an entire range of nitrate concentrations (0.010 to 2.00 mg/L).
5. Method selection for a new monitoring project for nitrate
Analytical performance is the basis for method comparisons, as indicated by method detection limit, precision, and accuracy (i.e., meta data; NELAC, 2000; USEPA, 1997a, b; 1999). These three parameters are generally regarded as key attributes of any chemical method (APHA, 1997; ASTM, 2000) and are considered to be critical for judging comparability of different analytical methods, and the ability of a method to meet MQOs and DQOs for a specific project. Performance information for analytical methods is documented for a variety of water methods in the National Environmental Methods Index (www.nemi.gov), maintained by the Methods and Data Comparability Board and the National Water Quality Monitoring Council, with database and web support provided by the U.S. Geological Survey. NEMI is a web-based methods compendium that provides information to compare and contrast performance and relative cost of analytical methods for water quality monitoring (Peters et al., 2000; Brass et al., 2000). A total of 16 methods for nitrate are available in NEMI, 11 of which have sufficient performance information documented. The DQOs and MQOs provide a framework for method selection, which is discussed using the eleven nitrate methods listed in Table 8. The eleven methods of interest are from fourdifferent sources (USEPA; USGS; ASTM; and Standard Methods). For method selection, it is important to note that the methods are differentiated by using three different types of instrumentation: capillary ion electrophoresis with UV detection (CIE-UV); ion chromatography with conductivity detection (IC-CD); nitrate reduction with colorimetric detection (RD-Vis).
Detection levels differ greatly among methods ranging from 0.002 - 0.42 mg/L (Table 8). Method sensitivity was the only factor that could be adequately addressed for all eight methods with the information available. Precision and accuracy information, as well as the spiking level used to derive method precision and accuracy, is nonexistent in some methods. EPA methods 300.0, 300.1,and 352.1, one ASTM method (D4327), and Standard Method 4500 all have accuracy and precision data. ASTM method D5996 has neither precision nor accuracy information. Standard Method 4110C (with direct conductivity detection) has accuracy but not precision information. USGS method I-2057 has only precision and not accuracy data (USGS, 1985).
Given the information in Table 8, only four methods have sufficient performance information with which to be able to make an informed choice. Standard Method 4500 NO3–E appears to satisfy all MQOs for both the compliance and the ambient monitoring DQOs (Table 8). EPA methods (300.0, 300.1, and 352.1) also appear to satisfy most of the suggested MQOs for both types of monitoring given their high sensitivity (i.e., the low detection levels), relatively high precision ( <1 – 14% RSD), and satisfactory accuracy (95-103% recoveries). Thus, the three EPA methods should theoretically be able to detect a 30% difference in nitrate levels between samples, and accurately detect a 0.1 mg/L nitrate concentration, as required in the ambient monitoring DQOs. However, note that the spiking concentration used to derive precision and accuracy for two of these EPA methods was 10 mg/L nitrate and the third was 0.5 mg/L. These concentrations are greater than the MQO for the ambient monitoring DQOs, in which we desired high precision and accuracy at a 0.1 mg/L nitrate concentration. Thus, available method performance information for these EPA methods indicates that they should be satisfactory for the compliance monitoring DQOs and perhaps satisfactory for the ambient monitoring DQOs, pending further laboratory evaluation.
Of the remaining methods in Table 8, only the two ASTM methods (D4327and D6508) have both precision and accuracy data. Accuracy and precision of method D4327 meet the MQOs as evidenced by the low spiking concentration. However, this method is less sensitive then either of the two EPA methods or Standard Method 4500, and the detection limit is higher than the desired MQO of 0.1 mg/L nitrate (Table 8). Therefore, this ASTM method may not meet all of the desired MQOs for ambient monitoring but it should satisfy the compliance monitoring DQOs. ASTM method D6508 meets the sensitivity and precision MQOs for ambient monitoring, however the reported accuracy does not meet the MQO for perhaps either monitoring program as defined by our DQOs and MQOs (140% recovery, Table 8). Also, the spiking concentration for this method was 1.99 mg/L, which is somewhat higher than that desired for the MQO for either monitoring program. Thus, neither of the ASTM methods appear to meet all of the ambient monitoring MQOs in this example. The remaining methods in Table 8 have insufficient performance information with which to evaluate their appropriateness, regardless of the MQOs selected.
For the compliance monitoring scenario, essentially all of the methods listed could produce acceptable data that would meet the project DQOs and MQOs. All of the methods are capable of quantitation above 0.5 mg/L nitrate, with greater than 80 percent accuracy and better than (less than) 20 percent precision . From a project manager’s viewpoint, all of the methods should yield comparable data for application to the compliance monitoring project. Cost would be a major factor in method selection. One of the methods (RD-Vis) costs less than the others, as long as nitrate is the major constituent, and it does not have to be determined by difference from nitrite, a minor constituent. However, a decision based on cost could be modified by site-specific requirements: a review of historical or pilot data would indicate the need for a high resolution method (CIE-UV or IC-CD) capable of distinguishing nitrate and nitrite in a single analysis. IC-CD methods can also measure multiple analytes using the same sample. Thus, there may be additional monitoring advantages of the ion chromatographic methods, depending on the program DQOs and what other related analytes a program needs to measure. Analyses of historical data presented earlier in this paper, provide several examples of sites for which separation of nitrate and nitrite is unnecessary.
For the ambient monitoring scenario, only a few of the methods listed could produce acceptable data that would meet the project DQOs and MQOs. Only the IC-CD methods are capable of accurately quantifying nitrate from 0.010 to 2.00 mg/L, with a detection limit of 0.005 mg/L. A particular IC-CD method is not chosen from those listed, because the performance data cited in Table 8 were obtained under different conditions, by a mix of single and multiple laboratories [and not all methods give all performance information]. An actual monitoring project would need to evaluate the IC-CD methods in detail, and verify performance capabilities of the chosen method in the laboratory that would conduct the analyses. From a project manager’s viewpoint, the IC-CD methods are potentially comparable for application to the ambient monitoring project.
6. Discussion: comparability revisited; development of nutrient criteria
This case study using nitrate raises a number of issues which merit further discussion, regarding comparability and the DQO/MQO process, in general, and as applied to the development of nutrient criteria, in particular.
Comparability A concept of comparability emerges from the nitrate case study as the overall requirement of data quality in hypothesis-based monitoring projects. The following definition is proposed:
comparability (of data): the data meet the criteria specified in the DQOs (representativeness and completeness) and MQOs (precision, accuracy), so that the project hypotheses can be tested (data can be compared) and statistical interpretations are valid at the desired level of confidence
This definition gives greater significance to comparability than found in previous documents, which imply comparability is only one of the factors by which to judge data quality, together with representativeness, completeness, precision, and bias (APHA, 1998) and measurability (USEPA 2000d). The above definition assigns comparability a more useful role, as a descriptor for data that satisfy all of the DQOs and MQOs in a particular project.
Representativeness Similarly, the development of DQOs argues in favor of a more specific role for representativeness and a clear link to sample design. As suggested in guidelines to the DQO process, DQOs were written for the nitrate case study to explicitly state the confidence level required for data interpretation and allowed rate of false positives and negatives. In the example given, the DQOs were also written to include a statement of the proposed times and numbers (Ni) for samples to be collected, which recognized real-world variability in nitrate concentrations (s2), and the uncertainty (± ts/ÖN) of statistical evaluations to be made. In effect, the sample design has been incorporated in the project DQOs, and representativeness is identified as the primary DQO for the adequacy of data based on the sample design. The following definition is proposed:
representativeness (of data): the results adequately represent the project sites in time and location: the samples collected and analyzed are sufficient in number N to i) make the required interpretations, at the level of statistical significance that is specified in the DQOs, and ii) allow for the overall uncertainty (± ts/ÖN) and rate of false positives (a) and negatives (b)
As defined above, representativeness is assigned the key role in determining that a sample design is sufficient in yielding data with the desired level of confidence for a particular project.
Other DQO/MQO criteria: Completeness of the sample design is essentially implied by representativeness, but can be given a common-sense definition:
completeness: a sufficient percentage of valid results is obtained for the project sites to make the decisions required in the DQOs.
For project planning, it should suffice to incorporate completeness into DQOs for representativeness as the primary goal, and simplify the terminology used to track DQO/MQO development. The case study for nitrate listed precision and accuracy as the major MQOs, and definitions were understood from project requirements:
precision: operationally defined by requirements for analysis of replicate samples and replicates of spiked controls
accuracy: operationally defined by requirements for analysis of standards, blanks, spiked controls and field samples, and by specifications for interpretation and reporting of data
This paper proposes use of validation as the overall goal of MQOs, with the following definition:
validation: operationally defined by required demonstrations of precision, accuracy (bias), proficiency, and quality control
Therefore, recommended terminology is simplified to: comparability for the overall DQO/MQO process; representativeness for the DQO process; validation for the MQO process (method selection, method performance verification).
Sample
design and sample collection:
variability Standard Methods (APHA, 1998) makes an
interesting comment that, in practice, the impact of sample design on
comparability is often not determined.
This paper suggests using representativeness to clarify the link between
DQOs and the sample design. The current
debate between probabilistic versus deterministic sample designs suggests that
this issue needs to be explored further.
Moreover, methods for sample collection need to be included in defining
performance criteria for methods of analysis.
The sample design (time, location, frequency of sampling) is the primary
tool for addressing environmental variability (senvir), whereas
laboratory measurements determine the analytical variability (smeas);
both contribute to the overall variability s2 and error 
– m (assuming the same
number of samples, N, for each term; otherwise each is divided by the
appropriate Ni; cf Appendix):
s2 = s2envir+ s2meas
Only the overall variability (s2) can be deduced from historical data for a single analyte, such as nitrate. A formal analysis of variance (ANOVA) requires additional measurements for sources of variability in both the field and laboratory, generally implying an expanded dataset for other conditions and analytes.
Collaboration between monitoring programs – DQOs/MQOs, method selection and comparability
The case study for nitrate described a relatively simplistic situation in which the compliance and ambient monitoring programs were independent. Collaborations between monitoring programs provide a means of increasing data quantity and quality for reduced cost, but require consideration of DQOs and MQOs. In the examples given, the compliance monitoring program was unrestricted in method selection, but the ambient monitoring program was restricted to a subset of acceptable nitrate methods. Clearly, the compliance program, with the least restrictive DQOs and MQOs, could use all data from the ambient monitoring program. However, the ambient monitoring program, with more narrowly defined DQOs and MQOs, could only accept limited results from the compliance program, if they were obtained by a method of analysis (IC-CD) with the required range of analysis and low limit of detection. In effect, the DQOs and MQOs for each monitoring program define the acceptance criteria for data comparability:
|
|
Monitoring program (DQOs/MQOs) |
|
Data type |
compliance |
ambient |
|
compliance |
data accepted (by definition) |
compliance data comparable, only if acquired using methods meeting ambient DQOs/MQOs |
|
ambient |
all data comparable |
data accepted (by definition) |
Therefore, the DQO/MQO process not only defines performance-based criteria for method selection, as seen in the nitrate case study, but also provides a framework for determining data comparability across monitoring programs. Potential collaborators can objectively compare their respective DQOs and MQOs to determine how best to match their needs and resources.
Evolution of monitoring programs – DQOs/MQOs, method selection and comparability As individual monitoring programs expand their historical databases, they can also be expected to want to preserve comparability for future assessments. This need will also influence the development of DQOs/MQOs and method selection.
Development of nutrient criteria: USEPA’s nutrient program depends on representative, unbiased data with which “reference” or minimally-impaired water quality conditions, including nutrient concentrations, are characterized and used as a baseline for developing ecoregional criteria (USEPA, 2000c). Clearly this requires analytical methods with known performance characteristics to make correct management decisions. The nutrient criteria program needs to define the level of certainty and acceptable rate of false positives and negatives that are required to determine if water quality is impaired or unimpaired. This information would align the nutrient criteria program with the DQO/MQO-method selection process: the acceptability (comparability) of data could be determined, which are used to develop nutrient criteria. The increasing realization of the importance of data quality in 303(d) water body listings (i.e., impaired status) and in TMDLs underscores the difficulties encountered in environmental programs when method performance and DQOs/MQOs are not clearly documented (Heinz CSEE, 2002).
This paper has used nitrate as a case study of the DQO/MQO process, with little discussion of current method limitations. The Methods Board maintains an active interest in new technologies for a number of important analytes (nitrogen, phosphorus, chlorophyll, turbidity, suspended particles, planktonic and periphytic algae, microorganisms, macrophytes, macroinvertebrates), in advanced and alternate technologies for sample collection (in-situ probes; remote monitoring), and in the development of reference materials to assess method performance (Frankovich and Jones, 1998; NOAA, 2000). Pilot studies for any of these areas of interest would be welcome, to improve the quality and quantity of comparable analytical data used to develop nutrient criteria.
/
a
a vs 
b
/
a
a vs 
b
/
a
a vs 
b
and standard
deviation 
= 
(A.1)
:
(A.2)
). A confidence level
P = 95% means that 95% of measured values will be within the uncertainty limits
calculated for the mean. Likewise, 5% of measured values will lie outside the
uncertainty limits. For the confidence
level written as P = 100%(1-
+ t 
+ t 
> 
+ ts/
+ ts/
(A.5)
a and 
b are obtained with corresponding standard
deviations sa and sb, the difference 
of the two means is:
a - 
b) (A.8)
, 
a ± ts a /
b ± ts b /
(A.9)
Vb = 
(A.10)
(A.11)
(A.12)
(also denoted 
, but not the same as 
defined in eq. A.3):
= 
= 
(A.13)
= 
(A.14)
(A.15)