Methods and Data Comparability Board Meeting
USGS Center, Reston Virginia
January 26 - 27, 1998
An electronic version of the ITFM report/appendices can be obtained from the following web site:
http://water.usgs.gov/public/wicp/itfm.html or alternatively contact Jerry if you need a hard copy.
Jerry will make sure all Board participants get the updated Water Quality Monitoring Conference brochure (July 1998, Reno)
By February 18, delegates and alternates need to tell Herb or Merle their preferred term length (1, 2, or 3 years) after which Herb and Merle will evaluate the responses and determine the need to make any adjustments so as to fulfill the Charter requirements.
Bill Battaglin and Herb will distribute to the Board information on the CRADA including the summary and workplan, and previous responses of Task Group participants.
Bob Held will set up the initial conference call with the rest of the PBMS work group. Herb and Merle need to be contacted as well as one or both may sit in on the call.
Barbara agreed to help find an alternate delegate in California or Nevada.
Jerry will draft a paragraph for publicity in ASTM and other newsletters that could be distributed. The paragraph will also be sent to Bob Held and Keith McLaughlin.
The NCOD Workgroup (Bob Berger, Dennis McChesney, and Andy Eaton) will review the latest NCOD background document and report back to the Board and to EPA (Chuck Job).
Board participants need to submit their top 5 priorities for Board activities in FY098 to Jerry by February 27.
The Board Publicity Workgroup (Bob Berger, Jerry Diamond, and Bill Potter [did not attend]) will develop ideas for publicity and outreach in FY098. More participants for the Workgroup will be solicited.
Ann Strong’s Technical Appendix on PBMS from the ITFM Final Report will be distributed to PBMS Workgroup participants.
Rob Henry will produce a table of parameters often measured, including different matrices, to help begin identifying and prioritizing methods in need of comparability measurements (objective #1 in the Board charter).
A total of 31 persons participated in the Methods Board meeting on January 26-27 in Reston, Virginia. The following is a brief summary of that meeting.
Performance Based Methods
All agreed that PBMS is central to the Board’s mission and its priorities. In this light, the issue paper on PBMS written by Ann Strong (in the ITFM Technical Appendices) needs to be revisited and perhaps expanded by the Board. A Workgroup was formed which will prepare a scoping document discussing how the Board should deal with PBMS issues.
Publicity and Outreach
The suggestion was made that the Board needs to determine the most efficient way to distribute information to a variety of interested organizations. Ideas include: Announcements and Fact Sheets for newsletters and meetings (ASIWPCA, AMSA, etc.). A Workgroup was formed to provide publicity and outreach guidance to the Board. More participants will be solicited. Tetra Tech will draft a paragraph for publicity in ASTM and other newsletters.
National Occurrence Data Base
EPA’s NCOD (National Contaminant Occurrence Data Base for Drinking Water and Source Water) was discussed. Herb distributed Chuck Job’s (EPA, OGWDW) write-up on the NCOD. Herb requested Board review of the latest NCOD background document (to be provided). A workgroup was formed to review the current NCOD database design and report back to the Board and EPA.
USGS-DuPont Cooperative Research and Development Agreement
The Board heard a summary of the USGS-EPA Drinking Water Initiative and the Cooperative Research Agreement (CRADA) between USGS and DuPont. It was agreed that the DuPont CRADA was relevant to the Board’s mission and objectives. Copies of material related to the CRADA will be distributed to the Board and a review team will be formed. The role of the Board with respect to CRADAs in general warrants further discussion.
Continue re-examination of priorities
There was general consensus on the Board’s charter objectives. Many specific priorities were raised in the meeting which need to be associated with those listed in the charter.
Participants:See Attachment 1
Janice Ward (USGS, Acting Chief, Office of Water Quality, WRD) gave a brief introduction and welcome.
Herb Brass then presented background information on the ITFM Methods Task Group and the mission, scope, and structure of the Board. A handout was available at the meeting (contact Jerry Diamond if a handout is desired).
Following Herb’s presentation the following observations or issues were offered:
!There was general concurrence among participants that a central problem in water monitoring information in the U.S. is lack of data quality/comparability among programs. However, several people cited recent examples of cooperation among agencies on methods development, some of which stemmed from previous ITFM Methods Task Group activities.
!Barbara Erickson brought up the issue of organizations needing to take responsibility for data they collect: are the data really representative and do they address the questions asked? She suggested that the Board may eventually need legal and data validator advise regarding data quality and representativeness issues.
!Andy Eaton suggested that the Board needs to focus especially on the field side of data collection as historically, data variability is more influenced by field methods and conditions under which sampling is performed.
!Maria Gomez-Taylor indicated interest in prelaboratory certification — the sampling/field side of monitoring.
Participants were asked to volunteer what they saw as their top priorities for the Board, both short and long term. Attachment 2 summarizes the priorities mentioned (but not in priority order). Many of the priorities raised fell into one of three general categories: (a) database and data sharing issues; (b) method development, new methods, reference method selection, and method flexibility (PBMS); and (c) method validation, data quality objectives (DQOs), and quality control issues.
Delegate/Alternate Term Length
Herb noted that the Board is still soliciting delegates and alternates for some sectors. Delegates and alternates can serve up to a 3 year term according to the Charter, but terms need to be staggered so that only a subset (approximately one-third) will cycle off the Board in a given year. Therefore, in the first year, delegates and alternates need to have different length terms to create the staggered effect desired in the Charter.
By February 18, delegates and alternates need to tell Herb or Merle their preferred term length, after which Herb and Merle will evaluate the responses and determine the need to make any adjustments so as to fulfill the Charter requirements.
Barbara Erickson presented information on the National Environmental Laboratory Accreditation Conference (NELAC) meeting held recently. The purpose of NELAC is
to foster the generation of environmental laboratory data of known quality in a cost-effective manner through the development of national accreditation standards for environmental
laboratories. A central issue is comparative quality of data. Some states will have trouble joining NELAC due to legal problems. Virginia is one state that passed legislation so that they could be part of NELAC. Barbara indicated that 18 states have already applied for inclusion in NELAC. She observed that there is disagreement among EPA programs on the proper direction of NELAC. Lynn Bradley remarked that the Defense Department may not have bought in at this time, which could mean that contract laboratories working for DOD might not participate in NELAC. The consensus is that NELAC has already come a long way but there are many hurdles yet to overcome. Most feel that NELAC is likely to be a reality and it will be an important force.
Comments from National Council
Elizabeth Fellows and John Klein gave a brief history of ITFM, the evolution and purpose of the National Council, and its relationship with the Board. Elizabeth reiterated her belief that Methods issues and the mission of the Methods Board were central to the Council’s mission and to the National strategy for water monitoring.
Elizabeth remarked that the National monitoring strategy is voluntary but she and John recognized that the Methods arena also needs some bite to work. The Board needs to find the right mix of bite and voluntary actions. She suggested that the Board define the most important issues and figure out the best strategy to get us there. Elizabeth also reminded the participants that the Board answers to the Council. There is a hierarchy. The implementation package the Board develops is ultimately most important — not how to get the package through the bureaucracy.
What about resources for the Board’s activities? Elizabeth answered that the Board needs to set its priorities and determine required resources. The resource request will be evaluated by USGS and EPA in consultation with the National Council.
EPA’s NCOD (National Contaminant Occurrence Data Base for Drinking Water and Source Water) was discussed in response to Elizabeth’s remarks, as this project has natural links with the Board and Elizabeth is involved with it. Elizabeth concurred that the Board could have beneficial input on how this database is designed and used.
Dennis reviewed/summarized NCOD and potential applications. Data elements are being decided already. How accessible will the data be to the public, and for regulated entities? What about data quality caveats, interpretable form, etc.? It was noted that data going in the database will be by Rule, after notice and comment. Therefore, it is unlikely that there will be non-relevant or inappropriate data in the database.
Lynn and Barbara reported that stakeholder participation in NCOD database design is supposed to be completed by January 28, 1998. The project has widespread interest and links with many agencies and organizations. Elizabeth indicated that the NCOD needs to find a good balance in:
! metadata requirements — enough but not burdensome
! database sleekness — easy data in and out
Elizabeth remarked that there will not be a single "database in the sky" that satisfies all purposes. Rather, NCOD probably will be a separate database along with NAWQA, NWIS (USGS’s "STORET"), STORET, and others. Dennis McChesney observed that laboratory LIMS-type systems are at the front end of all large databases, including NCOD potentially. Potentially, this is a direct issue for the Board.
The Internet accentuates the fact that data are now more easily available and the accessibility of
high quality data is now even more of an issue. In this light, several participants expressed the concern about excluding bad data (poor quality) after they have already been incorporated in databases. Another issue is expressing the quality of data for public, non-technical people. It was pointed out that there are different requirements for each database. Agencies often need to prioritize where resources need to go, which is dependent on interpretation of various types of data in multiple databases.
Herb distributed Chuck Job’s (EPA, OGWDW) write-up on the NCOD. He noted that there are many similarities with current Board priorities and concerns. Herb requested Board review of the latest NCOD background document (to be provided).
Bob Berger, Dennis McChesney, and Andy Eaton agreed to serve on a workgroup to review the current NCOD database design and report back to the Board and EPA.
Roles of Delegates/Alternates
Herb noted that the Board is still soliciting delegates and alternates for some sectors. The Board needs alternates as a way of including more expertise, wider representation, and as John noted, to create a wider network to publicize the Board’s products. Barbara suggested that the delegate should get input on Board issues from their alternate and others in their organization or region. Barbara agreed to help find an alternate in California or Nevada.
The roles and responsibilities of delegates and alternates were reviewed, considering the Methods Board Charter. It is clear that the Board will need to rely on the expertise of alternates in most of its deliberations. However, the function of alternates is not well defined in the Charter; for example, how one designates an alternate to vote for a delegate that is absent from a meeting (should that become necessary). It was decided to defer this issue, assuming that the way in which the Board operates subsequently will better define the role of alternates. Potentially, the Charter can be modified to deal with this issue.
Presentations of Previous Board Products
Following Ann Strong’s presentation (a handout was available which can be obtained from Jerry Diamond if desired) the following questions/comments were expressed:
!How will reference methods be defined or decided? Ann suggested that the Board probably consider methods from consensus organizations first as these methods have already been validated in most cases.
!The Board needs to interact with EPA’s PBMS streamlining and reference method processes.
!Is a reference method needed? Some participants suggested that the larger issue is to convince users that a given method is truly validated; i.e., the method meets measurement quality objectives.
!States and federal agencies need to agree on a PBMS approach, otherwise labs will not be interested in developing methods or in supporting a PBMS approach. To a large extent, this may be a policy issue rather than a technical one.
!Richard Ayers noted that states are waiting for permission to recognize flexibility in methods for NPDES and drinking water monitoring. Right now, states can’t allow flexibility under current wastewater and drinking water regulations.
!Costs associated with lab development of new methods, and for regulatory evaluation of various methods, could be significant.
!Barbara admonished that we don’t get hung up in details; don’t be afraid to change the system; don’t fall back on: "that’s what we fight in court all the time".
!Problem with availability of reference materials, especially in different matrices commonly encountered. What can the Board do to address this situation?
!Bob Carlson noted that within academia, peer review and publications are the consensus process for methods development and acceptance. Within EPA and other agencies, what is the process? Couldn’t methods be published and peer-reviewed? There needs to be dissemination and review of methods and modifications. Cliff Annis and Tom McAninch noted, however, that the majority of PBMS modifications are site-specific, and there is not a great need or desire to publish such modifications. Additionally, Andrew Eaton commented that peer review is often not adequate because it still represents an evaluation of a single lab study.
!DQOs should be based on the method itself, not on sample results (because of matrix problems). The QA Plan should embody method requirements.
!A central problem currently is method compliance vs method performance; perhaps PBMS can address this problem.
Field and Biological PBMS
Jerry gave a summary of the conceptual PBMS framework developed by the Task Group for Biological collection methods and field methods in general. He also briefly summarized parts of the Wisconsin Pilot studies as Mike Talbot (WDNR) was unable to attend the meeting. The USGS fact sheet for the biological part of the Wisconsin Pilot can be downloaded from the following internet address: //wwwdwimdn.er.usgs.gov/nawqa/pubs/fs_216-96.html. A more visually pleasing version of the Fact Sheet (exactly like the printed version) can be obtained by accessing the pdf file (same address as above but using pdf in place of html). One needs Adobe Acrobat software to read and display this type of file.
The chemical/nutrients part of the Wisconsin Pilot study is still in draft form and not available. This will also be a USGS Fact Sheet some time in the near future.
Herb noted that Herb Garn of USGS in Wisconsin has submitted an Abstract for a platform presentation on the Wisconsin Pilot work for the upcoming National Monitoring Conference in Reno, Nevada, July 7-9, 1998.
Much of Jerry’s presentation on PBMS is available in the ITFM Technical Appendices and in a paper published in the 1996 Journal of the North American Benthological Society (15:713-279). Copies can be obtained from Jerry on request.
USGS - DuPont CRADA and Drinking Water Initiative
Bill Battaglin (USGS) and Mike Focazio presented a summary of the USGS-EPA Drinking Water Initiative and the Cooperative Research and Development Agreement (CRADA) between USGS and DuPont. The following questions and comments were raised in response to this presentation:
What about data interpretation? Bill noted that both USGS & DuPont can publish data — DuPont cannot use their split sample data as "USGS" data. What about health and environmental effects data for these sulfonyl urea pesticides? Bill responded that there are few data available. Limited data suggests that the lower detection limits being developed by USGS are relevant to potential effects concentrations.
A question was raised as to whether CRADA projects are something the Board wants or should do. After a brief discussion, it was agreed that the DuPont CRADA was relevant to the Board’s mission and objectives. Copies of material related to the CRADA will be distributed to the Board. Subsequently, a review team will be formed. The role of the Board with respect to CRADAs in general warrants further discussion.
Meeting Dates and Locations
The next meeting may be in Reston, preferably prior to the next Council meeting in April. The Board needs to decide on a date. Possibilities are the week of April 20, the week of April 13 , and the week of April 27. The week of April 13 appeared to be better for most participants (and will be proposed based on subsequent discussions).
The third meeting could be in a variety of locations. One suggestion was having the meeting in Reno in conjunction with the National Monitoring Conference or, Bob Held indicated that ASTM, in conjunction with the ASTM D-19 meeting, could offer room space at the end of July also in Reno. Another possibility is Sacramento, CA.
There was general consensus among participants on the Board’s charter objectives, and that the priorities suggested at the meeting and listed in Attachment 2 are by in large specific issues associated with the various Charter objectives. We still need to associate the priorities raised with those listed in the charter. All agreed that PBMS is central to the Board’s mission and to its priorities. In this light, the issue paper on PBMS written by Ann (in the ITFM Technical Appendices) needs to be revisited and perhaps expanded by the Board.
Many participants thought that the Board should devote a block of time at the next meeting to discuss a PBMS framework and to define the issues. All agreed that it would be a good idea to invite Bill Telliard, (EPA, OST), and other appropriate EPA spokespersons, to discuss PBMS issues with the Board.
A workgroup was set-up to work with EMMC/EPA and to scope out the PBMS workgroup charge: Bob Held , Ann Strong, Barbara Erickson, Cliff Annis, Andy Eaton, Dennis McChesney, Bob Carlson, Larry Keith and Jerry Diamond. The workgroup will develop a scoping document or issue paper on how the Board will deal with PBMS issues.
Jerry will get copies of Ann’s appendix to PBMS workgroup people.
Rob Henry will produce matrix of parameters often measured, including different matrices, to help begin identifying and prioritizing methods in need of comparability measurements (objective #1 in the Board charter).
The suggestion was made that the Board needs to determine the most efficient way to distribute information to EPA, USGS, and other large organizations. Ideas include:
!Board Fact sheet could be sent with PE samples by EPA to all labs.
!ASTM Standardization - News
!Announcement or presentations at Telliard’s meeting and Norfolk conference.
!Announcements, Fact Sheets at Association of State Water Pollution Control Agencies (ASWPCA), Association of Metropolitan Sewerage Authorities (AMSA) meetings and other related organizations.
!Bob Berger, Bill Potter and Jerry Diamond agreed to participate in a Publicity Workgroup which will help with publicity direction, guidance. More participants will be solicited.
!Andy suggested that the Board needs to consider press releases in journals, newsletters and other "traditional" avenues in addition to Internet.
!Herb or Merle should give presentation on the Board to the Source Water Protection Conference in April (week of 4/27 Dennis will confirm).
!Tetra Tech will draft a paragraph for publicity in ASTM and other newsletters and a press release that could be distributed. Send paragraph also to Bob Held and Keith McLaughlin.
Participants of Methods and Data Comparability Board Meeting,
January 26-27, 1998
USGS Center, Reston, Virginia
Herb Brass, Co-Chair EPA, OGWDW
Merle Shockey, Co-Chair USGS, NWQL/WRD
Richard Ayers VA DEQ
Michalann Harthill USGS, BRD
Andy Eaton Standard Methods/Montgomery Watson Lab
Bob Carlson Kent State Univ.
Katherine Alben NY Dept. Health/SUNY
Dennis McChesney EPA, Region 2
Maria Gomez-Taylor EPA, OST
Rob Henry VG Elemental/ASTM D-19
Bob Held ASTM
Bob Berger East Bay MUD
Tom McAninch Eastman Chemical/CMA
Cliff Annis, Jr. Merck & Co./CMA
Ann Strong ACOE
Barbara Erickson AZ Dept. Health
Scott Coates AOAC
Chuck Spooner EPA, AWPD
Janice Ward USGS, WRD
John Klein USGS, WRD
Elizabeth Fellows EPA, OGWDW
Howard Hankin USDA, NRCS
Steve Via AWWA
Mike Focazio USGS/WRD
Lynn Bradley Assoc. State Tribal Pub. Health Lab
Brad Fisher Wash. Sub. San. Comm.
Keith McLaughlin USDA, FS
Gary Cottrell USGS, NWQL
Bill Battaglin USGS, ORH
Jerry Diamond Tetra Tech
Giles Miller KY Division of Water
Priorities suggested for the Methods and Data Comparability Board
Uniform data format - share data, minimize data manipulation or promote uniform data manipulation methods
Emerging methods/indicators and methods updates
- Resolve and coordinate the development of new methods (older ones too)
- Microbiological methods / useful indicators of human health risk
- Recommendations for creating and using new methods
- National Technological Transfer Act and EPA methods development
- Role of Consensus organizations; Private sector (ASTM, etc.) methods development
- Dealing with outdated methods
PBMS: importance of reference methods; best affordable technologies, faster, cheaper, better methods consistent with DQOs; issue of regulatory-driven vs DQO driven; non-scientific rejection of data
- interlab studies costly and difficult
- low level methods, field methods
- harmonization of PBMS implementation across EPA programs, state agencies
Glossary: e.g., "comparability"; expand and refine definition of PBMS
Method Validation Process: Clarification and coordination
Matrix issues: methods validity (robustness), detection limit versus quantitation, wider variety of reference materials for different matrices; transferability of data from one matrix to another
DQOs: goal oriented monitoring menu
support the use of DQOs in all monitoring design/methods
Menu of methods meeting various DQOs: purposes for which data can be used, cannot
Guidance, recommendations on representative sampling: real time, frequency, timing, size of samples; when data are "accurate" reflection of reality; how to ensure that samples accurately reflect parameter of interest; low-level monitoring
Volunteer monitoring methods:
Robustness, correlation with more sophisticated methods
e.g., chlorophyll a or nutrient preservation methods
Public outreach, include groups interested in "comparability"; communicating Board recommendations
Pilot projects - to determine comparability among collection methods; data collection
Monitor NELAC Progress
Application of biomonitoring data to TMDL (Total Maximum Daily Load) process, clean-up goals, Clean Water Act 303(d) (impaired) water body designation
MDL (method detection level) definition / process, method
Observations / Considerations
!Explore other ways of "doing business"; keep framework simple, usable
!Explore built-in validation for instrumentation manufacture, other new parameters of interest or those not currently available
!Recognition of regional differences or site specific considerations in methods requirements and performance characteristics (e.g., Eh/pH, salinity)
!Awareness and incorporation of EPA’s upcoming Criteria, standards in methods issues: endocrine disruptors, Hg, disease (Pfisteria), pharmaceuticals, pesticides/herbicides, chemical/biological/radiation (quantification methods)