Methods and Data Comparability Board Meeting Minutes

January 28-30, 2002

Sandia Laboratories

Albuquerque, New Mexico

Participants:

**The Board thanks Wayne Einfeld and Sandia Labs for hosting the meeting, providing refreshments, and organizing a very stimulating tour of Sandia's microelectronics laboratory.**

NEMI Outreach

• Tetra Tech identified many potential websites as links to NEMI and approximately 25 journals/newsletters for advertising NEMI.
• Participants at meeting completed lists identifying conferences and journals with which they could help provide links or advertising for NEMI.
• NWQM Conference in Madison will be a major outreach venue for NEMI; need to have NEMI Phase 2 up and running by then.
• Demo of NEMI at the conference would be nice to have; need those experienced in using NEMI at the conference booth.
• Identified several potential conferences for publicizing NEMI: PittCon, ASWIPCA, Norfolk Conference, AWWA WQTC, WEF.
• An article on NEMI for ES&T needs to be prepared and submitted.
• There needs to be a central calendar and people responsible for maintaining the organization of conference outreach for NEMI and other Board products.
• A short questionnaire for developing the NEMI User Survey was prepared by Tetra Tech and distributed to the NEMI Workgroup for completion by mid-February.
• At the conference, and other public displays of NEMI, Board needs to get user feedback; outreach needs to clearly communicate potential benefits to users: what's in it for them.
• Tetra Tech produced a brief 1 page outreach announcement of NEMI that will be reviewed by February 20.

National Monitoring Conference, Madison, WI, May 21-23, 2002

• Abby briefed the Board on the Conference organization and platform presentations for each session. A draft of the conference organization is on the Albuquerque meeting page of the Boards' internal website.
• Moderators should have a conference call with their speakers prior to the Conference to coordinate presentations.

Public Website

• Each Workgroup needs to update their information and remove dead links, outdated information; respond to Charlie by February 28 th .

NEMI Workgroup - Phase 2 Completion

• Larry prepared a detailed schedule of tasks for completing Phase 2 NEMI (see Table 1, Attachment A).
• NEMI needs its own URL; NEMI.gov is a possibility. Glenn Patterson, Dan, Charlie will help work with USGS to obtain a URL by March 15.
• Any changes to NEMI will be completed by March 1 afterwhich NEMI will be frozen for Beta review.
• Dan will move software for NEMI to a separate website temporarily for Beta review (March).
• Merle will help get ~ 100 USGS methods and James will work with Katherine to get 21 microbiology and 24 immunoassay methods into Phase 2 NEMI prior to the Beta test.
• Dan is working on exporting tables for users (printable and otherwise).
• End of March is the projected completion date for Beta review. April will be used to make adjustments as required or recommended.
• Method providers will review methods for accuracy by the end of February, prior to the Beta review. James will send providers files originally submitted by them with the NEMI information. Larry will send providers a cover letter prepared by Tetra Tech and instructions for review.
• Twenty potential Beta reviewers were identified by participants (see Table 2, Attachment A); most of them will be people completely disassociated with NEMI and the Board and they will represent government, private, and university sectors. Board members who personally know a Beta reviewer will make the first contact afterwhich Larry will follow-up with a formal letter requesting their review.
• February 15 th is deadline for getting Beta review contacts to Larry.

Water Quality Data Elements

Implementation of Chemical/Microbiological WQDE

• USGS working on having NWIS data elements consistent with WQDE recommendation; EPA also working on incorporating WQDE in STORET. EPA and USGS working together on improving data exchange - still many details to resolve.
• Glenn reviewed status of four pilot projects to implement the WQDE (descriptions of the pilots are on the meeting internal website page).

+ Oklahoma: added some more elements from WQDE; looking for data to populate database and elements

+ Mohawk River, NY: Obtaining approval from EPA on QAPP to then sample and populate database and WQDE.

+ Milwaukee MSD: In progress; analyte names vary among and within databases.

+ Delaware River Basin: No update.

Outreach

• Chuck Job and Glenn drafted an article on the WQDE and the process in getting them approved, which has been distributed for review.

Standardized Formats for Data Elements

• WQDE were not developed with formatting in mind - instead up to user (i.e., database architecture up to organization). However, consistency in format may be critical to data sharing.
• EPA's Office of Information is in the process of standardizing data formats for EPA databases.
• WQDE tables could use a third column describing desired format for each element.

Biology WQDE

• Tetra Tech compiled data elements from 10 databases including STORET, NAWQA, EMAP, and several state databases.
• Tetra Tech prepared and distributed a first cut of incorporating field biology data elements into existing WQDE.
• Ed and DRBC prepared a first draft of WQDE for toxicity data.
• Participants tentatively agreed that Sections 1, 3, 4, and 5 of the current WQDE could be used for all biological data with little changes.
• Several assignments made to take lead on developing data elements regarding sampling, analysis, and QA/QC.

+ Fisheries: DRBC (Ed Santoro), NOAA (Ed Johnson), USFWS (Geoffrey Ekuchukwu)

+ Organism tissue: NOAA (Ed Johnson)

+ Macroinvertebrates: DNR (Mike Miller), Tetra Tech (Sam Stribling), USGS (Steve Moulton), ICPBR (LeAnne Astin)

+ Plankton: ICPRB (Claire Buchannan)

+ Periphyton: USGS (Pete Ruhl)

+ Amphibians: USFWS

· Update to ACWI needed by April. Draft core elements for biota to be distributed to Workgroup by early March.

· Goal is to have draft core elements for discussion at National Conference in May.

Nutrient Workgroup

• Jerry will distribute to the Board minutes from EPA conference call in November, 2002 concerning Board comments on proposed nutrient criteria.
• Workgroup needs to look at memo from Geoff Grubbs, EPA, clarifying EPA position concerning nutrient criteria: on EPA/OST/criteria web page. Jerry will email link.
• Tetra Tech modifying previous method comparison study for nutrient methods and developing a stand alone white paper.

Nutrient Pilot

• Ron indicated that more relevant data are available for Everglades nutrient work: Ron's total Nitrogen versus TKN + NO 3 + NO 2 analyzed by Florida DEP, and S Florida Management District.
• Ron's structured sampling data appears very relevant: 18 months of side-by-side analyses on split samples (biweekly) at ~ 16 sites ranging in nitrogen levels. Preservation different between two methods: FIU-chilled, FLDEP, acid preserved. This would be a real-world comparability assessment.
• Real world as well as lab study would be useful to test PBMS concept: NOAA wants to relate marine nitrogen measurements to USGS and other upstream nitrogen data. Comparability data would be very useful.
• Consider FIU Total N method the reference method.
• Some of the same advantages of Total N as noted for the alternate COD method in PBMS pilot: environmentally friendly, cheaper, more sensitive, faster.
• Objectives of pilot:

1) show Total N method is an appropriate method given DQOs

2) show comparability of Total N method with historical data based on

TKN + NO 3 + NO 2

3) demonstrate the usefulness of examining lab and field approaches to PBMS and comparability

4) Evaluate MQOs using a single lab

5) Address freshwater - estuary linkage with respect to nitrogen

· Phased approach:

Phase I: - Ron will determine availability of field data for TKN, NO 3 , NO 2

- Ron, Tetra Tech will evaluate field data

- Ron will identify MQOs used previously in the project

Complete Phase 1 by July 2002

Phase II: - Tetra Tech will help design a laboratory case study (like COD

Pilot) and identify MQOs that should be used

Start Phase 2 in May and complete by Fall, 2002.

Phase III: - Interlaboratory, expanded scope study.

- Submit proposal to AWWARF May 1, 2003, building on data collected in Phase I and II.*

- Include other methods (e.g., immunoassay N methods)

- One-two year effort

*Note: Katherine advocated preparing an AWWARF proposal for Phase I, II, and III work, which would supply resources if funded. Bob Berger, however, indicated that the project is more likely to be funded if some data were already collected and presented.

Phase IV: - Evaluate laboratory and field approaches to method

comparability

- Develop guidance on how to do performance/comparability assessments

Biology Workgroup

NEMI Activities

• Katherine refined extensive Excel tables for immunoassay and microbiological methods, including tables specifically for comparing procedures as well as performance characteristics. These are on the internal Board web page. Charlie is transferring them and other files to pdf format so that they view and print better. We are all grateful to Katherine for all her hard work.
• Katherine, Dan, James, and Jerry will hold a call to discuss how to get immunoassay and microbiological methods into Phase 2 for Beta review.
• Several approved microbiological methods still need to be incorporated into NEMI for Phase 3; Gil will help on this.
• Toxicity methods have been on hold due to other priorities but are fairly close to being distributed for wider review. EPA whole effluent toxicity methods will need to be checked with Bill Telliard and Marion before incorporating into NEMI due to current litigation.
• Alan Cherepon will help Katherine incorporate more immunoassay methods into Phase 3 of NEMI (e.g., Atrazine).

New Technologies

• Katherine compiled a list of various new biological analytes, methods, and contacts for potential inclusion in NEMI and other Board activities.

+ Botulin toxins: Bart can get help to incorporate methods

+ Katherine will distribute her NEMI input form to participants in the New Technologies workshop, prior to the Conference to obtain more method summaries.

+ New technology characterization is a "look-see" activity for now as some of these methods have not been published or peer reviewed.

Field macroinvertebrate assessment pilot

• Mike briefed the Workgroup on a proposed comparability pilot he developed with R-EMAP funding. Two different collection methods would be examined. DNR can collect the samples but taxonomy and analyses are limited by current funding.
• Bill suggested taking out "PBMS" and instead use "comparability" or "performance assessment". Most agreed this would not change the intent or the results of the pilot.
• Need to review what's available from the NAWQA program. Charlie will contact Peter.
• Jerry will again distribute MQOs Tetra Tech has compiled for macroinvertebrate assessments as part of their WERF grant. These MQOs could be used to judge performance of different methods.
• Katherine encouraged Mike to also measure stressors (e.g., sedimentation, nutrients) as well as biological attributes to help "validate" the different assessment methods.
• Mike and Jerry will flesh out the pilot design and talk with Susan Jackson and Bill Swietlik at EPA to see if any additional funding might be available.

PBMS Workgroup

• Tetra Tech will revise COD Pilot paper, incorporating comments from EPA/EAD, and distribute to recent commenters and the Board by mid-February.
• The manuscript will be submitted to ES&T for publication in March, 2002.
• Tetra Tech will revise the poster on the COD Pilot Study, incorporating revisions made in the paper, for presentation at the Conference in May. The poster will be sent out to the Board for review, sometime in March.
• The Workgroup will hold a conference call in approximately two weeks to discuss other issues (i.e., data validation issues, DQO activities).
• The GAO report, previously distributed to the Board, cites the need for encouraging better, less expensive methods/technologies and impediments to PBMS as a barrier discouraging incorporation of new technologies. Jack and Cliff noted that medical and pharmaceutical methods are performance-based. PBMS is not a new concept.
• EPA's ATP (Alternate Test Procedure) program was discussed as one approach to getting new methods/technologies approved. However, for the drinking water program, and for major method modifications, Herb noted that the approval process can take several years. Elements the Board considers important in PBMS should be incorporated into the ATP Program. All agreed that there should be a presentation of the ATP Program, including users of the program, at the next Board meeting. Bill, Marion, and Herb will coordinate.

NEMI - Phase 3

• The Workgroup discussed the desire to add a QC indicator and perhaps a ruggedness index to methods in the future.
• Multi-search and multi-parameter searches are also desirable.
• Additional search categories would also be useful (e.g., volatiles). James will attempt to incorporate relatively easy groupings into Phase 2, if possible.
• An Expert system on the front end of NEMI would help make NEMI more usable; e.g., when don't get a method returned on a search, the system gives user advice on how they might adjust their search to obtain methods.
• Eventually expand NEMI to include air, soil and other methods.
• Refine output formats so that users can easily compare those characteristics in which they're interested.

Many thanks to Larry, Dan, James for their outstanding job getting NEMI to this point.

Accreditation Workgroup

• Comments from EPA/EAD, Chuck Spooner, ACWI, and others on the Federal Lab Accreditation white paper were discussed, as well as Tetra Tech's handling of the comments in a revised paper (version 7.1).
• Tetra Tech will further revise the paper, based on discussions at the meeting, within the next week and distribute simultaneously to the Board and Council for approval.
• Herb, Bart will brief the Council on changes made to the accreditation paper.
• Plan to have the approved paper to ACWI by early March for their review and upcoming meeting later in April.
• Board products such as WQDE and NEMI will be distributed to the NELAC Board, ELAB, and NELAC Quality Systems Committee. Jack, Bart, Herb, and others will review proposed language on performance based standards and recommend changes to the NELAC Quality Systems Committee. Jack will recommend changes to NELAC Standards to conform with Water Quality Data Elements. More interaction with NELAC/ELAB is a critical aspect of the Board's mission and its accreditation recommendations.
• NELAP is currently restructuring to satisfy legal concerns and to potentially act as a non-profit organization; NELAP is revising its standards in compliance with ISO 17025 and is revising standards to incorporate performance-based testing.
• Jerry presented a request by members of the North American Benthological Society for the Board to consider helping them coordinate national accreditation or certification of macroinvertebrate taxonomy for aquatic studies. Bart indicated that either the Field Methods or Quality Systems committees of NELAC might be interested in considering this request. However, he noted that NELAC is not likely to consider standards for certifying individuals. They might, however, be interested in developing standards for laboratories that do taxonomic work. Jerry noted that several NABS members would probably be available to help write the standards. Bart will discuss the request with the NELAC Board of Directors at the next NELAP meeting in June. Perhaps it may be feasible to have draft taxonomy standard for 2003.

Outreach Workgroup

• Tetra Tech has developed an announcement for NEMI that will be distributed widely and also be available at the National Conference.
• A letter is going out shortly to various organizations, previously identified, to encourage links to NEMI and the Board in general.
• Tetra Tech will prepare draft handouts for the Conference by early March for Board review.
• For now, Tetra Tech, along with the Outreach chairs and Charlie, will compile and coordinate the list of future meetings at which the Board will be featured.

Peer Review Issues

• Herb distributed EPA's Technical Support Center peer review instructions as a guide for developing more formal Board peer review practices (document is also on Board internal website).
• Those Board products that have author's names should be peer-reviewed by others within the organizations represented on the paper.
• Peer reviewers should have no affiliation with the Board or Council.
• Peer review implies reciprocal responsibilities; if an organization plays a role in helping to produce a Board product, then that organization has a responsibility to consider the results or recommendations of the product. This doesn't' mean that the organization must implement the recommendations. However, they should seriously consider the recommendations and respond to the Board in some fashion.
• Tetra Tech will draft a brief protocol for Board peer review practices and distribute for review and comment by late February.

Meeting Follow-Up and Announcements

• A mini-Board meeting will be held the week of April 15 in the Washington, D.C. to review NEMI Beta review results; Biology WQDE progress, and the Nutrient Pilot in anticipation of the National Conference.

A full Board meeting will be held June 3-5, probably in Annapolis, MD. The USFW will be contacted to determine meeting space availability.

ATTACHMENT A

MDCB and NEMI Steering Committee Report

Albuquerque, New Mexico

January 28 - 30, 2002

The focus of this meeting was on the actions and decisions that need to be completed in order for NEMI to go public this spring. Numerous Action Items were identified and a rigorous schedule was established to complete them. Table 1 summarizes the time line items that must be completed, along with the responsible person(s), between February 5 and May 20, 2002. Table 2 summarizes Sponsors and their contacts for potential Beta Reviewers. Table 3 summarizes additional supporting action items required to be completed (or that were completed) along with the responsible person(s) or entity.

Table 1. NEMI Public Release Schedule

The two Action Items in bold are considered show stoppers. The public announcements will NOT go out to the world if these show stoppers are not fully addressed.

Table 2. Beta Reviewer Candidates and Sponsors

There are 28 potential candidates listed in Table 2. Each sponsor should contact their candidate by phone or e-mail no later than by February 15, 2002 and then send an email to Larry Keith ( larrykeith@earthlink.net ) to confirm whether or not the candidate will accept the task of being a beta reviewer. Each candidate should be told that the beta review will take about an hour and that they will be contacted by Larry Keith by March 1 st to confirm that they agree to participate. Then they will be provided with an evaluation and comment form and the URL for NEMI by March 15 th and this form MUST be returned by no later than April 1 st in order to be reviewed and their responses collated with the other reviewers. Their instructions will provide Abby's email address for return of their review as an attached file. Candidates with asterisks by their name have already agreed to be beta reviewers and do not need to be contacted again prior to confirmation by Larry that they will participate.

Table 3. Other Action Items From the NEMI Steering Committee Meeting