This web site was copied prior to January 20, 2005. It is now a Federal record managed by the National Archives and Records Administration. External links, forms, and search boxes may not function within this collection. Learn more.   [hide]
Home    About MDCB    Tools    Strategy    News

Methods and Data Comparability Board Meeting Minutes

January 28-30, 2002

Sandia Laboratories

Albuquerque, New Mexico


Herb Brass, Co-Chair, EPA/OW

Charlie Peters, Co-Chair, USGS/WRD

Chuck Spooner, Co-Chair, National

Council, EPA/OW

John Klein, USGS/WRD

Rock Vitale, Envir. Standards, Inc.

Cliff Annis, Merck & Co., CMA

Ed Santoro, DRBC

Jack Kruiger, Maine Dept. Health

Ed Johnson, NOAA

Steve Posavec, Standard Methods, AWWA

Mike Miller, WDNR

Dennis McChesney, EPA Region 2

Katherine Alben, NYDH


Wayne Einfeld, Sandia

Larry Keith, Instant References

Dan Sullivan, USGS, WRD

Marion Kelly, EPA/EAD

Bill Telliard, EPA/OST

Mary Verwolf, DOE

Ron Jones, FIU

Alan Cherepon, TNRCC

Glenn Patterson, USGS/WRD

Geoffrey Ekuchukwu, USFW

LeAnne Astin, ICPRB

Chris Faulkner, EPA/OW

Colin Hill, Tetra Tech, Contractor

Sam Stribling, Tetra Tech, Contractor

Steve Moulton, USGS/NAWQA

Merle Shockey, USGS NWQL

Gil Dichter, Idexx

Bart Simmons, CAEPA

Julie _______

Jerry Diamond, Tetra Tech, Contractor

Abby Markowitz, Tetra Tech, Contractor

Jim Boiani, DynCorps, Contractor

**The Board thanks Wayne Einfeld and Sandia Labs for hosting the meeting, providing refreshments, and organizing a very stimulating tour of Sandia's microelectronics laboratory.**

NEMI Outreach

  • Tetra Tech identified many potential websites as links to NEMI and approximately 25 journals/newsletters for advertising NEMI.
  • Participants at meeting completed lists identifying conferences and journals with which they could help provide links or advertising for NEMI.
  • NWQM Conference in Madison will be a major outreach venue for NEMI; need to have NEMI Phase 2 up and running by then.
  • Demo of NEMI at the conference would be nice to have; need those experienced in using NEMI at the conference booth.
  • Identified several potential conferences for publicizing NEMI: PittCon, ASWIPCA, Norfolk Conference, AWWA WQTC, WEF.
  • An article on NEMI for ES&T needs to be prepared and submitted.
  • There needs to be a central calendar and people responsible for maintaining the organization of conference outreach for NEMI and other Board products.
  • A short questionnaire for developing the NEMI User Survey was prepared by Tetra Tech and distributed to the NEMI Workgroup for completion by mid-February.
  • At the conference, and other public displays of NEMI, Board needs to get user feedback; outreach needs to clearly communicate potential benefits to users: what's in it for them.
  • Tetra Tech produced a brief 1 page outreach announcement of NEMI that will be reviewed by February 20.
National Monitoring Conference, Madison, WI, May 21-23, 2002
  • Abby briefed the Board on the Conference organization and platform presentations for each session. A draft of the conference organization is on the Albuquerque meeting page of the Boards' internal website.
  • Moderators should have a conference call with their speakers prior to the Conference to coordinate presentations.
Public Website
  • Each Workgroup needs to update their information and remove dead links, outdated information; respond to Charlie by February 28 th .

NEMI Workgroup - Phase 2 Completion

  • Larry prepared a detailed schedule of tasks for completing Phase 2 NEMI (see Table 1, Attachment A).
  • NEMI needs its own URL; is a possibility. Glenn Patterson, Dan, Charlie will help work with USGS to obtain a URL by March 15.
  • Any changes to NEMI will be completed by March 1 afterwhich NEMI will be frozen for Beta review.
  • Dan will move software for NEMI to a separate website temporarily for Beta review (March).
  • Merle will help get ~ 100 USGS methods and James will work with Katherine to get 21 microbiology and 24 immunoassay methods into Phase 2 NEMI prior to the Beta test.
  • Dan is working on exporting tables for users (printable and otherwise).
  • End of March is the projected completion date for Beta review. April will be used to make adjustments as required or recommended.
  • Method providers will review methods for accuracy by the end of February, prior to the Beta review. James will send providers files originally submitted by them with the NEMI information. Larry will send providers a cover letter prepared by Tetra Tech and instructions for review.
  • Twenty potential Beta reviewers were identified by participants (see Table 2, Attachment A); most of them will be people completely disassociated with NEMI and the Board and they will represent government, private, and university sectors. Board members who personally know a Beta reviewer will make the first contact afterwhich Larry will follow-up with a formal letter requesting their review.
  • February 15 th is deadline for getting Beta review contacts to Larry.

Water Quality Data Elements

Implementation of Chemical/Microbiological WQDE
  • USGS working on having NWIS data elements consistent with WQDE recommendation; EPA also working on incorporating WQDE in STORET. EPA and USGS working together on improving data exchange - still many details to resolve.
  • Glenn reviewed status of four pilot projects to implement the WQDE (descriptions of the pilots are on the meeting internal website page).

+ Oklahoma: added some more elements from WQDE; looking for data to populate database and elements

+ Mohawk River, NY: Obtaining approval from EPA on QAPP to then sample and populate database and WQDE.

+ Milwaukee MSD: In progress; analyte names vary among and within databases.

+ Delaware River Basin: No update.

  • Chuck Job and Glenn drafted an article on the WQDE and the process in getting them approved, which has been distributed for review.
Standardized Formats for Data Elements
  • WQDE were not developed with formatting in mind - instead up to user (i.e., database architecture up to organization). However, consistency in format may be critical to data sharing.
  • EPA's Office of Information is in the process of standardizing data formats for EPA databases.
  • WQDE tables could use a third column describing desired format for each element.
Biology WQDE
  • Tetra Tech compiled data elements from 10 databases including STORET, NAWQA, EMAP, and several state databases.
  • Tetra Tech prepared and distributed a first cut of incorporating field biology data elements into existing WQDE.
  • Ed and DRBC prepared a first draft of WQDE for toxicity data.
  • Participants tentatively agreed that Sections 1, 3, 4, and 5 of the current WQDE could be used for all biological data with little changes.
  • Several assignments made to take lead on developing data elements regarding sampling, analysis, and QA/QC.

+ Fisheries: DRBC (Ed Santoro), NOAA (Ed Johnson), USFWS (Geoffrey Ekuchukwu)

+ Organism tissue: NOAA (Ed Johnson)

+ Macroinvertebrates: DNR (Mike Miller), Tetra Tech (Sam Stribling), USGS (Steve Moulton), ICPBR (LeAnne Astin)

+ Plankton: ICPRB (Claire Buchannan)

+ Periphyton: USGS (Pete Ruhl)

+ Amphibians: USFWS

· Update to ACWI needed by April. Draft core elements for biota to be distributed to Workgroup by early March.

· Goal is to have draft core elements for discussion at National Conference in May.

Nutrient Workgroup

  • Jerry will distribute to the Board minutes from EPA conference call in November, 2002 concerning Board comments on proposed nutrient criteria.
  • Workgroup needs to look at memo from Geoff Grubbs, EPA, clarifying EPA position concerning nutrient criteria: on EPA/OST/criteria web page. Jerry will email link.
  • Tetra Tech modifying previous method comparison study for nutrient methods and developing a stand alone white paper.
Nutrient Pilot
  • Ron indicated that more relevant data are available for Everglades nutrient work: Ron's total Nitrogen versus TKN + NO 3 + NO 2 analyzed by Florida DEP, and S Florida Management District.
  • Ron's structured sampling data appears very relevant: 18 months of side-by-side analyses on split samples (biweekly) at ~ 16 sites ranging in nitrogen levels. Preservation different between two methods: FIU-chilled, FLDEP, acid preserved. This would be a real-world comparability assessment.
  • Real world as well as lab study would be useful to test PBMS concept: NOAA wants to relate marine nitrogen measurements to USGS and other upstream nitrogen data. Comparability data would be very useful.
  • Consider FIU Total N method the reference method.
  • Some of the same advantages of Total N as noted for the alternate COD method in PBMS pilot: environmentally friendly, cheaper, more sensitive, faster.
  • Objectives of pilot:

1) show Total N method is an appropriate method given DQOs

2) show comparability of Total N method with historical data based on

TKN + NO 3 + NO 2

3) demonstrate the usefulness of examining lab and field approaches to PBMS and comparability

4) Evaluate MQOs using a single lab

5) Address freshwater - estuary linkage with respect to nitrogen

· Phased approach:

Phase I: - Ron will determine availability of field data for TKN, NO 3 , NO 2

- Ron, Tetra Tech will evaluate field data

- Ron will identify MQOs used previously in the project

Complete Phase 1 by July 2002

Phase II: - Tetra Tech will help design a laboratory case study (like COD

Pilot) and identify MQOs that should be used

Start Phase 2 in May and complete by Fall, 2002.

Phase III: - Interlaboratory, expanded scope study.

- Submit proposal to AWWARF May 1, 2003, building on data collected in Phase I and II.*

- Include other methods (e.g., immunoassay N methods)

- One-two year effort

*Note: Katherine advocated preparing an AWWARF proposal for Phase I, II, and III work, which would supply resources if funded. Bob Berger, however, indicated that the project is more likely to be funded if some data were already collected and presented.

Phase IV: - Evaluate laboratory and field approaches to method


- Develop guidance on how to do performance/comparability assessments

Biology Workgroup

NEMI Activities
  • Katherine refined extensive Excel tables for immunoassay and microbiological methods, including tables specifically for comparing procedures as well as performance characteristics. These are on the internal Board web page. Charlie is transferring them and other files to pdf format so that they view and print better. We are all grateful to Katherine for all her hard work.
  • Katherine, Dan, James, and Jerry will hold a call to discuss how to get immunoassay and microbiological methods into Phase 2 for Beta review.
  • Several approved microbiological methods still need to be incorporated into NEMI for Phase 3; Gil will help on this.
  • Toxicity methods have been on hold due to other priorities but are fairly close to being distributed for wider review. EPA whole effluent toxicity methods will need to be checked with Bill Telliard and Marion before incorporating into NEMI due to current litigation.
  • Alan Cherepon will help Katherine incorporate more immunoassay methods into Phase 3 of NEMI (e.g., Atrazine).
New Technologies
  • Katherine compiled a list of various new biological analytes, methods, and contacts for potential inclusion in NEMI and other Board activities.

+ Botulin toxins: Bart can get help to incorporate methods

+ Katherine will distribute her NEMI input form to participants in the New Technologies workshop, prior to the Conference to obtain more method summaries.

+ New technology characterization is a "look-see" activity for now as some of these methods have not been published or peer reviewed.

Field macroinvertebrate assessment pilot
  • Mike briefed the Workgroup on a proposed comparability pilot he developed with R-EMAP funding. Two different collection methods would be examined. DNR can collect the samples but taxonomy and analyses are limited by current funding.
  • Bill suggested taking out "PBMS" and instead use "comparability" or "performance assessment". Most agreed this would not change the intent or the results of the pilot.
  • Need to review what's available from the NAWQA program. Charlie will contact Peter.
  • Jerry will again distribute MQOs Tetra Tech has compiled for macroinvertebrate assessments as part of their WERF grant. These MQOs could be used to judge performance of different methods.
  • Katherine encouraged Mike to also measure stressors (e.g., sedimentation, nutrients) as well as biological attributes to help "validate" the different assessment methods.
  • Mike and Jerry will flesh out the pilot design and talk with Susan Jackson and Bill Swietlik at EPA to see if any additional funding might be available.

PBMS Workgroup

  • Tetra Tech will revise COD Pilot paper, incorporating comments from EPA/EAD, and distribute to recent commenters and the Board by mid-February.
  • The manuscript will be submitted to ES&T for publication in March, 2002.
  • Tetra Tech will revise the poster on the COD Pilot Study, incorporating revisions made in the paper, for presentation at the Conference in May. The poster will be sent out to the Board for review, sometime in March.
  • The Workgroup will hold a conference call in approximately two weeks to discuss other issues (i.e., data validation issues, DQO activities).
  • The GAO report, previously distributed to the Board, cites the need for encouraging better, less expensive methods/technologies and impediments to PBMS as a barrier discouraging incorporation of new technologies. Jack and Cliff noted that medical and pharmaceutical methods are performance-based. PBMS is not a new concept.
  • EPA's ATP (Alternate Test Procedure) program was discussed as one approach to getting new methods/technologies approved. However, for the drinking water program, and for major method modifications, Herb noted that the approval process can take several years. Elements the Board considers important in PBMS should be incorporated into the ATP Program. All agreed that there should be a presentation of the ATP Program, including users of the program, at the next Board meeting. Bill, Marion, and Herb will coordinate.

NEMI - Phase 3

  • The Workgroup discussed the desire to add a QC indicator and perhaps a ruggedness index to methods in the future.
  • Multi-search and multi-parameter searches are also desirable.
  • Additional search categories would also be useful (e.g., volatiles). James will attempt to incorporate relatively easy groupings into Phase 2, if possible.
  • An Expert system on the front end of NEMI would help make NEMI more usable; e.g., when don't get a method returned on a search, the system gives user advice on how they might adjust their search to obtain methods.
  • Eventually expand NEMI to include air, soil and other methods.
  • Refine output formats so that users can easily compare those characteristics in which they're interested.

Many thanks to Larry, Dan, James for their outstanding job getting NEMI to this point.

Accreditation Workgroup

  • Comments from EPA/EAD, Chuck Spooner, ACWI, and others on the Federal Lab Accreditation white paper were discussed, as well as Tetra Tech's handling of the comments in a revised paper (version 7.1).
  • Tetra Tech will further revise the paper, based on discussions at the meeting, within the next week and distribute simultaneously to the Board and Council for approval.
  • Herb, Bart will brief the Council on changes made to the accreditation paper.
  • Plan to have the approved paper to ACWI by early March for their review and upcoming meeting later in April.
  • Board products such as WQDE and NEMI will be distributed to the NELAC Board, ELAB, and NELAC Quality Systems Committee. Jack, Bart, Herb, and others will review proposed language on performance based standards and recommend changes to the NELAC Quality Systems Committee. Jack will recommend changes to NELAC Standards to conform with Water Quality Data Elements. More interaction with NELAC/ELAB is a critical aspect of the Board's mission and its accreditation recommendations.
  • NELAP is currently restructuring to satisfy legal concerns and to potentially act as a non-profit organization; NELAP is revising its standards in compliance with ISO 17025 and is revising standards to incorporate performance-based testing.
  • Jerry presented a request by members of the North American Benthological Society for the Board to consider helping them coordinate national accreditation or certification of macroinvertebrate taxonomy for aquatic studies. Bart indicated that either the Field Methods or Quality Systems committees of NELAC might be interested in considering this request. However, he noted that NELAC is not likely to consider standards for certifying individuals. They might, however, be interested in developing standards for laboratories that do taxonomic work. Jerry noted that several NABS members would probably be available to help write the standards. Bart will discuss the request with the NELAC Board of Directors at the next NELAP meeting in June. Perhaps it may be feasible to have draft taxonomy standard for 2003.

Outreach Workgroup

  • Tetra Tech has developed an announcement for NEMI that will be distributed widely and also be available at the National Conference.
  • A letter is going out shortly to various organizations, previously identified, to encourage links to NEMI and the Board in general.
  • Tetra Tech will prepare draft handouts for the Conference by early March for Board review.
  • For now, Tetra Tech, along with the Outreach chairs and Charlie, will compile and coordinate the list of future meetings at which the Board will be featured.

Peer Review Issues

  • Herb distributed EPA's Technical Support Center peer review instructions as a guide for developing more formal Board peer review practices (document is also on Board internal website).
  • Those Board products that have author's names should be peer-reviewed by others within the organizations represented on the paper.
  • Peer reviewers should have no affiliation with the Board or Council.
  • Peer review implies reciprocal responsibilities; if an organization plays a role in helping to produce a Board product, then that organization has a responsibility to consider the results or recommendations of the product. This doesn't' mean that the organization must implement the recommendations. However, they should seriously consider the recommendations and respond to the Board in some fashion.
  • Tetra Tech will draft a brief protocol for Board peer review practices and distribute for review and comment by late February.

Meeting Follow-Up and Announcements

  • A mini-Board meeting will be held the week of April 15 in the Washington, D.C. to review NEMI Beta review results; Biology WQDE progress, and the Nutrient Pilot in anticipation of the National Conference.

A full Board meeting will be held June 3-5, probably in Annapolis, MD. The USFW will be contacted to determine meeting space availability.


MDCB and NEMI Steering Committee Report

Albuquerque, New Mexico

January 28 - 30, 2002

The focus of this meeting was on the actions and decisions that need to be completed in order for NEMI to go public this spring. Numerous Action Items were identified and a rigorous schedule was established to complete them. Table 1 summarizes the time line items that must be completed, along with the responsible person(s), between February 5 and May 20, 2002. Table 2 summarizes Sponsors and their contacts for potential Beta Reviewers. Table 3 summarizes additional supporting action items required to be completed (or that were completed) along with the responsible person(s) or entity.

Table 1. NEMI Public Release Schedule


Responsible Person

Action Item


Larry @ Instant Ref.

Albuquerque meeting notes with deadlines distributed


Abby @ Tetra Tech

List of press release & general release contacts to Steering Committee


Abby @ Tetra Tech

Beta Review letter and instructions distributed for approval


Glen Patterson @ USGS

Update of URL approval to MDCB


Larry @ Instant Ref.

All Methods Sources contacted to set up QC reviews


Abby @ Tetra Tech

Draft "world announcement" emailed to Outreach Committee


James @ DynCorp

Original information sent to all methods sources by email


Steering Committee

All Beta Reviewers contacted by sponsors & email addresses provided to NEMI chair (See Table 2).


Abby @ Tetra Tech

Provides draft world announcement & press release for MDCB review


Abby @ Tetra Tech

Beta Reviewer letter and instructions completed & approved


Dan @ USGS

Freeze NEMI target date - this could slip if necessary to 3/13/02


Larry @ Instant Ref.

Beta Reviewers contacted using email by NEMI chair


Dan @ USGS

Confirms approval of URL for NEMI of


Larry @ Instant Ref.

Letter and instructions sent by email to all Beta Reviewers


Methods Sources

All QC of Internet and Original Source Materials completed


Outreach Committee

Contacts for world announcement & press release approved


Dan @ USGS

URL at must be established for QC


James and Dan

Complete all Source Materials corrections in NEMI


Outreach Committee

 World announcement letter & press release approved


Beta Reviewers

Beta reviews sent to Tetra Tech for review and sorting


Abby @ Tetra Tech

EPA/USGS co-chairs given email lists for announcements


Abby @ Tetra Tech

Beta review report to MDCB - NO SHOW STOPPERS


Dan @ USGS

Provides schedule to address all beta reviewer issues


Herb & Charlie

NEMI world announcement emailed by EPA/USGS co-chairs


Herb & Charlie

NEMI press release emailed by EPA/USGS co-chairs


Abby @ Tetra Tech

Finalizes numbers of lines available for Conference Workshop


Larry @ Instant Ref.

Finalizes PowerPoint slides for Conference Workshop


Dan @ USGS

NEMI web site goes live


Dan, Larry, James, & Katherine

NEMI debut at NWQMC conference

The two Action Items in bold are considered show stoppers. The public announcements will NOT go out to the world if these show stoppers are not fully addressed.

Table 2. Beta Reviewer Candidates and Sponsors


Candidate / Classification

Candidate's E-mail

Steering Committee

Ed Johnson * / Fed

Steering Committee

Rock Vitale * / Lab

Steering Committee

Jack Kruger * / State

Charlie Peters

Linda Green / Univ.

Charlie Peters

Ellen McCarron / Fed.

Charlie Peters

? / International person

Herb Brass

Reenie Paris -? /Fed.

Herb Brass

? / International

Herb Brass

? / International

Ed Santaro

Monitoring Advisory / State?

Ed Santaro

Monitoring Advisory / State?

Rock Vitale

Skip Kingston / Univ.

Rock Vitale

Alan Hewitt / Fed

Rock Vitale

Richard Burrows / Lab

Mary Verwolf

Cliff Watkins * / Lab

Mary Verwolf

? / International person

Marion Kelly

Richard Turle / Canada

Cliff Annis

John Treffy / Univ

Steve Posavec

Lenore Clesceri / Private

Dennis McChesney

Larry Bailey * / State

Dennis McChesney

Jack Ryan * / State

Dennis McChesney

Carlton Hunt / Private (Battelle)

Dennis McChesney

? German / International

Katherine Alben

? / Env. Engineer/Scientist

Katherine Alben

? / Env. Engineer/Scientist

Bart Simmons

Silky Labie ? / State

Larry Keith

Martha Wells / Univ.

Larry Keith

Bill Glaze / Univ.

There are 28 potential candidates listed in Table 2. Each sponsor should contact their candidate by phone or e-mail no later than by February 15, 2002 and then send an email to Larry Keith ( ) to confirm whether or not the candidate will accept the task of being a beta reviewer. Each candidate should be told that the beta review will take about an hour and that they will be contacted by Larry Keith by March 1 st to confirm that they agree to participate. Then they will be provided with an evaluation and comment form and the URL for NEMI by March 15 th and this form MUST be returned by no later than April 1 st in order to be reviewed and their responses collated with the other reviewers. Their instructions will provide Abby's email address for return of their review as an attached file. Candidates with asterisks by their name have already agreed to be beta reviewers and do not need to be contacted again prior to confirmation by Larry that they will participate.

Table 3. Other Action Items From the NEMI Steering Committee Meeting

Action Item

Status or Comment

Methods Disclaimer

Done - James will e-mail disclaimer to Dan to post in the web page

Criteria for Methods Submission

Done - James will e-mail disclaimer to Dan to post in the web page

Prepare guidance materials for Beta Reviewers

Abby will complete and distribute draft by 2/8/02 and final by 2/22/02. Larry will send it by 3/15/02 as per Table 1

Prepare "world announcement" and press releases plus associated e-mail contact lists

Abby and Dennis will complete draft by 2/12/02 and distribute final by 3/20/02. Herb and Charlie will e-mail them on 4/8/02 as per Table 1

Obtain USGS approval for

Glen to provide update of contact with USGS officials by 2/8/02 and show stopper deadline of 3/18/02 to be confirmed by Dan as per Table 1

Define relationships between NEMI workshop and Tracks 2 & 3 at conference

Issues to be resolved with Abby's help during future NEMI Steering Committee calls in April

QA/QC of NEMI content

Source materials and Internet report format content will be reviewed by method sources between 2/15/02 and 3/15/02. Corrections are expected to be minor and completed by 3/3/20/02

Complete regulation additions

James and Dan expect to complete this by the end of February but it must be finished before NEMI is frozen March 1 st .

Prepare draft on Past Development and Future Directions for background information on NEMI page and to stress that the current version is only the beginning and will continue to evolve and grow.

Larry will prepare and circulate drafts for review prior to March 1 st . Dan will add "Version 1.0" and also "Future Enhancements" to the NEMI page and make a link from each to this piece.

Develop "Useful Queries" links similar to those provided by DOE. Two lists: easy list for Phase 2 (e.g., organics, inorganics, nutrients, radiochemical, biological) and difficult list for Phase 3 (e.g., pesticides, PAHs, PCBs, Dioxins and Dibenzofurans, volatiles, phenols, semivolatiles, etc.).

James has the lead for the easy lists and may have completed some already. Steering Committee will tackle the hard lists in Phase 3. Easy list needs to be provided to Dan so that he can incorporate it prior to freezing NEMI on March 1 st .

Move web site home page to another server to address instability problems.

Dan will complete moving the home page to the Wisconsin District Sun Solaris web server prior to the March 1 st freeze of NEMI.

Complete misc. small fixes to NEMI (add an icon to the search link, put list of methods in a multiple column table, replace feedback email link, delete "-999", fix empty DOE methods, add more descriptive text to query options, fix forbidden access error in Search Option 1, temporarily remove Search Option 2, delete link "Contributing Organizations," determine which options in "Useful Queries" page should be deleted

Dan will complete these prior to the March 1 st freeze of NEMI.

WICP | ACWI | NWQMC | Feedback
Home | About MDCB | Tools | Strategy | News
Contact Eric Vowinkel ( or Herb Brass (
Maintained by
Last Update: September 15, 2004 September 15, 2004