Methods and Data Comparability Board Meeting Minutes

June 21 - 23, 2000

Narragansett, RI

 

 

Participants

 

Herb Brass                        USEPA/OW/Co-chair

Charlie Peters                    USGS/WRD, Co-chair

Harold Aroudel                USGS/WRD

Katherine Alben               NYDH/SUNY

Gil Dichter                         Idexx

John Klein                         USGS/WRD

Ed Santoro                        DRBC

Dennis McChesney         EPA Region 2

Jerry Diamond                  Tetra Tech, Inc.

Abby Markowitz              Tetra Tech, Inc.

Matt Brinn                         ETCC

Jack Krueger                     Maine DEP

Bart Simmons                    CA EPA

Ellen McCarron                FL DEQ

Glenn Patterson                USGS, WRD

Dario DelSanto                 RTI

Linda Green                       URI

 

Andy Eaton                      Std Methods/Montgomery Watson

Larry Keith                        WPI

Rick Dunn                         Hach

Cliff Annis                         Merck/CMA

Elane Streets                     Argonne, Nat’l Lab

Steve Moulton                 USGS/NAWQA

Dan Sullivan                     USGS

Richard Ayers                  VDEQ

Sam Stribling                     Tetra Tech, Inc.

Sherwin Beck                    Tetra Tech, Inc.

Jim Boiani                          DynCorp

Chuck Job                         USEPA/OW

Adriana Cantillo               NOAA

 

 

Austin National Water Quality Monitoring Conference, April 25-27, 2000

 

Herb and others noted that conference participants were very supportive of Board products and activities.  Many state people and those with volunteer monitoring interests expressed interest in the Board.  Participants were generally supportive of the preliminary framework on method and data comparability presented at the break out session.  Suggestions made by participants included:

 

·       The MDCB needs to take a more general, integrated look at comparability issues – consider developing guidance on the DQO process for monitoring programs using specific examples and concrete steps that others could follow. Include information regarding the importance of communication between the upper level managers, project leaders, lab managers, and data users.

 

·       Consider developing an “expert system” to help guide users through the DQO process, MQOs, and choosing appropriate methods depending on DQOs.

 

·       The MDCB needs a place for people to access information about methods issues – provide a clearinghouse of methods information.  Greater outreach in general is needed.

 

 

 

Advisory Committee on Water Information (ACWI) Meeting May 16-17, 2000

 

Herb and John noted that ACWI was very receptive to the progress of the Council and the Board and agreed to take up recommendations from the Council and the Board.  It was agreed that decisions could be made by ACWI in between their annual meetings to help move the process more quickly.  Herb noted that the federal lab accreditation paper should be one of the first products and recommendations to go to ACWI.  NEMI should also be presented to ACWI with recommendations for further development and greater participation (resources) by other agencies.

 

Outreach and Communication

 

Both the NWQM conference in Austin and the ACWI meeting underscore the need for Board outreach and communication with other groups and the public.  Board participants recognized several vehicles for outreach: (1) web site and links with other sites, (2) clearinghouse for information (e.g., organized, annotated catalogue of web sites for different methods-related purposes), and (3) guidance documents/papers/presentations (e.g., NEMI, PBMS paper, accreditation paper).

 

Web Site

 

·       Need overarching integration framework on homepage so the public can easily see relationships among workgroups and Board activities/products.  Then lead them to more technical discussions elsewhere in the site.

 

·       Need lay explanation of Boards’ mission and objectives, particularly in the larger context of the Council and ACWI.

 

·       Each Workgroup needs a lay description of their mission, objectives, and how they are linked to other Board activities.

 

·       Perhaps consider a “tutorial” on how to use the web site.  The web site could then serve as a teaching tool as well as a clearinghouse of information .

 

·       Be careful that the web site is not a product in itself – it is a vehicle for communicating products to the public.

 

·       Abby will finalize link letter and package on the Board by the end of June for distribution to other organizations.

 

·       All information on internal site will be classified by workgroups as either: (a) move to public page, (b) move to public site but password protect, or (c) archive.

 

·       Letter needs to be finalized for obtaining permission to use organization logos on web site.

 

Products/Guidance

 

 

·       PBMS dog and pilot brief need to be published in newsletters (Environmental Laboratory, others), journals (e.g., ES&T).  Jerry will pursue.

 

·       PBMS dog needs to be published (AWWA journal?)  Andy and Jerry will coordinate.

 

·       An outreach strategy should be developed for activities (executive summaries, biology performance paper, accreditation paper, NEMI progress, etc.) should be prepared and distributed in appropriate newsletters, journals.

 

Other Outreach Activities

 

 

·       Abby will contact, via email, Austin conference participants who are interested in the Board.  This contact will let people know we put their forms showing their interest and that the Board will be following up shortly.  She will also send these people a “delegate profile form” and request a resume so that the Board can determine the best fit.

 

·       Board and Council outreach groups will meet together at the next Council meeting in August to coordinate efforts and to determine the most complimentary relationships between the respective web pages. ACWI should provide the broad public focus, NWQMC the senior monitoring manager level focus, and the Board the technical filed/lab level focus.

 

·       Tetra Tech is developing a slide bank for the Methods Board consisting of over 15 slide presentations made by various people for the Board and its activities.  Tetra Tech will organize these and distribute to Board participants for later use as needed.

 

·       A minutes template will be prepared and distributed to work group chairs and outreach liaisons to facilitate preparation and website posting of meeting minutes.

 

·       Develop an updated Board fact sheet from the MTM and WTQC papers being prepared by Charlie and Herb.

 

·       Develop an information packet that would include: Board fact sheet, Work Group fact sheets, Council brochure, poster tear off/URL sheet, Board roster, product blurbs and executive summaries, power point status report, board mission statement, etc.

 

·       Individuals from the conference “yellow pages” that were recommended as new work group participants are #’s 1, 12, and 20.

 

Accreditation

 

 

·       Executive summary was presented at the ACWI meeting and received support from ACWI members.

 

·       Paper needs to be revised in time for review by the Council meeting in mid-August.  Tetra Tech will coordinate with the workgroup to get a revised paper distributed by August 1 so Council members can review prior to their meeting.  Assuming Council approval, the paper then needs to be submitted to ACWI for approval and implementation of recommendations.

 

·       Some concerns about EPA support for NELAC.  The Accreditation paper could help build support for NELAC as well as helping to focus NELAC on accreditation issues important to many federal labs.

 

·       Tetra Tech and others presented preliminary revisions to the paper and all agreed that the paper should be brief and focused on federal lab accreditation.  Tetra Tech will submit a revised paper to the Workgroup by July 5, 2000.

 

·       A conference call is planned July 6 to discuss progress on revisions and information still needed from workgroup members.  Harold will set-up the call.

 

·       Harold, Jerry Parr, and Bart Simmons will be at the NELAC meeting the week of 6/26/00.  They will consult with others there on the Boards’ recommendations and where NELAC is headed.

 

·       Individuals from the conference “yellow pages” will be selected for review of the position paper.

 

 

Biological Methods

 

 

·       Sam Stribling discussed revisions to the performance paper and will revise the paper by July 31, 2000 for wider peer review.

 

·       Members discussed the need for reference voucher specimen collections as a tool analogous to reference materials for chemistry.  Guidance from the Board may be needed in this area.  Steve Moulton is interested in coordinating efforts on this.

 

·       A focus group for field biological methods will be formed to reach consensus on NEMI fields and data dictionary, performance-based approach for determining method and data comparability for field methods, and develop pilot study ideas following a DQO approach.  Mike Miller, Sam Stribling, and Jerry Diamond will be coordinating the focus group.  Potential group participants will be contacted within the next 2-3 weeks.  A similar type of focus group may be warranted for toxicity methods.

 

·       Jerry informed the Workgroup that Tetra Tech received a grant from WERF for designing a study to examine whole effluent toxicity data in relation to bioassessment data.  Central to the grant is Board peer review and some coordination tasks.  An inherent part of the approach is formulating and using DQOs for the study design.  This work will dovetail with the focus group activities as well as those in other parts of the Board.

 

·       The group agreed that we need a compilation of comparability pilots that have already been done and studies that have evaluated performance characteristics of biological methods, or sources of variability.  Ed, Mike, Steve, Sam will coordinate this effort.

 

·       The Workgroup will provide NEMI with answers to questions regarding data fields by mid-July.

 

·       Individuals from the conference “yellow pages” selected for involvement in biology work group efforts are: #’s 3 and 11.

 

PBMS

 

COD Pilot

 

 

·       Matt is getting signatures for the CRADA.  Herb and Larry Fradkin are assisting.  Matt will email version 4 of the SOW to Andy and Rick for their review.

 

·       Rick prepared a detailed study scope for discussion.  Andy will be the primary contact for the project.  Cliff will be the contact for data auditing.  Jerry will manage data analysis and report preparation.

 

·       Training for the pilot will be on 8/23/00 at the Hach facility in Loveland, CO.  Region 3 (and may be Region 2) may not be able to attend.  Herb will look into travel support for a Region 3 person.  However, they may not need training as they’re running the new method now.  Rick will contact Region 2 lab about training.

 

·       Sample collection should occur between 9/11/00 and 9/22/00.  Analytical work would then be finished by 10/6/00.  Data will be submitted for auditing by 10/13/00 and audited by 10/20 00 in time for the Board meeting on 10/23 - 24.  Tetra Tech will finish analyzing data by 11/15 and Workgroup data evaluation will be completed by 11/30.  Tetra Tech will prepare report for Board meeting in January 2001.

 

·       Cliff will determine the data package required from labs for auditing.

 

·       Rick will revise the detailed protocol by mid-July.

 

·       Jerry will make minor changes to the Pilot Protocol and distribute version 3.2 by mid-July: Merck will not be conducting analyses, they will be auditing data; spike concentration needs to be 200 mg/L not 100 mg/L as stated to avoid possibility of spike within the accepted precision range at high COD concentrations; labs should run 3^rd party PT samples as well in the initial capability demonstration.

 

·       The pilot needs a “Tier 1" lab.  Jerry will contact Hampton Roads POTW and perhaps others to solicit participation.

 

·       The Pilot should not be held up by the CRADA.  Any labs that participate later do not have to sign the CRADA or could sign the CRADA afterwards.

 

Other Issues

 

 

·       Herb needs a brief on PBMS Workgroup activities and the COD Pilot for an EPA senior managers meeting in mid-July.

 

·       ACWI encourages the Board to move recommendations forward.  For PBMS, develop a strategy for PBMS implementation based on the Dog and the Pilot.  Perhaps dovetail effort with ACS pilot results.  All agreed that we should wait for results of the COD pilot before going too far on the implementation strategy paper.

 

·       Brief used for ACWI needs to be published in ES&T, Environmental Laboratory, Environmental Testing and Analysis, Washington Report, and other publications.  Jerry will take the lead.

 

·       The Dog needs to be published.  Jerry will talk to Bob Berger about prospects for publishing in a WEF journal.

 

·       Individuals from the conference “yellow pages” selected for review of the COD pilot effort are Martin and Ingersoll.

 

 

WQDE

 

Microbiological and chemical tables have been accepted by the Board.  Workshop notes will be reviewed and WQDE tables revised based on that review. The tables will be combined and integrated as possible – include a new column in the table for chemical, microbiological, or both. Chuck will send them out for wider review (NOAA, NRCS, EPA OWOW/Storet, USGS NWIS/NAWQA, Board, Council, ACWI, yellow pages volunteers).  Jerry will provide Chuck with a statement concerning the type of parameters addressed in the microbiological table and what parameters are not covered in the table.  Chuck will prepare a Federal Register announcement of regional work shops to review the WQDE lists.

 

 NEMI

 

 

·       Progress was reviewed to date with respect to a task-related timeline prepared by DynCorp. All tasks were completed on time with the exception of those involving biological methods. These tasks were not completed due to lack of technical information on those methods.

 

·       The list of about 100 chemical methods to use for Phase I testing of the business rules and prototype structure of NEMI was reviewed and approved by the full board. Previous review and acceptance by the NEMI steering committee occurred earlier this month.

 

 

·     Gil Dichter, Katherine Albine, and Jerry Diamond will review the draft table of initial biological methods and lead the refinement of it. Chair of the workgroup will email the initial table of biological methods to gil-dichter@idexx.com.

 

·     Chair of the workgroup, USGS, and DynCorp will provide a list of questions to Tetra Tech to distribute to the biological methods workgroup by mid July. Answers will be provided to the NEMI workgroup by mid August. The first questions are:

 

 

51.              When biological methods have meta data generated on an "analyte" (i.e., species, etc.) from multiple matrices (i.e., tap water, waste water, estuaries, etc., can one matrix that is most commonly used among methods for that "analyte" be prioritized as the matrix of choice for inclusion of data in the database? Provide a table of most commonly occurring matrices used for biological methods.

52.               What unique identifier will be used with each biological "analyte" entity?

53.              What fields of information will comprise the data dictionary (fields of information) for the biological methods? Provide a table of fields.

54.              Which of the fields in the biological methods data dictionary should be searchable?

55.              What units of concentration will be used with macrobiological analytes?

 

 

·     Significant progress was made in determining a first draft of business rules for the software program for chemical methods. Chemical methods include radiochemical methods, non-radiochemical methods, and nutrient methods. The initial business rules include:

 

 

(57)           IF a method has a publication date THEN enter that date as part of the method reference information in the data dictionary field for method date.

 

(58)           IF there exist multiple classes of methods for a particular analyte (e.g., uranium can be measured several ways as an inorganic analyte, with a unit of _g/mL or _g/L or it can be measured several ways as the radionuclide, with activity units of pCi/mL or Bq/L), THEN the most commonly used unit for that analyte will be consistently used for the given class of methods.

 

(59)           IF a single method measures different analyte concentrations with different mass units (e.g., Cu as ug/L and Ca as mg/L) THEN use the unit most commonly associated with the analyte rather than using a consistent unit across all analytes whose concentration is measured as mass.

(60)            IF a method measures an analyte concentration in non-mass units (e.g., pH and specific conductivity), THEN include those units with the measured values.

 

(61)           IF a method measures the activity of radionuclides, THEN use the International System of Units (SI) recommended  concentration units of Becquerels (Bq)/volume or mass unit.

 

(62)           IF a method measures microbiological "analytes" THEN use concentration units of CFU/100 mL.

 

(63)           IF a method measures macrobiological "analytes" THEN use concentration units (to be determined by the biological methods workgroup).

 

(64)           IF a method contains precision data from both a single source and from multiple sources THEN use both types of precision data up to a maximum of two different sources.

 

(65)           IF a precision value is entered THEN also include an entry in the precision descriptor dictionary field that describes whether the data is from a single source, multiple sources (with a description if possible), or from an unknown source.

 

(66)           IF a method contains precision data generated from multiple concentration levels (e.g., low, medium, and high concentration ranges) THEN include precision data from the lower concentration range unless it is not available or is not deemed representative of the method performance.

 

(67)           IF a method contains detection level data THEN use available values rather than converting them to a single normalized concentration level unit.

 

(68)           IF a method contains detection level data THEN include a description of what kind of detection level data is in the data field (i.e., Method Detection Limit, Minimum Detection Limit, etc.) in a data dictionary field describing detection level data.

 

(69)           IF a method contains equations for calculating precision and accuracy THEN use the lowest concentration of the stated range if there is one, or else use the detection level to calculate values for precision and accuracy and enter the resulting calculated values rather than the equations.

 

(70)           IF a method contains accuracy data THEN enter it in either a field for % recovery or in a field for % false positives and % false negatives or if both are present.

 

(71)           IF a biological method contains accuracy data THEN the common terms are selectivity and specificity and these shall be the data dictionary value field. However, selectivity = % false negatives and specificity = % false positives. With biological methods the terms selectivity and specificity shall be the priority term used since people are used to seeing them but the terms % false positives and % false negatives will also be included in parenthesis after each.

 

(72)           Unique ID #: Use CAS #s when available, or otherwise derive one from best practice.

 

 

·     Larry will contact Scott Coates to determine his interest in being a liaison to AOAC for discussions involving method summary input from that organization.

 

·     Larry will coordinate preparation of a quarterly progress report. Information will be obtained from portions of DynCorps monthly reports plus input from Dan Sullivan at USGS. An initial draft will be circulated for approval and then submitted to Jerry Diamond and Herb Brass.  Draft report target dates will be mid July, mid October, and mid January.

 

 

·     The first report is needed by Herb by July 14^th in text format. A PowerPoint format will be prepared by July 19^th for a July 25^th meeting of EPA's Office of Water Senior Advisor Management Group.

 

 

·     A report containing initial plans for Phase I construction of NEMI will be prepared for circulation to government agencies (through ACWI) and individuals from the April 2000 NWQMC meeting who expressed interest in serving as peer reviewers. The report will include:

 

 

(77)           The initial list of methods whose summary information will be included,

(78)           The presentation format of information from NEMI,

(79)           The initial list of data dictionary fields of information, and

(80)           The initial business rules that will be used to input data.

 

 

·     Government agencies will be asked to comment on whether these initial plans meet their needs and how they can contribute and be more active with NEMI's continued development (including potential funding for NEMI phases II and III).

 

·     Individuals will be asked to comment on the presentation format in addition to the other described technical content. Larry has evaluated the responses from individuals expressing interest in NEMI, and categorized them in the table below. This table will be used for further action, including initial contacts.

People Interested in NEMI From the NWQMC Meeting

 

Priority	Name	Comments
1	Lindsay Martin	Statistics and monitoring experience
1	Bill Ingersoll	Chemist, Navy lab, works with Jackie Sample
1	Karen Williams	QA/QC, monitoring, volunteer work, only category
1	James Bauer	Chemist, education
2	Diane Wilson	Monitoring, volunteer work - Select her or Bauer
2	Teresa Bauer	HS Teacher, volunteer work - Select her or Wilson
2	Alan Cherepon	TNRCC, pesticide monitoring