United States Department of Health and Human Services
Decorative bullet image: Home
Decorative bullet image: Questions?
Decorative bullet image: Contact Us
Decorative bullet image: Site Map
HHS Logo Bottom
spacer image
    

Guidelines for Ensuring the Quality of Information Disseminated to the Public

G.   Health Resources and Services Administration

I.   Agency Mission

The Health Resources and Services Administration (HRSA) is the lead Federal agency in promoting access to health care services that improve the Nation's health. With a statutory emphasis on special needs, underserved, and vulnerable populations, HRSA mobilizes its bureaus, programs, staff, and partners to assure access to quality health care. The four main strategies to achieve this goal are to: eliminate barriers to care; eliminate health disparities; assure quality of care; and, improve public health and health care systems. To fulfill HRSA's mission, its programmatic portfolio includes a range of programs or initiatives designed to increase access to care, improve quality, and safeguard the health and well-being of the Nation's most vulnerable populations.

Collectively, HRSA programs work to improve access to care for the more than 38.7 million Americans who are uninsured and the 40 million who live in medically undeserved areas. HRSA supports over 700 community health centers; funds services for people living with HIV/AIDS through the Ryan White CARE (Comprehensive AIDS Resources Emergency) Act; assists States and health care organizations in improving services to mothers and children; oversees the National system that allocates organs, tissue, and bone marrow for transplantation; and works with academic health centers and other training programs to enhance the diversity and distribution of the Nation's health care workforce.

The primary operating units in HRSA each contribute to the overall mission and major goals and objectives:

  • The Bureau of Health Professions: provides national leadership to assure a health professions workforce that meets the health care needs of the public.

  • The HIV/AIDS Bureau: provides leadership in the delivery of high quality HIV primary care and supporting services for uninsured and underinsured individuals and families affected by HIV/AIDS.

  • The Maternal and Child Health Bureau: works on behalf of America's mothers, children, and families in ways that will assure continued improvement in their health, safety, and well-being.

  • The Office of Rural Health Policy: serves as the leading Federal proponent for better rural health care services.

  • The Office of Special Programs: ensures access and capacity to scarce resources including assurance of access to organ and bone marrow transplantation programs.

  • The Bureau of Primary Health Care: increases access to comprehensive primary and preventive health care and works to improve the health status of underserved and vulnerable families and individuals.

More information about HRSA, its programs and activities is available on the HRSA web site www.hrsa.gov.

II.   Scope and Applicability of Guidelines for Agency/Office

HRSA will ensure that disseminated information meets the standards of quality set forth in the OMB, HHS and HRSA guidelines.  It is HRSA's policy to ensure and maximize the quality, objectivity, utility, and integrity of information that it disseminates to the public. We strive to provide information that is accurate, reliable, clear, complete, unbiased, and useful. We are committed to integrating the principle of information quality into every phase of information development, including creation, collection, maintenance, and dissemination.

The guidelines apply to substantive information that is disseminated by Federal agencies subject to the Paper Reduction Act (PRA), excluding information that is someone's opinion rather than fact or the agency's views. More specifically, it applies to agency initiated or sponsored information. The HRSA information to which these guidelines would apply are aggregate statistical types of information that are abstracted from grantee applications, policy and program information (i.e., data and state profile information, community information, official reports), as well as other electronic documents, newsletters, and brochures that are provided to the public. For example, this includes the HIV/AIDS State Profiles, the Title V Information System web site, the current findings from the National Sample Survey of Registered Nurses, the Area Resource File, and the HRSA State Profiles.

The pre-dissemination review described in the guidelines only applies to information first disseminated on or after October 1, 2002. The administrative mechanism for correction applies to information that the agency disseminates on or after October 1, 2002, regardless of when the agency first disseminated the information.

The guidelines would not include data provided directly to HRSA by grantees, such as individual grantee application forms, reporting requirements, performance plans, etc., but would apply to information that is disseminated from such collections of information. Examples of other types of information collections to which these guidelines would not apply is the information that is collected for the National Practitioner Data Bank (NPDB), since this information is not disseminated to the public but is restricted in access to those entities and persons who, under specified conditions may request information from the NPDB. Other information that is not covered includes papers, books, journal articles, and other documents that have the disclaimer that the information contained therein does not represent the agency's views. Information that is not covered also includes press releases, correspondence with individual persons, archival records, or opinions.  In addition, the guidelines do not apply to press releases that support the announcement or give public notice of information that HRSA has disseminated elsewhere.

III.   Types of Information Disseminated by the Agency to the Public

HRSA disseminates a range of information to the public via the HRSA web site, the HRSA Information Center, and the Office of Communications. The HRSA Information Center is a gateway to information about HRSA programs. All information supplied through this center has been cleared through HRSA and ASPA. Through the Information Center, health care professionals, policymakers, researchers, and members of the public can obtain material on HRSA-supported health programs. The HRSA website also provides information, data, and reports on community health resources, health professions training, HIV/AIDS, managed care, maternal and child health, minority health, oral health, primary health care, rural health, and women's health. Within the HRSA web site there is a wealth of public information. Users of this site may review information on-line or download it for future use.

    1. Examples of Program, Policy, and Administrative data:

        1. The HRSA State Profiles features resources granted by the Agency within each state targeted to serve low-income, uninsured, and medically undeserved populations. To provide an understanding of each state's unique environment, the Profiles also include demographic data and health status indicators. Each State Profile features:

          • An overview of funds granted within a state
          • Selected demographic data
          • Health status and health care access indicators
          • Health care provider resources
          • Highlights of HRSA resources targeted in the areas of primary health care, health care provider resources, maternal and child health, public health, rural health, and HIV/AIDS.

          The State Profiles are six-page reports highlighting the health status of each State's population and HRSA's investment of health resources within each State and the District of Columbia for the years 1997 through 1999. Two categories of data are provided: program data describing resources provided to States by HRSA, and demographic and health status indicator data. For each data element, there are data definitions, source information, types of calculations and derivations, value ranges, and a description of the HRSA Program from which the data were collected. Users can create customized reports by selecting specific data elements, and can generate data to be imported into a spreadsheet or database.

        2. The HRSA HIV/AIDS Bureau State Profiles describe spending and service information of the Ryan White CARE Act programs. Information is provided on location of grantee, clients served, grantee accomplishments, and the characteristics of the HIV epidemic in the state (e.g., co-morbidities and other funding sources such as Medicaid, the largest payer of HIV/AIDS services in the nation), reporting requirements and progress reports for the Title I, II, III, and IV grantees; CDC Surveillance Reports; and other sources including Census and GAO.

        3. The Title V Information System (IS) provides information on the status of maternal and child health in the United States. Data are compiled from annual Title V Block Grant applications and reports submitted by all U.S. States, Territories, and Jurisdictions. Information is provided on key measures of maternal and child health, budgets and expenditures, program data, performance measures, and summary data. Users may download prepared tables, graphs, brochures, fact sheets, and reports on maternal and child health, as well as the Electronic Reporting Package (ERP) that States use to report their data.

        4. The HRSA Preview: the HRSA Preview provides the general public with a single source of program and application information related to the Agency's competitive grant offerings. It contains a description of competitive and other grant programs scheduled for awards in Fiscal Year 2002, and includes instructions on how to contact the Agency for information and receive application kits for all programs. The following specific information is included in the HRSA Preview: (1) program title; (2) legislative authority; (3) purpose; (4) eligibility; (5) funding priorities and/or preferences; (6) estimated dollar amount of competition; (7) estimated number of awards; (8) estimated project period; (9) Catalog of Federal Domestic Assistance (CFDA) identification number; (10) application availability date; (11) letter of intent deadline (if any); (12) application deadline; (13) projected award date; (14) programmatic content, with telephone and e-mail addresses. Certain other information, including how to obtain and use the HRSA Preview and grant terminology, can also be found in the HRSA Preview.

        5. HRSA's Bureau of Health Professions Area Resource File (ARF): the ARF is a county level data set with information on health professions, health facilities, utilization, expenditures, population characteristics, and geographic environment. The data set includes county level estimates for the entire U.S. of the number of physicians, including specialties and limited demographic information, dentists, dental hygienists, optometrists, pharmacists, podiatrists, nurses, physician assistants, and other health professions. Information is available on hospitals, nursing homes, HMOs, inpatient days and other utilization measures, hospital and Medicare expenditures, and various selected variables containing information on the resident population and environmental characteristics. The ARF utilizes secondary data obtained from Census, the American Medical Association, National Center for Health Statistics, and others and compiles a wide array of data sources. As such, the limitations of these data sources all apply, as do any cross source comparability issues.

    2. Example of statistical data:

    3. The National Sample Survey of Registered Nurses (NSSRN): The Seventh NSSRN was conducted in 2000 and is the nation's most extensive and comprehensive source of statistics on all those with current licenses to practice in the United States. It provides information on the number of registered nurses, their educational background and specialty areas; their employment settings, position levels, and salaries; their geographic distribution; their personal characteristics including gender, racial/ethnic background, age, family status, and satisfaction with their job. The substantial database resulting from the 2000 study may be used for many different types of analyses concerning a variety of subjects. The report presents an overview of the personal, professional, and employment characteristics of the almost 2.7 million registered nurses in the country as of March 2000. A summary of the findings from the study and some comparisons to the findings of prior studies in this series, are presented in the report. Appendix A contains a series of tables summarizing the data. A review of the survey methodology and the statistical techniques used in sample selection, response weighting, and identification of sampling errors are found in Appendix B. The survey instrument is included in Appendix C of the report.

IV.   Types of Dissemination Methods

Information is disseminated by HRSA by a variety of mechanisms: the HRSA Office of Communications (OC) serves as the Agency point of contact for clearing and producing HRSA information products. Information is available from the HRSA Information Center in the form of publications, brochures, fact sheets, and other resources on health care services. Through the Information Center, health care professionals, policymakers, researchers, and members of the public can obtain material on HRSA-supported health programs. Information on HRSA publications is provided in the Information Center catalog, and may also be ordered directly from the HRSA web site.

For each of the earlier examples given above of information disseminated by the Agency to the public, the following dissemination methods are used:

    1. The HRSA State Profiles are available through the HRSA web site on stateprofiles.hrsa.gov and the information can be viewed and downloaded using Adobe Acrobat and provides a formatted six page profile for each state.

    2. The HIV/AIDS Bureau State Profiles can be accessed at hab.hrsa.gov. The State Profiles can be viewed or downloaded in Adobe Acrobat. The site has a phone number and email contact information for further requests and feedback.

    3. The Title V Information System is available on the HRSA Bureau of Maternal and Child Health web site at www.mchdata.net. Program materials can be downloaded to obtain prepared tables and graphs, brochures, fact sheets, annual reports, and more. Hard copy versions are available from the Information Center. The web site also includes specific contact information for questions regarding the content of the site and questions, problems or comments regarding the functionality of the site.

    4. The HRSA Preview was published in the Federal Register on 8/9/2001 and is also available for downloading on the web at www.hrsa.gov/grants.htm. Additional information is available at:

            HRSA Grants Application Center
            Attention: Grants Management Officer
            901 Russell Avenue, Suite 450
            Gaithersburg, MD 20879

      or by calling the HRSA Grants Application Center at 1-877-477-2123.

    5. The Area Resource File is available in electronic format only. Contact information is provided on the HRSA web site and includes an address, phone number, fax number, email, and web site specific to ARF.

    6. Findings from the 2000 National Sample Survey are provided on the HRSA web site at bhpr.hrsa.gov/healthworkforce/rnsurvey and can be downloaded in Adobe Acrobat. Contact information for the Nursing Data and Analysis Staff in the Bureau of Health Professions, HRSA, is also provided. Preliminary findings were published in a report and are available from the HRSA Information Center.

IV.   Agency Quality Assurance Policies, Standards and Processes for Ensuring the Quality of Information Disseminated to the Public

HRSA reviews the quality (including the objectivity, utility, and integrity) of information before its is disseminated and treats information quality as integral to every step of the development of information, including its creation, collection, maintenance and dissemination.

    1. Program information:


    2. Program information provided by grantees in the form of application data, reporting requirements, performance reports and progress reports, receives a series of reviews prior to any dissemination to the public. Grantees conduct internal reviews in order to provide data that are accurate, consistent, and complete. The HRSA Office or Bureau conducts a standard review to ensure data quality, completeness, and reliability.
    3. Reports, publications, and other products:

    4. HRSA publications, audiovisual products, and exhibits are required to be consistent with government-wide and HHS public affairs policies. HRSA has published guidelines which closely follow the regulations described in the HHS Public Affairs Management Manual and Government Printing and Binding Regulations to assist program staff in obtaining appropriate review and clearance of information products. To produce a publication, a complete HHS-615 (Publication Planning and Clearance Request) form must be submitted to, and approved by, ASPA.

      Publications to be produced at HRSA's request under contract are subject to HHS clearance and the Joint Committee on Printing (JCP) printing procedures. An HHS-524 clearance is required for publications to be developed as part of a larger public affairs services contract. The HHS-524 must be approved before a Request for Proposals (RFP) can be issued.

    5. Statistical data disseminated to the public:

    6. At HRSA, the quality assurance process begins at the inception of the information development process.

      For information disseminated from sample surveys and other research related or evaluative activities, the widely accepted standards of technical and scientific review are utilized to ensure data quality. These standards include, where appropriate, peer review, internal expert review, Institutional Review Board (IRB) review and OMB review.

      For each of the examples provided earlier, the following quality assurance procedures are employed.

        1. The HRSA State Profiles are reviewed by program staff from each Bureau providing input into the profile prior to dissemination. This is an interval review by each Office and Bureau from which data are obtained. The originating office is responsible for ensuring that all necessary internal review, approval, and clearance is obtained prior to dissemination. Where indicators are taken from other sources, that source is clearly indicated and the limitations of the data are described. For the demographic and other data derived from other sources, HRSA provides an explicit list of the source information, the year the data were obtained, citations, and related URLs. For example, for health indicators such as percent of people insured in the State who had private insurance, the source listed is the U.S. Census Bureau, Health Insurance Statistical Tables, and the citation is Table HI-4, Health Insurance Coverage Status and Type of Coverage by State-All Persons, 1998. Each data element contains such a source listing along with definitions.

        2. The HIV/AIDS Bureau uses CAREWare, a software package for use by the HIV/AIDS Bureau providers to track clients and services. The software and manual are available online, and the system captures all the data items required for the reports needed. CAREWare contains consistency and edit checks on inputted data for quality assurance. This system was designed specifically to collect the exact data elements and generate complete reports. The HIV/AIDS program staff review all data for verification prior to dissemination. The CARE Act State Profiles data provides, for each element, a source list and year of data, as well as notes relating to limitations, rounding, and restrictions, where appropriate. For example, for the Title II Profile Report, Clients Served by gender, age, race/ethnicity, exposure category, the source is listed as the Annual Administrative Report to HRSA, with a note that Accounts are rounded to the nearest 10. Data are not reported from cells of fewer than six clients.

        3. The Maternal and Child Health Bureau Title V annual report uses an electronic reporting package that was developed in collaboration with the states in order to improve accuracy and completeness of the report. This reporting package provides for automatic calculations of ratios, rates, and percentages, and carries data over from year to year, and assures that data used in multiple tables is entered only once. This system has in place a mechanism that warns states when conflicting or unacceptable figures appear as a quality check on the final data. Knowledgeable MCHB program staff monitor and review all information submitted in the grant annual report for completeness, accuracy, and reliability.

        4. The HRSA Preview undergoes appropriate internal Division, Bureau, and Office review and approval prior to dissemination. HRSA adheres to all accepted standards applying to the publication of information in the Federal Register. The HRSA Preview undergoes extensive Office/Bureau, and Agency wide review to ensure accuracy, reliability, and comprehensiveness prior to dissemination to the pubic.

        5. The Area Resource File provides county level data from a variety of data sources. As a result the data have a number of limitations and users of the file are provided with a disclaimer as to the applicability and detail of the data. The ARF utilizes secondary data only, and the years and detail for which all county level data are available are considerably limited. Each data source conducts its own review prior to the release of information. Knowledgeable staff within the originating office and across offices conduct standard reviews for completeness and reliability of the data.

        6. The National Sample Survey of Registered Nurses received extensive review during each phase of operation: survey design, sample selection, fielding of the survey, data editing, and analysis. Clearance was required of all proposed sampling methods, survey procedures, the data collection instrument, and analytic plans by HRSA, HHS, and OMB. This clearance requires a series of expert review. A scientific review, an internal review, and IRB review were conducted to ensure that survey procedures, data collection, editing, and analyses would adhere to the highest standards.

          As in all research activities involving human subjects, it is HRSA's policy that adequate protection of participants be ensured in accordance with the provisions of 45 CFR Part 46. Project plans and procedures must be submitted to the appropriate IRB for approval, and the program must execute all necessary Assurances of Compliance with OPRR. The NSSRN obtained IRB review and approval for all survey methods and procedures.

          For the National Sample Survey of Registered Nurses, widely accepted standards as established by the statistical community are utilized to assure data survey quality. For example, the median design effect for the survey was 1.66, an accepted level by the statistical community for a large national survey. The information disseminated in the report provides extensive detail on the analytic issues (sampling and nonsampling errors, standard error calculation, variance estimation, etc.). As with any sample survey, the results are subject to sampling error. The report provides detail on measures of variability, standard errors, information on the computation of the sampling variance and the design effect.

          Further, HRSA strives to demonstrate in its Paper Reduction Act (PRA) clearance packages that each draft information collection will result in information that will be collected, maintained, and used in a way that is consistent with OMB, HHS and HRSA information quality guidelines.

V.   Agency Administrative Complaint Procedures


    1. Responsibility of the Complainant

    2. To seek a correction of information disseminated by the agency, individuals should follow the procedures described below.

        1. A complaint or request for review and correction of information shall be in written hard copy or electronic form;
        2. it shall be sent to the agency by mail or electronic-mail(e-mail); and
        3. it shall state that an information quality request for correction is being submitted.

      The complaint shall contain

        1. a detailed description of the specific material that needs to be corrected including where the material is located, i.e. the publication title, date, and publication number, if any, or the website and web page address (url), or the speech title, presenter, date and place of delivery; and
        2. the specific reasons for believing the information does not comply with OMB, HHS or HRSA guidelines and is in error and supporting documentation, if any;
        3. the specific recommendations for correcting the information;
        4. a description of how the person submitting the complaint is affected by the information error; and
        5. the name, mailing address, telephone number, e-mail address, and organizational affiliation, if any, of the individual making the complaint.

      Complainants should be aware that they bear the ‘burden of proof’ with respect to the necessity for correction as well as with respect to the type of correction they seek.

      Complaints by mail should be directed to:

            HRSA Reports Clearance Officer
            HRSA/OPE, Room 14-45
            5600 Fishers Lane
            Rockville, MD. 20857

      E-Mail complaints should be sent to www.infoquality@hrsa.gov

    3. Responsibility of the Agency

    4. Based on a review of the information provided, the agency will determine whether a correction is warranted and if, so what action to take. The agency will respond to the requestor by letter or e-mail. The agency's response will explain the findings of the review and the actions that the agency will take, if any. The response will consider the nature and timeliness of the information involved and such factors as the significance of the correction on the use of the information and the magnitude of the correction. The response will describe how the complainant may request reconsideration. The agency will respond to all requests for correction within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.

    5. Appeals

    6. If the individual submitting the complaint does not agree with the agency's decision (including the corrective action, if any), the complainant may send a written hard copy or electronic request for reconsideration within 30 days of receipt of the agency's decision. The appeal shall state the reasons why the agency response is insufficient or inadequate. Complainants shall attach a copy of their original request and the agency response to it, clearly mark the appeal with the words, Information Quality Appeal, and send the appeal to the specific agency appeals address:

            Division of Information and Analysis
            Office of Planning and Evaluation
            HRSA, Room 14-45
            5600 Fishers Lane
            Rockville, MD. 20857

      The agency official who resolved the original complaint will not have responsibility for the appeal.  The agency will respond to all requests for appeals within 60 calendar days of receipt. If the request requires more than 60 calendar days to resolve, the agency will inform the complainant that more time is required and indicate the reason why and an estimated decision date.

    7. Comments and Requests for Information:


    8. HRSA's Office of Communications (OC) has maintained a toll-free phone number to answer questions and address comments on subject specific information. This office has served as the point of entry for queries regarding HRSA information, requests for publications, comments and corrections to information disseminated to the public. To date this Office has not received any complaints about information disseminated by HRSA. Calls have been made, however, by States and grantees to correct and/or update information that is maintained on the HRSA web site. These infrequent calls are received by the OC and directed toward the appropriate Bureau for review, verification, and revision. The States/grantees verify the correction/modification of their data to Bureau staff and after confirmation with program staff the revisions are implemented. In addition, the HRSA web site contains a detailed list of HRSA phone numbers by Office and Bureau for comment or for additional information. Comments and questions can be mailed to ask@hrsa.gov or comments@hrsa.gov, or by calling 1-888-275-4772.

VI.   Influential Scientific, Financial and Statistical Information

From time to time HRSA disseminates information that would generally be regarded as influential. In those circumstances, we use the highest standards of reproducibility and transparency for such information to be disseminated to the public.

VII.   References

(Currently HRSA is in the process of updating these references)

HRSA Circular, No. 95-01, Identification of HRSA in Communications Materials Logo Guidelines. HRSA, OC, 5600 Fishers Lane, Rockville, Md. 20857

HRSA Policy Circular, No. 96.05, Protection of Participants in HRSA Research Programs. HRSA, 5600 Fishers Lane, Rockville, MD. 20857

Planning, Clearing, and Producing a HRSA Information Product. 1999, HRSA, Office of Communications, 5600 Fishers Lane, Rockville, MD. 20857

Publications Catalog, 2001. HRSA, Information Center, 5600 Fishers Lane, Rockville, MD. 20857

Skinner, CJ. Aggregated analysis: standard errors and significance tests. In: Skinner, CJ, Holt, D. Smith TMF, eds. Analysis of complex surveys. New York: John Wiley and Sons, Inc. 1989.

The Registered Nurse Population, Preliminary Findings, 2001. The National Sample Survey of Registered Nurses, 2002. HRSA, Division of Nursing, 5600 Fishers Lane, Rockville, MD. 20857

VIII.   Other Agency Specific Policies and Procedures

HRSA, through its Bureaus and Offices, administers a variety of service delivery and demonstration programs. HRSA Bureaus and Offices also administer some specific research programs and a variety of epidemiological and service utilization studies, as well as evaluations which might be considered research. Participants must be protected from potential risks that may be associated with any such research projects. HRSA Policy Circular No. 96.05 establishes policies and procedures to protect human subjects in research programs conducted or supported by HRSA.

This circular gives guidance for compliance with HHS regulations dealing with protection of human research subjects. It states HSRA policy for various levels of protection applicable to HRSA programs, and explains how to determine the protection level appropriate to each program. It provides guidance on exemption for certain research activities or public benefit or service programs, requiring evaluation of such claims by program staff and approval by a HRSA Human Subjects Protection Committee.

If the public checks the HRSA resources for public information but does not find adequate information, they may file a Freedom of Information Act (FOIA) request. The HRSA FOIA Home Page, newsroom.hrsa.gov/efoia.htm has the location and contact information for making such requests. The site contains a guide for requesting information and records from HRSA as well as a copy of the Freedom of Information Act, As Amended. In addition, this site provides the HRSA Annual FOIA Report for Fiscal Years 1998 through 2001.

Last revised: May 14, 2004

Home | Questions? | Contact Us | Site Map | Accessibility | Privacy Policy | Freedom of Information Act | Disclaimers

The White House | FirstGov