Meeting Summary

Exploring a National Standard for Measuring Patients' Hospital Care Experiences

Participants

Agency for Healthcare Research and Quality (AHRQ): Carolyn Clancy, M.D., Gregg Meyer, M.D., Chuck Darby, M.A., Katherine Crosson, M.P.H.
American Association of Retired Persons (AARP): Joyce Dubow
California Healthcare Foundation (CHCF): Lisa Payne-Simon, M.P.H.
Centers for Medicare and Medicaid Services (CMS): Elizabeth Goldstein, Ph.D., Beth Kosiak, Ph.D., Rachael Weinstein, Aucha Prachanronqrong
Department of Defense (DoD) Tricare: Lt. Col. Thomas Williams, Ph.D.
Indian Health Service (IHS): Katherine Ciacco Palatianos, M.D., M.P.H.
Joint Commission on Accreditation of Healthcare Organizations (JCAHO): Margaret VanAmringe
National Cancer Institute (NCI): Steven Clauser, Ph.D.
National Institutes of Health (NIH) Clinical Center (CC): David Henderson, M.D., Laura Lee, R.N.
Office of Personnel Management (OPM): Abby Block, M.A., M.S.W., M.B.A.
National Research Corporation/Picker (NRC): Mike Hays, Steve Larson, Paul Huelskamp, Joe Carmichael
The Commonwealth Fund: Anne-Marie Audet, M.D., M.Sc.
Veteran's Health Administration (VHA): Everett Jones, M.D.

Contents

Welcome, Introductions, Meeting Purpose
Need for a United States Standard
CAHPS® Overview
Patients' Evaluation of Hospital Performance in California
Great Britain's National Health Care Service
Centers for Medicare and Medicaid Services: Arriving at a National Standard
Next Steps

Welcome, Introductions, Meeting Purpose

Mike Hays, NRC, and Chuck Darby, AHRQ, formally welcomed the participants and provided an overview of the process followed to develop survey instruments for placement in the public domain. Specifically, they addressed how this process might be used to develop a measure of patient experiences with hospital care. AHRQ has been the Nation's leader in establishing standards for measuring experiences with the U.S. health care system and the Consumer Assessment of Health Plans (CAHPS®) continues to be the ground-breaking work providing the foundation for similar initiatives. To date, CAHPS® has not addressed the hospital patients' experiences.

In the recent past, leaders from across the U.S. health care industry have called for a response to the need to assess the experiences of hospital patients:

• In Crossing the Quality Chasm, the Institute of Medicine listed patient-centered care as one of six aims for improvement. The dimensions of patient-centered care are:
  1. Respect for patients' values, preferences, and expressed needs.
  2. Coordination and integration of care.
  3. Information, communication, and education.
  4. Physical comfort.
  5. Emotional support—relieving fear and anxiety.
  6. Involvement of family and friends.
  7. Continuity and transition.
  8. Access to care.
  Measuring these dimensions will require a standard instrument that produces reliable, consistent results.
• The National Quality Forum (NQF) has recommended several priorities for research, development, and further investigation for suitable performance measures for acute hospital inpatients. One NQF priority area is the need to measure patient experiences with inpatient care.

AHRQ's Acting Director, Carolyn Clancy, M.D., joined the group later in the morning and extended her welcome and thanks to the participants for their insights, suggestions, and willingness to work with the Agency on establishing a national standard to measure hospital patient experiences.

Participants introduced themselves and indicated their specific interest in the development of a national standard.

Return to Contents

Need for a United States Standard

Gregg Meyer, M.D., AHRQ

Gregg Meyer extended his welcome and opened by stating simply that the current U.S. "dashboard" for hospital performance was not balanced. Clinical performance measures exist but have not been merged into a core instrument. Data on patients' experience with care has been advanced by CAHPS® and the CAHPS® II grantees are being asked to consider development of instruments and reporting templates for other than health plans, e.g., group practice and individual level CAHPS® instruments. There is clearly a need for a hospital standard; however, a number of external factors have prevented this from happening. Critics have asked what benefit would accrue from the use of a national standard. Dr. Meyer believes that a single survey could serve multiple purposes, including:

• Providing information to the public that, in turn, would fuel research efforts.
• Unleashing the power of benchmarking.
• Driving quality improvement efforts.

Questions/Comments

1. Why would JCAHO be interested in a national standard?

   Selfishly, a national standard is needed to fill out the full range of hospital performance measures. The JCAHO may introduce standards in the pain management area; however, if we don't know how patients reacted to their pain management plans, we don't have a good measure of the patients' hospital experience.

2. A national standard would help avoid the situation where dueling measures exist and patients and vendors may not know who or what to believe. On the balance, we may learn a great deal from the use of several surveys.

3. What tools are currently in the public domain? Could questions be added to existing instruments?

4. When reports are released, what differences really make a difference for audiences such as patients and vendors? We need to spend some time identifying how the hospital data would be used by a variety of audiences. The education process is going to be a very important part of the reporting task. How often should reports be released?

5. How does AHA present reports?

   AHA usually develops two formats: a brief overview of the report findings for the consumer, and a much more detailed version for the hospital management, clinical, and administrative staff.

Return to Contents

CAHPS® Overview

Chuck Darby, AHRQ

Chuck Darby reviewed the highlights in the process that the CAHPS® Consortium used to develop an integrated set of carefully tested and standardized questionnaires and reporting formats that could be used to collect and report meaningful and reliable information from consumers about their health care experiences. The effort was initiated in l995, when the Agency awarded 5-year grants to consortia headed by the Harvard Medical School, Research Triangle Institute, and RAND. These three organizations, along with AHRQ and Westat, formed teams to focus on questionnaire development, reporting products development, and evaluation. In June 2002, CAHPS® II will launch a 5-year cooperative agreement with AIR, Harvard Medical School, and RAND.

Key lessons learned during CAHPS® I that may be applicable in the development of a national standard for measuring patients' hospital care experiences include:

• Identify the stakeholders and include them as early as possible in the development and testing process.
• Identify how stakeholders will use the information in the resulting survey.
• Make the process for developing the survey very transparent.
• Identify barriers to the process that may be related to resources, politics, and/or technology and how you will address them.
• Strive for methodologically sound instruments.
• Provide good user support.
• Develop reports concurrently with the instrument.
• Develop one package to house the instruments, instructions, and complementary resources.
• Build a database to enable the development of benchmarks.
• Include evaluations throughout the process.

If a national standard hospital survey were to become part of CAHPS®, it would be placed in the public domain in keeping with other AHRQ-sponsored initiatives. Additionally, the survey would provide the basis for development of a benchmarking database.

Return to Contents

Patients' Evaluation of Hospital Performance in California

Lisa Payne Simon, California Healthcare Foundation

Lisa Payne-Simon briefed the meeting participants on The Patients' Evaluation of Hospital Performance in California (PEP-C), a project that has established a scientifically valid survey of patients' (n = 21,151) experience in 113 general acute care California hospitals and publicly reported the results. She explained that the PEP-C (Picker Institute) survey focused on specific processes of care and critical aspects of patient-provider interaction among adult medical, surgical, and maternity patients. The survey was available in English, Spanish, and Chinese. The Picker Institute developed the survey that focused on seven key aspects of hospital performance, or "dimensions of care":

1. Respect for patients' preferences and expressed needs.
2. Coordination of care and integration of services.
3. Information and education.
4. Physical comfort and pain relief.
5. Emotional support and alleviation of fears and anxieties.
6. Involvement of family and friends.
7. Transition to home and continuity of care.

Survey results from PEP-C were publicly reported in August 2001 and the CHCF believes that the reports reached about 4 million individuals. Survey findings were reported in multiple formats for a wide range of audiences:

• A consumer report, with easy-to-read charts and project overviews in multiple languages for patients and family members.
• A detailed technical report for participating hospitals (and others upon request) that addresses patient care issues raised in the survey.
• An executive summary report for a cross-section of constituencies.
• A policy report, including an analysis and discussion of survey results and policy implications raised by the PEP-C project.

Survey results showed that patients believe California hospitals did a good job attending to the patients' physical comfort, including pain, but could improve patients' transition to the home environment. Maternity and surgical patients reported more positive experiences than those admitted for general medical care.

Lisa Payne-Simon reported that the PEP-C Project team faced many challenges:

• Despite the large number of respondents, only 30 percent of California hospitals participated in the survey during Cycle 1.
• Southern California, especially the Los Angeles area, was not well represented in the sample.
• Financial incentives to participate were limited.
• Many hospitals resisted public disclosure of performance data.
• The absence of a national standard for measuring patient experience hampered the ability to gain momentum around a specific instrument.

The PEP-C Project is actively recruiting participants for Cycle II and plans to implement a second public report this summer. She concluded her remarks by expressing strong support for the development of a national standard and was pleased that this first meeting was occurring.

Questions/Comments

1. Has there been a fall out in the number of hospitals who are signing up to participate in Cycle II from Cycle I?

   No, most of the Cycle I hospitals have signed up to participate in Cycle II. Since the recruitment for Cycle II is ongoing, it is difficult to predict how many new hospitals will participate.

2. Does CHCF have any sense of whether changes have been implemented at the hospital level as a result of hospitals receiving their reports or from public reporting efforts?

   We know hospitals have implemented changes as a result of PEP-C and that they have a heightened awareness of patient experience. Anecdotally, we've heard that a number of hospitals improved from the confidential cycle to the public report cycle.

3. How did CHCF recruit hospitals to participate in PEP-C?

   The recruitment period for Cycle I ran about 18 months and included a confidential data collection round for PEP-C hospitals. Recruitment strategies for Cycle I included numerous outreach and promotional efforts to hospitals administrators, hospital staff, and key purchasers. The recruitment period for Cycle II runs 4-5 months. Recruitment efforts have been widened to include other influential groups such as consumer advocacy organizations, policymakers at the local and State levels, and other grass roots level organizations.

4. Did you receive feedback from consumers?

   Over 100,000 PEP-C reports have been sent out in response to consumer requests. We've heard from consumers that they consider PEP-C data to be useful and valuable. The media has been highly responsive to PEP-C. Media coverage of PEP-C has, for the most part, tended to reward hospitals' participation rather than narrowing in on participants' specific results.

5. What will the PEP-C Project team do differently in Cycle II?

   We are expanding our recruitment strategies (outlined above). We are also able to make some significant improvements to the core survey program owing to NRC's purchase of the Picker Institute last year. Most notably, we're adding:

   • New questions on use of interpreters and primary language spoken.
   • More frequent sampling.
   • Web-based analysis and reporting.
   • Ability to add custom questions to the survey.

   Lastly, due in part to the amount of editorial and earned media attention received in Cycle I, paid advertising will not be used for recruitment purposes in Cycle II.

Return to Contents

Great Britain's National Health Care Service

Mike Hays, NRC/Picker

Mike Hays provided a brief overview of Great Britain's experience implementing and using a national standard. He opened his remarks by stating that while the United Kingdom's national health care service (NHS) is highly centralized, consumers have expressed their dissatisfaction with health care services and have been quite vocal with the British Parliament and the Parliament has responded.

In response to consumer unrest and a major safety incident, made public in the NHS- published Bristol Report, performance measurement and management has been placed front and center. As a result, more emphasis is naturally placed on the outcome and accountability of the system by consciously managing it while retaining some elements of the private market. Although the definition of the term "performance management" can be vague, the British have adopted a multi-prong strategy for improving the performance of the NHS based on empirical evidence, concrete goals, and quantifiable results.

One element of the strategy has been that standardization of measuring the patients' experience with the care received. The Picker family of surveys is administered in all 250 hospital Trusts. The public has embraced the reporting concept and the first set of reports will be released in late 2002, via print medium and the Web. Hospitals are rewarded for good performance with financial incentives and less NHS oversight, while poorer performing Trusts have greater oversight and financial restrictions.

Questions/Comments

1. How did patients use the information from The Picker Report?

   It is too early to know since there has not been widespread distribution of the reports; however, the health care service is becoming less restrictive so as to allow patients that really don't have the option to move easily between hospitals of their choice.

Return to Contents

Centers for Medicare and Medicaid Services:
Arriving at a National Standard

Rachael Weinstein, CMS

As the largest purchaser of health care services and a regulatory agency, CMS is key in the process of adopting a national standard for measuring patients' hospital care experiences. CMS is considering various approaches to helping arrive at a national standard. Rachael Weinstein presented two possible approaches that could be taken to formalize the development of a national standard and the subsequent reporting activities:

1. CMS could develop a Notice of Proposal for Rule Making (NPRM) proposing that hospitals report, in a manner and form specified by the Secretary, Department of Health and Human Services (HHS), consumer satisfaction measures on yet to be determined basis and to yet to be determined parties, e.g., CMS, HHS Secretary level, third party vendors, etc. The preamble of that NPRM would state that the development and consensus on the tool would be conducted by a process that conforms to the National Technology Transfer and Advancement Act (NTTAA). Then, the Secretary would announce the tool that Medicare/Medicaid participating hospitals would use via the Federal Register.
2. CMS could develop an NPRM that actually names the tool to be used. This would mean that the public would "get a second chance" at commenting on the tool itself. The first opportunity would occur during the consensus process used to develop the tool, and the second through the NPRM described above.

Both approaches could take from 1 to 2 years to complete and involve considerable approvals at numerous levels within CMS, HHS, and the Office of Management and Budget.

Questions/Comments

1. What about the JCAHO?

   The JCAHO will take their lead from the Government and would not go out on their own and develop a national standard.

2. The process sounds very burdensome. Is there any way that we can accelerate the process?

   It is a regulatory process but some hospitals may want to move forward in the absence of standards. We are going to have to work hard to create the influence that will be needed to sell a national standard. We are going to need to make a distinction between the opportunity to participate in the development of the standard and the requirement to report.

3. How important are States in the process?

   States will want to get on board quickly since State requirements are very important. The private sector, in turn, is heavily influenced by what is happening at the Federal level.

4. What about public reporting?

   Every effort does not need to be reported; however, there is value in reporting since it usually stimulates change. Quality health care has often been hidden but reporting patients' hospital care experience would turn the spotlight.

   Present challenges include:

   • Community hospitals who are often very resistant to public reporting because of their reluctance to be compared against competing institutions.
   • Hospitals who fear that reporting efforts may weaken their market share and/or image.
   • Vendor communities who have a host of issues.

   Reports need to be shared with a variety of audiences and through multiple modes of delivery: print and Web-based report-card style reports, fact sheets, press announcements, and media stories. Marketing is extremely important in the process and must be initiated at the outset of any data gathering effort.

After considerable discussion, it was recommended that the group consider going with Option 1, Notice of Proposal for Rule Making; however, the actual survey instrument would not be included in the proposed document. This approach would allow time for public buy in and would inform a wider community of the intent to develop a national standard.

Return to Contents

Next Steps

AHRQ's path forward includes the following:

1. Hold a series of public meetings with stakeholders to identify the issues, concerns, and interests of the health care community. Summaries of all meetings will be posted on the AHRQ Web site: www.ahrq.gov/qual/cahpsix.htm.
2. Identify potential sponsors who will fund, assist in the development, and ultimately manage a process for implementing and maintaining the national standard in the public domain.
3. Identify the process through which measures will be defined and a call for applicable instruments made. Instruments submitted by industry will be evaluated to determine if they meet the measurement needs and if additional development is necessary. Once consensus on the national standard has been reached, actions will be taken to place the resulting work in the public domain and instrument testing will be completed.
4. In parallel with these efforts, work should be initiated to establish a process for implementing the standard Nation-wide. This will include information related to data collection, analysis, and public reporting. Additionally, Federal rules requiring the use of the standard will be published.

Return to Contents

Current as of May 2002

Internet Citation:

Meeting Summary: Exploring a National Standard for Measuring Patients' Hospital Care Experiences. May 9, 2002. National Research Corporation and Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/qual/mtg5902.htm

Return to Measuring Hospital Patient Experiences
CAHPS®—Consumer Assessment of Health Plans
AHRQ Home Page
Department of Health and Human Services