Agency for Healthcare Research and Quality (AHRQ): Carolyn Clancy, M.D., Gregg Meyer, M.D., Chuck Darby, M.A., Katherine Crosson, M.P.H.
American Association of Retired Persons (AARP): Joyce Dubow
California Healthcare Foundation (CHCF): Lisa Payne-Simon, M.P.H.
Centers for Medicare and Medicaid Services (CMS): Elizabeth Goldstein, Ph.D., Beth Kosiak, Ph.D., Rachael Weinstein, Aucha Prachanronqrong
Department of Defense (DoD) Tricare: Lt. Col. Thomas Williams, Ph.D.
Indian Health Service (IHS): Katherine Ciacco Palatianos, M.D., M.P.H.
Joint Commission on Accreditation of Healthcare Organizations (JCAHO): Margaret VanAmringe
National Cancer Institute (NCI): Steven Clauser, Ph.D.
National Institutes of Health (NIH) Clinical Center (CC): David Henderson, M.D., Laura Lee, R.N.
Office of Personnel Management (OPM): Abby Block, M.A., M.S.W., M.B.A.
National Research Corporation/Picker (NRC): Mike Hays, Steve Larson, Paul Huelskamp, Joe Carmichael
The Commonwealth Fund: Anne-Marie Audet, M.D., M.Sc.
Veteran's Health Administration (VHA): Everett Jones, M.D.
Welcome, Introductions, Meeting Purpose
Need for a United States Standard
CAHPS® Overview
Patients' Evaluation of Hospital Performance in California
Great Britain's National Health Care Service
Centers for Medicare and Medicaid Services: Arriving at a National Standard
Next Steps
Mike Hays, NRC, and Chuck Darby, AHRQ, formally welcomed the participants and provided an overview of the process followed to develop survey instruments for placement in the public domain. Specifically, they addressed how this process might be used to develop a measure of patient experiences with hospital care. AHRQ has been the Nation's leader in establishing standards for measuring experiences with the U.S. health care system and the Consumer Assessment of Health Plans (CAHPS®) continues to be the ground-breaking work providing the foundation for similar initiatives. To date, CAHPS® has not addressed the hospital patients' experiences.
In the recent past, leaders from across the U.S. health care industry have called for a response to the need to assess the experiences of hospital patients:
AHRQ's Acting Director, Carolyn Clancy, M.D., joined the group later in the morning and extended her welcome and thanks to the participants for their insights, suggestions, and willingness to work with the Agency on establishing a national standard to measure hospital patient experiences.
Participants introduced themselves and indicated their specific interest in the development of a national standard.
Gregg Meyer extended his welcome and opened by stating simply that the current U.S. "dashboard" for hospital performance was not balanced. Clinical performance measures exist but have not been merged into a core instrument. Data on patients' experience with care has been advanced by CAHPS® and the CAHPS® II grantees are being asked to consider development of instruments and reporting templates for other than health plans, e.g., group practice and individual level CAHPS® instruments. There is clearly a need for a hospital standard; however, a number of external factors have prevented this from happening. Critics have asked what benefit would accrue from the use of a national standard. Dr. Meyer believes that a single survey could serve multiple purposes, including:
Why would JCAHO be interested in a national standard?
Selfishly, a national standard is needed to fill out the full range of hospital performance measures. The JCAHO may introduce standards in the pain management area; however, if we don't know how patients reacted to their pain management plans, we don't have a good measure of the patients' hospital experience.
A national standard would help avoid the situation where dueling measures exist and patients and vendors may not know who or what to believe. On the balance, we may learn a great deal from the use of several surveys.
What tools are currently in the public domain? Could questions be added to existing instruments?
When reports are released, what differences really make a difference for audiences such as patients and vendors? We need to spend some time identifying how the hospital data would be used by a variety of audiences. The education process is going to be a very important part of the reporting task. How often should reports be released?
How does AHA present reports?
AHA usually develops two formats: a brief overview of the report findings for the consumer, and a much more detailed version for the hospital management, clinical, and administrative staff.
Chuck Darby reviewed the highlights in the process that the CAHPS® Consortium used to develop an integrated set of carefully tested and standardized questionnaires and reporting formats that could be used to collect and report meaningful and reliable information from consumers about their health care experiences. The effort was initiated in l995, when the Agency awarded 5-year grants to consortia headed by the Harvard Medical School, Research Triangle Institute, and RAND. These three organizations, along with AHRQ and Westat, formed teams to focus on questionnaire development, reporting products development, and evaluation. In June 2002, CAHPS® II will launch a 5-year cooperative agreement with AIR, Harvard Medical School, and RAND.
Key lessons learned during CAHPS® I that may be applicable in the development of a national standard for measuring patients' hospital care experiences include:
If a national standard hospital survey were to become part of CAHPS®, it would be placed in the public domain in keeping with other AHRQ-sponsored initiatives. Additionally, the survey would provide the basis for development of a benchmarking database.
Lisa Payne-Simon briefed the meeting participants on The Patients' Evaluation of Hospital Performance in California (PEP-C), a project that has established a scientifically valid survey of patients' (n = 21,151) experience in 113 general acute care California hospitals and publicly reported the results. She explained that the PEP-C (Picker Institute) survey focused on specific processes of care and critical aspects of patient-provider interaction among adult medical, surgical, and maternity patients. The survey was available in English, Spanish, and Chinese. The Picker Institute developed the survey that focused on seven key aspects of hospital performance, or "dimensions of care":
Survey results from PEP-C were publicly reported in August 2001 and the CHCF believes that the reports reached about 4 million individuals. Survey findings were reported in multiple formats for a wide range of audiences:
Survey results showed that patients believe California hospitals did a good job attending to the patients' physical comfort, including pain, but could improve patients' transition to the home environment. Maternity and surgical patients reported more positive experiences than those admitted for general medical care.
Lisa Payne-Simon reported that the PEP-C Project team faced many challenges:
The PEP-C Project is actively recruiting participants for Cycle II and plans to implement a second public report this summer. She concluded her remarks by expressing strong support for the development of a national standard and was pleased that this first meeting was occurring.
No, most of the Cycle I hospitals have signed up to participate in Cycle II. Since the recruitment for Cycle II is ongoing, it is difficult to predict how many new hospitals will participate.
We know hospitals have implemented changes as a result of PEP-C and that they have a heightened awareness of patient experience. Anecdotally, we've heard that a number of hospitals improved from the confidential cycle to the public report cycle.
The recruitment period for Cycle I ran about 18 months and included a confidential data collection round for PEP-C hospitals. Recruitment strategies for Cycle I included numerous outreach and promotional efforts to hospitals administrators, hospital staff, and key purchasers. The recruitment period for Cycle II runs 4-5 months. Recruitment efforts have been widened to include other influential groups such as consumer advocacy organizations, policymakers at the local and State levels, and other grass roots level organizations.
Over 100,000 PEP-C reports have been sent out in response to consumer requests. We've heard from consumers that they consider PEP-C data to be useful and valuable. The media has been highly responsive to PEP-C. Media coverage of PEP-C has, for the most part, tended to reward hospitals' participation rather than narrowing in on participants' specific results.
We are expanding our recruitment strategies (outlined above). We are also able to make some significant improvements to the core survey program owing to NRC's purchase of the Picker Institute last year. Most notably, we're adding:
Lastly, due in part to the amount of editorial and earned media attention received in Cycle I, paid advertising will not be used for recruitment purposes in Cycle II.
Mike Hays provided a brief overview of Great Britain's experience implementing and using a national standard. He opened his remarks by stating that while the United Kingdom's national health care service (NHS) is highly centralized, consumers have expressed their dissatisfaction with health care services and have been quite vocal with the British Parliament and the Parliament has responded.
In response to consumer unrest and a major safety incident, made public in the NHS- published Bristol Report, performance measurement and management has been placed front and center. As a result, more emphasis is naturally placed on the outcome and accountability of the system by consciously managing it while retaining some elements of the private market. Although the definition of the term "performance management" can be vague, the British have adopted a multi-prong strategy for improving the performance of the NHS based on empirical evidence, concrete goals, and quantifiable results.
One element of the strategy has been that standardization of measuring the patients' experience with the care received. The Picker family of surveys is administered in all 250 hospital Trusts. The public has embraced the reporting concept and the first set of reports will be released in late 2002, via print medium and the Web. Hospitals are rewarded for good performance with financial incentives and less NHS oversight, while poorer performing Trusts have greater oversight and financial restrictions.
It is too early to know since there has not been widespread distribution of the reports; however, the health care service is becoming less restrictive so as to allow patients that really don't have the option to move easily between hospitals of their choice.
As the largest purchaser of health care services and a regulatory agency, CMS is key in the process of adopting a national standard for measuring patients' hospital care experiences. CMS is considering various approaches to helping arrive at a national standard. Rachael Weinstein presented two possible approaches that could be taken to formalize the development of a national standard and the subsequent reporting activities:
Both approaches could take from 1 to 2 years to complete and involve considerable approvals at numerous levels within CMS, HHS, and the Office of Management and Budget.
The JCAHO will take their lead from the Government and would not go out on their own and develop a national standard.
It is a regulatory process but some hospitals may want to move forward in the absence of standards. We are going to have to work hard to create the influence that will be needed to sell a national standard. We are going to need to make a distinction between the opportunity to participate in the development of the standard and the requirement to report.
States will want to get on board quickly since State requirements are very important. The private sector, in turn, is heavily influenced by what is happening at the Federal level.
Every effort does not need to be reported; however, there is value in reporting since it usually stimulates change. Quality health care has often been hidden but reporting patients' hospital care experience would turn the spotlight.
Present challenges include:
Reports need to be shared with a variety of audiences and through multiple modes of delivery: print and Web-based report-card style reports, fact sheets, press announcements, and media stories. Marketing is extremely important in the process and must be initiated at the outset of any data gathering effort.
After considerable discussion, it was recommended that the group consider going with Option 1, Notice of Proposal for Rule Making; however, the actual survey instrument would not be included in the proposed document. This approach would allow time for public buy in and would inform a wider community of the intent to develop a national standard.
AHRQ's path forward includes the following:
Current as of May 2002
Internet Citation:
Meeting Summary: Exploring a National Standard for Measuring Patients' Hospital Care Experiences. May 9, 2002. National Research Corporation and Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/qual/mtg5902.htm
Return to Measuring Hospital Patient Experiences
CAHPS®—Consumer Assessment of Health Plans
AHRQ Home Page
Department of Health and Human Services