The National CAHPS® Benchmarking Database (NCBD) is a new resource that provides benchmarks to facilitate comparisons across health plans by users of the CAHPS® survey. In the U.S. health care system, use of the Consumer Assessment of Health Plans survey is widespread.
The NCBD also provides an important source of data for research on consumer assessments of health care of interest to multiple audiences, including policymakers, health plans, providers, purchasers, and consumers.
Select for PDF File (884 KB). PDF Help.
Acknowledgments
Message from the Director
Summary
Overview
Composition of the NCBD 2000
Summary of Findings
Using the NCBD for Benchmarking
Using the NCBD for Research
Appendix A. Detailed NCBD 2000 Benchmark Tables by Sector
Appendix B. NCBD Advisory Group Members
It is with great pleasure that the Agency for Healthcare Research and Quality (AHRQ) presents this first annual report of findings from the National CAHPS® Benchmarking Database (NCBD).
When the Agency embarked on the Consumer Assessment of Health Plans (CAHPS®) project in 1995, we set out to develop and test a new, standardized approach for surveying people about their experiences with their health plans and medical care. We also wanted to develop methods for presenting survey findings clearly and fairly to consumers and purchasers for their use in making health plan choices.
Now, 6 years later, we are proud to witness the widespread use of the CAHPS® survey throughout the U.S. health care system. CAHPS® has been adopted by the Medicare program, the Nation's largest purchaser, as well as by growing numbers of State Medicaid agencies and public and private employers. With its inclusion as part of the accreditation program of the National Committee for Quality Assurance, CAHPS® has become the industry standard for use by health plans when seeking accreditation.
The NCBD is a new resource that provides value to CAHPS® survey users by providing national and market-level benchmarks to facilitate comparisons across health plans. It also provides an important source of data for research on consumer assessments of health care of interest to multiple audiences, including policymakers, health plans, providers, purchasers, and consumers.
High-quality comparative data are essential not only to guide choice of health plans, but also for effective quality improvement and research. We extend our thanks to the many thousands of Americans who have helped create this database by responding to the CAHPS® survey, and to the many sponsors who have submitted their survey data to the NCBD. We look forward to continuing to work together to build the NCBD as an integral part of the Agency's commitment to reporting on the quality of the Nation's health care system.
John Eisenberg, M.D.
Director, Agency for Healthcare Research and Quality
The National CAHPS® Benchmarking Database (NCBD) is the national repository for data from the CAHPS® family of surveys. The NCBD was initiated in 1998 to support benchmarking and research related to consumer assessments of care.
All sponsors of CAHPS® surveys that are administered according to CAHPS® specifications are invited to participate in the NCBD. Participating sponsors receive a customized report that compares their own results to appropriate benchmarks derived from the NCBD. Survey sponsors include public and private purchasers (employers, State Medicaid agencies, and Medicare) and individual health plans.
NCBD has experienced tremendous growth during its first 3 years. Participation has grown from 85 health plans to 793 for the adult population and from 33 to 148 plans for the child population.
For the adult population, NCBD has grown steadily each year and now includes substantial numbers from all sectors—commercial, Medicaid, and Medicare. For the child population, growth in the Medicaid sector has been steady, but participation by the commercial sector declined in 2000 and is expected to remain low until 2002, when a revised survey focusing on children with special health care needs should become more widely implemented.
Examination of NCBD 2000 data has revealed the following key findings:
NCBD provides benchmark data through this report and through the Sponsor Reports. Annual evaluations through focus groups, interviews, and surveys have indicated that sponsors are pleased with the process and the reports. Sponsors have indicated they use the data in a variety of ways including evaluating health plan performance, internal management, targeting opportunities for performance improvement, reporting to consumers, and informing policymakers.
NCBD data are available for researchers who submit an application and sign a data release agreement that ensures the confidentiality of the data. To date, 12 research projects have been approved covering topics from differences in consumer assessments by race and ethnicity to the effects of case-mix adjustment on reports of patient experiences by health plans' profit status. A complete list of projects, including contact information and current status, is included in this report.
In its first 3 years, NCBD has been established as a valuable source of CAHPS® information for both sponsors and researchers. In the near future, NCBD will also be an important source for the National Healthcare Quality Report (NQR) currently under development by AHRQ. The NQR will be published annually beginning in 2003 and will provide information to policymakers to monitor the Nation's progress toward improved health care quality.
In future years, the NCBD will expand to include new CAHPS®-derived survey data from the following instruments:
NCBD staff is also exploring linkages to other datasets to enhance the research value of the data. In the years to come, NCBD will continue to provide valid and reliable data for benchmarking and research related to consumer assessments of care.
Further information about the NCBD is available through the NCBD Web site at http://ncdb.cahps.org.
In 1995, the Agency for Healthcare Research and Quality (AHRQ) began an initiative to build an integrated set of standardized questionnaires and reporting formats that could be used to collect and report meaningful and reliable information about the experiences of consumers enrolled in health plans. The goals of CAHPS® were to:
To implement CAHPS®, AHRQ awarded three cooperative agreements to consortia at:
A contract was also awarded to Westat for user support.
In the first stage of the project, the teams designed, tested, and refined the survey instruments, report formats, and accompanying Kit to assist those who use the CAHPS® materials. In the second stage, they implemented and evaluated large-scale demonstration projects and used the results to further refine the CAHPS® materials.
The CAHPS® survey is designed to measure important dimensions of health plan performance from the consumer's point of view. By responding to a standardized set of questions administered through a mail or telephone questionnaire, health plan enrollees report on their experiences and rate their health plans and providers in several areas. CAHPS® surveys are administered to a random sample of health plan enrollees by independent survey vendors, following standardized procedures.
The CAHPS® survey is designed for use with all types of health insurance enrollees (commercial, Medicaid, and Medicare) and across the full range of health care delivery systems from fee-for-service to managed care plans. A core survey questionnaire is available for adults concerning their own experiences and for parents concerning the experiences of their children. Supplemental questions have been developed as modules for people with chronic conditions and special health care needs.
Further information on CAHPS® is available through the CAHPS® Survey Users Network Web site at www.cahps-sun.org.
As CAHPS® was implemented, AHRQ supported the development of the National CAHPS® Benchmarking Database (NCBD) to serve as the repository for all CAHPS® data. All sponsors of CAHPS® surveys that are administered independently according to CAHPS® specifications are invited to participate in the NCBD. Survey sponsors include public and private purchasers (employers, State Medicaid agencies, Medicare) and individual health plans.
The NCBD is a national database of CAHPS® survey data; it is intended to support benchmarking and research related to consumer assessments of care.
Initiated in 1998, the NCBD currently includes 3 years of CAHPS® data. The database for 2000 includes CAHPS® survey data from nearly 400,000 survey respondents distributed over more than 900 health plan sampling units. The NCBD is available as a resource for CAHPS® survey sponsors, researchers, and others interested in using comparative CAHPS® survey findings and detailed benchmark data. The NCBD also offers an important source of primary data for conducting specialized research that will enable improvements in future survey design and a better understanding of health plan and enrollee characteristics that influence performance.
The NCBD is funded by AHRQ and administered by Westat. A formal NCBD Advisory Group, composed of representatives from survey sponsor organizations and other interested groups, provides oversight and direction for the project. Select for a list of NCBD Advisory Group members.
A central purpose of the NCBD is to facilitate comparisons of CAHPS® survey results by survey sponsors. By compiling CAHPS® survey results from a variety of sponsors into a single national database, the NCBD enables purchasers and plans to compare their own results to relevant national benchmarks, in order to identify performance strengths as well as opportunities for improvement.
Survey sponsors may participate in the NCBD simply by submitting their CAHPS® survey results according to specified guidelines. In return, participating sponsors receive a customized report that compares their own results to appropriate benchmarks derived from the NCBD. Comparisons include national averages by population sector and the 90th percentile score for CAHPS® results.
An example of the type of chart used in the NCBD sponsor reports is in this report, along with information on use of reports by sponsors. As a service to CAHPS® survey sponsors not participating in the NCBD, Appendix A includes detailed tables of national averages and benchmark data for CAHPS® survey scores in the commercial, Medicaid, and Medicare populations.
Researchers may gain authorized access to NCBD data needed to help answer important health services research questions related to consumer assessments of quality as measured by CAHPS®. A list of current NCBD research projects is included in this report.
This first annual report of the NCBD includes the following sections:
The NCBD consists of 3 years of CAHPS® survey data: Phase I in 1998, Phase II in 1999, and Phase III in 2000. As evidenced by the tables below, the number of participating health plans has increased markedly over that time from 85 to 793 for the Adult population. For the Child population, participation has steadily increased among Medicaid plans while commercial plan participation peaked in Phase II (1999).
Data for the Phase I feasibility stage in 1998, shown in Table 1, include CAHPS® 1.0 survey results submitted by six commercial and seven Medicaid sponsors.
| Adult Population | Child Population | |||
| Population Sector | Sampling Units* | Respondents | Sampling Units* | Respondents |
| Commercial | 54 | 34,965 | 0 | 0 |
| Medicaid | 31 | 23,519 | 33 | 9,871 |
| Total | 85 | 58,484 | 33 | 9,871 |
Select for Text Version.
Phase II of the NCBD was implemented in 1999 to further develop and demonstrate the feasibility and usefulness of a national comparative database of CAHPS® survey data. Data for Phase II, shown in Table 2, include CAHPS® 2.0 or 2.0H survey results submitted by 15 commercial sponsors and 14 Medicaid sponsors.
| Adult Population | Child Population | |||
| Population Sector | Sampling Units* | Respondents | Sampling Units* | Respondents |
| Commercial | 307 | 168,234 | 149 | 42,879 |
| Medicaid | 77 | 28,420 | 66 | 14,106 |
| Total | 384 | 196,654 | 215 | 56,985 |
Select for Text Version.
Phase III of the NCBD was conducted in 2000. Data for Phase III, shown in Table 3, include CAHPS® 2.0 or 2.0H survey results submitted by 39 commercial sponsors, 17 Medicaid sponsors, and Medicare.
| Adult Population | Child Population | |||
| Population Sector | Sampling Units* | Respondents | Sampling Units* | Respondents |
| Commercial | 270 | 135,479 | 8 | 2,760 |
| Medicaid | 156 | 49,327 | 140 | 41,400 |
| Medicare Managed Care | 367 | 166,072 | — | — |
| Total | 793 | 350,878 | 148 | 44,160 |
Select for Text Version.
*A sampling unit refers to an individual health plan product, such as a Health Maintenance Organization (HMO) or Point of Service (POS) product, whose enrollees are randomly sampled for administering the CAHPS® survey.
Composition of the 2000 NCBD
Participating Sponsors. All sponsors of CAHPS® surveys that are administered independently according to CAHPS® 2.0 or HEDIS 2.0H specifications are invited to participate in the NCBD. Survey sponsors include public and private purchasers (employers, State Medicaid agencies, Medicare) and individual health plans.
Table 4 shows the distribution of sponsors by sponsor type in the NCBD 2000. Readers should note that each sponsor may represent a range of plans or sampling units. For example, individual health plan sponsors account for one health plan within the NCBD, while Medicare managed care (the Centers for Medicare & Medicaid Services [CMS], formerly the Health Care Financing Administration [HCFA],) accounts for 367 plans or sampling units.
Public Employers: 2 sponsors
Public/Private Purchasing Coalitions: 4 sponsors
Health Plan Coalitions: 1 sponsors
State Medicaid Agencies: 14 sponsors
State Data Agencies: 3 sponsors
Individual Health Plans: 32 sponsors
Medicare Managed Care: 1 sponsors
Total Sponsors: 57 sponsors
Health Plan Types. The majority of health plans represented in the NCBD 2000 are HMO plans, as indicated in Table 5.
| Plan Type | Commercial | Medicaid | Medicare |
| Health Maintenance Organization (HMO) | 186 | 146 | 367 |
| Point of Service (POS) | 29 | 0 | 0 |
| HMO/POS | 36 | 0 | 0 |
| Preferred Provider Organization (PPO) | 17 | 0 | 0 |
| Primary Care Case Management (PCCM) | 0 | 10 | 0 |
| Indemnity/Fee-for-Service | 2 | 0 | 0 |
| Total | 270 | 156 | 367 |
Select for Text Version.
Response Rates. Response rates vary by population sector in the NCBD 2000, with Medicare health plan enrollees showing the highest rate of response, followed by commercial enrollees and Medicaid enrollees.
| Population Sector | Adult Population (%) | Child Population (%) |
| Commercial | 48.4 | 41.7 |
| Medicaid | 37.6 | 38.9 |
| Medicare Managed Care | 82.0 | Not applicable |
Select for Text Version.
Representativeness of NCBD 2000 Data. Although the NCBD includes a broad cross-section of commercial, Medicaid, and Medicare health plan enrollees, the extent to which NCBD 2000 data are representative of the general population varies by sector:
Medicaid: The NCBD 2000 database includes 14 of the 50 State Medicaid agencies in the United States.2 In terms of total enrollment, in 1999 these Medicaid programs enrolled 7,813,160 beneficiaries in managed care plans, or roughly 44 percent of the total number of Medicaid managed care enrollees.3
Within each of the 14 State Medicaid programs participating in the NCBD 2000, the survey data are highly representative of the Medicaid adult and child population enrolled in the Medicaid managed care plans offered in these states.
Data were not available at the time of publication of this report to determine the extent to which the types of plans included in the NCBD 2000 are representative of the mix of plan types in general for the commercial population. Nor was it possible based on the data available to determine the representativeness of survey respondents to the population as a whole.
In the future, it should be possible to assess the overall representativeness of the NCBD by comparison to CAHPS® results for the general population obtained through the Medical Expenditure Panel Survey (MEPS). MEPS is an annual survey of approximately 15,000 U.S. households sponsored by AHRQ, related to health care cost and utilization behavior. Beginning in 2000, AHRQ has included selected CAHPS® questions in the MEPS survey, thus providing a population-based point of reference for the plan-based CAHPS® data contained in the NCBD.
The NCBD has become a rich source of data on the CAHPS® survey. It includes data from different population sectors (commercial, Medicaid, and Medicare) and from a variety of plan types including Health Maintenance Organization (HMO), Preferred Provider Organization (PPO), and primary care case management.
As implementation of the CAHPS® survey broadens to include other entities and special population groups (e.g., provider group-level CAHPS®, behavioral health CAHPS®, and children with special health care needs), the NCBD will also expand to include data gathered through these new CAHPS® instruments.
1. Bruce E. Landon; Alan M. Zaslavsky; Nancy Dean Beaulieu; James A. Shaul; Paul D. Cleary. "Health Plan Characteristics and Consumers' Assessments of Quality," Health Affairs (Volume 20 Number 2) March 2001, pp. 274-86.
2. The 14 State Medicaid programs participating in the 2000 NCBD are Arizona, California (San Diego County only), Colorado, Hawaii, Kansas, Michigan, New York, Ohio, Oklahoma, Pennsylvania, Texas, Utah, Vermont, and Washington.
3. 1999 Medicaid Managed Care Enrollment Report, Health Care Financing Administration.