This chapter is intended to provide realistic guidance to community organizations interested in conducting health access surveys. Often the first activity that local health access coalitions and others interested in improving health care think of doing is a survey. This makes sense; local coalitions reason that if decisionmakers are confronted with concrete information about local populations and local health care safety net providers, they will do a better job of marshaling resources to address access to care problems. For their part, local officials responsible for wisely appropriating resources to meet health needs are often also supportive of using surveys to assess needs and monitor spending.
The impulse to sponsor surveys is often given momentum because doing surveys seems easy—write some questions, ask some people the questions, tabulate the answers, and report the findings. The trouble is that each of these steps takes a good deal longer, costs a great deal more, and is more complex than most expect. Worse, when not well executed, surveys can be misleading and can sour relationships among survey sponsors, the populations they are trying to help, and the decisionmakers they are trying to inform. The focus of this chapter is on conducting health access surveys the right way regardless of the level of resources available to support survey work. For the purposes of this chapter, doing surveys right means two things:
This chapter is intended to provide the reader with guidance for doing local health access surveys right.
Health access survey research has a venerable history (Berk and Schur, 1997). In the early 1930s, the Falk Commission on the Cost of Medical Care, a group of private health care stakeholders, commissioned the first modern American survey of health services use and cost. Since that time, health survey research has evolved into a complex endeavor, engaging highly trained statisticians and social scientists. Today, the Federal Government supports no fewer than four large-scale, ongoing national surveys that measure some aspect of health care access, utilization, or coverage, not to mention dozens of special studies addressing particular aspects of health access. Private foundations have gotten into the act as well. For instance, The Robert Wood Johnson Foundation, the Nation's largest health philanthropy, sponsored four major national access surveys between 1976 and 1994; since 1996, it has supported the 60-city Community Tracking Survey, which involves surveys of households, physicians, and employers on health care topics. States also sponsor surveys that can be helpful for community health assessment. (For more information, go to the chapter by Lynn Blewett and Timothy Beebe).
With all of these rich health survey resources, why are any new local surveys needed? Perhaps a new survey is not needed in some circumstances, but in other circumstances the limitations of major national and State health surveys can weigh heavily. First, even large population surveys do not provide data for small local areas. (The main exception is the decennial U.S. Census, but the Census does not provide information on the indicators of greatest value for community health assessment.) Second, national surveys are designed to meet national (and sometimes regional or State) information needs, and local needs might be very different. By necessity, national surveys often lump data for diverse populations into broad categories; this can be misleading. For instance, most analysts reporting rates of uninsurance by ethnicity do not provide data for Hispanic subgroups, but a close look reveals that the uninsured rate among Hispanic subgroups varies greatly. In 1989, uninsurance rates varied more than twofold, from 17.3 percent among Puerto Rican residents of the United States to 37.4 percent among Mexican Americans (Berk, Albers, and Schur, 1996). In addition, local surveys can ask about local institutions and can be tailored to identify the needs of local populations, such as a particular immigrant group, in ways that are not feasible on national surveys. Finally, local surveys can be helpful tools for engaging local volunteers and decisionmakers. Local surveys can seem more real than big national studies and can be more effective tools for local system improvement.
Should you conduct a local health access survey? Box 1 summarizes the major questions that should be asked before embarking on a survey. To answer whether you should commit to a local health access survey, ask yourself these questions as you read this and other chapters in this book.
Before embarking on a local access survey, ask these six questions:
|
The primary aim of this chapter is to provide readers with guidance for developing and conducting local health access surveys. The material presented here is intended as a starting point, not as a cookbook for doing surveys. If this chapter is successful, it will motivate you to seek other resources for meeting your specific survey needs and give you the tools you need to find these resources.
This chapter is divided into nine sections. After this introduction, the second section provides an overview of what local health access surveys are, for what purposes they are and are not valuable for evaluating access to care and the local safety net and when other information and assessment tools might be more valuable. The third section describes the major survey resources that provide access and safety net-related information. Existing surveys are vital sources of tried-and-true survey questions and can provide benchmarks for interpreting findings from local survey efforts. The fourth through eighth sections discuss fundamental concepts of survey research for local survey developers, covering how to get a survey development process under way, who to survey, what to ask (and how to ask it), and other important methodological considerations. The final section summarizes the information presented in the chapter and emphasizes basic principles for successfully executing a local survey.
"Assessing the safety net" can engender many different objectives. Even defining the safety net is a complex endeavor, although local population survey developers are generally most interested in defining population access needs and understanding how well institutions whose mission it is to serve the uninsured and other vulnerable populations perform. But even within these broad goals, more clarity is needed before an appropriate survey strategy can be designed. This section outlines the initial steps that local groups should take to get their survey strategy started, and subsequent sections provide more detailed guidance for each step of the survey process. Throughout, references are given for more detailed sources of guidance for each step. (An excellent comprehensive, albeit somewhat technical, source for methods specific to health surveys can be found in Aday, 1996).
Before deciding how best to utilize surveys to assess the local health care safety net, it is important to clarify the objectives of the assessment. Only in the context of those objectives can the sources of information that are of greatest value to the assessment effort be identified. The broad aim of assessing the adequacy of the local health care safety net can embody many possible specific information objectives. Box 2 provides examples of some basic information objectives. There are, of course, many other possible information needs. Effective use of information tools depends first and foremost on starting with clear objectives. The survey development process involves dozens of detailed decisions. At each stage of the development process, it is advisable to keep a clear list of information objectives on hand.
|
Surveys are only one of many possible sources of good information about access to care and the safety net. The Institute of Medicine, part of the prestigious National Academy of Sciences, has identified available sources of information for monitoring access to health care. These sources, and the kinds of data they can provide, are shown in Box 3. Finding data from these sources for local areas can be challenging. A good place to start the search for available access-related data is the center for health statistics in each State's department of health. (For links to related Web sites, visit http://www.cdc.gov/states.htm, accessed April 8, 2003). Given the substantial expense and complexity of conducting surveys, utilizing existing sources of information can be much more cost effective.
| Data Source | Examples of Measures |
|---|---|
| Vital Statistics | Use of prenatal care Death rates by cause |
| Hospital discharge data | Preventable hospitalization rates |
| Disease registries | Cancer rates Rates of HIV infection and AIDS Sexual transmitted disease rates |
Source: Millman 1993, 24, box 1-1.
Note: For additional information on the analysis of vital statistics and hospital discharge data, go to the chapter by John Billings in Part II of Tools for Monitoring the Health Care Safety Net.
Once it is determined that local health access information cannot be obtained from sources other than a local survey effort, it is important to take a close look at what type of survey can generate the desired information. Surveys are simply tools for systematically asking people questions. To be useful, members of the survey population must be equipped to provide well-informed answers to the survey questions. This observation seems self-evident, but it is often the case that what we need to know about health care access is not easily translated into questions that individuals can readily answer. For example, identifying how many people in a given group are medically uninsured is a seemingly simple task. However, getting accurate estimates of the uninsured is very difficult and involves asking numerous questions (Swartz, 1986). Simply asking, "Do you have health insurance?" may fail to count people enrolled in public programs that policymakers would consider medical coverage but may not be considered health insurance by the respondent. Thus, surveys about coverage typically ask detailed questions about each possible source of coverage, counting a respondent as uninsured only if he or she reports no coverage by any of the different sources.
Individuals are likely to be able to provide good answers for questions about how they feel and what they do. But most people could not answer questions about many of the things we would want to know about the safety net. For example, many people do not know the specialty of the physicians they visit, and they certainly may not know about services that are available but that they have not used. While there are many possible definitions of the safety net (Lewin and Altman, 2000), one thing is clear: the safety net involves patients and health care providers. Thus, a comprehensive assessment of the safety net in a local area might involve surveys of potential patients (the population) and surveys of providers. A list of possible topics about which individuals and health care providers can be expected to report accurately in a survey is long (Box 4). In considering which group to survey, compare your information objectives to the topics in Box 4.
| General population | Health care providers |
|---|---|
|
|
A wealth of health care survey resources are available through the Internet and other sources. These resources are critical for helping survey developers avoid reinventing the wheel; they are especially valuable in the early planning stages of local surveys. Existing surveys can be excellent sources of questions for local surveys and can provide national, statewide, or regional benchmarks against which to compare findings of community survey efforts. Examining existing survey methods and questionnaires can also provide important clues about the feasibility of obtaining certain kinds of information from surveys.
With the aim of jump-starting local survey development efforts, this section provides an overview of six exemplary health-related population surveys and several important provider data collection efforts and surveys. Most of the surveys reviewed here are national population studies, but examples of health care provider surveys are included as well. Basic information about survey content and design is presented here. In subsequent sections of the chapter, these concepts are further explained and applied to the context of local survey efforts, which generally have fewer financial and technical resources than the major national surveys. Nevertheless, much can be learned from examining the methods used at the leading edge of health survey research.
Tables 1 and 2 provide an overview of six major population health surveys. The purpose of these surveys varies, but they all provide basic information about population health and health care access. The Federal Government sponsors four of the surveys discussed here:
Private foundations sponsor two others:
These surveys provide a rich collection of useful questionnaire items that have been carefully developed and refined through years of experience. Also, importantly, the questionnaires, methods, and findings (including the actual data) of each of these surveys are in the public domain. There are many other surveys, including many State-sponsored efforts that can be of considerable value to local groups. (For an excellent source of information about State surveys, go to http://www.shadac.org/collecting/).
The Current Population Survey-March Supplement (CPS), also called the Annual Demographic Survey, is the main national survey used to track the number of people with health insurance coverage and the uninsured. It also includes in-depth measures of income, demographics, and employment, which enable detailed analysis of the characteristics of the uninsured. The Federal Bureau of Labor Statistics and the U.S. Census Bureau sponsor the CPS, and have administered the CPS-March Supplement annually since 1980.
The CPS has two important limitations that diminish its value for local safety net assessment. First, it lacks some key measures. For instance, the CPS does not ask about health care utilization and it offers only limited health status measures. Second, the CPS sample is not optimally designed for producing State or local estimates. Although analysts use the CPS to make estimates for individual States, large cities, counties, and metropolitan areas, the survey sponsors do not vouch for the reliability of these estimates. The Federal Government does publish State estimates about health coverage from the CPS by pooling 3 years of data to increase reliability of estimates, a method that reduces the timeliness of these data.
A second major Federal survey, the Medical Expenditure Panel Survey—Household Component, includes many in-depth questions about health insurance coverage, household medical expenditures, health status, access to care, and service utilization. The MEPS-HC, sponsored by the Federal Agency for Healthcare Research and Quality, began in 1996 and is one of the few sources that tracks health coverage over time (e.g., measuring the duration of uninsured spells). The MEPS-HC has several other design features and supplements, such as the companion employer survey, that enrich this source of information. The MEPS-HC does not support State- or local-area estimates, but is an excellent source of well-validated questions that can be adopted for local survey efforts, and it provides good national and regional measures of health access and coverage.
The MEPS-HC samples its respondents from another major Federal survey, the National Health Interview Survey (NHIS), which is conducted by the National Center for Health Statistics at the U.S. Centers for Disease Control and Prevention (CDC). The NHIS, which has been conducted for nearly 50 years, is used primarily for monitoring trends in illness and disability in the United States and tracking national disease control objectives. In recent years, the NHIS has included measures of health insurance coverage. Although its sample is large, confidentiality and sample design considerations preclude use of NHIS for making State or local-area estimates.
The final Federal survey described in this chapter, the Behavioral Risk Factor Surveillance System (BRFSS), also sponsored by the CDC, provides State and sub-State estimates of insurance coverage and other indicators relevant to safety net assessment. The BRFSS is conducted continuously and can provide trend information through comparison of monthly samples. However, the focus of this survey is on health risks rather than health insurance, and the BRFSS measures only limited demographic and other variables that are related to health insurance and access to care. Importantly, the BRFSS focuses only on adults and lacks vital information about children and families.
Two major privately sponsored surveys are also of considerable value to those interested in health care safety net performance. The Community Tracking Survey—Household Survey (CTS) is conducted by the Center for Studying Health System Change and sponsored by The Robert Wood Johnson Foundation. The CTS has been conducted on a 2-year cycle since 1996 in 60 communities that were selected to represent the nation. Among the 60 CTS study sites are 13 metropolitan case study sites for which especially rich information is available. CTS is a source of in-depth data on health coverage, access to care, health status, and other important measures. Importantly, CTS includes two companion surveys of employers and physicians, which can be of great value in safety net assessment. Results of the survey are available for each case study site, as well as for the Nation and regions of the United States. In addition, reports based on in-depth interviews with key health care stakeholders in each of the case study sites are also available.
The second privately sponsored survey, the National Survey of America's Families (NSAF), was conducted in 1997, 1999, and 2001-02 and is sponsored by the Urban Institute with a consortium of private funders. The focus of this survey is on the overall well-being of adults and children, and it was developed to track the impact of changes in welfare, social service, and health policy. The NSAF includes a number of measures of health care coverage and access to care. The NSAF is nationally representative and provides large representative samples for 13 individual States. Like the CTS, MEPS-HC, and other sources, the NSAF is a good source of questions for local surveys, and it provides national and State benchmarks of important measures.
Table 3 provides a summary of major data elements available in each of the population surveys discussed here. For most, descriptive tabulations of coverage, health care use, and access variables are readily available on the respective sponsor's Web site. The sponsors make modifications to questionnaires over time, thus it is important to consult the respective Web pages of each for the most current information. In general, reports based on these surveys include tabulations of major health measures according to demographic, employment, and other family and individual characteristics. The content of each survey varies, as do survey methods and question wording. Thus, survey findings on seemingly identical topics can vary from source to source. Consequently, it is important to select one source when presenting statistical trends. Before developing a new local health survey, it is worthwhile reviewing the existing questionnaires and survey results. The information in Table 3 can help in identifying the surveys of greatest interest. More guidance for selecting survey questions is provided in the section "What to Ask."
As discussed, patients and members of the general public can provide some of the information of value for assessing the safety net. However, accurately measuring the capacity of the health care system to deliver services to uninsured and other vulnerable populations may require surveying health care providers. Like surveys of general populations, surveys of providers can be quite challenging, and care should be taken to review existing sources of information on provider supply before launching a health care provider survey. If surveying local health care providers is necessary, existing provider data sources can be instructive. Table 4 describes valuable sources of information on physicians, clinics, and other health care providers. Some of these sources rely on formal surveys, while others are derived from administrative data or other sources. Although it is beyond the scope of this chapter to describe each of these sources in detail, Table 4 provides ways to find additional information.
Once a decision has been made that a local survey is needed and the objectives for the survey project are clearly identified, it is time to start the survey planning process. Survey planning entails some basic decisions:
Before delving into these decisions in the following sections, it is essential that you take stock of the resources available for developing and implementing a survey project.
The oft-repeated theme of this chapter is that administering health surveys is complex. An industry has evolved to manage these complexities. Thus, early on, budding local survey sponsors must confront "make or buy" decisions. Although it is almost always easier and better to hire experts, survey firm capabilities vary widely and they can be quite expensive given the level of resources available to many local groups. Also, once engaged, survey firms will generally move quite quickly, and mid-project design changes can lead to unexpected costs and delays. Thus, regardless of whether a survey firm is used, careful advance planning is essential.
If resources are available, survey firms can offer quite a lot: scientific sample selection, questionnaire design and testing, computer-assisted questionnaires that enable efficient administration of even the most complex surveys, highly trained interviewers, survey quality control, and data tabulation and reporting services. It is important to remember, however, that survey professionals are expert at developing and administering surveys, but they are most often not health care content experts. In selecting a survey firm, it is important to interview prospective contractors, check references (good survey firms will give you a list of prior clients), and carefully review and negotiate the specific tasks that they propose. Be sure that the firm you hire subscribes to standards of the American Association of Public Opinion Research. (For more information, go to http://www.aapor.org/default.asp?page=survey_methods/standards_and_best_practices, accessed April 8, 2003). Most survey firms are for-profit vendors, although many local universities also provide excellent survey research services.
The survey research industry is quite competitive, and it is useful and wise to ask a number of firms to submit competitive bids (within many organizations, competitive bidding may even be required). When soliciting bids, you should outline a request for proposals (RFP) that provides details of your survey plan. The sections below outline the specific parameters of the survey that may be specified in the plan. Although RFPs often ask bidding firms to identify the price for delivering a fixed product, depending on your local services, crafting an RFP that requests the survey firms to bid the survey design for a fixed price may be advisable. Whether or not a formal request for proposals is used, be sure to interview potential survey contractors (State Health Access Data Center, 2003).
To illustrate this strategy, consider an example. If a local access coalition has raised $40,000 to conduct an access-to-care survey of the local low-income population, then the RFP would specify that this amount is available and it would identify "musts" of the survey design, such as offering interviews Spanish and English, completing data collection by a fixed date, and a list of specific data elements that are to be collected. The RFP would then ask bidders to identify other features of the design, such as the number of interviews to be completed and how interviewers will be trained. In using this bidding strategy, it is important for the local coalition to specify all of the survey design "musts" that are important to them (again, underscoring the importance of clear planning). This process enables the survey firms that are bidding to be creative and propose designs that utilize their strengths. It is also a way for the local coalition to obtain (free) advice for its survey design.
Before proceeding further, a note on conducting research on human subjects is in order. Although beyond the scope of this chapter, maintaining high ethical standards is essential in all forms of social research, including conducting surveys. These standards involve obtaining the informed consent of survey respondents before proceeding with interviewing and taking appropriate measures to protect respondent confidentiality, among other things. (For a good succinct discussion of survey ethics, see Fowler, 1993). Depending on the research setting, survey plans may be subject to the approval of Institutional Review Boards or Privacy Boards; therefore, survey sponsors should contact the appropriate staff (e.g., office of legal council) within their institutions about necessary clearances early in the development process. The following sections discuss the major decisions that must be made in designing local surveys and provide useful advice for each step. Unavoidably, this section uses survey research jargon (shown in italics) along with definitions of these terms.