Introduction: Evaluation of IHC |
Wired for Health and Well-Being: The Emergence of Interactive Health CommunicationEditors: Thomas R. Eng, David H. Gustafson Suggested Citation: Science Panel on Interactive Communication and Health. Wired for Health and Well-Being: the Emergence of Interactive Health Communication. Washington, DC: US Department of Health and Human Services, US Government Printing Office, April 1999. Download in PDF format: [Entire Document] [References] Chapter VII. The following recommendations of the Panel stem from reviews of published data and studies, discussions among Panel members, interactions with other participants at Panel meetings and presentations, and comments and suggestions from the liaisons to the Panel during the course of more than 2 ½ years. Although the ideas presented here resulted from careful and systematic evaluation of available information and data about IHC by many experts, the science base on IHC is limited, and these recommendations should be viewed as informed guidance rather than as definitive directives. The Panels recommendations are intended to promote and focus debate and discussion among IHC stakeholdersthose who develop applications, those who use them, those who purchase them for others to use, and those who establish policies that affect their use. Additional consensus-building processes could build on and further refine the Panels suggested guidance in this evolving field. Disagreements about some of the Panels recommendations may emerge because some of the issues around IHC are controversial and challenge traditional roles and systems. In these cases, colleagues are encouraged to offer differing perspectives on these issues and engage in a constructive public dialog to advance the field of IHC. Vision The Panel developed the following vision for IHC:
The Panel believes that IHC has the potential to contribute substantially to the health of the public. Individual health status can be enhanced through the use of IHC for health promotion and disease prevention. By reaching, informing, and motivating people in innovative ways, these technologies can augment public health efforts and minimize the total burden of illness and its economic and social costs. By providing peer and emotional support, IHC has the potential to alleviate the adverse social and emotional consequences of illness, and, perhaps, improve functional status. The Panel believes that IHC will not replace health care professionals, but complement what they do. Indeed, IHC could strengthen the relationship between patients and health professionals in several ways. For example, by providing appropriate health information to patients outside the context of an office visit, patient-provider encounters may be better focused and more efficient. By enabling and facilitating shared decisionmaking processes, patients are more likely to achieve their desired outcomes and express satisfaction with their care. Roles and Responsibilities of Stakeholders A preferred future for IHC will not materialize without a concerted effort to nurture its potential benefits and to minimize its potential for harm. As a first step, the roles and responsibilities of stakeholders need to be clarified. The Panel suggests a basic framework as presented in Table VII-1, not as a fixed model, but rather as a starting point for further discussions among stakeholders.
There is considerable overlap in these roles and responsibilities. In some cases, an individual stakeholder may have multiple roles as part of his/her job. For example, the same health professionals may function as developers when they serve on the health plans Web site development committee, may assume purchaser roles when they participate on the information technology committee on software/hardware procurement, and may play the roles of information intermediaries when they help patients interpret information downloaded from the Internet. The potential roles and responsibilities of stakeholders in evaluation of IHC deserve special emphasis. Although frequently considered the purview of researchers and developers, all stakeholders including consumers, health professionals, purchasers, and policymakers can play an active role in evaluation. Consumers and purchasers can use evaluation information to help in decisions regarding the purchase and use of IHC applications. They can help specify performance criteria, reasonable costs, and outcomes for use of the product in specific circumstances. Consumers and purchasers also should benefit from evaluating the extent to which the application can "stand alone" as opposed to augmenting existing health information and support strategies. Does the application add value or is it merely more "bells and whistles?" Policymakers can ensure that financial and clinical environments are conducive to IHC evaluation, including promoting incentives for use of evaluated products. Policymakers also have a role in evaluating the impact of IHC on health care and public health systems. The role of clinicians and other health professionals deserves special mention because their training in scientific methods enables them to promote evaluation and quality improvement. As societal and health care trends pressure consumers to assume more responsibility for their health care, they will need assistance in this capacity. For example, rapid advances in medical technology require people with chronic health conditionsand their caregiversto have fairly high-level skills in, and knowledge of, technology and health. Clinicians can help in this regard by helping patients and caregivers identify and evaluate the quality and utility of IHC applications and other information resources (Engstrom and Brown, 1996; Eytan, 1997; Krishna et al., 1997; Renner, 1997). This is particularly important for complex health problems where patients and families may have difficulty navigating and evaluating the spectrum of health information resources. Clinicians also are in a position to confirm that patients understand the health information they find regardless of its source. Suggested specific activities for clinicians in promoting an "information-friendly" practice are presented in Appendix F. Similarly, an appropriate role for public health professionals is to assess IHC applications and other information resources available to the community and determine how online health resources can be used to complement other community health resources. One of the traditional "safety net" roles of public healthto address community health problems and needs of population groups that are not being served by the established health care systemalso can be applied to IHC. In this context, public health professionals, in collaboration with other stakeholders, should take responsibility for ensuring that all populations have access to appropriate health information and support. Major Strategies In deliberating about the initiatives that could move the field of IHC forward and achieve the vision proposed in this chapter, the Panel was impressed with the need for a multidisciplinary, multipronged approach. The Panel proposes four overarching strategies to achieve the Panels vision for IHC:
The Panel chose to put a primary emphasis on promoting evaluation because it believes that widespread evaluation and dissemination of results will drive the widespread availability of high-quality and effective applications. Because of the newness of the field, scientific knowledge about many aspects of IHC is very limited. Greater knowledge is needed to improve the effectiveness of IHC, inform application design and implementation, and, ultimately, further appropriate public policy. The Panel also identified major gaps in resources available to application developers and shortcomings in the skills of users. These deficiencies will need to be addressed to ensure the continued development of innovative applications and the ability of intended users to take full advantage of IHC. Finally, enhancing access to IHC for all populations is essential because, without equitable access, evaluation efforts will be incomplete and the potential for IHC to provide benefits to entire communities is not likely to be realized. In the remainder of this chapter, the Panel identifies the major areas within each strategy and outlines specific tactics that could be used to achieve a preferred future for IHC. The main recommendations of the Panel within each area are in bold italics. Strategy 1. Strengthen Evaluation and Quality of Applications Evaluation of Applications The Panel believes that promoting widespread evaluation of IHC applications should be a central strategy for improving their quality and effectiveness. Because of the emerging nature of the technology and the potential for harm, there is a need to adapt an evidence-based approach to IHC. However, in applying this approach, it should be recognized that it may not be sufficient in all situations and that all types of data and evidence, not just statistical significance, should be considered when appropriate. There also is a need to define what is a "gold standard" for adequate evidence. Ultimately, a balanced approach to evaluationone that promotes quality and considers the potential risks of an applicationis indicated. In light of these issues, the Panel believes that evaluation of IHC applications should: 1) be practical in that evaluation methods reflect real-world limitations; 2) be proactive in that it seeks to prevent problems; 3) have a clear purpose of how results will be used to improve application quality; 4) be a shared responsibility in that all stakeholders participate; and 5) be integrated into the product development cycle in that evaluation methods are used to strengthen the software quality assurance process and are implemented throughout the product development process, from conceptualization through post-marketing surveillance. To spur widespread evaluation activities, new evaluation models that facilitate evaluation for developers and that help users and purchasers assess quality and appropriateness need to be developed. These approaches may be necessary because current evaluative strategies were developed for relatively static health interventions that do not lend themselves well to the dynamic nature of new and emerging media. Agile evaluation methods that can adapt to the evolving field of IHC are essential. It is possible that appropriate evaluation systems will need to rely on many of the same advanced technologies that IHC applications are using. For example, software agents may be used to detect system changes and alert potential users. They may be used to extract or identify specific changes so that users, developers, and others can periodically examine the modifications without having to evaluate the whole system again. These technologies could identify, assuming accurate and timely reporting, when changes occur in an application, including sponsorship, content, and skills of the development team.
Disclosure of Information About Applications Users and purchasers need access to essential information about an application (e.g., identity of the developer and sponsor, purpose and content of the product) in order to make informed decisions about whether the application is appropriate for their needs. In other sectors, such as the food processing and motor vehicle industries, essential information about the producer and components of the product is routinely disclosed to consumers. Currently many IHC applications do not publicly disclose such information about their products. In some cases, even basic information about the identity and background of the developers or sponsors of the product is not revealed. Providing full disclosure of the essential elements of their application may be in the best long-term interests of developers.
Voluntary Quality Standards In the last few years, there has been much discussion about quality standards for health-related information on the Internet and other IHC applications. By implementing such standards, developers may be able to avert government regulation. While recognizing the need for quality assurance mechanisms, many policymakers are reluctant to regulate this emerging industry for fear of hindering innovation and infringing on free speech. Several organizations have called for the adoption of voluntary quality standards for health-related Web sites (Silberg et al., 1997; HON, 1998). Quality standards should include consideration of the use and development of new search engines that rank or evaluate Web sites by reviewing content and links.
Reviews of Web Sites and Other IHC Applications Ensuring the quality of health information on Web sites is a major interest of policymakers and consumers. Currently, many organizations conduct reviews of health-related Web sites and some produce ratings based on a variety of criteria (Jadad and Gagliardi, 1998). Most of these organizations, however, do not focus their assessments on evidence of effectiveness. A review of explicit criteria proposed by rating organizations and others revealed that none suggested using evidence of application effectiveness as a criterion for quality (Kim et al., 1999).
Strategy 2. Improve Basic Knowledge and Understanding of IHC Research and Development It is unclear if current market forces will appropriately focus the activities of developers on the high-priority, health areas for the nation. Because IHC development is occurring through the independent and uncoordinated efforts of many developers, both overlap and gaps in research and application development exist. In addition, priorities for demonstration projects have not been elucidated. For the field of IHC to move forward, many stakeholders, including the government, academia, developers, and consumer groups, will need to reach a consensus regarding a comprehensive agenda for basic and applied research, application development, and demonstration projects.
Examples of areas in need of basic and applied research include: impact of IHC on behavior change and quality of life; population and individual differences in use and effects of IHC; effectiveness of different media in health communication and behavior change; effective user interface design; measurement approaches for assessing application efficacy and effectiveness; understanding concordance between patients preferences and health choices as a measure of decision quality; relative effectiveness of IHC and alternative interventions (e.g., virtual versus in-person support groups); and IHC implementation models as they relate to professional and institutional adaptation and integration. Examples of worthy application and demonstration projects include: community wide projects to improve population health; integration of IHC applications in health care settings; implementation of IHC in settings outside the context of the clinical encounter; projects that integrate delivery of care, preventive health, and access to information and support; models for implementing and maintaining IHC applications in organizations including workplaces; applications for the workplace, underserved populations, and other groups of low commercial interest; and projects that expand public access to IHC. Public Funding Agencies A variety of government agencies currently support research and development in IHC. As with other health issues, coordination of funding programs is sometimes lacking. In a rapidly evolving field such as IHC, it is particularly important for public agencies to coordinate their efforts to ensure that priorities are sufficiently addressed and that redundancy is minimized.
Impact of IHC Some researchers have attempted to document the impact of specific IHC interventions in select populations, but there are no studies of the impact of IHC on the health, economic, and social outcomes of large populations. Data from such studies would be extremely useful as a basis for policy and funding decisions of policymakers, purchasers, and investors.
Policies and Practices of IHC There is no formal activity to monitor the impact of IHC-related policies at the local or national levels. Such a program would be a means by which consistency of policy and regulatory oversight might develop. For example, consistency or compatibility of standards for technical interfaces and terminology can help foster innovation of IHC application development by clarifying the conditions under which they would be expected to operate. Unless some means exists to objectively monitor policies and practices influencing IHC, the totality of the policy environment is not likely to be understood or improved.
Privacy and Confidentiality Related to IHC
Strategy 3. Enhance Capacity of Stakeholders to Develop and Use IHC IHC Application Development Resources Many developers, especially individuals, small companies and nonprofit organizations, have limited resources. Development and dissemination of public domain application development resources can benefit the field by reducing duplicative efforts and avoiding reinvention of commonly used objects, tools, and other materials. This will free up resources and allow developers to focus on innovative application design, and enable small or nonprofit developers to more easily create IHC applications for underserved or low-commercial interest groups.
Common functions for which public domain objects and tools are needed include those that:
Examples of public domain information resources needed for IHC development include:
Collaboration and Partnerships Many developers have limited expertise or experience in technical or topic-specific areas that are necessary for application development and evaluation. There is often a disconnection between basic research conducted at academic institutions and application development activities of commercial companies. The demands and pressures of the commercial sector to bring products quickly to the market may sometimes run counter to the deliberate pace of academic institutions. Examples of collaborative projects exist but it is clear that many projects are duplicative (e.g., many Web sites offer similar information on the same health topics) and some may even replicate failed approaches. This phenomenon, unfortunately, also is common in many other areas of science and engineering. Increased exchange of information among developers, and between developers and other stakeholders, may result in more efficient uses of unusual expertise and limited resources, and ultimately improve the quality of applications. Of course, some developers will be constrained in participating in collaborations or partnerships because of their desire to preserve proprietary approaches and maintain a competitive edge.
Health and Technology Literacy Components of health and technology literacy include computer and technology use skills, reading ability, and ability to comprehend health- and science-related concepts. Many individuals have inadequate technology skills, such as the ability to use a computer or navigate the Internet. Although young people are becoming increasingly adept in using computer-based communication technology, many older individuals and others who are at high risk for adverse health conditions, will need training to be proficient. In addition, some cannot utilize IHC applications because they do not understand health information or the medium used is not appropriate for them (Williams et al., 1995; Baker et al., 1996; Eng et al., 1998). About half of the US population has rudimentary or limited reading skills (NWGLH, 1998). People with inadequate health literacy have a variety of communication difficulties that may interact to influence health outcomes (AMA, 1999). Improving these skills is important because most IHC applications are primarily text-based and designed for educated and literate audiences. IHC applications should take advantage of interactive media to address the needs of low literacy audiences. In addition, the ability to interpret and personalize scientific information will be increasingly important as individuals assume more responsibility for their health-related decisions and seek health information and support outside of the traditional health care setting.
IHC Intermediaries Some consumers do not have the skills or experience to successfully and completely use IHC applications. These individuals will require assistance from health professionals, peers, or others who are experienced and skilled in using technology and interpreting health information. Within the context of the health care setting, much of this responsibility may fall on health professionals, including health educators and clinicians, many of whom will need additional training to assume this role. Educators, medical and other librarians, and other information intermediaries also will play an increasing role in helping the public select and interpret health information. Within health care organizations, performance measurement tools can help focus institutional resources and attention on ensuring that members have access to quality health information and support.
Implementation and Financial Models for IHC The effective dissemination of IHC applications has a number of obstacles and will not occur unless there are appropriate implementation and financial models to support them. Little is known about implementation barriers, their impact, or how to overcome them. A central implementation hurdle is in modifying the roles of those who play a key role in system implementation. An attractive and content-rich application is only one component of a successful system. The successful implementation of applications may require complex organizational changes. Job descriptions, reporting relationships, and organizational structures need to be considered in implementation activities. For example, issues related to modifying routine clinical practice so that clinicians refer patients to these systems and support them in their use are important. In addition, how can mechanisms be established to identify, recruit, and train potential users who are likely to benefit from such systems in a timely manner? Other implementation challenges include: overcoming clinician resistance for varying reasons; developing appropriate support mechanisms and allocating resources; involving key personnel early in the adoption decision; clarifying organizational priorities for the system; and supporting infrastructure and equipment costs of providing computers and access devices.
Another major obstacle to widespread dissemination of IHC applications is the relative lack of appropriate financial models used by developers. While consumer demand for IHC applications seems to be substantial, the appropriate business model is not clear. Moreover, perceived costs of adopting and integrating IHC applications may deter some purchasers and providers who are considering their implementation.
Strategy 4. Ensure Equitable Access to IHC for All Populations Public Awareness and Appropriate Use of IHC Although public awareness of technology, such as the Internet, is widespread, many people, including health professionals, do not know which health-related applications are available or which ones are appropriate for their needs. Greater awareness of the appropriate selection and use of IHC applications may lead to widespread use of high-quality products and discourage use of poor-quality ones, thus, maximizing the benefits and minimizing the potential for harm from IHC.
Access to Infrastructure Both public and private sector stakeholders, particularly government agencies and corporations, will need to collaboratively reduce the gap between the health information "haves" and the "have-nots." For this to happen, a range of activities will be necessary including supporting health information technology access in homes, workplaces, and public places, developing applications for the growing diversity of potential users, and improving literacy in health and technology, especially among underserved populations.
Applications for the Underserved Applications that address rare health conditions or target certain populations (e.g., disabled communities, low-income groups, certain racial/ethnic groups, and non-English speaking populations) are not a focus for commercial developers. Many of these orphan applications are not being produced in the current market environment because developers lack the skills or experience in developing applications for these communities. These products also are not economically viable because the target audiences are too small or because the potential users cannot afford to pay for the application.
Conclusion There is little doubt that IHC applications will continue to grow and that consumers will increasingly turn to them for health information, communication, support, and transactions. IHC has the potential to dramatically improve the ways in which people prevent disease, maintain their health, and recover from illness. Rapidly evolving technologies may impact health and health care in ways that few of us imagine. However, for IHC to play a pivotal and positive role in creating a healthier society, a range of initiatives is needed and many stakeholders must participate in application development and evaluation. Consumers will need to assess applications before using them and avoid using unevaluated ones, developers will need to implement evaluation methods throughout the development process, health professionals will need to become involved in application development and refer their patients to high-quality products, information intermediaries will need to help consumers select applications and interpret health information, health care purchasers will need to demand evaluated products, and policymakers will need to implement policies that support the development and use of high-quality applications. An evidence-based approach to the development and diffusion of IHC applications is central to the process of ensuring that IHC applications are of high quality, effective, and accessible to all. National initiatives that focus on strengthening evaluation and quality of IHC, improving knowledge and understanding of IHC, enhancing capacity of stakeholders to develop and use IHC, and improving equal access to IHC, could enable us to reap its many potential benefits to individual and public health.
Return to Table of Contents
Comments: SciPICH@nhic.org Updated: 05/09/01 |
|||||||||||||||||||