Objective 11-4. Disclosure of Information
to Assess the Quality of Health Web Sites
Drafter
Cynthia Baur, Ph.D., Office of Disease Prevention and Health Promotion,
U.S. Department of Health and Human Services
Small Group Participants
Chuck Bell, Consumers Union
Carol Cronin, Markle Foundation
Guy D'Andrea, URAC
Robert Denniston, Office of National Drug Control Policy
Joan Dzenowagis, Ph.D., World Health Organization
Note taker: Marianne Glass Duffy, National Institute of Nursing
Research, National Institutes of Health, U.S. Department of Health
and Human Services
Back to Top
Text of Objective
Increase the proportion of health-related World Wide Web sites that
disclose information that can be used to assess the quality of the
site.
Scope
The objective is intended to represent an important aspect of quality
improvement for health Web sites but cannot encompass all issues
raised by and related to the subject. The objective means to serve
the specific purpose of providing data on progress for health Web
site quality improvement and the general purpose of stimulating
discussion about a wide range of health Web site quality issues.
Measurement
The objective is developmental. Several elements are needed to make
this a measurable objective. These elements include a reliable estimate
of the number of health-related Web sites and the number of health-related
Web sites that disclose information that can be used to assess quality;
consensus about which information should be disclosed to assess
quality; and an organization or governmental entity to undertake
assessment and measurement. It may be possible to work with Internet
search engines and directories to develop a methodology to create
a proxy number for the total number of health-related Web sites.
The explicit purpose of the objective is to ensure that Web sites
properly disclose certain information about their operations. The
supporting text for the objective identifies six types of information
that should be disclosed to users of health Web sites.
- Identity of the developers and sponsors of the site (and how
to contact them) and information about any potential conflicts
of interest or biases (Note: "Bias" is not intended
to mean simply a point of view or perspective but a systematic
prejudice or predisposition that is intended to influence a user
unfairly.)
- Explicit purpose of the site, including any commercial purposes
and advertising
- Original sources of content on the site
- How the privacy and confidentiality of any personal information
collected from users is protected
- How the site is evaluated (in terms of impact on users as a
result of interacting with a Web site's functions)
- How the content is updated
Although not a matter of information disclosure, Web sites should
also be accessible to all users, including those with disabilities
and low-end technology.
Back to Top
Issues Pertaining to Measurement
The inclusion of this objective in the Nation's public health agenda
represents the belief that the implementation and measurement of
the diffusion of disclosure and quality standards for health Web
sites is vital, not only to improve individual Web sites, but also
for the credibility of the e-health field as a whole and the overall
improvement of the public's health. The challenges to achieve improvements
should not be underestimated, although numerous groups have made
substantial progress to identify ethical practices, disclosure mechanisms,
and quality standards for health Web sites.
Defining the universe of health-related Web sites is difficult.
An infinite and ever-expanding number of Web sites offers an ever-increasing
variety of functions. Some Web sites are clearly providers of mainstream
health information and services; others mix health with other subject
areas, leaving the boundaries on "health" information unclear. For
example, should all lifestyle information be considered "health-related"?
How should the categories of "alternative and complementary medicine"
be treated? Should personal Web sites or even pages posted by an
individual to express a point of view or personal experience with
a health issue be considered health-related Web sites? If the most
expansive definition of "health-related" is used, even more Web
sites may need to be considered, and it is not yet clear how this
term should be applied in either the short term or the long term.
The objective is based on the principle that public disclosure
of critical information about Web site operations will benefit users.
Disclosure is considered an essential first step in the process
of quality improvement. However, disclosure alone is unlikely to
accomplish the larger goal of the objective, which is to improve
the quality of health Web sites, not just increase the amount of
information disclosure. In lieu of an oversight and enforcement
mechanism for Web site quality standards, disclosure is used as
a proxy for quality, not quality itself. Quality is constituted
by the actual characteristics of a Web site, its practices and policies,
and the value of the site and its content to users. Moreover, disclosure
doesn't necessarily change consumer behavior to seek out the highest
quality Web sites, although disclosure may make the behavior more
likely. One of the limitations of a disclosure approach for Web
sites is that, to date, it has been a purely voluntary effort, and
it is not clear how broad and deep the support among Web site sponsors
is for disclosure. Also, little effort has been made to make information
about Web sites easily understandable for consumers.
An issue that underlies disclosure is the difficulty associated
with developing and implementing Web site quality standards. Elaborating,
operationalizing, and maintaining core elements and definitions
of quality standards require considerable work. Once the standards
are developed, they need to be implemented consistently, presented
understandably, and disclosed publicly. Mechanisms to monitor their
use and enforce their proper application do not yet exist and will
require a substantial commitment of resources and effort. An additional
issue is the applicability of standards, particularly voluntary
ones, across the spectrum of Web sites. Government Web sites, for
example, may have specific obligations or standards based on legal
or regulatory requirements or the mandate of the agency or office
that sponsors the Web site. Private-sector Web sites that engage
in a specifically regulated activity, such as the sale of prescription
medication, may be subject to their own sets of criteria. It is
not clear how many different sets of standards can be meaningfully
applied to a given Web site or a set of content, or if a single
set of criteria could be meaningfully applied to all health Web
sites.
The proliferation of Web site standards may present another problem
for creating an environment for meaningful disclosure. Inundating
users with information about every aspect of a Web site's operations
will likely create a situation of information overload, resulting
in user frustration and lack of interest or time to process all
the different approaches and pieces of information. Standard criteria
and templates, whether in human or machine-readable languages, have
been proposed as a solution by some organizations. Web site sponsors,
search engines, and directories may be reluctant to agree on standards,
however, if they perceive that their own ability to respond to market
conditions and their user base will be limited as a result.
Back to Top
State of Knowledge About
Issues Represented by the Objective
The objective intends to promote greater disclosure of information
about health Web site content and operations so that users can determine
for themselves the degree of quality and appropriateness of the
site and its content or features. A disclosure approach is not intended
to assess whether Web site content is accurate. Rather, disclosure
of key information allows potential users to compare Web sites and
decide which they can trust and which are the most suitable for
their needs. Disclosure of key information has been used in situations
where users may have differing perceptions of what constitutes "appropriate"
and "high quality." In the context of Web sites, "quality" is not
limited to the caliber of information on the site; it extends to
multiple elements that bear on reliability, value, and user protections.
The idea of disclosure of information about Web site operations
as a means to assess appropriateness and quality borrows from other
consumer sectors and health interventions. The assumption is that
consumers—whether of manufactured products, medical procedures,
or Web sites—need information to make the most informed decisions.
Examples include the practice of giving patients information about
the risks and benefits of medical treatments or experiments or giving
potential purchasers information about consumer products or services.
The nutritional food label is one of the most well-known implementations
of the disclosure approach. The label is required to appear on most
types of packaged and prepared foods and to provide specific information
about the nutritional characteristics of food items so that consumers
can decide for themselves if the food fits their own dietary needs.
The label and its constituent elements have been subject to extensive
public comment and are standardized. The Food and Drug Administration,
the Federal agency with oversight responsibility of the food label,
has put a great deal of effort into research and evaluation of consumers'
understanding and use of the label to make the label "consumer-friendly."
Research has shown that consumers do use information from food labels
to make purchase choices and that those who do can positively influence
their nutritional intake.
In the Internet context, much of the debate about disclosure of
information has occurred in relation to Web sites' handling of users'
personal information. Congressional committees have held numerous
hearings to discuss the information practices of Web sites. The
Federal Trade Commission (FTC) and numerous research organizations
and advocacy groups have conducted studies and issued reports documenting
that in general Web site users are ill-informed about Web sites'
information practices. Even savvy Web users often have a difficult
time understanding the privacy policies that Web sites post to disclose
their information practices. The FTC has proposed that Web sites
adopt fair information practices to inform consumers of the uses
of their personal information. Notably, the Health Insurance Portability
and Accountability Act of 1996 (HIPAA), which is the latest set
of Federal regulations on health information privacy, will not affect
most health Web site operations. Technology tools, such as the Platform
for Privacy Preferences Project (P3P) standard for matching users'
preferences with the elements of Web sites' privacy policies, are
also emerging and contributing to the advancement of a disclosure
approach. However, credible research is lacking on the impact of
making users more aware of privacy and disclosing specific practices
(e.g., disclosure, fair information practices, P3P).
Although not specific to Web-based information or the health sector,
information quality standards will achieve a new level of rigor
and prominence with the implementation of Federal "Guidelines for
Ensuring and Maximizing the Quality, Objectivity, Utility, and Integrity
of Information Disseminated by Federal Agencies" (Section 515, Public
Law 106-554; H.R. 5658). As of October 1, 2002, all Federal agencies
are obligated to develop and abide by information quality standards
that will apply to almost all information disseminated to the public,
including Web-based health information. The guidelines are likely
to have an impact on information practices beyond the Federal Government.
Federal agencies will need to review not only the information they
create but also third-party information that agencies disseminate
as their own. Non-Federal and private-sector organizations that
supply information to Federal agencies may need to change their
own information practices to meet agency standards. In this way,
the standards may shift expectations and practices across all sectors,
including health information.
Back to Top
Stakeholders
The ultimate goal of the objective is to increase the proportion
of reliable health Web sites. Those who would benefit from this
are numerous and varied. However, some Web site sponsors benefit
from the continued dissemination of misleading or false information
because it helps to sell their products, however ineffectual or
dangerous. Although these types of Web sites can never be completely
eliminated, their visibility may be diminished if the proportion
of reliable Web sites increases.
The following list includes any group that has a reasonably substantial
interest in the disclosure of information about health Web sites.
- Health information seekers (consumers, patients, families, and
caregivers)
- Healthcare professionals and their professional societies
- Healthcare provider organizations (private and public)
- Health insurance providers
- Managed care organizations
- Public health organizations
- Third-party reviewers and accrediting bodies
- Government—regulators, information providers, protectors
of the public interest, consumer protection groups, healthcare
providers, and researchers
- Academic and research institutions
- Health info-mediaries—organizations that help their constituencies
access and use health information
- Literacy and health literacy experts
- Health industry groups—pharmaceuticals and medical products
and devices
- Sellers of health information—online and offline bookstores
(e.g., Amazon, Barnes & Noble)
Some groups will be more interested and motivated than others.
Web site sponsors want to be perceived as credible and reliable
providers of information and services and need mechanisms to signal
their credibility and reliability to consumers. Internet search
engines enhance their own popularity, credibility, and economic
viability when they direct searchers to the information they need
in the most efficient, reliable manner, which may be easier to do
with disclosure and quality criteria. Health information seekers
need credible and reliable information to make informed decisions
and high-quality services to realize the maximum improvements in
their health, and disclosure and quality criteria may be important
tools to help them. Healthcare professionals are also information
seekers, as well as sources for referrals, and they need quality
information so that they know they can rely on the Web sites they
use and to which they refer their patients. Health services and
insurance providers need their own Web sites to be perceived as
credible, and they need to know that the patients they treat and
insure are using reliable information to make decisions that affect
their health. In addition to their functions as health insurers
and service providers, governments are responsible for providing
health information in a reliable manner, protecting consumers from
harm, and promoting the public interest in having a safe Web environment
and reliable health information and services.
Back to Top
Status of Selected Stakeholder
Activities
Various mechanisms have been proposed to help users assess Web site
quality, which include mechanisms to protect users' privacy. All
mechanisms rely at least in part on Web sites' disclosure of information
about their operations. Without disclosure, it would be virtually
impossible to answer most of the questions on the quality checklists,
seal assessment forms, and other tools, or to check for compliance
with quality standards that have been developed by the initiatives
described below. Numerous groups have created checklists that consumers
can use to assess Web sites. Seal programs, such as the HON code
and TRUSTe, provide minimum criteria that Web sites must meet to
receive the right to display the seal.
Some e-health companies collaborated to draft a code of ethics
for health Web sites (Hi-Ethics). The Internet Healthcare Coalition
(IHC) convened a diverse group of interested parties, including
some Hi-Ethics members, to draft a separate code of ethics. URAC,
an accrediting body for healthcare organizations, has created a
program to allow health Web sites to apply for accreditation. The
URAC standards encompass elements of the Hi-Ethics and IHC codes.
Professional societies have created their own referral programs
and networks to direct consumers to reputable organizations and
Web sites. The American Medical Association (AMA) has created quality
guidelines that it uses to govern operations on AMA-affiliated Web
sites.
Government agencies have created portals and Web sites to distribute
reliable information and links to other organizations and sources
of information. The World Health Organization proposes to create
and manage a dot-health domain name that would function as a label
for trustworthy health information. The European Union funded an
initiative called MedCERTAIN, envisioned as a self- and third-party
rating system with metadata tagging technology, standard quality
vocabulary, and content filtering labels. MedCIRCLE, a new initiative,
expands on MedCERTAIN and aims to realize the application of standard
vocabularies and the interoperability of rating services and gateways.
The U.S. Department of Health and Human Services (HHS) is implementing
several ongoing activities. In response to the Federal law on "Guidelines
for Ensuring and Maximizing the Quality, Utility, and Integrity
of Information Disseminated by Federal Agencies," HHS has drafted
departmental and agency-specific guidelines on information quality.
HHS also has draft guidance on developing agency policies about
linking to Web sites external to the Department. The Multimedia
Technology Health Communication small business grant program at
the National Cancer Institute (NCI) has funded two projects to develop
Web assessment tools. NCI staff has developed a tracking model for
grantees and contractors that includes consistent procedures for
usability testing and for assessing the quality of site content.
Back to Top
Factors That Can Influence
Change
The Internet is a very fluid environment, and the factors that can
influence the disclosure of information about Web sites and their
content and the diffusion of quality standards can emerge and recede
quickly. The following list identifies the most likely reasons that
Web sites may begin to disclose information, the likely leverage
points to encourage disclosure, or potential barriers that could
impede disclosure or make disclosure less effective. Some of these
factors may be specific to health Web sites, although many reflect
the broader Internet environment and public perceptions of the Web
and Web site practices.
Pending or actual legislation
- Self-regulation. Web site sponsors and their business
partners have been concerned about the potential for regulation
of Web site operations, specifically regulation of the use of
consumers' personal information. The industry has responded by
launching numerous self-regulatory efforts and testifying frequently
to Congress about the viability of self-regulatory mechanisms.
More robust disclosure could be an important element in convincing
legislators that Web sites are providing enough information to
consumers to make self-regulation workable.
- Legislation in other sectors. The notices mandated by
the Gramm-Leach-Bliley Act (GLB) have brought attention to the
process of disclosing information to consumers. Under GLB, financial
institutions are required to provide notices to their customers
about their information-collection and information-sharing practices.
The financial services sector has been publicly criticized for
the quality of these notices. The FTC held a workshop in December
2001 for companies to learn more about disclosure and providing
notices about privacy for consumers. Privacy regulations under
HIPAA will also be a high-profile test of how to give consumers
notice about information-handling practices. In addition to increasing
the amount of notice in the financial services and healthcare
sectors, GLB and HIPAA may become important models of how to disclose
information about many types of business practices, including
Web site operations.
Marketplace considerations
- Failure of dot-coms. As a result of difficult market
conditions, Web site sponsors may be more interested in finding
ways to retain customers and develop long-term relationships with
them. Companies may perceive disclosing information about their
Web operations as one way to enhance their reputations and attract
and retain customers.
- Image and public relations. The ability to enhance Web
sites' reputations through positive publicity, or fear of potential
damage to image from adverse publicity, may motivate information
providers to adopt disclosure mechanisms and quality standards.
- Transparency. Disclosing practices that Web site sponsors
may be uncomfortable defending in public may motivate them to
change and improve their practices.
- Burden of disclosure. Web site sponsors may view the
perceived financial and administrative burdens of reporting information
and redesigning Web sites to make information easy to find and
use as a barrier to disclosure of information.
- Financial costs of disclosure. Disclosure may impose
financial costs on Web sites for several reasons. Web sites may
need to be redesigned to accommodate the disclosure of information
about quality criteria. Web site sponsors that decide to seek
accreditation through URAC or other means have to pay a fee to
the accrediting body as well as cover the costs associated with
staff time to prepare for the review and to redesign business
functions to meet standards.
Government Web sites
- Applicability of disclosure and standards to Government Web
sites. Government Web sites may be reluctant to participate
in disclosure and quality standard mechanisms that either are
designed to fit the operations of non-Government Web site operations
or require submitting the site to marketplace considerations,
such as paying for accreditation or being listed in a directory
or search engine. Several outcomes are possible if Government
Web sites decide not to use the same quality standards and disclosure
mechanisms as those used by non-Government Web sites. Non-Government
Web sites may decide that disclosure and quality criteria cannot
be that important if Government Web sites are not willing to use
them. An artificial distinction could be made between Government
Web sites and non-Government Web sites because Government information
is available on many different types of Web sites, some of which
are clearly dot-gov and some of which aren't, including those
that are clearly commercial.
Media activities
- Media activism. The media have reported numerous electronic
mishaps related to the unauthorized release of identifiable personal
information. Mistaken e-mails, files that should have been erased
but were left on computers, and unauthorized sharing of personally
identifiable health information are a few examples that have appeared
in the headlines. In national surveys, consumers repeatedly report
concern about how much information is collected about them while
they are on the Internet and who has access to that information.
Public comment about the recently enacted privacy regulations
for healthcare transactions also indicates concern by consumers
about the electronic information flow among organizations that
provide health care. If media coverage feeds public fears about
abuses, disclosure may be seen as insufficient to deal with the
magnitude of the problem.
- Media framing of quality issues. Consistent use of media
for advocacy of quality standards could enhance the media's watchdog
role and promote consumer awareness regarding Web content. Responsible
(instead of inflammatory) stories on fraudulent and unethical
practices could promote outrage and action on the part of consumers,
rather than broadly dampen enthusiasm for using the Web as a health
resource.
Attention on quality
- Break-out of one (or more) of the quality initiatives.
If some of the most popular health Web sites begin to use one
of the more robust approaches described above and they receive
large amounts of favorable publicity, public demand for disclosure
of information and adoption of quality standards may increase.
- Best practices. The development of best practices for
disclosure and quality standards could reinforce the behavior
of those Web sites that are implementing the practices and be
a prod for those sites that aren't.
- Healthcare providers' adoption of Web tools. Healthcare
providers could take on more of an opinion leader role in recommending
the best Web resources. This would help dilute the typically negative
opinions that many healthcare providers express about resources
for consumer health information and provide an important conduit
for awareness and education activities with patients.
- Emergence of a market leader for Web site seals and recommendations.
Seals and recommendations are based on the ability to assess Web
sites' activities. If a small number of seals and related identifying
marks or logos emerge as credible and meaningful to consumers,
disclosure would consequently be enhanced.
- Visible support. Employers, other sponsors of health
insurance, and information intermediaries could insist that their
agents—such as health plans, employee assistance programs, and
disease management programs—use only health information services
that disclose information about their Web site operations and
meet quality standards.
Technical factors
- Emergence of technical solutions. Standards, such as
P3P, and one-click access to an "About Us" section that
relays core information to users could make it easier for Web
site sponsors to engage in disclosure.
- Filtering software. The proliferation of tools to recommend
sites could direct users to sites that disclose meaningful information
in an easy to understand way and direct users away from sites
that don't.
User considerations
- Information overload. In the interest of full disclosure
or perceived liability considerations, Web sites may provide so
much information about themselves and their operations that users
become overwhelmed by the amount of information that they need
to assess the quality of a Web site. Or, because of the lack of
standardization in disclosure, users may find they need to learn
multiple approaches to seek out information on Web pages as they
move among sites, which could also create information overload.
The likely consequences are frustration and an eventual lack of
interest or at least attention to the information provided.
- Disparities. The success of disclosure as a tool to help
consumers will be influenced by the characteristics of Web site
users. Users differ greatly in their health status, their fluency
in the language used to present information, and their skills
in information processing, general literacy, technology literacy,
and health literacy. Also, the sophistication of the information
presented on Web sites, in terms of both health content and information
about Web site operations, tends to be very high, which would
make the site difficult to use for a large number of health information
seekers.
Back to Top
Strategies and Solutions
Premise: Promoters of disclosure and quality standards need
to find solutions that put the least amount of burden on users.
Three general strategies have been identified.
Create market advantage for quality (of interest to search
engines, directories, other aggregators of Web content, and evaluators
of Web sites and their content).
- Encourage dissemination of information about trusted sites.
- Promote, publicize, and reward good information practices; recognize
the best and the worst.
- Promote editorial "brands" for the most reputable
sites.
Build demand for quality.
- Encourage public education to alert Web site users to disclosure,
quality criteria and standards, how to identify them, and how
to interpret seal programs or other disclosure mechanisms.
- Make communicating with Web site users about quality a priority.
- Engage health info-mediaries: they are the conduits to particular
user groups and audiences to discuss quality and how to use the
Web.
- Engage in counteradvertising: Web advertising could be used
to direct users away from unreliable and low-quality sites and
toward reliable and high-quality sites. One method might be ads
that remind users of corporate and other influences on site content.
Build capacity to use quality criteria.
- Rewrite Web site policies in a simple and focused manner to
convey only the most relevant information to consumers.
- Develop a template to allow users to identify the information
being disclosed easily and quickly.
- Develop users' critical information-seeking skills.
- Collaborate with K-12 educators to integrate health content
into school-based programs that teach technical skills.
- Educate Web publishers about the importance of incorporating
quality criteria.
Back to Top
Action Steps
This section identifies the necessary steps for measuring the objective
or supporting the three strategies outlined above. They are not
in sequential order. Many of these actions could occur simultaneously
and in coordination with each other.
- Define and characterize major categories of health-related sites
for identification and quantification.
- Identify core elements and definitions of quality standards.
- Conduct consumer testing to find out what consumers understand
about disclosure and quality standards on health Web sites.
- Create a communication tool (e.g., label) to help Web site users
understand the information disclosed about Web site operations.
- Explore U.S. Department of Education programs for teaching "skills
for the digital age" to see if health content can be included.
- Mobilize health info-mediaries to help stimulate consumer demand
for disclosure and quality.
- Engage groups involved in health literacy in skill building
around the use of Web resources.
- Identify potential template approaches (e.g., P3P, the food
label).
- Identify industry leaders in quality improvement for Web sites.
- Create a mechanism to monitor adherence to quality standards.
- Develop self-applied tools for Web publishers to enhance their
understanding of quality issues.
- Research the operations and practices of Internet search engines
and directories to understand their methodologies and market influences.
- Open discussions with search engines about the possibility of
incorporating quality criteria and disclosure templates in the
search and directory processes.
- Identify ways to publicize search engines and directories that
steer users to more reputable sites.
Back to Top
For More Information
American Medical Association
www.ama-assn.org/ama/pub/category/1905.html
Hi-Ethics
www.hiethics.com
Health on the Net
www.hon.ch
Internet Healthcare Coalition
www.ihealthcoalition.org
MedCERTAIN
www.medcertain.com/english/metadata/overview.htm
MedCIRCLE
www.medcircle.org
URAC
http://webapps.urac.org/websiteaccreditation/default.htm
Back to Top
References
Baur C, Deering MJ. Proposed frameworks to improve the quality of
health Web sites: review. Med Gen Med. September 26, 2000:E35.
Available online at http://www.medscape.com/viewpublication/122_toc?vol=2&iss=3&templateid=2.
Baur C, Deering MJ. Commentary on the "Review of Internet health
information quality." J Med Internet Res 2001;3(4). Available
online at www.jmir.org/2001/4/e29/index.htm.
Eysenbach G. An ontology of quality initiatives and a model for
decentralized, collaborative quality management on the (semantic)
World Wide Web. J Med Internet Res 2001;3(4). Available online
at www.jmir.org/2001/4/e34/index.htm.
Mack J, ed. The new frontier: exploring ehealth ethics.
Washington, DC: URAC/Internet Healthcare Coalition, 2001.
Risk A, Dzenowagis J. Review of Internet health information quality.
J Med Internet Res 2001;3(4). Available online at www.jmir.org/2001/4/e28/index.htm.
Back to Top |
 |
|
