Box V-1
A Developer’s Dilemma

The PDM company was formed by the Jones brothers, Paul (a programmer), Bob (a business school professor), and Greg (a physician), in response to the news that their father had been diagnosed with prostate cancer. As their father informed each of them about his condition, he also expressed his confusion and doubt about what to do. Each brother commiserated with him about the difficulty of working through the available options for prostate cancer treatment. One evening not long afterwards, the brothers were having dinner together and talking about their father. They agreed that men like him needed a program to help sort through their options in a way that would help them make the best possible decisions during a difficult time. Collectively, they felt that their unique assemblage of talents gave them a special understanding of what was needed and how to do it. As a result, they created the PDM Company (for prostate decisionmaking) with a goal of creating (and selling) an interactive application that would help men decide about treatment options–thus making some money while meeting a real need.

With expert medical guidance recruited by the physician brother and decisionmaking expertise from the business school brother, the programmer began creating their flagship program. Due to major disagreements among the medical experts about which treatment options actually were most desirable, the project schedule started slipping. Meanwhile, Bob was holding focus groups to determine what men thought would be most useful and to get their reactions to mock-ups of the program. Enthused by the reactions of the men he spoke with, Bob began encouraging Paul to expedite development of the program. Greg was very concerned about the potential to lead men into the wrong decisions. He was worried about both legal liability and the moral/ethical issues of encouraging specific decisions, even inadvertently. Bob, an expert in marketing, explained how easy it is to influence decisions while providing seemingly neutral information. Paul suggested an extensive program evaluation to make sure the program actually helped men make the right decisions. They all agreed this sounded like a good idea.

Paul was coordinating the production of multimedia assets (graphics, audio files, and video) as well as programming. The video subcontractor kept delaying production and Paul could not finish the programming without the assets. Then, a major new microcomputer operating system (OS) version was announced and Paul realized his program would need to be changed if it was to run under the new OS. Each delay meant more days and weeks of effort, rent, subcontracts running over budget, and other expenses. Money was becoming a real worry. Finally, Paul announced that a pre-beta version was available for testing. Bob tried to run it on his office computer but he did not have the right multimedia software installed. He gave the program to others but all of them ran into some type of bug or installation glitch. Paul tried to create new versions for each problem that was encountered. Bob began to worry about just what types of computers could run the program. Was the installed base a big enough market? Maybe they should simplify the program so that it would run on more computers or make it a Web-based application? Paul was beginning to feel stressed out. Greg kept asking about the big delay.

Not much later, Paul produced a "final candidate" program and they began pilot testing with five men recently diagnosed with prostate cancer. Despite some glitches, the program worked pretty well and the men generally liked it. They felt it definitely was useful and had helped them make good treatment decisions. Greg was called in and the three Jones brothers began discussing an evaluation that would include enough men to have a statistically meaningful sample and would use a randomized control trial design. Greg and Paul agreed that anything less would not really tell them whether the program worked well or lead to any bad decisions. It would take about six months, but it was all going to be worth the effort. The euphoria of being so close to completion lasted three days. Then, Bob was at a conference where he overheard two men talking about the prostate decisionmaking program they had recently seen demonstrated. "Very slick," said one man. "Yes, they’ve got a winner I’d guess," added the other. "Do you really think they can ship it within the next three months like they claimed?" asked the first. "I think so," was the reply. Bob ran to a phone and called Paul. To his surprise, Greg answered and immediately interrupted his brother to announce that Paul had completed the "final" program. "Great news!" he added. "We’re ready to begin the randomized clinical trial. My friends, the ones with that huge urology practice I told you about, have agreed to start referring patients Monday." Bob brusquely replied, "Forget the trial, we’ve got to ship right away or we’ll get scooped! I just heard about another program coming out." What ensued was a bitter argument about whether or not they had sufficient data to begin shipping the program.

"We can get data from actual users," said Bob. "Without random assignment, we cannot be sure what that kind of data would mean," said Greg. Paul added, "I don’t want to do any harm but I don’t really see how we could." Greg explained how leading a man into the wrong decision could mean an unnecessary end to his sex life, on top of urinary incontinence. "Think about no more intercourse and being incontinent the rest of your life," Greg said. "But we probably have been more careful about creating a balanced program than our competitors. If we don’t go to market now we may not make anything out of this!" Bob said with emotion. "I don’t think we should market it unless we have data to back up its efficacy and safety," said Greg. "But that’s what the pilot data says, isn’t it?" asked Paul. "It’s not conclusive, there were only five men," said Greg. "Yes, but it’s the best we can do for now," added Bob. "We can run the trial while we start marketing, can’t we?" said Paul. Silence followed as each brother thought about the choices and implications. Paul spoke, "Well, what about it? We could start the trial as planned and Bob can start the marketing plan. By the time we ship a significant number, we should have preliminary data from enough patients to see if there are any major problems. We will keep analyzing the trial data as they come in and if we see any problems we yank the program and fix whatever has come up." Greg shook his head, "If we cause a problem I’d have a hard time living with it. First we test, then we ship." Bob looked glum. He said, "You do understand we could lose everything we’ve put into this?" Greg sighed and said softly, "Yes, I do."

The next three months were tense ones for PDM. After enrolling the first 20 men, they discovered that about half of the users were missing a critical section of the program due to a confusing interface element. The experienced computer users understood the interface metaphor Paul had programmed, but the others missed it. It was a serious problem, but easily fixed. Moreover, their competitor did release a comparable program. They heard about it while reading an industry newsletter. There was a review of their competitor’s program by a physician. Their dismay turned to glee as they read the review. The gist of the review was that the program had offered terrible advice, "evidently based on faulty algorithms," concluded the reviewer. The reviewer strongly suggested that the program be pulled off the market before the developers (and any clinics making it available to patients) faced lawsuits. The brothers carefully reviewed their own program to make sure it did not have similar problems. Once the randomized clinical trial had produced enough data, Greg gave a peer-refereed presentation of the scientific data at a national meeting. A medical director for a very large HMO approached Greg after the talk and asked if he was prepared to negotiate for "the purchase of a rather large number of copies to be used across all of their sites."

Key Considerations

• The first guiding principle must be: "Do no harm."
• Two key ingredients are content expertise and accurate knowledge of the target population (i.e., their desires, capabilities, and needs). Both are necessary for optimal program development.
• Key design decisions (e.g., specific behavioral goals, target platform, structure) must be made carefully—and early in the development process—to avoid wasted efforts and costly delays.
• Adequate testing of the programming and the content are critically important—both to ensure efficacy and to avoid liability.
• Premature marketing can result in problems worse than reduced market share.
• Good efficacy data can be an important sales factor.

Box V-2
The Proactive Health Care Professional

Dr. Emily Chen is a senior physician with more than 20 years of experience in a large oncology group practice. During the last year or so, the clinicians in the practice have noticed a dramatic increase in the number of people who have been coming in with realms of printouts from Medline searches and other online resources. The clinical staff have been overwhelmed by these patients because much clinician time is required to address all their questions and concerns. Some of the clinicians also believe that most of the time spent with these patients is spent correcting inaccurate information or explaining why the information is not relevant to the patient’s situation. As a result, these clinicians have been discouraging their patients from using online health resources. In addition, because the practice receives a fixed pre-negotiated reimbursement rate from the insurance company for most services regardless of the actual time involved, the practice administrator has strongly encouraged time limitations on many types of visits.

Dr. Chen has been reluctant to accommodate the more information-savvy patients in the practice and will sometimes only brusquely answer these patients’ questions. Then, one day, a long-time patient of Dr. Chen’s abruptly announces that, while he respects the quality of clinical care at the practice, his information and support needs are not being met. He is switching to a new practice across town because they not only answer all his questions, but they actually encourage him to conduct online research about his illness. In fact, this practice encourages him to e-mail the clinic staff with any questions he might have at any time. Asking around, Dr. Chen finds out that several other clinicians in the practice have had similar experiences recently. Concerned, Dr. Chen asks the practice’s accountant to review the clinic logs. She finds out that there has been a significant drop in the number of new patients coming into the practice during the last year, and an alarming number of long-time patients have elected to leave the practice.

At their next meeting, the practice management team decides to conduct a survey to assess the information and support needs of patients. The team learns that an overwhelming number of patients are dissatisfied with the information and support they receive from their clinician. It becomes clear to the team that the policy of not fully supporting patients who use the Internet to access health information and support is shortsighted and potentially ill-fated. Using the survey results, the team sets out to develop a comprehensive "patient information support" policy with input from an IT consulting company.

The team decides that the new policy would revolve around a proactive and comprehensive approach to managing patient information and support needs. Their goal is that all patients will be fully supported in seeking information and support, and fully satisfied with their clinical encounters.

The management team quickly realizes that, although most of the office staff have used personal computers, only a few of them have experience in searching for health information on the Internet. They hire a local medical librarian to hold a series of in-service training sessions on locating and evaluating health information on the Internet. The librarian also identifies several well-respected online cancer support groups that the clinicians start to use to get a sense of what questions and concerns are circulating among cancer patients and caregivers. The nurses, technicians, and other office staff are included in these training sessions because they will assume much of the responsibility for supporting patient information and support needs under the new policy.

A subgroup of the team is tasked with building the Web site for the practice. They start by locating and conducting reviews of relevant, health-information Web sites. This group carefully evaluates these Web sites using standardized evaluation criteria and creates links to sites that seem to be of highest quality and relevance to the practice’s patient population. Sites that inform and guide cancer patients to appropriate clinical trials of new therapies are highlighted. They also contract with a regional cancer center to use some of their online content and adapt it to their patient population. To complement the health-information resources on their site, they purchase licenses for several risk-assessment and clinical decision-support tools from a university group after examining several peer-reviewed research articles documenting the effectiveness of these tools. In addition, during the course of the next few months, each clinician submits several "frequently asked questions" from patients and the answers are posted on the practice’s Web site. All of the clinicians are now available by e-mail to handle nonemergency questions. Patients really like this because they no longer feel that they have to think of all their questions during their visits, and many have been reluctant to call their clinician for nonurgent questions.

The team also wants to make sure that all patients have some access to their Web site. On a trial basis, several interactive kiosks are installed in the practice waiting room. An "information assistant" is available to show patients how to use the system. New patients are instructed to use the kiosks to provide a detailed medical history so that the clinicians can focus on their responses rather than administering the history during their visit. Because the kiosks are linked to the Internet, patients also can access the online resources on the practice Web site while they wait. The office finds that the kiosks are well received, and that they actually free up time that the clinicians would have otherwise spent interviewing the patients and explaining basic concepts about cancer.

Several months after implementing their new "patient information support" policy, the team implements structured and open-ended surveys of patients to get their feedback on the new approach. They find that most patients are satisfied with the information and support they receive through the practice. They use the survey data to further refine their quality improvement initiatives. Dr. Chen and the other clinicians also have the following observations: 1) most patients seem to be well informed about their illnesses and their questions during visits are better focused; 2) many patients have become active participants in their care and seem to be more satisfied with their care; and 3) the number of patients in the practice has increased.

Key Considerations

• Clinicians have important roles in helping patients locate, assess, and interpret health information.
• Many health professionals will need training and skills-building in health information technology to adequately address the information and support needs of their patients.
• A proactive and comprehensive approach to addressing patient information and supporting needs can help improve patient satisfaction.
• Encouraging patients to use online resources does not necessarily increase the amount of time that clinicians have to spend with patients during visits.

Box V-3
Implementing IHC in a Health Plan

Janice Rodriguez, an accomplished project manager, has just been recruited from an interactive health software company to be the regional chief information officer for a large progressive health plan with several facilities in the area. The CEO of the health plan wants the organization to be on the "cutting-edge" of using IHC applications to improve member health and to reduce costs of delivering care.

Eager to please, Janice embarks on an ambitious plan to eventually implement a comprehensive suite of Web-based applications to provide seamless health information and support for members, allow electronic member transactions, provide clinical decision support for members and clinicians, and facilitate electronic communication between clinicians and members. For the first phase of her plan, she focuses on selecting and implementing a shared decision-support application for members and clinicians.

Janice ponders whether to outsource the development of the application or to do it in-house. After an assessment of her department’s resources and an environmental scan of available applications, she decides on the former option. She surmises that, with a pilot project, it is better to minimize uncertainty. The health plan’s IT department has some expertise in general Web site development, but they are unfamiliar with decision-support applications. If she outsources to a company with an established product, she is assured of prompt delivery and can more easily predict costs. In addition, she is worried about potential liability issues. Relying on an established and rigorously evaluated application, she concludes, may help in that regard.

She sends out notice of what she is looking for to her former company and a few other contacts. Within a week, Janice is deluged with calls from more than a dozen companies that want to sell their product. Each of them claims that their product is of high quality and very effective.

Janice understands that she needs to base her decision on more than surface appeal or which salesperson can do a better presentation, because her boss is looking for bottom-line benefits to the health plan, such as increased member satisfaction and retention. She promptly assembles a team of people from the IT and clinical research departments with experience in evaluation. They review the literature on evaluation of IHC, talk to colleagues in other plans, and brainstorm, coming up with a list of questions for themselves and prospective vendors. The questions include: What are reasonable objectives for such an application within the context of the organization? Was the application evaluated with regards to these objectives, and what were the results? Has it been shown to be cost-effective? Is the product flexible and adaptable to the plans changing needs?

After reviewing additional information from the vendors, the team decides to purchase a hundred licenses of the application from a company that presented very convincing data on effectiveness which was published in a respected scientific journal. Janice and the team make plans to deploy the application in the primary care department in one of the plan’s facilities because their medical director happens to be a "techie" and is very enthusiastic about the program. Things look positive; the application runs well with their existing clinical information system, the training session is well attended, and several of the primary care staff say that they are impressed with the technology.

Janice and her team check in weekly to see how things are going. After a month, only a small proportion of the clinicians has used the application with their patients. Janice interviews the staff. "I don’t want to support using something that may eventually replace me," says one physician. "I like the program but I don’t have time to use it during my appointments…I am also afraid that it will actually increase the amount of time I have to spend with some patients," says another. Yet a third comments, "It’s too much of a hassle for me to use it because it crashes my personal scheduler program when I do. When I ask our IT administrator to help, he says he will not support it because he is too busy troubleshooting the clinical information system that the plan just purchased last year."

Janice’s team realizes that they have formidable implementation problems on their hands, so they start again from square one, and recruit more troops. They fan out, interview additional stakeholders, hold extensive "town meetings," and form a plan-wide implementation team.

The team is divided into a number of smaller groups that are dedicated to a specific aspect of the implementation process. The development team takes responsibility for assessing the value of available applications, rating them according to compatibility with the health plan, and modifying and designing interfaces to reflect the organizational culture. The implementation working group will oversee the process of defining constituencies and determining how best to represent their needs in the dissemination process. The operations working group considers how the application will be merged with existing clinical care, how quality will be assessed, and how the application will be maintained. The evaluation group defines how the impact of the program will be determined, designs usability testing protocols, analyzes potential return on the investment, and determines how study results will be disseminated. The technology working group hammers out the "nuts and bolts" of the installation and designs the architecture to support the smooth operation of the application. Finally, a planning working group is formed to examine future opportunities and directions for new applications and development.

Meanwhile, a communication plan is set into action so that support for the program can be gathered from the stakeholders and opinion leaders throughout the organization. Key decisionmakers are identified to ensure continued financial support; they are encouraged to join the working groups and inject their concerns into the development process. Clinical staff members from all the facilities are asked to visit and report on changes in medical care and health plan policies. In addition, organization leaders continually address the working groups to inform them of organizational changes so that the applications are consistent with organizational goals.

Finally, the working groups meet every few months with their customers, including members of the health plan, clinical staff, and plan administrators. They define and refine the dissemination process, provide feedback on the success of the program, and brainstorm on future options for added technology.

A year later, a survey shows that 70 percent of members who have used the decision-support application are highly satisfied with their experience and believe that it not only helped them make a better decision, but also improved their relationship with their health care provider. A similar survey among providers shows that more than half have already adapted the application into their routine practice and believe that it improves the quality-of-care they deliver.

Key Considerations

• Health care organizations and other potential purchasers should carefully assess their internal capabilities for application development and maintenance before attempting to develop their own applications.
• Purchasing decisions require careful consideration of the objectives for the application and evidence of application effectiveness.
• Substantial attention to, and planning of, the implementation process are critical. The process should be consistent with the organization’s structure and culture, and address institutional barriers.
• Continuous evaluation and quality improvement are essential.
• Successful implementation is a team process; involvement of, and communication with, all stakeholders are necessary.

Box V-4
A Consumer’s Experience with IHC Applications

Mary Smith returned home from the hospital with her week-old twins, a girl and a boy. Like many first-time mothers, Mary was both enchanted and overwhelmed with the challenges of her new family. The practical advice offered during the childbirth preparation classes seemed to escape her awareness at the very moment that questions about nursing the babies entered her mind. Her questions seemed too minor and insignificant to warrant a call to the nurse midwife. However, she remained worried, and a bit unsure about many of the day-to-day rituals of mothering twins. Encouraged by her partner, Mary decided to use some of the health information resources available on the Internet. During one of her first trips away from home, Mary stopped at the public library to use their Internet computer.

First Time: Finding Resources

Mary began looking for health information at her hospital’s Web site. She brought along the address for the hospital’s Web site, which she had found in her discharge instructions. Mary typed the address into the browser, and soon pages appeared on the computer screen presenting information by her hospital. Most of the information addressed logistics such as how to get to the hospital or where to park; however, several sets of keywords suggested that specific clinical advice might be available. Selecting the phrase "Help for New Parents" led Mary to discover a series of pages with helpful hints and explanations about taking care of infants.

One page described common sleep patterns for newborns, noting that smaller babies had more sleep difficulties than larger babies. This information confirmed Mary’s experience with her five-pound twins, neither of whom slept more than three hours at a time. Mary read with eagerness the suggestions regarding how to establish successful sleep routines with small infants. Another page addressed bathing infants. Mary was able to view a short movie demonstrating how to hold small infants while bathing them. She learned that many soaps caused skin irritation in newborns, and reviewed a list of recommended bathing products.

None of the material presented on the hospital’s Web site specifically addressed the needs of twins. Mary noticed the message "Search For Other Resources" just as she needed to leave the library to return home. She made a note to come back and check the new information.

Second Time: Follow-up and Exploration

One week later Mary returned to the library. By this time, she had begun to feel a bit more confident feeding and bathing the babies, but had found herself becoming increasingly weepy and sad over very small events. The babies seemed to be nursing successfully, but she continued to worry about her ability to feed them. Mary launched another session to her hospital’s Web site.

Mary located the appropriate section within the hospital’s Web site and looked through the listing until she found the message "Search For Other Resources." At this point Mary initiated a search. In response to a question requesting her topic, Mary typed in the words "newborn twins." She received a list of 27 different Web sites, all denoting "twins" somewhere in the title of the pages. She began reviewing the pages one at a time.

A commercial baby food company presented the first page that Mary discovered. Included on this page were pointers to descriptions about various formula preparations and strategies for promoting health nutrition. Pretty colors and pleasing pictures of well-nourished babies made the page seem interesting; however, with her commitment to breast-feeding, Mary determined that the content on the page was not relevant to her current situation.

A second page led Mary to the Web site developed and supported by a national voluntary society, Parents of Twins. This page differed from the ones Mary had already viewed. In addition to the usual helpful reference materials, this page had a public bulletin board area where interested persons could post or read messages. Mary focused her attention here.

Mary discovered several hundred messages with topics ranging from "Twins—double trouble" to "Help—where can I find cheap diapers." As Mary reviewed the messages, she noticed that parents of twins posted most of the messages. Many messages offered humorous or unnerving stories about child care experiences. Some posed specific questions, such as "Has anyone tried to nurse both babies at the same time?" to which other participants offered suggestions or advice. Occasionally a user identified himself or herself as a health care professional; their comments seemed different from those posted by the parents, offering clinical descriptions or recommendations to consult with clinical care providers.

Mary observed that the quality of advice in the postings ranged from very believable to almost ludicrous. Some postings clearly were advertisements in disguise, suggesting that a specific product may help with a given problem. Others offered helpful strategies, such as how to use pillows to position the babies comfortably during feedings. Occasionally a poster would direct readers to additional Web sites, which Mary herself sometimes subsequently viewed.

Mary noticed that a small set of individuals seemed to do most of the posting. Five or six users seemed to post several times a week, sometimes addressing their messages directly to other individuals or to the group as a whole. Phrases such as "Twin Moms Unite" or "Parents in the Know" created a sense of community that Mary knew she would like to join. Since no funds were needed to use the computer or to participate in the conversations in the bulletin board area, Mary resolved to return when she could and partake in this shared experience of parenting twins.

Maintenance: Linking IHC With Professional Care Services

When the twins were 4-weeks-old, Mary returned to the midwife for her post-partum checkup. In the conversation about her emerging role as mother, Mary described for the midwife the assistance and peer support she had discovered on the Web. Mary talked about the range of helpful (and sometimes unhelpful) advice provided by fellow users, and about the different Web sites that addressed child-care and parenting. Her midwife listened attentively, expressing support for Mary’s abilities to seek out and interpret health-related information for herself. She reviewed Mary’s strategies for determining the quality and relevance of the health information provided and identified some additional Web sites that she already reviewed and believed Mary would find helpful.

Key Considerations

• IHC applications are an important resource for consumers and public access to these resources is critical.
• Consumers need to be critical of the quality and relevance of online health information and support.
• The use of IHC applications can be more effective when their use is linked with regular care from a health professional.

Home | IHC | Publications | About SciPICH | Glossary | Feedback | Disclosure Statement | Site Map