Research in Action Issue 16

Programs and Tools to Improve the Quality of Mental Health Services

The Agency for Healthcare Research and Quality (AHRQ) has a broad portfolio of mental health research. This report focuses specifically on AHRQ-funded research that has led to the development of programs, methods, and tools for evaluating and improving the quality of mental health services and improving the education of mental health professionals.

Select for print version (PDF File, 275 KB). PDF Help.

Contents

Introduction / Background / Impact of AHRQ Research / Ongoing Research / Conclusion / For More Information / References

By Barbara L. Kass-Bartelmes, M.P.H., C.H.E.S.

Introduction

Americans of all ages suffer from mental illness, and many do not receive the care they need. For example, about 44 million people 18 and over suffer from mental disorders, which can result in disability and death.¹ Major depressive disorder is the leading cause of disability among adults, and suicide ranks as the second leading cause of death among U.S. adolescents ages 10-17.² In any given year, over 2 million Americans suffer the symptoms of schizophrenia, but more than half do not receive appropriate treatment.^3,4 Furthermore, little is known about the quality of mental health programs, how well they work, or how to measure their effectiveness.⁵ Mental health professionals have expressed concern that education and training programs have not kept up with changes in health and health care over the past 15 years.⁶ As a result, those receiving care may not be benefiting from recent advances.

AHRQ-funded research has led to the development of programs, methods, and tools for evaluating and improving the quality of mental health services and improving the education of mental health professionals. AHRQ's mental health services research covers areas such as:

• Depression—The Partners In Care program increases treatment for depression and improves outcomes.
• Schizophrenia—Two toolkits were developed to improve treatment for schizophrenia.
• Adolescents at Risk—New tools can be used to screen adolescents at risk for suicide and to evaluate school mental health programs.
• Quality Assessment—Tools have been developed that promote quality improvement programs at managed behavioral health care organizations.
• Professional Education—Researchers have identified solutions that can improve education and training for mental health care professionals.

Return to Contents

Background

Research in mental health services has long indicated the need to improve care for people who have or are at risk for developing mental health disorders.

Depression

Between 1987 and 1997, the rate of outpatient treatment for depression tripled.⁷ However, depression is difficult to treat even in diagnosed patients because depressed patients are less likely to complain of their problem than patients diagnosed with physical ailments. In fact, depressed patients may not even know that they are depressed or that they need treatment.⁸ Furthermore, treatment itself requires careful management for up to 1 year.⁸ Research shows that patients often do not receive the best quality of care for depression.⁹

For example, research sponsored by AHRQ has found that:

• About 20 percent of patients in primary care settings have depression and 6 percent require treatment, yet it often is not detected.^10,11
• Less than half of patients who were diagnosed as depressed actually received treatment.¹²
• Only 29 to 43 percent of patients with symptoms of depression received counseling when they were initially screened for depression, and only 35 to 42 percent had used the correct dosage of antidepressant medication.⁹

Schizophrenia

Schizophrenia is a chronic, disabling medical illness, and those who suffer from it have high rates of under- and unemployment, often resulting in poverty and reliance on publicly funded health care, such as Medicaid.⁴ No specific cause of schizophrenia has been identified4 nor is there any known cure for this disease.³ About 10 percent of people diagnosed with schizophrenia eventually commit suicide.³ Even though effective medications and therapies exist to treat people with schizophrenia, the Schizophrenia Patient Outcomes Research Team (PORT) funded by AHRQ found that:

• Less than half of patients suffering from schizophrenia received proper doses of antipsychotic medications or appropriate psychosocial interventions.
• Younger schizophrenic patients were more likely than people ages 65 and over to be offered psychotherapy and vocational rehabilitation.
• African-American patients were less likely to report being diagnosed with schizophrenia and, once diagnosed, more likely to be on higher doses of antipsychotic medications. However, they were less likely to receive individual therapy or to be prescribed an antidepressant medication when depressed.^4,13,14

Adolescents at Risk

The suicide rate in young people has increased dramatically since 1980; among adolescents ages 10-14 years, the rate increased 100 percent.¹⁵ In 2000, over 1,900 adolescents ages 10-19 years committed suicide.¹⁶ It is estimated that, for every young person who completes a suicide, 8 to 25 others have attempted suicide.¹⁶ Research indicates that it is imperative to identify and treat both suicidal thoughts and depression among adolescents.

Findings from AHRQ-funded studies include the following:

• One study showed that about 20 percent of high school students admitted to having thoughts of suicide and 10 percent had attempted suicide.¹⁷
• Forty-seven percent of physicians in another study stated that at least one of their adolescent patients attempted suicide in the prior year. However, only 23 percent of these physicians routinely screened adolescent patients for suicide risk factors.¹⁸

Depression increases the risk of adolescent deaths from suicide.^19-21 But even if an adolescent is diagnosed with depression or found to be at risk for suicide, access to mental health services can be difficult. For example:

• Attempts to reduce health costs have resulted in decreased access to mental health services for children.²⁰
• Expanded school mental health programs that offer assessment, treatment, case management, referral services, and preventive services can help fill the gap, but they need evaluation to demonstrate their value and effectiveness to help ensure funding and continued growth.^22-24

Quality Assessment

Insurance plans often provide mental health services through managed behavioral health care organizations (MBHOs).⁵ Consumer opinions of the services they receive from MBHOs are used to evaluate the quality of these services.²⁵ However, research indicates that:

• Evaluation instruments that have not been standardized or validated are often used.²⁵
• The National Committee for Quality Assurance (NCQA) accreditation process requires that MBHOs show that they collect, analyze, and use consumer opinions to develop quality improvement programs, but few instruments have been developed to collect this information.⁵

Professional Education

Education and training of behavioral health practitioners have not kept up with changes in health care.²⁶ As a result, students are not trained to work under the conditions they will face in actual practice, such as using evidence-based guidelines and working in managed care practices.⁶ Updating graduate school curricula competes with other demands that are made on faculty, such as competition for research funds.²⁷ In addition, continuing education requirements for licensed professionals are not standardized.²⁸ As a result:

• Community-based mental health services are often not available to people with mental illness, partially because mental health professionals do not know about these services.^26,29
• New practices based on evidence are not implemented.²⁶
• Students do not receive training in new methods and treatment and get little experience working in outpatient settings, substance abuse treatment, community services, or primary care settings.²⁷

Return to Contents

Impact of AHRQ Research

Collaborative Care Program Improves Care and Outcomes for Depression

Under the AHRQ-funded Partners in Care (PIC) collaborative-care program conducted by the PORT II for depression, the rates of antidepressant medication use and psychotherapy increased among patients who were diagnosed with depression.^10,30

The PIC program conducted a randomized trial comparing quality improvement programs in clinics for medications (QI-meds) and for psychotherapy (QI-therapy).⁸ Two groups of clinics were randomly assigned to either the QI-meds or QI-therapy group. A third group of clinics served as a control by offering usual care for depression.⁸

Patients who enrolled in the PIC intervention were encouraged by nurses and providers to choose which treatment they preferred—either the QI-meds or the QI-therapy, or no treatment.^10,30,31

• The QI-meds intervention provided patients with nurse followup assessments and support for medication compliance for 6 or 12 months.^8,30,32 QI-meds patients could still receive psychotherapy, but they had to pay the normal insurance copayment.^8,30,32
• The QI-therapy intervention provided patients with 12-16 sessions of cognitive behavioral therapy on a reduced copayment basis.^8,30 QI-therapy patients could still take antidepressant medications, but they did not receive the nurse followup.^8,30
• Clinics that were providing usual care received depression practice guidelines.³⁰

Local expert leaders in primary care, nursing, and mental health specialties were selected to implement the PIC interventions and received 2-day training on implementation.^8,10,31,33 Primary care expert leaders then hired depression nurse specialists,who received a 1-day training session.⁸ The depression nurse specialists initially assessed patients for both the QI-meds and QI-therapy programs and handled case management for the QI-meds only patients.⁸ Psychotherapists were also hired and trained in short-term psychotherapy for depression in patients who chose the QI-therapy only program.⁸

When compared to patients who received usual care, patients in the PIC program were twice as likely to begin treatment within the first 6 months of being diagnosed, were less likely to be depressed during the first year, were more likely to remain employed after 1 year, and were more likely to state that they had good quality of life.^10,33 Results showed that:

• Rates of counseling and appropriate use of antidepressant medications were 30-40 percent higher during the first 6 months and 20-30 percent higher after 1 year for intervention patients than for patients who received usual care (Figures 1 and 2).³³
• At 6 and 12 months, 7-10 percent fewer intervention patients had depression compared to usual care patients (Figure 3).³³
• Of patients who were employed at the beginning of the intervention, 89.7 percent of intervention patients and 84.7 percent of usual care patients were still employed after 1 year.³³
• Of patients who were unemployed at the beginning of the intervention, 16.4 percent of intervention patients and 11.4 percent of usual care patients were working at 6 months; at 1 year, 17-18 percent of all patients had started working.³³
• After 6 months, 72 percent of patients who received appropriate treatment (i.e., counseling or medication) were employed, compared with 53 percent of patients who did not receive appropriate treatment.³²
• At the end of 2 years, patients who chose the QI-therapy intervention had improved emotional well-being compared to patients who received usual care.³⁴

The PIC program was also successful in treating depressed patients who had additional medical conditions such as back problems, arthritis, hypertension, asthma, diabetes, and heart disease.³⁵ After 6 and 12 months, these intervention patients were less likely to report depressive disorders.³⁵ In addition, after 18 months, the clinicians who participated in the intervention training programs demonstrated more knowledge about assessing and treating depression than clinicians who did not participate in the intervention training.³¹

More information on the Partners in Care program can be found at www.rand.org/publications/RB/RB4528/. PIC materials, including user's guide, clinician's guide, and implementation guides, are available through the PIC Web site at www.rand.org/health/partners.care/portweb/.³⁶

Toolkits to Improve Schizophrenia Care Are Based on Treatment Guidelines

Based on a complete and thorough review of studies conducted on the outcomes of treatment for schizophrenia, AHRQ's PORT developed evidence-based recommendations, published in 1995, to improve the quality and cost effectiveness of care for patients with schizophrenia.³⁷ Specifically, the recommendations address interventions with:

• Antipsychotic medications.
• Drug therapy for anxiety, depression, and aggression/hostility.
• Electroconvulsive therapy.
• Psychological interventions.
• Family interventions.
• Vocational rehabilitation.
• Assertive community treatment and case management.³⁷

They have been used in 9 States to improve statewide treatment approaches and are the basis of two toolkits for treatment of schizophrenia developed by the Human Services Research Institute (HSRI) and the Department of Health and Human Services of the State of New Hampshire.

Using AHRQ's evidence-based recommendations on schizophrenia, HSRI, in Cambridge, MA, developed a toolkit with two primary purposes. First, the toolkit measures how well an organization implements a set of practice recommendations. Second, it illustrates how evaluation evidence can be used to develop recommendations for specific disorders. Specifically, the HSRI toolkit shows how designing a measurement strategy based on recommendations can help promote implementation of those recommendations.

While the toolkit is not a guide to the treatment of schizophrenia (primarily because the recommendations will be updated), the process it describes illustrates how to build a recommendation-based quality assessment system. The PORT tools for assessing care (the PORT Mental Health Survey and the Schizophrenia PORT Inpatient and Outpatient Record Review forms) are included in the toolkit. The resulting performance measures and evaluation studies should contribute to quality improvement, enhancing outcomes, and preventing adverse events. The toolkit is available through HSRI's Web site at www.hsri.org.³⁸

The New Hampshire Department of Health and Human Services used the Schizophrenia PORT recommendations for key policy initiatives. The New Hampshire Division of Public Health used the PORT findings to highlight racial disparities in the provision of care, directly influencing New Hampshire's Healthy People 2010 plan, which is aimed at eliminating health care disparities. Partnering with the Dartmouth College Psychiatric Research Center, the Division of Mental Health restructured care protocol for patients with schizophrenia based on the PORT findings. The restructuring effort included the development of evidence-based toolkits for retraining community mental health staff.

A Screening Tool Helps Identify Children and Adolescents at Risk for Suicide

A Risk of Suicide Questionnaire (RSQ) that can be used during emergency room visits takes 2 minutes to detect children and adolescents at risk for suicide.^39,40 An AHRQ-funded study showed that these 4 questions identified 98 percent of children at risk:

• Are you here because you tried to hurt yourself?
• In the past week, have you been having thoughts about killing yourself?
• Have you ever tried to hurt yourself in the past other than this time?
• Has something very stressful happened to you in the past few weeks?³⁹

The RSQ was the result of a clinical practice guideline (CPG) program conducted at Children's Hospital Boston.⁴⁰ The program provided training to emergency room nurses on psychiatric issues such as how to perform patient searches, de-escalating techniques, and risk factors for suicide.⁴⁰ At the beginning of the training, emergency room nurses at Children's Hospital stated that they were uncomfortable dealing with children who had psychiatric problems or asking parents or children about thoughts of suicide.⁴⁰ In addition, 94.4 percent of nurses stated that suicidal patients were the most difficult patients to treat.⁴⁰

Two years into the CPG program, nurses reported a significant increase in their confidence and less stress when dealing with psychiatric patients.^39,40 They also stated that they preferred using the screening tool to the previous method of simply judging when to ask about suicidal behavior.^39,40 Parents also expressed relief that emergency room clinicians were asking about suicide.³⁹

Evaluation Shows That a School Mental Health Program Lowers Costs

AHRQ-supported studies helped to develop a systematic approach to evaluating expanded school mental health (ESMH) programs in middle and high school^22,23 and discovered that the costs of ESMH programs were low compared to the costs of programs in the community or private sectors.²⁴ A cost-of-care evaluation conducted for one school mental health program found that clinician services cost less than $50 per hour—much less than private services, which were estimated to cost $100 to $120 per hour.²⁴

A typical program provides:

• Assessment.
• Individual, group, and family counseling.
• Crisis intervention.
• Prevention and case management activities.
• Teacher consultation.
• Collaboration with school staff and the community.²⁴

According to researchers funded by AHRQ, an ESMH evaluation plan should systematically measure the outcomes of the program's goals, such as identifying emotional, behavioral, and academic problems early and improving school attendance. The following steps are recommended in conducting ESMH program evaluations:

• Focus the evaluation on both quantitative measures (such as changes in behavioral problems) and qualitative measures (such as satisfaction ratings) obtained from adolescents, parents, teachers, school personnel, etc. These measures should be associated with a positive outcome (for example, improved grades or attendance).
• Ensure cultural competence, taking into consideration program philosophy, staff ethnicity, cultural competence training, and cultural values of the target population.
• Obtain support from the school and community by selecting goals that are important to stakeholders.
• Use outcome indicators that address life stressors and risk factors, protective or resilience factors, emotional and behavioral problems, and life functioning.
• Use specific measurement strategies, which can include self-report, reports by parents and teachers, and clinician ratings.
• Measure the cost effectiveness of the ESMH program in relation to traditional mental health programs.²²

An advisory board of stakeholders (families, school staff and teachers, mental health professionals, community leaders, the community health/mental health system, and funders) should review and revise the evaluation plan and provide feedback on the results. This feedback should be used for program improvement.^22,23

Consumer Ratings Can Improve Behavioral Health Services

An AHRQ-funded study that used the Consumer Assessment of Behavioral Health Services (CABHS) prompted quality improvement efforts in several managed behavioral health care organizations.⁵ CABHS is based on CAHPS®^a and was developed to collect ratings from consumers about services received from MBHOs.²⁵ Consumer ratings of behavioral health care services can help to evaluate and identify ways to improve behavioral health insurance plans.⁵

The 56 questions in the CABHS survey ask consumers about their experience in finding a clinician, getting approval for treatment, getting prompt counseling or treatment, communication with clinicians, and customer service, as well as staff, paperwork burden, written materials, and how much the treatment helped patients.⁵ CABHS also collects information from clinicians about medications, treatment options, and patient rights.⁵ Examples of questions and responses from consumers are shown in Table 1.

The survey assessed patient satisfaction in 5 commercial and 5 public assistance plans.⁵ Based on the results of these surveys, 3 MBHOs implemented quality improvement efforts.

• Patients at one MBHO reported problems in getting help from customer service, telephone answer response time, finding a clinician, getting care while waiting for treatment approval from the plan, and filling out paperwork. As a result, the MBHO added more staff and trained its customer service personnel to answer calls for information, check patient eligibility, authorize an initial treatment session, help patients find a clinician, and ensure that patients receive appropriate and timely care. The MBHO also introduced centralized scheduling and increased the number of clinician hours to reduce waiting times for new patient evaluations.⁵
• A second MBHO found that 13-30 percent of its members were not given adequate information about patient rights. The MBHO implemented projects to improve dissemination of information about patient rights, including consulting with members and clinicians, posting patient rights in clinics, and interviewing providers to find out if patient rights information is being provided.⁵
• A third MBHO developed educational materials for its members and clinicians to improve the patients' ability to get timely appointments.⁵

Based on the CABHS data collected in this AHRQ-funded study, along with data from a previous study using the Mental Health Statistics Improvement Program (MHSIP), researchers developed the Experience of Care and Health Outcomes (ECHO™).⁵

The ECHO™ 3.0 has two surveys—one for MBHOs and one for health plans.^5,41 Both surveys ask the same questions, but the health plan survey includes questions about administrative services, such as filling out paperwork and finding information in written materials.⁴² The surveys can be downloaded for free from the Web site: www.hcp.med.harvard.edu/echo/home.html. Consumers, clinicians, MBHOs, health care plans, purchasers, States, and Federal agencies can collect consumers' ratings of their behavioral health treatment, including mental health and alcohol, drug, and other substance abuse services.⁴² ECHO™ assesses specific aspects of care, including getting care quickly, communication with clinicians, information provided by clinicians on medication side effects, family involvement in care, information about self-help groups and treatment, cultural competence, perceived improvement in functioning, patient rights, and health plan or MBHO administrative services.⁴²

ECHO™ has been adopted as a CAHPS® instrument.⁴³ In addition, NCQA adopted ECHO™ 1.1H, which excludes 13 questions from the original MBHO version,⁴¹ and made it a part of the NCQA Quality Assurance Plan HEDIS® (the Health Plan Employer Data and Information Set) 2003 Survey Measures.⁴⁴ ECHO™ 1.1H was designed to target quality assessment by MBHOs and assesses aspects of care such as overall satisfaction with counseling and treatment, getting treatment quickly, communication with clinicians, access to treatment and information from the MBHO, office wait times, ability to refuse treatment, and receipt of information about treatment options, medication side effects, managing conditions, and patient rights.⁴⁴

^a CAHPS® (Consumer Assessment of Health Plans) is a family of rigorously tested and standardized questionnaires and reporting formats that can be used to collect and report meaningful and reliable information about the experiences of consumers with a variety of health services. For more information, see AHRQ's Web site at www.ahrq.gov/qual/cahpsix.htm.

A Conference Provides Solutions To Improve Provider Training and Continuing Education

AHRQ cosponsored the National Conference on Behavioral Health Workforce Education in Annapolis, MD September 10-11, 2001. The proceedings from this conference resulted in a special double-issue volume of Administration and Policy in Mental Health in May 2002. The proceedings present problems and solutions to educating behavioral health professionals, including:

• Clarifying the issues of the behavioral health care system and the need for change.^6,45,46
• Educational best practices for practitioners, patients, consumers, and families.^26,47
• Recommendations to enhance graduate education so that practitioners develop the competencies necessary for current practice.²⁷
• Methods to improve continuing education practices.²⁸

Return to Contents

Ongoing Research

AHRQ continues to fund and sponsor many research projects involving mental health services. For a comprehensive list and summary of past and ongoing projects, please see the Compendium of AHRQ Research Related to Mental Health (AHRQ Pub. No. 03-0001). The HTML version can be found on AHRQ's Web site at www.ahrq.gov/research/mentalcomp/. The PDF version can be downloaded at www.ahrq.gov/research/mentalcomp/mentalcompend.pdf (297 KB; PDF Help). The PDF version is indexed by topic as well as by principal investigators and authors.

Return to Contents

Conclusion

AHRQ's wide range of mental health research has produced programs, methods, and tools that can improve the quality of mental health services. The PIC program allows people suffering from depression to collaborate in their care with their providers and has been shown to improve outcomes. Providers who treat schizophrenia patients have toolkits that use evidence-based treatment recommendations to assess the care and treatment they provide. AHRQ research supported the development of a simple tool to screen adolescents at risk for suicide. Research is also directed toward finding ways to evaluate and promote quality improvement programs for mental health services in school-based programs and MBHO's. In addition, AHRQ's research helps identify solutions that can improve education and training for mental health care professionals.

Return to Contents

For More Information

For further information on AHRQ's mental health research, contact:

Charlotte Mullican, B.S.W., M.P.H.
E-mail: CMullica@ahrq.gov
Phone: (301) 427-1495

Return to Contents

References

1. National Institute of Mental Health. The numbers count. Mental disorders in America. NIH Pub No. 01-4584. Web site: www.nimh.nih.gov/publicat/numbers.cfm. Accessed Apr 7, 2003.

2. Centers for Disease Control and Prevention. Web-based Injury Statistics Query and Reporting System (WISQARS™) [online database]. National Center for Injury Prevention and Control; 2002. Web site: www.cdc.gov/ncipc/wisqars. Accessed Apr 8, 2003.

3. National Institute of Mental Health. Schizophrenia. Web site: www.nimh.nih.gov/publicat/schizoph.cfm. Accessed Apr 2, 2003.

*4. Lehman AL. Quality of care in mental health: the case of schizophrenia. Health Aff 1999; 18(5):52-65.

*5. Shaul JA, Eisen SV, Stringfellow VL, et al. Use of consumer ratings for quality improvement in behavioral health insurance plans. Jt Comm J Qual Improv 2001; 27(4):216-29.

*6. Hoge MA. The training gap: an acute crisis in behavioral health education. Adm Policy Ment Health 2002; 29(4/5):305-17.

*7. Olfson M, Marcus SC, Druss B, et al. National trends in the outpatient treatment of depression. JAMA 2002; 287(2):203-9.

*8. Rubenstein LV, Jackson-Triche M, Unutzer J, et al. Evidence-based care for depression in managed primary care practices. Health Aff 1999; 18(5):89-105.

*9. Wells KB, Schoenbaum M, Unutzer J, et al. Quality of care for primary care patients with depression in managed care. Arch Fam Med 1999; 8:529-36.

10. RAND Health. Partners In Care. Hope for those who struggle with hope. Research Highlights. Web site: www.rand.org/publications/RB/RB4528. Accessed Oct 23, 2002.

*11. Garrard J, Rolnick SJ, Nitz NM, et al. Clinical detection of depression among community-based elderly people with self-reported symptoms of depression. J Gerontol 1998; 53A(2):M92-M101.

*12. Callahan CM, Dittus RS, Tierney WM. Primary care physicians' medical decision making for late-life depression. J Gen Intern Med 1996; 11:218-25.

*13. Dixon L, Green-Paden L, Delahanty J, et al. Variables associated with disparities in treatment of patients with schizophrenia and comorbid mood and anxiety disorder. Psychiatr Serv 2001; 52(9):1216-22.

*14. Dixon L, Lyles A, Smith C, et al. Use and costs of ambulatory care services among Medicare enrollees with schizophrenia. Psychiatr Serv 2001; 52(6):786-92.

15. Public Health Service [U.S.]. The Surgeon General's call to action to prevent suicide, 1999. Web site: www.surgeongeneral.gov/library/calltoaction/default.htm. Accessed Apr 15, 2003.

16. National Institute of Mental Health. Suicide facts. 2003. Web site: www.nimh.nih.gov/research/suifact.htm. Accessed Apr 15, 2003.

17. Hacker K, Drainoni ML. Mental health and illness in Boston's children and adolescents: one city's experience and its implications for mental health policy makers. Public Health Rep 2001; 116(4):317-26.

18. Frankenfield DL, Keyl PM, Gielen A, et al. Adolescent patients—healthy or hurting? Missed opportunities to screen for suicide risk in the primary care setting. Arch Pediatr Adolesc Med 2000; 154(2):162-8.

19. Nock MK, Kazdin AE. Examination of affective, cognitive, and behavioral factors and suicide-related outcomes in children and young adolescents. J Clin Child Adolesc Psychol 2002; 31(1):48-58.

20. Sampson SM, Mrazek DA. Depression in adolescence. Curr Opin Pediatr 2001; 13:586-90.

21. Catallozzi M, Pletcher JR, Schwarz DF. Prevention of suicide in adolescents. Curr Opin Pediatr 2001; 13:417-22.

*22. Weist MD, Nabors LA, Myers CP, et al. Evaluation of expanded school mental health programs. Community Ment Health J 2000; 36(4):395-411.

*23. Nabors LA, Weist MD, Reynolds MW. Overcoming challenges in outcome evaluations of school mental health programs. J Sch Health 2000; 70(5):206-9.

*24. Nabors LA, Leff SS, Mettrick JE. Assessing the costs of school-based mental health services. J Sch Health 2001; 71(5)199-200.

*25. Eisen SV, Shaul JA, Clarridge B, et al. Development of a consumer survey for behavioral health services. Psychiatr Serv 1999; 50(6):793-8.

*26. Stuart GW, Burland J, Ganju V, et al. Educational best practices. Adm Policy Ment Health 2002; 29(4/5):325-33.

*27. Hoge MA, Jacobs S, Belitsky R, et al. Graduate education and training for contemporary behavioral health practice. Adm Policy Ment Health 2002; 29(4/5):335-57.

*28. Daniels AS, Walter DA. Current issues in continuing education for contemporary behavioral health practice. Adm Policy Ment Health 2002; 29(4/5):359-76.

29. Department of Health and Human Services [U.S.]. Mental health: a report of the Surgeon General—executive summary . Rockville, MD: Substance Abuse and Mental Health Services Administration, Center for Mental Health Services, National Institutes of Health, National Institute of Mental Health. 1999. Web site: www.surgeongeneral.gov/library/mentalhealth/summary.html. Accessed Apr 9, 2003.

*30. Dwight-Johnson M, Unutzer J, Sherbourne C, et al. Can quality improvement programs for depression in primary care address patient preferences for treatment? Med Care 2001 Sep; 39(9):934-44.

*31. Meredith LS, Jackson-Triche M, Duan N, et al. Quality improvement for depression enhances long-term treatment knowledge for primary care clinicians. J Gen Intern Med 2000 Dec; 15(12):868-77.

*32. Schoenbaum M, Unutzer J, McCaffrey D, et al. The effects of primary care depression treatment on patients' clinical status and employment. Health Serv Res 2002; 37(5):1145-58.

*33. Wells KB, Sherbourne C, Shoenbaum M, et al. Impact of disseminating quality improvement programs for depression in managed primary care. JAMA 2000; 283(2):212-20.

*34. Sherbourne CD, Wells KB, Duan N, et al. Long-term effectiveness of disseminating quality improvement for depression in primary care. Arch Gen Psychiatry 2001; 58:696-703.

*35. Koike AK, Unutzer J, Wells KB. Improving the care for depression in patients with comorbid medical illness. Am J Psychiatry 2002; 159:1738-45.

*36. Vaiana M. RAND. E-mail correspondence to Barbara Kass-Bartelmes, Agency for Healthcare Research and Quality. July 17, 2003.

*37. Agency for Healthcare Quality and Research. The Schizophrenia Patient Outcomes Research Team (PORT) treatment recommendations. Rockville, MD. 1998. Web site: www.ahrq.gov/clinic/schzrec.htm. Accessed Sept 9, 2003.

*38. Lehman A, Steinwachs D. Measuring conformance to treatment guidelines: the example of the schizophrenia PORT. Aug 2001. Pub. Code PN-46. Schizophrenia Patient Outcomes Research Team, University of Maryland School of Medicine. Web site: www.hsri.org. Accessed Apr 2, 2003.

*39. Horowitz LM, Wang PS, Koocher GP, et al. Detecting suicide risk in a pediatric emergency department: development of a brief screening tool. Pediatrics 2001; 107(5):1133-7.

*40. O'Neill K, Horowitz LM, Smith MF, et al. Recognizing suicide risk in a pediatric emergency department: a change in nursing care. Pediatr Emerg Care 2001; 17(4):306-9.

41. Shaul J. Department of Health Care Policy, Harvard Medical School. E-mail communication to: Barbara Kass-Bartelmes. Mar 18, 2003.

42. Department of Health Care Policy, Harvard Medical School. ECHO™ Experience of Care and Health Outcomes Survey. Web site: http://www.hcp.med.harvard.edu/echo/home.html. Accessed June 18, 2004.

43. Darby C. Agency for Healthcare Research and Quality. E-mail communication to: Barbara Kass-Bartelmes. Mar 18, 2003.

44. National Committee for Quality Assurance. NCQA Quality Assurance Plan. HEDIS® 2003 Survey Measures. Web site: http://www.ncqa.org/programs/hedis/2003qap.doc. Accessed Mar 18, 2003.

*45. Huey LY. Problems in behavioral health care: leap-frogging the status quo. Adm Policy Ment Health 2002; 29(4/5):403-19.

*46. O'Connell MJ, Gill D, Artar A, et al. Student voices: perspectives on behavioral health education. Adm Policy Ment Health 2002; 29(4/5):421-34.

*47. Morris JA, Stuart GW. Training and education needs of consumers, families, and front-line staff in behavioral health practice. Adm Policy Ment Health 2002; 29(4/5):377-402.

*AHRQ-funded/sponsored research.

Return to Contents

AHRQ Publication No. 04-0061
Current as of July 2004

Internet Citation:

Kass-Bartelmes, BL. Programs and Tools to Improve the Quality of Mental Health Services. Research in Action Issue 16. AHRQ Publication No. 04-0061, July 2004. Agency for Healthcare Research and Quality, Rockville, MD. http://www.ahrq.gov/qual/menttoolria/menttoolria.htm

Return to Mental Health
Research Findings
AHRQ Home Page
Department of Health and Human Services