Demonstrations of high-quality services have surpassed cost control as the primary task of health care. In this study we evaluated the relationship between several traditionally accepted standards of quality of children's mental health care and client mental health outcomes. Three customary performance guidelines (therapeutic relationship, satisfaction, and parent involvement) that are believed to be indicators of quality care were tested for their ability to account for improvement of mental health outcomes among adolescent clients. In addition, we examined service utilization to determine whether it represents an intermediate outcome of the treatment process.The results showed statistically significant, yet weak and inconsistent, relationships between these commonly accepted measures of quality and client outcomes among adolescents receiving outpatient treatment. We did not find service utilization to be a mediator. Limitations of the study and implications of the results are discussed.
References
1.
Abelson, R.P. (1984). A variance explanation paradox: When a little is a lot. Psychological Bulletin,97(1), 129-133.
2.
Achenbach, T.M. (1991). Manual for the Child Behavior Checklist and 1991 Profile. Burlington: University of Vermont, Department of Psychiatry.
3.
Achenbach, T.M., & Edelbrock, C. (1987). Manual for the Youth Self Report and Profile. Burlington: University of Vermont, Department of Psychiatry .
4.
Armbrister, P., & Kazdin, A.E. (1994). Attrition in child psychotherapy. In T. H. Ollendick & R. J. Prinz (Eds.), Advances in clinical child psychology (Vol. 16, pp. 81-108). New York: Plenum Press.
5.
Bickman, L. (1997). Resolving issues raised by the Fort Bragg evaluation: New directions for mental health services research. American Psychologist, 52(2), 562-565.
6.
Bickman, L.Guthrie, P.R., Foster, E.M., Lambert, E.W., Summerfelt, W.T., Breda, C.S., & Heflinger, C.A. (1995). Evaluating managed mental health services: The Fort Bragg experiment. New York: Plenum Press.
7.
Bickman, L., Heflinger, C.A., Pion, G., & Behar, L. (1992). Evaluation planning for an innovative children's mental health system. Clinical Psychology Review, 12, 853-865.
8.
Bickman, L., Karver, M.S., & Schut, L.J.A. (1997). Clinician reliability and accuracy in judging appropriate level of care. Journal of Consulting and Clinical Psychology, 65(3), 515-520.
9.
Bickman, L., Noser, K., & Summerfelt, W.T. (1999). Long-term effects of a system of care on children and adolescents. The Journal of Behavioral Health Services and Research, 26, 185-202.
10.
Bickman, L., & Peterson, K.A. (1990). Using program theory to describe and measure program quality. In L. Bickman (Ed.), Advances in program theory: New directions for program evaluation, No. 47 (pp. 61-72). San Francisco: Jossey-Bass.
11.
Bickman, L., & Salzer, M. (1997). Measuring quality in mental health services [Special issue]. Evaluation Review, 21(3).
12.
Bickman, L., Summerfelt, W.T., & Noser, K. (1997). Comparative outcomes of emotionally disturbed children and adolescents in a system of services and usual care. Psychiatric Services,48(12), 1543-1548.
13.
Bilbrey, J., & Bilbrey, P. (1995). Judging, trusting, and utilizing outcomes data: A survey of behavioral health care payers. Behavioral HealthCare Tomorrow,4(4), 62-65.
14.
Bordin, L. (1979). The generalizability of the psychoanalytic concept of the working alliance. Psychotherapy: Theory, Research, and Practice, 16, 252-260.
15.
Brannan, A.M., Sonnichsen, S., & Heflinger, C.A. (1996). Measuring satisfaction with children's mental health services: Validity and reliability of the satisfaction scales. Evaluation and Program Planning, 19(2), 131-141.
16.
Brook, R.H., & Williams, K.N. (1975). Quality of health care for the disadvantaged. Journal of Community Health, 1, 132-156.
17.
Burnam, M.A. (1996). Measuring outcomes of care for substance use and mental disorders. New Directions for Mental Health Services , 71, 3-17.
18.
Campbell, J. (1996). Toward collaborative mental health outcomes systems . New Directions for Mental Health Services, 71, 69-78.
19.
Cohen, J., & Cohen, P. (1983). Applied multiple regression/correlation analysis for the behavioral sciences. Hillsdale, NJ: Erlbaum.
20.
Deming, W.E. (1982). Quality, productivity, and competitive position . Cambridge, MA: Massachusetts Institute of Technology, Center for Advanced Engineering Study.
21.
Deming, W.E. (1986). Out of the crisis. Cambridge, MA: Massachusetts Institute of Technology, Center for Advanced Engineering Study.
22.
Donabedian, A. (1980). Explorations in quality assessment and monitoring: The definition of quality and approaches to its assessment. Ann Arbor, MI: Health Administration.
23.
Donabedian, A. (1986). Criteria and standards for quality assessment and monitoring. Quality Review Bulletin, 12, 99-108.
24.
Donabedian, A. (1995). The quality of care: How can it be assessed? In N. O. Graham (Ed.), Quality in health care: Theory, application, and evolution (pp. 32-46). Gaithersburg, MD: Aspen.
25.
Dror, Y. (1968). Public policymaking reexamined. San Francisco: Chandler.
26.
Freeman, M.A. (1995). Behavioral at-risk contracting in a changing healthcare environment. In G. L. Zieman (Ed.), The complete capitation handbook: How to design and implement at-risk contracts for behavioral healthcare. Tiburon, CA : CentraLink.
27.
Greenberg, L.S., & Pinsof, W.M. (1986). (Eds.). The psychotherapeutic process: A research handbook. New York: Guilford Press .
28.
Heflinger, C.A., Bickman, L., Northrup, D., & Sonnichsen, S. (1997). A theory-driven intervention and evaluation to explore family caregiver empowerment. Journal of Emotional and Behavioral Disorders, 5(3), 184-191.
29.
Hodges, K., Bickman, L., Ring-Kurtz, S., & Reiter, M. (1991). A multi-dimensional measure of level of functioning in children and adolescents. In A. Algarin & R. M. Friedman, (Eds.), The Fourth Annual Research Conference: A system of care for children's mental health; Building a research base. Tampa: University of South Florida, Florida Mental Health Institute, Research and Training Center for Children's Mental Health.
30.
Horvath, A.O., & Symonds, B.D. (1991). Relation between working alliance and outcome in psychotherapy: A meta-analysis. Journal of Counseling Psychology , 38(2), 139-149.
31.
Joint Commission on Accreditation of Hospitals. ( 1996). [Web page] Available: www.jcaho.org
32.
Lambert, W., Salzer, M., & Bickman, L. (1998). Clinical outcome, consumer satisfaction, and ad hoc rating of improvement. Journal of Consulting and Clinical Psychology, 66(2), 270-279.
33.
Markson, L.E., & Nash, D.B. (1995). Accountability and quality in health care: The new responsibility. Oakbrook Terrace, IL: The Joint Commission on Accreditation of Healthcare Organizations .
34.
Martin, G., & Allison, S. (1993). Therapeutic alliance: A view constructed by a family therapy team. Australian and New Zealand Journal of Family Therapy, 14(4), 205-214.
35.
McGlynn, E.A., Halfon, N., & Leibowitz, A. (1995). Assessing the quality of care for children: Prospects under health reform. Archives of Pediatric and Adolescent Medicine, 149, 359-368.
36.
McGlynn, E.A., Norquist, G.S., Wells, K.B., Sullivan, G., & Liberman, R.P. (1988). Quality-of-care research in mental health: Responding to the challenge. Inquiry, 25, 157-170.
37.
Nash, D. (1995, October). Quality and utilization management: The cutting edge for 2000. Presentation at the American Managed Care and Review Associations Quality and Utilization Management Conference , Nashville, TN.
38.
National Committee for Quality Assurance. ( 1996, April). Draft Accreditation Standards for Managed Behavioral Healthcare Organizations.
39.
Pinsof, W. (1994). An integrative systems perspective on the therapeutic alliance: Theoretical, clinical, and research implications. In A. O. Horvath & L. S. Greenberg (Eds.), The working alliance: Theory, research, and practice (pp. 173-195). New York: Wiley .
40.
Rust, R.T., & Oliver, R.L. (1994). Service quality: Insights and managerial implications from the frontier. In R. T. Rust & R. L. Oliver (Eds.), Service quality: New directions in theory and practice (pp. 1-19). Thousand Oaks, CA: Sage.
41.
Salzer, M.S., & Bickman, L. (1996). Delivering effective children's services in the community: Reconsidering the benefits of systems of care. Applied and Preventive Psychology, 6(1), 1-13.
42.
Salzer, M.S., Nixon, C.T., Schut, J.A., Karver, M.S., & Bickman, L. (1997). Validating quality indicators: Quality as relationship between structure, process, and outcome. Evaluation Review , 21(3), 292-309.
43.
Science Applications International Corporation. ( 1995). Mental health quality monitoring screens and utilization review criteria. Beaverton, OR: Author.
44.
Shaffer, D., Gould, M.S., Brasic, J., Ambrosini, P., Fisher, P., Bird, H., & Aluwhalia, S. (1983). A children's global assessment scale (GCAS). Archives of General Psychiatry,40, 1228-1231.
45.
Sheps, M.C. (1955). Approaches to the quality of hospital care. Public Health Reports, 9, 877-886.
46.
Shirk, S.R., & Saiz, C.C. (1992). Clinical, empirical, and developmental perspectives on the therapeutic relationship in child psychotherapy. Development and Psychopathology, 4, 713-728.
47.
Smith, G.R. (1996). State of the science of mental health and substance abuse patient outcomes assessment. New Directions for Mental Health Services, 71, 59-78.
48.
Stroul, B.A., & Friedman, R.M. (1986). A system of care for severely emotionally disturbed youth. Washington, DC: CASSP Technical Assistance Center.
49.
Weisz, J.R., & Weiss, B. (1990). The impact of methodological factors on child psychotherapy outcome research: A meta-analysis for researchers. Journal of Abnormal Child Psychology, 18, 639-670.
50.
Wells, K.B., & Brook, R.H. (1989). The quality of mental health services: Past, present, and future. In C. A. Taube, D. Mechanic, & A. A. Hohmann (Eds.), The future of mental health services research (pp. 203-224). Washington, DC: U.S. Government Printing Office.
51.
Zieman, G.L. (1995). The complete capitation handbook: How to design and implement at-risk contracts for behavioral healthcare. Tiburon, CA: CentraLink.
