ALERT! Warning: your browser isn't supported. Please install a modern one, like Firefox, Opera, Safari, Chrome or the latest Internet Explorer. Thank you!
Log inLog in to ACORN Wiki. | RegisterSign on to ACORN Wiki.

You are here: COMMONS » OutcomesInformedCare

Outcomes Informed Care

  • Jeb Brown, PhD; Center for Clinical Informatics
  • Takuya Minami, PhD; University of Massachusetts-Boston

Outcomes informed care is known by a variety of terms, such a routine outcomes measurement, feedback informed treatment and outcomes management.

Outcomes are measured on all patients receiving treatment by means of patient self-report questionnaires administered in frequent intervals (preferably every session) during the course of treatment. The practice aspect of outcomes informed care is from the perspective of the therapists. Therapists are provided with the means to monitor and manage outcomes at the time of service, thus offering opportunities to improve outcomes for their patients. The system aspect of outcomes informed care is from the perspective of the larger systems, as in the case of payers identifying high performing providers. As the organization is informed about their patient outcomes at system-wide levels, outcomes-informed care offers the tools to empirically validate their treatment across the entire system of care.

How does outcomes informed care care lead to improved outcomes?

First and foremost, outcomes informed care is built on the premise that patients deserve the best care possible. As such, whether the treatment is “working” for them or not should be assessed by their actual treatment outcomes. By focusing directly on the most important factor, outcomes informed care attempts to directly impact the quality of service, both in individual practice and as a system.

Unlike traditional psychotherapy research (i.e., clinical trials), outcomes informed care is concerned with the realities of non-research clinical settings. Well-designed clinical trials investigate the efficacy of various treatments by carefully controlling for characteristics of patients included in the treatment, the matter in which the treatment is delivered, and the dose/frequency of treatment (1,2). Therefore, although it may seem difficult to dispute that these treatments studied in clinical trials may be among the best of choices under these restrictive conditions, it has yet to be demonstrated that these treatments yield the best outcomes under real-world conditions. By contrast, outcomes informed care directly assesses the effectiveness of various treatments that are currently delivered by therapists working with a diverse mix of patients that are seen in routine delivery of behavioral healthcare treatment services. Thus, outcomes informed care is not concerned with whether or not the therapists are using “brand-name” therapies that are studied in the clinical trials; what it is concerned with is whether or not the provided treatments actually lead to clients’ improvements. However, this in no way implies that outcomes informed care is at odds with empirical research. Rather, outcomes informed care is firmly grounded on research evidence.

It is well known that individual patients vary widely in their response to treatment. Both clinical trials and large studies of treatment as usual in the community find that 5-15% of patients end treatment with significantly increased symptoms than when they began. A growing body of published studies demonstrates that therapists who have access to self-report treatment outcomes provided by patients are able to proactively identify patients who are most at risk for treatment failure (3-10). Early identification of these at-risk cases permits therapists to proactively work to keep the patients engaged in treatment while evaluating future treatment options. A number of published studies have demonstrated that routine use of patients’ self-report treatment outcome questionnaires results in both longer treatment length and significantly improved outcomes for at-risk patients as compared to similar patients treated by therapists who are not provided with patients’ self-reported outcomes (11-15). Therefore, outcome-informed care leads to improved patient outcomes by reducing treatment failures.

What is the difference between outcomes informed care and evidence-based treatments?

Evidence-base treatmennts refers to the guideline for clinical practice that rests on the assumption that outcomes are best improved by choosing the most effective treatment for a specific diagnosis as studied in well-controlled clinical trials 1. Therefore, evidence-based treatment dictates that treatment methods that have been extensively studied and “empirically supported” in clinical trials should be used in real-world practice whenever possible. Evidence-based treatment also rests on the assertion that some treatment methods are better than others—in particular, that treatments studied in clinical trials are better than those practiced in the real world (16-20). However, this assertion is highly questionable. Decades of research have demonstrated that a wide variety of treatment methods have been shown to be equally effective, and that much of the variance in outcomes can be attributed to factors that are thought to be common to all treatments, including the working alliance with a caring and expert professional (1,21-27). Outcomes informed care focuses on patient-reported outcomes rather than therapist-reported treatment method because it also incorporates this body of research.

Outcomes informed care can clearly be described as “evidence based practice” because of its foundation on the substantial body of published research demonstrating the effectiveness of patient-focused measurement and feedback system. However, outcomes informed care is fundamentally different from the common usage of the term because outcomes informed care is a “meta-method”. It allows for outcomes improvement across all treatment methods, and is not dependent on the unsubstantiated assertion that certain treatments yield better outcomes than others. Outcomes-informed care, which encompasses variability in treatments, therapists, and patients, is therefore an “evidenced based practice” as opposed to the treatment-specific “evidence-based treatment.”

Is there an association between outcomes-informed care and the role of common factors in treatment outcomes?

The methods associated with outcomes informed care were first developed by psychotherapy researchers that were well known for their work in evaluating evidence for the effectiveness of psychotherapy through more sophisticated methods such as meta analysis and hierarchical linear modeling (7,10,12,24). Through comprehensive reviews of the literature, these researchers realized the apparent importance of common factors across treatment methods and began to investigate methods to improve patient outcomes that were independent of the utilized treatment.

Outcomes informed care recognizes that the effect of any treatment is largely dependent on who delivers the treatment. Recent advances in research have led to a rapidly growing body of studies concerned with the role of the individual therapist in the outcome of care (28-39). This body of evidence from both clinical trials and real world settings demonstrate that the individual therapist is the single most important “ingredient” in the effects of treatment outcome. Statistically speaking, significantly more of the variance in outcomes is explained by who delivered the treatment than by the treatment method per se (36,39). The effect of the therapist does not diminish even if the patient is on medication.

How widely is outcomes informed care practiced?

Some of the earliest adopters of outcomes informed care were large managed behavioral health care organizations (MBHOs) that began to systematically collect treatment outcome data for as many patients as possible within their systems of care (10,40-42). Published reports of findings from these large-scale outcomes informed care projects have led to rapidly increasing adoption of the methods by therapists and behavioral healthcare organizations throughout the United States.

In these early outcomes informed care initiatives, the Outcome Questionnaire (OQ) family of outcome measures, including the OQ-30, YOQ-30, and OQ-45 are the most widely used (43-46). Among brief measures, variations on the Institute for the Study of Therapeutic Change’s four-item Outcome Rating Scale have been used throughout the world in various outcomes informed care initiatives. More recently, the ACORN collaboration has developed a suite of questionnaires are are widely used in a variety of settings.

Much of the data collected by the MBHOs have shared with academic researchers from a variety of institutions, and this ongoing collaboration between the MBHOs and research institutions have resulted in increasingly sophisticated use of the information obtained from the routine collection of treatment outcome data.

What are the results for the ACORN collaboration?

ACORN participating providers are seeing a significant upward trend on outcomes during the first two to three years of participation. Results vary from clinician to clinician, and from site to site. The principal predictors of improved outcomes over time are the number of patients involved and the amount of time the clinician spent viewing their data via the ACORN Decision Support Toolkit. For more information on this analysis, see: Measurement + Feedback = Improved Outcomes.

What are the implications for therapists and MBHOs?

Therapists can be heartened by the evidence from MBHOs for the effectiveness of their psychotherapy services. In general, psychotherapy as practiced in the real world results in significant improvement in 6 to 12 weeks for patients with moderate to severe symptoms. This evidence of the effectiveness of psychotherapy permits MBHOs to make a case to employers and health plans to continue to include behavioral health benefits that allow easy access to Employee Assistance Programs and other psychotherapy services.

Highly effective behavioral healthcare providers are also able to make a compelling case for the value of their services, both to the MBHO and to health care system in general. Interestingly, it appears that the therapists are normally distributed with regards to their effectiveness. While a small minority of therapists appear to be of very little help to their patients, the vast majority of therapists appear to be, on average, providing at least some relief for their patients. Many therapists appear to be providing highly effective treatment, and the value of their services is evident.

Outcomes informed care provides therapists with the tools to improve their own outcomes. Through use of outcome measures alone or in combination with a measure of therapeutic alliance, therapist can elicit useful feedback from patients. Although the use of these measures in no way substitutes for clinical judgment, evidence suggests that these measures provide additional information which could be used by skilled therapists to complement their own clinical judgment in seeking how to best engage their patients in therapy and tailor the treatment to fit their individual needs while continuously monitoring the impact of treatment. Research strongly suggests that therapists who incorporate outcomes informed care into their own practice are likely to achieve better treatment outcomes than those who do not (3-9,12-15).

The evidence of the potential for outcomes informed care to improve outcomes for large populations of patients provides a compelling reason for MBHOs to implement these methods. In addition to reducing the frequency of treatment failures by proactively identifying at-risk patients, evidence of variability in outcomes across therapists suggests that MBHOs can substantially improve the probability of successful outcomes be assisting clinicians to achieve realize the benefits of outcomes informed care and by steering referrals towards therapists with demonstrated track records of highly effective services.


1. Wampold BE. 2001. The great psychotherapy debate: Models, Methods, and Findings. Mahwah NJ: Lawrence Erlbaum Associates. 272 pp.

2. Westen D, Morrison K. 2001. A multidimensional meta-analysis of treatments for depression, panic, and generalized anxiety disorder: An empirical examination of the status of empirically supported therapies. J Consul Clin Psychol 69:875-99.

3. Sapyta J, Riemer M, Bickman L. 2005. Feedback to therapist: theory, research, and practice. J Clin Psychol 61(2):145-53.

4. Hannan C, Lambert MJ, Harmon C et al. 2005. A lab test and algorithms for identifying clients at risk for treatment failure. J Clin Psychol 61(2):155-63.

5. Lambert MJ, Harmon C, Slade K et al. 2005. Providing feedback to psychotherapists on their patients progress: Clinical results and practice suggestions J Clin Psychol 61(2):165-74.

6. Harmon C, Hawkins, Lambert MJ et al. 2005. Improving outcomes for poorly responding clients: The use of clinical support tools and feedback to clients. J Clin Psychol 61(2):175-85.

7. Brown GS, Jones ER. 2005. Implementation of a feedback system in a managed care environment: What are patients teaching us? J Clin Psychol 61(2):187-98.

8. Miller SD, Duncan BL, Ryan S, et al. 2005. The Partners for Change Outcome Management System. J Clin Psychol 61(2):199-208.

9. Claiborn CD, Goodyear EK. 2005. Feedback in psychotherapy. J Clin Psychol 61(2):209-21.

10. Brown GS, Burlingame GM, Lambert MJ, et al. 2001. Pushing the quality envelope: A new outcomes management system. Psychiatr Serv 52(7):925-34.

11. Lueger RJ. 1998. Using feedback on patient progress to predict the outcome of psychotherapy. J Clin Psychol 54:383-93.

12. Lambert MJ, Whipple JL, Smart DW, et al. 2001. The effects of providing therapists with feedback on patient progress during psychotherapy: Are outcomes enhanced? Psychother Res 11(1):49-68.

13. Lambert MJ, Whipple JL, Vermeersch DA, et al. 2002. Enhancing psychotherapy outcomes via providing feedback on client progress: A replication. Clin Psychol Psychother 9:91-103.

14. Whipple JL, Lambert MJ, Vermeersch DA, et al. 2003. Improving the effects of psychotherapy: The use of early identification of treatment failure and problem-solving strategies in routine practice. J Counsel Psychol 50(1):59-68.

15. Lambert MJ, Whipple JL, Hawkins EJ, et al. 2003. Is it time for clinicians to routinely track patient outcome? A meta-analysis. Clin Psychol Sci Prac 10:288-301.

16. Addis ME. 2002. Methods for disseminating research products and increasing evidence-based practice: Promises, obstacles, and future directions. Clin Psychol Sci Prac 9:367-78.

17. Chorpita BF, Yim LM, Donkervoet JC, et al. 2002. Toward large-scale implementation of empirically supported treatments for children: A review and observations by the Hawaii Empirical Basis to Services Task Force. Clin Psychol Sci Prac 9:165-90.

18. Herschell AD, McNeil CB, McNeil DW. 2004. Clinical child psychology’s progress in empirically supported treatments. Clin Psychol Sci Prac 11:267-88.

19. Manderscheid RW, Henderson MJ. 2004. Mental health, United States, 2002 executive summary. Admin Policy Mental Health 32:49-55.

20. Stirman SW, Crits-Christoph P, DeRubeis RJ. 2004. Achieving successful dissemination of empirically supported psychotherapies: A synthesis of dissemination theory. Clin Psychol Sci Prac 11:343-59.

21. Rosenzweig S. 1936. Some implicit common factors in diverse methods of psychotherapy: “At last the Dodo said, ‘Everybody has won and all must have prizes.’” Am J Orthopsychiatry 6:412-5.

22. Shapiro DA, Shapiro D. 1982. Meta-analysis of comparative therapy outcome studies: A replication and refinement. Psychol Bull 92:581-604.

23. Robinson LA, Berman JS, Neimeyer RA. 1990. Psychotherapy for treatment of depression: A comprehensive review of controlled outcome research. Psychol Bull 108:30-49.

24. Wampold BE, Mondin GW, Moody M, et al. 1997. A meta-analysis of outcome studies comparing bona fide psychotherapies: Empirically, “All must have prizes.” Psychol Bull 122:203-15.

25. Ahn H, Wampold BE. 2001. Where oh where are the specific ingredients? A meta-analysis of component studies in counseling and psychotherapy. J Counsel Psychol 48:251-7.

26. Chambless DL, Ollendick TH. 2001. Empirically supported psychological interventions: Controversies and evidence. Annual Rev Psychol 52:685-716.

27. Martindale C. 1978. The therapist-as-fixed-effect fallacy in psychotherapy research. J Consult Clin Psychol 46:1526-30.

28. Luborsky L, Crits-Christoph P, McLellan T, et al. 1986. Do therapists vary much in their success? Findings from four outcome studies. Am J Orthopsychiatry 56:501-12.

29. Crits-Christoph P, Baranackie K, Kurcias JS, et al. 1991. Meta-analysis of therapist effects in psychotherapy outcome studies. Psychother Res 1:81-91.

30. Crits-Christoph P, Mintz J. 1991. Implications of therapist effects for the design and analysis of comparative studies of psychotherapies. J Consul Clin Psychol 59:20-6.

31. Wampold BE. 1997. Methodological problems in identifying efficacious psychotherapies. Psychother Res 7:21-43,

32. Elkin I. 1999. A major dilemma in psychotherapy outcome research: Disentangling therapists from therapies. Clin Psychol Sci Prac 6:10- 32.

33. Wampold BE, Serlin RC. 2000. The consequences of ignoring a nested factor on measures of effect size in analysis of variance designs. Psychol Methods 4:425-33.

34. Huppert JD, Bufka LF, Barlow DH, et al. 2001. Therapists, therapist variables, and cognitive-behavioral therapy outcomes in a multicenter trial for panic disorder. J Consul Clin Psychol 69:747-55.

35. Luborsky L, Rosenthal R, Diguer L, et al. 2002. The dodo bird verdict is alive and well—mostly. Clin Psychol Sci Prac 9:2-12.

36. Okiishi J, Lambert MJ, Nielsen SL, et al. 2003. Waiting for supershrink: An empirical analysis of therapist effects. Clin Psychol Psychother 10:361-73.

37. Brown GS, Jones ER, Lambert MJ, et al. 2005. Identifying highly effective psychotherapists in a managed care environment. Am J Managed Care 11(8):513-20.

38. Wampold BE, Brown GS. 2005. Estimating therapist variability: A naturalistic study of outcomes in private practice. J Consul Clin Psychol.73(5): 914-923.

39. Kim DM, Wampold BE, Bolt DM. 2006. Therapist effects and treatment effects in psychotherapy: Analysis of the National Institute of Mental Health Treatment of Depression Collaborative Research Program. Psychother Res. 16(2):161-172.

40. Matsumoto K, Jones E, Brown GS. 2003. Using clinical informatics to improve outcomes: A new approach to managing behavioral healthcare. J Info Tech Health Care 1(2):135-50.

41. Brown GS, Jones ER, Betts W, et al. 2003. Improving suicide risk assessment in a managed-care environment. Crisis 24(2):49-55.

42. Brown GS, Herman R, Jones ER, et al. 2004. Improving substance abuse assessments in a managed care environment. Joint Commission J Quality Safety 30(8):448-54.

43. Wells MG, Burlingame GM, Lambert MJ, et al. 1996. Conceptualization and measurement of patient change during psychotherapy: Development of the Outcome Questionnaire and Youth Outcome Questionnaire. Psychother 33:275-283.

44. Lambert MJ, Hatfield DR, Vermeersch DA, et al. 2001. Administration and scoring manual for the LSQ (Life Status Questionnaire). East Setauket, NY: American Professional Credentialing Services.

45. Burlingame GM, Jasper BW, Peterson G, et al. 2001. Administration and scoring manual for the YLSQ. East Setauket, NY: American Professional Credentialing Services.

46. Vermeersch DA, Lambert MJ, Burlingame GM. 2002. Outcome Questionnaire: Item sensitivity to change. J Pers Assess 74:242-61.



This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding ACORN Wiki? Send feedback
Syndicate this site RSS