ALERT! Warning: your browser isn't supported. Please install a modern one, like Firefox, Opera, Safari, Chrome or the latest Internet Explorer. Thank you!
Log inLog in to ACORN Wiki. | RegisterSign on to ACORN Wiki.

Outcomes Measurement 2.0

The outcomes measurement methods employed by the ACORN community rely on the use of sophisticated statistical methodologies to assure that the questionnaires provide a high "yield" of useful information while recognizing the constraints on time and money in real world clinical settings. In order to achieve a high degree of measurement "efficiency", the psychometricians and statisticians involved in the analysis of data from ACORN questionnaires are employing measurement methods that are unfamiliar to most clinicians who have been exposed to research based on tradition outcomes research methodology, which we refer to as "Outcomes Measurement 1.0." This term refers to a tradition of measurement that started with Francis Galton and intelligence testers such as Terman and Wechsler, extending to the present in the huge catalogue of copyrighted tests available for purchase. Tests in this tradition were generally static, with updates every few years, and they were "one size fits all," with the same one or two forms that everyone had to use. Tests were distributed in a fee-based proprietary model like book publishing.

The arrival of the 21st century with ubiquitous computers is changing many things, and the measurement of client progress is unlikely to be an exception. We call the old tradition "Measurement 1.0" and what seems to be coming, "Measurement 2.0," following the popular terms "Internet 1.0" and "Internet 2.0." Like what has come to be called the Web 2.0 (of which http://en.wikipedia.org is a prime example), Outcomes Measurement 2.0 depends upon the ability of large communities of clinicians and researchers to collaborate in developing measurement methods. Such natural communities of collaboration are able to generate data, perform analyzes, and share information on a scale unheard of a decade ago. For example, the organizations utilizing the ACORN COMMONS currently generate data on between between 30,000 and 40,000 psychotherapy sessions monthly (a number that is steadily rising) using a wide variety of patient self report questionnaires.

The National Institute of Health's Patient-Reported Outcomes Measurement System is an exciting example of measurement 2.0 at work.

For a more complete discussion of Measurement 2.0 see: Outcomes_Measurement_2.0-White_Paper

Following are comparisons and contrasts between Measurement 1.0 and 2.0 methods across key domains of activity. Comments and contributions are welcome!

Questionnaire ownership and licensing

Measurement 1.0 Measurment 2.0
Reliance on copyrighted and published questionnaires Item bank and resulting questionnaires belong to community of users. See CopyRightCopyLeft
Copyright holder may charge fees for the use of questionnaires No fees for questionnaires constructed from items in the shared item bank
Copyright holder may place conditions or restrictions on the use of questionnaires Each organization is responsible for their own measurement methods and determines the appropriate content and use of questionnaires

Questionnaire development

Measurement 1.0 Measurment 2.0
A pool of items are tested in various samples A pool of items is tested in various samples
Item analysis used to select items for final questionnaire Item analysis used to select items for multiple versions of the questionnaires, depending on the needs of the users
Questionnaire validated, usually in correlation studies with questionnaires measuring the same construct Construct validity determined by factor analysis and comparison of results to known constructs. Various organizations may conduct correlations studies to satisfy internal skeptics.
Questionnaire published in final form Questionnaires are constantly evolving as data accumulates and measure needs change
Manual published Online manual constantly updated as data accumulates and needs of users dictate
Questionnaire published in peer reviewed journal Information on questionnaires shared among community of users and the interested public via a web site
Several years pass before a new version is published Multiple versions available, with the community of users determining which versions offer the greatest utility

Normative data

Measurement 1.0 Measurement 2.0
Normative data collected by test developer Normative data collected by community of users
Test developer may or may not continue to have access to data from wide spread field use of the questionnaire Community of users continuously generate data from the field use of the measures
Normative information provided in manual and remains static Normative information available through online manual and is updated continuously

Case Mix Adjustment

Measurement 1.0 Measurement 2.0
Manual may include some information on norms for different categories of patients Online manual includes information requested by users
Proprietary software products associated with the proprietary questionnaires may include case mix adjustment algorithms Case mix adjustment calculations performed dynamically using the General Linear Model and related methods. This is made possible through use of enterprise level data warehousing and analytic software such as products from SAS Institute
Case mix adjustment algorithms are static, until next version of software is released Algorithms are constantly evolving based on continuous analysis of a growing number of cases data repository

Outcomes Benchmarking

Measurement 1.0 Measurement 2.0
Questionnaire vendor may offer proprietary benchmarking reports Community of users collaborate on development and implementation of benchmarking methodologies
Benchmarking methodology (if it exists) tends to be static Benchmarking method is dynamic, based on continuous revision to case mix adjustment models as the data dictates
Offers benchmarks against reference samples external to the organization Offers benchmarking based on both external and internal reference samples, i.e. normative samples generated by the organization or community of users using the questionnaire

Comments

 

Primary topics:

You may also view all topics in alphabetical order in WebTopicList.
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding ACORN Wiki? Send feedback
Syndicate this site RSS
^