Current written tools of assessment are mostly measuring the capacity to solve well-defined problems by the application of rules and principles, while the essence of expertise in the professions lies in the capacity to solve illdefined problems, that is, reasoning in contexts of uncertainty. The purpose of this study is to describe an approach that allows assessing ill-defined problems and to present and discuss research findings related to this approach. The tool has been used up to now mainly in medicine, however it can be applied in all health professions. The approach is based on three principles: (a) examinees are faced with a challenging authenticsituation in which several options are relevant; (b) the response format is a Likert-type scale that reflects the way information is processed in problem-solving situations, according to the script theory; and (c) scoring is based on the aggregate scoring method to take into account the variability of reasoning processes among experts. Research findings suggest that the approach permits one to reliably discriminate examinees across their level of experience, and so in very different domains. It makes it possible to measure skills or domains that were up to now difficult to measure.
Barrows, H. S.
, Feightner, J. W., Neufeld, V. R., & Norman, G. R. (1978). Analysis of the clinical methods of medical students and physicians (Final Report, Ontario Department of Health, Grants ODH-PR-273 and ODH-DM-226). Hamilton, Canada: McMaster University.
2.
Brailovsky, C.
, Charlin, B., Beausoleil, S., Côté, S., & van der Vleuten, C. (2001). Measurement of clinical reflective capacity early in training as a predictor of clinical reasoning performance at the end of residency: An exploratory study on the Script Concordance Test. Medical Education, 35, 430-436.
3.
Brazeau-Lamontagne, L.
, Charlin, B., Gagnon, R., Samson, L., & van der Vleuten, C. (in press). Measurement of perception and interpretation skills along radiology training: utility of the script concordance approach. Medical Teacher.
4.
Charlin, B.
, Brailovsky, C. A., Brazeau-Lamontagne, L., Samson, L., & Leduc, C. (1998). Script questionnaires: Their use for assessment of diagnostic knowledge in radiology. Medical Teacher, 20, 567-571.
5.
Charlin, B.
, Brailovsky, C. A., Leduc, C., & Blouin, D. (1998). The Diagnostic Script Questionnaire: A new tool to assess a specific dimension of clinical competence. Advances in Health Sciences Education, 3, 51-58.
6.
Charlin, B.
, Desaulniers, M., Gagnon, R., Blouin, D., & van der Vleuten, C. (2002). Comparison of an aggregate scoring method with a consensus scoring method in a measure of clinical reasoning capacity. Teaching and Learning in Medicine, 14, 150-156.
7.
Charlin, B.
, Roy, L., Brailovsky, C. A., & van der Vleuten, C. P. M. (2000). The Script Concordance Test: A tool to assess the reflective clinician. Teaching and Learning in Medical Education, 12, 189-195.
8.
Charlin, B.
, Tardif, J., & Boshuizen, H. P. A. (2000). Scripts and medical diagnostic knowledge: Theory and applications for clinical reasoning instruction and research. Academic Medicine, 75, 182-190.
9.
Cohen, L. (2003). The development and validation of the Seating and Mobility Script Concordance Test (SMSCT). Unpublished doctoral dissertation, University of Pittsburgh, PA.
10.
Dauphinee, W. D.
(1995). Assessing clinical performance: Where do we stand and what might we expect?Journal of the American Medical Association, 274, 741-743.
11.
Elstein, A. S.
, Shulman, L. S., & Sprafka, S. A. (1978). Medical problem solving: An analysis of clinical reasoning. Cambridge, MA: Harvard University Press.
12.
Elstein, A. S.
, Shulman, L. S., & Sprafka, S. A. (1990). Medical problem solving, a ten-year retrospective. Evaluation & the Health Professions, 13, 5-36.
13.
Fox, R. C.
(2000).Medical uncertainty revisited. In G. L. Albrecht, R. Fitzpatrick, & S. C. Scrim-shaw (Eds.), Handbook of social studies in health and medicine (pp. 409-425). London: Sage.
14.
Grant, J.
, & Marsden, P. (1988).Primary knowledge, medical education and consultant expertise. Medical Education, 22, 173-179.
15.
Irby, D.
(1997). Editorial. Academic Medicine, 72, 116-116.
16.
Johnson, E. J.
(1988). Expertise and decision under uncertainty: Performance and process. In M. T. H. Chi, R. Glaser, & M. J. Farr (Eds.), The nature of expertise (pp. 209-228). Hillsdale, NJ: Lawrence Erlbaum.
17.
Khonputsa, P., Besinque, K., Fisher, D., & Gong, W. C. (2004). The Pharmacy Script Concordance Test: A pilot study assessing clinical pharmacy competence in Thailand. Manuscript submitted for publication.
18.
Labelle, M.
, Beaulieu, M., Thivierge, R. L., Paquette, D., & Choquette, D. (2001, January 27-30). Best practice in CME: An innovative problem-based learning model based on the use of the Script Concordance Test. Poster presented at the Alliance for Continuing Medical Education’s 25th Annual Conference. San Francisco.
19.
Llorca, G.
(2003). Évaluation de résolution de problèmes mal définis en éthique clinique: variation des scores selon les méthodes de correction et selon les caractéristiques des jurys [Illdefined problem assessment in clinical ethics: Score variation according to scoring method and jury characteristics]. Pédagogie Médicale, 4, 80-88.
20.
Nelson, K.
(1986). Event knowledge: Structure and functionin development. Hillsdale, NJ: Lawrence Erlbaum.
21.
Norcini, J. J.
, Shea, J. A., & Day, S. C. (1990). The use of the aggregate scoring for a recertification examination. Evaluation and the Health Professions, 13, 241-251.
22.
Norman, G.
, Swanson, D. B., & Case, S. M. (1996). Conceptual and methodological issues in studies comparing assessment formats. Teaching and Learning in Medicine, 8, 208-216.
23.
Norman, G. R.
(1985). Objective measurement of clinical performance. Medical Education, 19, 43-47.
24.
Schmidt, H. G.
, Boshuizen, H. P. A., & Hobus, P. P. M. (1988). Transitory stages in the development of medical expertise: The “intermediate effect” in clinical case representation studies. In V. L. Patel & G. J. Grogen (Eds.), Proceedings of the 10th Annual Conference of the Cognitive Science Society (pp. 139-145). Hillsdale, NJ: Lawrence Erlbaum.
25.
Schmidt, H. G.
, Norman, G. R. & Boshuizen, H. P. A. (1990).A cognitive perspective on medical expertise: Theory and implications. Academic Medicine, 65, 611-621.
26.
Schön, D. A.
(1983). The reflective practitioner: How professionals think in action. New York: Basic Books.
27.
Sibert, L.
, Charlin, B., Corcos, J., Gagnon, R., Grise, P., & van der Vleuten, C. (2002).Stability of clinical reasoning assessment results with the script concordance test across two different linguistic, cultural and learning environments. Medical Teacher, 24, 537-542.
