Methodologies for assessing human judgment in complex domains are important for the design of both displays that inform judgment and automated systems that suggest judgments. This paper uses the n-system lens model to evaluate the impact of displays on human judgment and to explicitly assess the similarity between human judgments and a set of potential judgment algorithms for use in automated systems. First, the need for and concepts underlying judgment analysis are outlined. Then the n-system lens model and its parameters are formally described. This model is then used to examine a previously conducted study of aircraft collision detection that had been analyzed using standard analysis of variance methods. Our analysis found the same main effects as did the earlier analysis. However, n-system lens model analysis was able to provide greater insight into the information relied upon for judgments and the impact of displays on judgment. Additionally, the analysis was able to identify attributes of human judgments that were---and were not---similar to judgments produced by automated systems. Potential applications of this research include automated aid design and operator training.
References
1.
Bisantz, A. M., Kirlik, A., Gay, P., Phipps, D., Walker, N., & Fisk, A. D. (2000). Modeling and analysis of a dynamic judgment task using a lens model approach. IEEE Transactions on Systems, Man, and Cybernetics---Part A: Systems and Humans, 30(6), 1–12.
2.
Brehmer, B. (1976). Social judgment theory and the analysis of interpersonal conflict. Psychological Bulletin, 83, 985–1003.
3.
Brehmer, B., & Joyce, C. R. B. (Eds.). (1988). Human judgment: The SJT view.Amsterdam: North-Holland.
4.
Brunswik, E. (1955). Representative design and probabilistic theory in a functional psychology. Psychological Review, 62, 193–217.
5.
Castellan, N. J. (1972). The analysis of multiple criteria in multiplecue judgment tasks. Organizational Behavior and Human Performance, 8, 242–261.
6.
Cooksey, R. W. (1996). Judgment analysis: Theory, methods, and applications.San Diego: Academic.
7.
Cooksey, R. W., & Freebody, P. (1987). Cue subset contributions in the hierarchical multivariate lens model: Judgments of children's reading achievement. Organizational Behavior and Human Decision Processes, 39, 115–132.
8.
Cooksey, R. W., Freebody, P., & Davidson, G. R. (1986). Teacher's prediction of children's early reading achievement: An application of social judgment theory. American Educational Research Journal, 23, 41–64.
9.
Dalgleish, L. I. (1988). Decision making in child abuse cases: Application of social judgment theory and signal detection theory. In B. Brehmer & C. R. B. Joyce (Eds.), Human judgment: The SJT view (pp. 317–360). Amsterdam: North-Holland.
10.
Dawes, R. M. (1979). The robust beauty of improper linear models in decision making. American Psychologist, 34, 571–582.
11.
Einhorn, H. J., Kleinmuntz, D. N., & Kleinmuntz, D. N. (1979). Linear regression and process-tracing models of judgment. Psychological Review, 86, 465–485.
12.
Goldberg, L. R. (1968). Simple models or simple processes? Some research on clinical judgments. American Psychologist, 23, 483–496.
13.
Gupta, N., Bisantz, A. M., & Singh, T. (2002). The effects of adverse condition warning system characteristics on driver performance: An investigation of alarm signal type and threshold level. Behaviour and Information Technology, 21, 235–248.
14.
Hammond, K. R. (1996). Human judgment and social policy.New York: Oxford University Press.
15.
Hammond, K. R., Stewart, T. R., Brehmer, B., & Steinmann, D. O. (1975). Social judgment theory. In M. F. Kaplan, & S. Schwartz (Eds.), Human judgment and decision processes (pp. 271–312). New York: Academic.
16.
Kirlik, A. (2001). Human factors. In K. R. Hammond, & T. Stewart (Eds.), The essential Brunswik (pp. 400–403). New York: Oxford University Press.
17.
Kirlik, A., & Strauss, R. (2001). Medical uncertainty and the older patient: An ecological approach to supporting judgment and decision making. In W. Rogers, & A. D. Fisk (Eds.), Human factors interventions for the health care of older adults (pp. 69–90). Mahwah, NJ: Erlbaum.
18.
Kuchar, J. K., & Carpenter, B. D. (1997). Airborne collision alerting logic for closely-spaced parallel approach. Air Traffic Control Quarterly, 5(2), 111–127.
19.
Pritchett, A. (1999). Pilot performance at collision avoidance during closely-spaced parallel approaches. Air Traffic Control Quarterly, 7(1), 47–75.
20.
Pritchett, A. (2000). Display effects on shaping apparent strategy: A case study in collision detection and avoidance. International Journal of Aviation Psychology, 10, 59–83.
21.
Rohrbaugh, J. (1988). Cognitive conflict tasks and small group processes. In N. Brehmer & C. R. B. Joyce (Eds.), Human judgment: The SJT view (pp. 199–226). Amsterdam: North-Holland.
22.
Schmitt, N., Ciyle, B. W., & Saari, B. B. (1977). Types of task information feedback in multiple-cue probability learning. Organizational Behavior and Human Performance, 18, 316–328.
23.
Seong, Y. (2002). Modeling human operator's judgment and trust in conjunction with automated decision aids. Unpublished doctoral dissertation, University at Buffalo, The State University of New York, Amherst, NY.
24.
Seong, Y., & Bisantz, A. M. (2002). Judgment and trust in conjunction with automated decision aids: A theoretical model and empirical investigation. In Proceedings of the Human Factors and Ergonomics Society 46th Annual Meeting (pp. 423–427). Santa Monica, CA: Human Factors and Ergonomics Society.
25.
Stewart, T. R., Heideman, K. F., Moniger, W. R., & Reagan-Cirincione, P. (1992). Effects of improved information on the components of skill in weather forecasting. Organizational Behavior and Human Decision Processes, 53, 107–134.
26.
Wigton, R. S. (1988). Applications of judgment analysis and cognitive feedback to medicine. In B. Brehmer & C. R. B. Joyce (Eds.), Human judgment: The SJT view (pp. 427–442). Amsterdam: North-Holland.
27.
Wong, G. A. (1993, May). Development of precision runway monitor system for increasing capacity of parallel runway operations. Paper presented at the Advisory Group for Aerospace Research and Development (AGARD) Meeting on Machine Intelligence in Air Traffic Management, Berlin, Germany.
