Activity Sampling Applied to Interactive System Design

In the design of interactive systems, it is desirable to obtain both qualitative and quantitative estimates of combined human/system performance as early as possible. Recent methodological extensions to the well-known technique of activity sampling can make simulation a more effective source of both kinds of data. Experience with these enhancements demonstrates that through the use of intensive videotape sessions, computer-naive subject-matter experts can be trained reasonably quickly to reliably exercise highly complex classification schemes that include simultaneous categories with multiple subcategories. In the context of knowledge engineering, this measurement technique, combined with scenario-driven simulations of proposed forms of interaction, can also be employed to spin off a variety of synergistic benefits, such as (a) critical-incident generation, (b) early “hard” estimates of personnel requirements and training costs, (c) explanatory (hypothesis-generating) power of a very application-specific nature, and (d) conversion of user-group resistance into user-group support.