Iterative Usability Testing of a Security Application

This paper reports the results of three iterative usability tests of a security application as it evolved through the application development process and highlights the use of several methodological techniques: 1) reusable color foil prototypes of application panels as an alternative to developing online prototypes during short development cycles, 2) field tests as a complement to laboratory tests, 3) iterative testing of an evolving prototype, and 4) analysis of dollar value of usability work. The techniques used represent an attempt to apply usability engineering to system design (Whiteside, Bennett, and Holtzblatt; 1988) and to provide management with a dollar value estimate of human factors work (Mantei, 1988). Significant improvements in end user performance and satisfaction occurred across the three iterative tests (field prototype test, laboratory prototype test, and laboratory integration test) conducted across 7 months with 27 participants. The product usability objective was met during the third test. By using the reusable foil prototypes of the interface panels, usability staff were able to efficiently and effectively identify problems, make design changes, and retest the panels. The field test furnished unique data necessary to understanding end user issues. Iterative testing provided the opportunity to test the impact of changes made to the interface and a reliability check on previous results. The methodology for computing the value of usability work provided a feasible way of analyzing the cost benefit of the human factors work.