Requirements for Effective Function Allocation

Abstract

In this paper, we identify the requirements for effective function allocation within teams of human and automated agents. These functions include all the activities in the team’s environment required to meet collective work goals, that is, taskwork functions. In addition, the allocation of taskwork functions then creates the need for additional teamwork functions to coordinate between agents. Key requirements include that each agent must be capable of each individual function it is allocated and must be capable of its collective set of functions, including teamwork. Of note, many important attributes may be observed only within the detailed dynamics of simulation or actual operations, particularly when a function allocation requires tightly coupled interactions and when teamwork (including human–automation interaction) may support or detract from effective performance. Finally, we note that function allocation is a key design decision that should be made deliberately. By addressing function allocation early in design, before technologies and interfaces are created, key trade-offs can be considered and fundamental concerns with human factors addressed.

Keywords

function allocation topics human–automation interaction human–system integration design methods methods literature review

Introduction

In this paper, we examine the requirements for effective function allocation, that is, the allocation of work within teams of human and automated agents. Function allocation is an important design decision that must be undertaken early in the design process as it forms the basis for machine logic and automation interfaces. The functions that are divvied up during function allocation include all the activities performed in the team’s environment required to meet collective work goals, that is, taskwork functions. The allocation of taskwork functions in turn creates the need for additional teamwork functions to coordinate between agents. All of these functions are performed in a complex work environment in which each agent’s behavior is driven by the dynamics of the domain and by the actions of other agents.

The five requirements presented in this paper are derived from a critical literature review in which the concept of function allocation was specifically examined from multiple perspectives in an effort to triangulate and synthesize the thinking on this topic. The resultant requirements described here form the basis for two subsequent companion papers, in which we investigate methods to measure and model function allocation in a computational framework.

Throughout, our intention in identifying key requirements for function allocation is practical: We wish to guide design such that key trade-offs can be considered and fundamental concerns with human factors addressed before technologies and interfaces are created. Indeed, from a human factors perspective, function allocation is the earliest possible design decision from which training, procedures, technological functions, and interface specifications all flow.

Further, we explicitly acknowledge conceptual debates about the relative roles of human and automation, and the purposes that should be ascribed to machines. However, we also note the importance of including perspectives spanning the dynamics of the work, the interactions within the team, and the agents’ collective ability to perform the work that is required of the team. Thus, the function allocation should be designed for its operating context, and no one function allocation is assumed to be optimal—or sufficient—for all types of operations.

Instead of seeking an optimal function allocation, we identify those requirements that any function allocation should meet. These requirements are derived by examining problems with function allocation as identified in operations and in the literature. Likewise, the origins of these problems are then examined as part of first-principle arguments for key requirements for function allocations.

Requirement 1: Each Agent Must Be Allocated Functions That It Is Capable of Performing

This requirement is met by a function allocation in which every agent in the team is able to perform each of the functions assigned to him/her/it, viewing each function in isolation relative to the agent’s capabilities. A common method of addressing this requirement follows what Bailey (1982) has termed a comparison strategy for function allocation: Each function is individually compared to the capabilities of each agent and assigned to the most capable. Such a comparison has been fostered by constructs such as the oft-called Fitts list, which articulates capabilities according to those that “Men Are Better At” versus those that “Machines Are Better At” (also called the MABA-MABA list; Fitts, 1951). Sometimes an agent’s capabilities are articulated relative to those of others. For example, distinctions have been made between automation that is capable of being a tool for a human versus a prosthesis, depending on whether it enables a human’s normal capabilities or extends a human’s capabilities beyond that normally achievable (Reason, 1987). As automation has become capable of initiating or conducting its own actions, a further distinction has been made of automation as an agent (Lee, 2006). Likewise, it is common to see one agent—human or automated—described as a backup or monitor for another agent who is presumed to be fallible (Bainbridge, 1983; Wiener & Curry, 1980) or to propose that the agents should provide complementary functions (Jordan, 1963).

Constructs such as the MABA-MABA list have been criticized for the extent to which human capabilities are framed in machine terms (see, for example, Sheridan, 2000). Indeed, the general definition of automation as “machines capable of taking over functions normally conducted by humans” has served as a challenge to increase autonomy through machines capable of more functions (see, for example, calls for the U.S. Air Force to developed increased autonomy; Dahm, 2010).

Unfortunately, with the common focus on the automation’s capabilities, one can forget to consider what the human’s corresponding functions will need to be. For example, categorizations of levels of automation as defined by Billings (1997) or Sheridan and Verplank (1978) or, on a more multidimensional scale, by Parasuraman, Sheridan, and Wickens (2000) are defined by what the machine is capable of doing. Thus, although a higher level of automation suggests the automation is a greater driver of total system performance, counterexamples have also been noted whereby the human may consistently override or second-guess high levels of automation or may be required (by procedures or regulations) to act as the unthinking servo-actuator for a lower level of automation that alerts or suggests decisions (Pritchett, 2005, 2009).

Function allocations established around machine capabilities can implicitly apply a function allocation strategy termed by Bailey (1982) as leftover allocation: Automate as many functions as technology will permit, and assume the human will pick up whichever functions are leftover. Commonly automated functions typify standard processes within conditions that can be clearly defined, such as nominal operations or established emergency shutdown procedures. However, this strategy makes for brittle automation that can fail in an unexpected manner or provides little support in off-nominal conditions, which is generally when the human needs the most support (Norman, 1990). Authors of operational studies have noted that function allocations based on this approach often result in designs in which humans are assigned to monitor automation or to monitor the environment for conditions beyond which the automation can operate (Bainbridge, 1983; Wiener & Curry, 1980), despite consistent findings that humans are ineffective at this task (Lee & Moray, 1992; Molloy & Parasuraman, 1996). Indeed, the researcher authoring the MABA-MABA list suggested instead that “machines should monitor men” (Fitts, 1951).

A related concern is that with human–automation function allocation, one often considers only the agents’ capability to assume authority for a function, unlike with human–human function allocation, in which one also examines responsibility. Specifically, whereas authority is generally used to describe who is assigned the execution of a function in operational sense, responsibility is used to identify who will be held accountable in an organizational and legal sense for its outcome. Except when automation is proven to provide safety in all foreseeable operating conditions, humans remain vested with the responsibility for the outcome of its actions, a situation termed the “responsibility-authority double-bind” (Woods, 1985). If the human cannot knowledgably oversee the automation, he or she is forced to trust the automation. However, without a concrete basis for assessing whether the automation is correct, humans often overtrust or undertrust the automation (Parasuraman & Riley, 1997). Either way, incorrect trust is viewed as human error, despite its basis in the function allocation. This view suggests that our assessment of whether a machine is capable of a function is commonly assessed by a lower standard—one of authority alone, and only in expected conditions—than we typically use when allocating functions to human team members.

Requirement 2: Each Agent Must Be Capable of Performing Its Collective Set of Functions

Assuming that the previous requirement is met, for this requirement, one now examines whether each agent can perform his/her/its collective set of functions under realistic operating conditions. This emphasis on the each agent’s collective set of functions underlies methods for global function allocation and for understanding the agents as limited resources (see Dearden, Harrison, & Wright, 2000, for a review). One potential obstruction may be that the set of functions is too large, that is, its taskload overwhelms an agent’s ability to process and execute. This possibility has been a concern with overloading automated systems, leading to technical developments such as real-time embedded systems capable of prioritizing demands on processors and communication devices (see Storey, 1996, for a review of the field).

Likewise, the set of functions assigned to a human agent may create a taskload that corresponds to excessive workload. Indeed, a common driver for adding a human team member is a need to share—to divvy up—the work. Further, it is important to examine the distribution of taskload through time. Taskload that is consistently excessive is usually readily apparent—and quickly remedied. However, workload spikes are not as easily predicted, as they represent brief durations when an agent is suddenly required to perform too many functions. Such spikes are endemic to emergencies, such that the number of agents in a team may be driven not by nominal conditions but by off-nominal or emergency conditions; take, for example, the staffing of a fire station that is (hopefully) not fully dispatched at all times.

Examining concerns with human workload in human–automation teams, automation is generally good at reducing the average workload of human operators, often by reducing the manual control or execution functions that the humans must perform. This characteristic has been touted as a contribution of automation and has been used to reduce staffing requirements, such as the reduction of flight crew in the flight deck from three (or more) to two. However, authors of operational studies also note the unfortunate prevalence of cases in which automation creates a workload spike for a human team member (Bainbridge, 1983; Billings, 1997; Wiener, 1985). This spike can occur in situations in which the automation is not capable of the function required, particularly, off-nominal or emergency situations outside automation’s boundary conditions, as noted in Requirement 1. For example, many autoflight systems are not rated to control the aircraft in unusual attitudes or when there may be ice accretion on vital sensors. Such situations can also be created—or compounded—by automation that demands significant interaction from a human team member at brief points in time, as also discussed next in Requirement 3. For example, highly automated autoflight systems can suddenly require a significant amount of programming from their human flight crew in response to a reroute commanded by an air traffic controller, even as the flight crew must also respond to the controller and potentially execute other tasks, such as finding charts appropriate to the new routing.

The corollary of workload spikes is the problem of excessively low workload during normal operations in between the spikes. Low workload by itself corresponds to concerns with task engagement, boredom, and the human becoming “out of the loop” (Bainbridge, 1983; Endsley & Kiris, 1995). Further, this low workload typifies passive activities, such as monitoring, discussed earlier as being not well suited to the human.

Estimates of the workload of any human agent in the team also must involve making sure to weigh properly the cognitive workload allocated to the human, even as the manual workload is allocated to the machine (Bainbridge, 1983; Billings, 1997; Wiener, 1985). These cognitive functions can demand significant information gathering, judgment, detection, and decision-making activities, sometimes even up to the point of reverse-engineering or second-guessing the output of the automation. In some instances, such as pilots’ responses to emergency alerts and decision aids, pilot reports suggest that they may be cognitively railroaded into reliance on the machine’s decisions, due to the immediate difficulty in performing all the requisite cognitive functions directly (Pritchett, 2005).

Estimates of the workload incurred by such cognitive functions need to account for myriad effects. One is the human’s expertise at the task and thus his or her ability to apply less-effortful strategies, rules, and skills while maintaining performance (Rasmussen, 1983). Further, nonlinear effects arise when several cognitive functions rely on awareness of the same information or on interdependent judgments of similar phenomena. In such cases, adding or removing one cognitive function may seem to have little impact, yet another function may implicitly require a significant, unique set of supporting activities and thus appear to require a disproportionate amount of effort.

Interdependence between tasks suggests that Requirement 2 can be better achieved when the function allocations establish coherent roles within the team. One attribute of a coherent function allocation can be viewed from the bottom up: Its functions share (and build upon) obvious, common constructs underlying all their activities, such as a shared information and knowledge basis within each agent; meanwhile, the distribution of functions prevents conflicts over low-level actions and resources between agents. Another attribute can be viewed from the top down: The functions collectively contribute toward work goals in a manner that not only is apparent to the human but can be purposefully coordinated and adapted in response to context. As noted earlier, however, with function allocation strategies that focus on automating as much as possible, the human’s allocation tends to be whatever is leftover, and the human must pick up any remaining functions throughout the work domain. When these leftovers are collectively an incoherent set, the human can neither coordinate underlying information search and judgment activities nor purposefully adapt his or her functions toward improved performance toward work goals (Dekker & Woods, 2002). Although this problem may be partly addressed by an interface that can illustrate the allocation, clearly coordinate human and automation tasks, and enable better monitoring of automation (M. Palmer, Rogers, Press, Latorella, & Abbott, 1995), the general problem of incoherent roles has its basis in a team design that does not allocate functions according to clearly specified roles.

Requirement 3: The Function Allocation Must Be Realizable With Reasonable Teamwork

The allocation of taskwork functions within any team—whether it includes automation or not—demands additional teamwork functions. These teamwork functions serve to coordinate the taskwork across agents. They include human-to-human communication, human–automation interaction, and coordination of the timing and conduct of taskwork, such as when an agent needs to wait after concluding one function for other agents to complete their parallel activities. Thus, each different function allocation of the same taskwork adds a unique set of teamwork functions, the impact of which must then be considered from the perspective of the previous two criteria: whether each agent can perform each of his/her/its teamwork activities in isolation and whether each agent can perform its entire set of both taskwork and teamwork functions.

A team perspective on function allocation involves considering automation as a team member and thus viewing human–automation interaction as similar to interaction within a human team. For example, models and measures of trust, social judgment, and roles have been extended from the social sciences and related studies of teams to human–automation interaction (see, for example, Bass & Pritchett, 2008; Muir, 1994; Muir & Moray, 1996; Woods, 1985; Woods & Hollnagel, 2006). However, automation does not have the same teamwork skills that humans naturally have. The automation does not have a sense of responsibility and, thus, cannot be delegated responsibility for its outcomes (Sarter & Woods, 1997). Automation does not worry about consequences. Automation has no motivation to live up to its obligations, does not experience shame or embarrassment, and cannot be assessed for attributes such as loyalty, benevolence, and agreement in values (Lee & See, 1994; Pitt, 2004). When placed outside its boundary conditions, automation cannot function properly, unlike a human team member, who will generally strive for effective performance in unfamiliar circumstances.

When automation is viewed as a team member, communication between humans and automation is of significant interest. It is important that team members are able to anticipate each other’s information needs and provide information at useful, noninterruptive times (Entin & Entin, 2001; Hollenbeck et al., 1995; Hutchins, 1995). Timing must be considered as well; interleaved tasks can leave one agent waiting on another, and poorly timed communication can disrupt performance. An interruption may be demanded by circumstances, in which case it can spur knowledge acquisition (Zellmer-Bruhn, 2003) and facilitate decision-making performance (Speier, Valacich, & Vessey, 1999). Unfortunately, too often automation is “clumsy”: It unduly interrupts its human team members because whereas humans can implicitly sense information about whether other team members would benefit from an interruption, automation historically cannot (Christoffersen & Woods, 2002; see Dorneich, Ververs, Mathan, Whitlow, & Hayes, 2012, for an example of efforts to improve this aspect of automation). With human–automated teams, predefined sets of function allocations may serve as more explicit coordination strategies, such as the playbook proposed by Miller and Parasuraman (2007) and the function allocations that the pilot of a modern aircraft may invoke.

Requirement 4: The Function Allocation Must Support the Dynamics of the Work

According to an ecological view, work is an ongoing response to, and action upon, the work environment. This definition can be viewed from the perspective of each agent and of the team as a whole. The physical environment has dynamics that can drive taskwork; at the same time, the teamwork actions of each agent modify the environment of other agents, creating a dynamic interplay.

In dynamic, complex work environments, many work activities are interdependent and heavily coupled. However, these couplings may be hidden. For example, in a function allocation with the pilot flying via the control column and with the autothrottle on, the pilot controls elevator and the automation controls throttle. However, pitch and speed are intrinsically coupled such that the actions of one will step on the actions of the other when, for example, a change in speed by the autothrottle requires the pilot to compensate for a resulting change in pitch. Thus, dynamic analysis is often required to identify situations in which the interleaving of functions assigned to disparate agents requires significant coordination or idling as one waits on another, or when workload may accumulate, or when one agent will be unduly interrupting another.

Further, function allocation should support how the dynamics of the environment are managed by the agents. For example, function execution is dictated by established procedures in some domains, such as aviation, and in other domains by less formally defined work practices. Whether explicitly established or informally defined, a function allocation should not interrupt these procedures, as they are proven to aid agent memory and guarantee consistency and safety (Ockerman & Pritchett, 2000; Palmer & Degani, 1991). Thus, a function allocation should involve considering when procedures are an intrinsic dynamic within the work environment and mirror their structures.

Among less prescribed constructs for managing complex, dynamic environments, some researchers have proposed the construct of resilience, which describes a team’s ability to work in unexpected situations as maintaining control of, or regulating, those situations (Hollnagel, 2004; Hollnagel, Woods, & Leveson, 2006; Sheridan, 2008). Resilience enables organizations to cope with unexpected variability in the environment in a “robust yet flexible” manner, thus mitigating risks (Chialastri & Pozzi, 2008). However, the brittleness of automation (i.e., degradation in its performance when the environment exceeds its boundary conditions) reflects how allocation of key functions to automation may degrade the team’s resilience in unexpected situations.

Likewise, resilience is fostered when a human agent may select strategies (courses of action) appropriate to the state of the environment and his or her own capabilities (Pritchett, 2010). For example, Hollnagel’s concept of cognitive control describes how humans adapt their activities (and sequence them) in response to their competency and their perception of resources available to them (such as information availability) and demands on them (such as subjective available time) (Feigh & Pritchett, 2006; Hollnagel, 1993). However, such adaptation can be constrained or eliminated by an overly prescribed (or proscribed) function allocation, particularly when human–automation interaction dictates a specific sequence of activities from the human (Feigh, 2011). The adverse effects of such overly prescribed function allocations have been found to manifest in workarounds or disuse of automation (Feigh & Pritchett, 2010; Kirlik, 1993; Parasuraman & Riley, 1997).

A related construct is the notion that humans work to develop and maintain a stable work environment. If the work environment maintains a certain level of regularity, the human can predict its dynamics and tailor his or her activity to the needs in the environment (Hollnagel, 2004). However, in the face of unexpected events, environmental predictability decreases, requiring the humans to spend more time to reacting to events (Hollnagel, 2002). A function allocation may aggravate inherent environmental unpredictability by, for example, limiting human agents’ ability to view important aspects of the environment or by distributing functions such that one agent will trigger the requirement for another to act. When considering adaptive function allocation, one finds that dynamic changes in function allocation is a further source of such unpredictability. Therefore, a trade-off exists when designing function allocations between maintaining predictability versus applying complex automated capabilities and dynamically allocating functions (Miller & Parasuraman, 2007).

In summary, the dynamics of the environment must be considered in several ways extending beyond historical approaches to function allocation using static values for cost and benefit (see, for example, Dearden et al.’s [2000] discussion of the limitations of simple economic bases for function allocation). The dynamics of the work may be interleaved or need to be coordinated between agents. In addition, we expect human agents to maintain a stable work environment and adapt their cognitive behavior relative to the demands placed upon them and the resources available to them. Broadening out to the system level, human organizations can adapt their collective behavior to maintain resilience to the unexpected. In contrast, automated agents are typically brittle relative to conditions that can arise in dynamic environments, and rigid function allocations limit the humans’ ability to adapt to intervene in those conditions.

Requirement 5: The Function Allocation Should Be the Result of Deliberate Design Decisions

Rather than proposing a separate method for designing function allocations, we propose the requirement of sensibly integrating function allocation decisions into the broader engineering design process. In many cases, initial specifications of function allocation arise from changes in the high-level concept of operation. Such changes may be incremental and constrained by current-day technologies, procedures, personnel, and/or policies; in other cases, such changes in concepts of operation may represent significant innovations in which constructs such as common work practices and relationships between tasks and tools must be significantly altered.

Therefore, the ability to meet the preceding requirements will be fostered if the function allocation is itself the product of deliberate design decisions—decisions that generally need to be made early in the (re)design of a concept of operations or sociotechnical system (Dearden et al., 2000). Many “engineering” models commonly exist at this stage of design to describe the dynamics of the work environment (for example, in designing an aircraft flight deck, reasonable models are established both in the literature and as simulation software for the flight dynamics of the aircraft and for atmospheric effects, such as turbulence and wind). Corresponding dynamic models of the work activities themselves, if not already available, would also be valued by, and contribute to, other engineering analysis and design activities.

Corresponding to general calls by the cognitive engineering community to get involved earlier in design, a function allocation modeling framework based on the same dynamic analysis used in evaluating concepts of operation and in developing automation can promote more integrated and deliberate approaches to function allocation within teams of humans and automated agents. This is particularly the case if such models enable designers to intrinsically integrate the economic and safety metrics by which the total system will be evaluated, the potential contributions of (or constraints on) technology and human performance, and the regulatory, policy, and procedural considerations in allowing access to (and defining interaction within) the collaborative functioning of the system. (See, for example, calls for more systematic evaluations of concepts of operation and function allocation in air transportation by a National Research Council, 2003, report and the wide range of criteria proposed for evaluating function allocations reviewed by Dearden et al., 2000, which reflects the broader system design considerations within which functions allocations are created.)

The value of examining function allocations early in design, that is, before technologies and interfaces have been created, may also be argued in terms of cost and development risk. Function allocation decisions made early in design can be used as a cost-effective method for defining the specifications for technologies and to identify the information and interactivity specifications for interfaces; this method minimizes the risk of significant problems being identified later. In contrast, once significant effort has been placed in creating these technologies—sufficient for human-in-the-loop evaluations, for example—fundamental problems with function allocation pose a significant risk to the project, as the cost of redesigning the technologies can be large. In some cases, the cost of redesigning the technologies is considered prohibitive, and cost-benefit analysis dictates that given the sunk costs in technology design and the notional value of a redesign, problems inherent to the system design at the function allocation level should be addressed operationally through procedures and training.

Conclusions

In this paper, we reviewed the fundamental requirements that any function allocation should be designed to meet. Of note, we did not propose methods for deciding upon function allocations, and we did not enter the debate as to whether function allocation is (or should be) considered science, engineering, or a black art (Clegg, Gray, & Waterson, 2000; Sheridan, 2000). Instead, this review focused on a first-principles analysis of the requirements, taken from perspectives examining the needs and capabilities of each team member, of the team as a construct, and of the mission relative work performance.

Several consistent threads were developed throughout this paper’s review of the requirements for effective function allocation. First, the construct of function allocation is not simple; indeed, the challenge of designing human teams is not solved, and the addition of concerns with human–automation interaction further complicates the challenge. As a result, there is no simple solution to the problem of function allocation. For example, design principles dealing with the role of the human and the machine are useful to some degree but further need to address the context of the entire team and its work as a whole, and the collective set of taskwork and teamwork assigned each agent.

Second, many issues with function allocation may be observed only within the dynamics of simulation or actual operations. This is particularly the case when a function allocation does not establish a clear-cut division in the work but instead requires tightly coupled interactions between agents and within work activities. For example, a poorly designed function allocation may artificially create the need for agents to interrupt each other by incoherently breaking up natural divisions within the taskwork; on the other hand, a well-designed function allocation may wisely foster effective interruptions whereby agents with different knowledge know when another agent would like notification. Thus, any particular function allocation may appear to work well in a context with one temporal dynamic and yet be inappropriate in another context.

Third, although some issues with function allocation may be completely described at a high level, such as categorizations of level of automation, other issues can be fully articulated (and predicted) only from a model with sufficient detail to capture specific interactions between agents and the coordinated dynamics of the work and the environment. For example, current-day flight deck function allocations assign all aspects of the trajectory to the autoflight system, with the exception of target altitude, which must be set by the pilot as it is issued by the air traffic controller. Further, such detailed analysis should allow for not only the machine capabilities commonly described by levels of automation but also the detailed human activities that act with it. For example, a function allocation described as a low level of automation may effectively command an action of its human teammate who is too busy or starved for appropriate information to judge whether it is correct; conversely, a function allocation described as a high level of automation may be constantly overridden by a human teammate who elects to do the task himself or herself.

Finally, although function allocations are commonly described as the allocation of authority over the execution of a function, the responsibility for the outcome is also an important construct that, implicitly or explicitly, results in additional monitoring activities for the human supervisor of automation. Thus, mismatches between responsibility and authority are a construct worth identifying in concept. Such mismatches can be particularly important when they infer monitoring activities that, due to competing activities or natural complacency, may not be performed as assiduously as a design might implicitly require.

Each requirement also has implications for how to model function allocations and measures by which they may be assessed. These implications are detailed in the following companion papers.

Footnotes

Acknowledgements

The models and simulation framework discussed here have been developed through cooperative agreements with NASA’s Aviation Safety Program’s Integrated, Intelligent Flight Deck project and System-wide Safety and Assurance project, under the technical supervision of Michael Feary, Paul Schutte, Michael Schafto, and Giullaume Brat.

Karen M. Feigh is an assistant professor in the Georgia Institute of Technology’s School of Aerospace Engineering. She earned a PhD in industrial and systems engineering from the Georgia Institute of Technology in 2008.

Amy R. Pritchett is the David S. Lewis Associate Professor in the Georgia Institute of Technology’s School of Aerospace Engineering and director of the Georgia Tech Cognitive Engineering Center. She earned an ScD in aeronautics and astronautics from MIT in 1997.

References

Bailey

R. W.

(1982). Human performance engineering. Englewood Cliffs, NJ: Prentice Hall.

Bainbridge

(1983). Ironies of automation. Automatica, 19, 775–779.

Bass

E. J.

Pritchett

A. R.

(2008). Human–automated judge learning: A methodology for examining human interaction with information analysis automation. IEEE Transactions on System Man and Cybernetics, Part A: Systems and Humans, 38, 759–776.

Billings

(1997). Aviation automation: The search for a human-centered approach. Mahwah, NJ: Lawrence Erlbaum.

Chialastri

Pozzi

(2008). Resilience in the aviation system. Lecture Notes in Computer Science, 5219, 86–98.

Christoffersen

Woods

D. D.

(2002). How to make automated systems team players. In Salas

(Ed.), Advances in human performance and cognitive engineering research (Vol. 2, pp. 1–12). Bingley, UK: Emerald Group. doi:10.1016/S1479-3601(02)02002-7

Clegg

Gray

Waterson

(2000). The “charge of the byte brigade” and a socio-technical perspective. International Journal of Human-Computer Studies, 52, 235–251.

Dahm

(2010). Technology horizons: A vision for air force science and technology during 2010–2030 (Tech. Rep. AF/ST-TR-10-01-PR). Washington, DC: Office of the USAF Chief Scientist. Retrieved from http://www.af.mil/shared/media/document/afd-100727-053.pdf

Dearden

Harrison

Wright

(2000). Allocation of function: Scenarios, context and the economics of effort. International Journal of Human–Computer Studies, 52, 289–318.

10.

Dekker

Woods

D. D.

(2002). MABA-MABA or abracadabra? Progress on human–automation co-ordination. Cognition, Technology & Work, 4, 240–244.

11.

Dorneich

M. C.

Ververs

P. M.

Mathan

Whitlow

Hayes

C. C.

(2012). Considering etiquette in the design of an adaptive system. Journal of Cognitive Engineering and Decision Making, 6, 243–265.

12.

Endsley

Kiris

(1995). The out-of-the-loop performance problem and level of control in automation. Human Factors, 37, 381–394.

13.

Entin

E. E.

Entin

E. B.

(2001, June). Measures for evaluation of team process and performance in experiments and exercises. Paper presented at the 6th International Command and Control Research and Technology Symposium, Annapolis, MD. Retrieved from http://aptima.biz/publications/2001_EntinEE_EntinEB.pdf

14.

Feigh

K. M.

(2011). Design of a cognitive work support system for multiple cognitive states. Journal of Cognition Technology and Work, 13, 259–279.

15.

Feigh

K. M.

Pritchett

(2006). Contextual control modes during an airline rescheduling task. In Proceedings of the Human Factors and Ergonomics Society 50th Annual Meeting (pp. 457–461). Santa Monica, CA: Human Factors and Ergonomics Society.

16.

Feigh

K. M.

Pritchett

A. R.

(2010). Modeling work for cognitive work support system design in operational control centers. Journal of Cognitive Engineering and Decision Making, 4, 1–26.

17.

Fitts

(Ed.). (1951). Human engineering for an effective air-navigation and traffic control system (DTIC Accession No. ADB815893). Washington, DC: National Research Council, Committee on Aviation Psychology.

18.

Hollenbeck

J. R.

Ilgen

D. R.

Sego

D. J.

Hedlund

Major

D. A.

Phillips

(1995). Multilevel theory of team decision making: Decision performance in teams incorporating distributed expertise. Journal of Applied Psychology, 80, 292–316.

19.

Hollnagel

(1993). Human reliability analysis: Context and control. London, UK: Academic Press.

20.

Hollnagel

(2002). Time and time again. Theoretical Issues in Ergonomic Science, 3, 143–158.

21.

Hollnagel

(2004). Barriers and accident prevention. London, UK: Ashgate.

22.

Hollnagel

Woods

D. D.

Leveson

(2006). Resilience engineering: Concepts and precepts. Aldershot, UK: Ashgate.

23.

Hutchins

(1995). Cognition in the wild. Cambridge, MA: MIT Press.

24.

Jordan

(1963). Allocation of functions between man and machines in automated systems. Journal of Applied Psychology, 47, 161–165.

25.

Kirlik

(1993). Modeling strategic behavior in human-automation interaction: Why an “aid” can (and should) go unused. Human Factors, 35, 221–242.

26.

Lee

Moray

(1992). Trust, control strategies and allocation of function in human-machine systems. Ergonomics, 35, 1243–1270.

27.

Lee

J. D.

(2006). Human factors and ergonomics in automation design. In Salvendy

(Ed.), Handbook of human factors and ergonomics (pp. 1570–1596). Indianapolis, IN: Wiley.

28.

Lee

J. D.

See

K. A.

(2004). Trust in automation: Designing for appropriate reliance. Human Factors, 46, 50–58.

29.

Miller

C. A.

Parasuraman

(2007). Designing for flexible interaction between humans and automation: Delegation interfaces for supervisory control. Human Factors, 49, 57–75.

30.

Molloy

Parasuraman

(1996). Monitoring an automated system for a single failure: Vigilance and task complexity effects. Human Factors, 38, 311-322.

31.

Muir

(1994). Trust in automation: Part I. Theoretical issues in the study of trust and human intervention in automated systems. Ergonomics, 37, 1905–1922.

32.

Muir

Moray

(1996). Trust in automation: Part II. Experimental studies of trust and human intervention in a process control simulation. Ergonomics, 39, 429–460.

33.

National Research Council. (2003). Securing the future of U.S. air transportation: A system in peril. Washington, DC: National Academies Press.

34.

Norman

D. A.

(1990). The “problem” of automation: Inappropriate feedback and interaction, not “over-automation.” Philosophical Transactions of the Royal Society of London, B, 327, 585–593.

35.

Ockerman

Pritchett

(2000). A review and reappraisal of task guidance: Aiding workers in procedure following. International Journal of Cognitive Ergonomics, 4, 191–212.

36.

Palmer

Degani

(1991). Electronic checklists: Evaluation of two levels of automation. In Jensen

R. S.

(Ed.), Proceedings of the Sixth Symposium on Aviation Psychology (pp. 178–183). Columbus: The Ohio State University.

37.

Palmer

Rogers

Press

Latorella

Abbott

(1995). A crew-centered flight deck design philosophy for high-speed civil transport (HSCT) aircraft (NASA Technical Memorandum 109171). Hampton, VA: NASA Langley Research Center. Retrieved from http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.51.1426

38.

Parasuraman

Riley

(1997). Humans and automation: Use, misuse, disuse, abuse. Human Factors, 39, 230–253.

39.

Parasuraman

Sheridan

T. B.

Wickens

C. D.

(2000). A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics–Part A: Systems and Humans, 14, 286–297.

40.

Pitt

(2004). Digital blush: Towards shame and embarrassment in multi-agent information trading applications. Cognition, Technology & Work, 6, 23–36.

41.

Pritchett

(2005). Reviewing the role of cockpit alerting systems. In Harris

Muir

H. C.

(Eds.), Contemporary issues in human factors and aviation safety (pp. 47–80). London, UK: Ashgate.

42.

Pritchett

(2009). Aviation automation: General perspectives and specific guidance for the design of modes and alerts. Reviews of Human Factors and Ergonomics, 5, 82–113.

43.

Pritchett

(2010). The system safety perspective. In Salas

& Maurino

(Eds.), Human factors in aviation (2 ed, pp. 65–94). London: Academic Press.

44.

Rasmussen

(1983). Skills, rules, and knowledge; Signals, signs, and symbols, and other distinctions in human performance models. IEEE Transactions on Systems, Man and Cybernetics, 13, 257–266.

45.

Reason

(1987). Cognitive aids in process environments: prostheses or tools? International Journal of Man-Machine Studies, 27, 463–470.

46.

Sarter

N. B.

Woods

D. D.

(1997). Automation surprises. In Salvendy

(Ed.), Handbook of human factors and ergonomics (pp. 1926–1943). Indianapolis, IN: Wiley.

47.

Sheridan

T. B.

(2000). Function allocation: Algorithm, alchemy or apostasy? International Journal of Human–Computer Studies, 52, 203–216.

48.

Sheridan

T. B.

(2008). Risk, human error, and system resilience: Fundamental ideas. Human Factors, 50, 418–426.

49.

Sheridan

T. B.

Verplank

W. L.

(1978). Human and computer control of undersea teleoperators (Tech. Rep. 780815025). Arlington, VA: Office of Naval Research. Retrieved from http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA057655%26Location=U2%26doc=GetTRDoc.pdf

50.

Speier

Valacich

Vessey

(1999). The influence of task interruption on individual decision making: An information overload perspective. Decision Sciences, 30, 337–360.

51.

Storey

N. R.

(1996). Safety critical computer systems. Boston, MA: Addison-Wesley Longman.

52.

Wiener

(1985). Beyond the sterile cockpit. Human Factors, 27, 75–90.

53.

Wiener

E. L.

Curry

R. E.

(1980). Flight-deck automation: Promises and problems. Ergonomics, 23, 995–1011.

54.

Woods

(1985). Cognitive technologies: The design of joint human–machine cognitive systems. AI Magazine, 6(4), 86–92.

55.

Woods

D. D.

Hollnagel

(2006). Joint cognitive systems: Patterns in cognitive systems engineering. Boca Raton, FL: CRC Press.

56.

Zellmer-Bruhn

(2003). Interruptive events and team knowledge acquisition. Management Science, 49, 514–528.