Abstract
Objective:
We investigated the effectiveness of a theoretically motivated intervention at reducing procedural errors in train yard operations, where making these errors could be costly or life-threatening.
Background:
A widespread and persistent memory error that people commit on a daily basis is the postcompletion error (PCE), that is, forgetting to complete the final step of a procedural task in which the final step occurs after completion of the task goal. PCEs occur in the railroad industry when a locomotive conductor changes the direction of a rail switch but fails to report this change. This particular error could contribute to unsafe conditions as another train traveling on the same track could derail. Although training can help reduce some of the factors leading to unsafe conditions on the rail, research has demonstrated that PCEs are different from other errors of omission in that they cannot be eliminated through training, which makes them a difficult problem to address.
Method:
Twenty-eight undergraduates completed train yard tasks within a mid-fidelity simulator used in commercial rail operations for training. Each participant received the behavioral intervention in one block and no intervention in another. Specifically, participants were required to perform an additional task designed to remind participants of the postcompletion step.
Results:
Our intervention significantly reduced PCE rates in the context of train yard operations, on average by 65%.
Conclusion:
We discuss implications of the effectiveness of our intervention at reducing train yard accidents and how this outcome can contribute to the literature on the cause of PCEs.
Application:
In cases such as the railroad industry, where redesigning technology is prohibitive, our behavioral intervention is an effective alternative for significantly reducing PCEs.
Keywords
Introduction
A widespread and persistent error that people commit on a daily basis is the postcompletion (PC) error (PCE; Back, Cheng, Dann, Curzon, & Blandford, 2007; Byrne & Bovair, 1997; Li, Blandford, Cairns, & Young, 2008; Ratwani, McCurry, & Trafton, 2008): driving down the road with the lid to the gas cap inadvertently left open, leaving the original in a photocopier, or walking away from a vending machine with the snack but not the change. PCEs can occur at the final step of a specific type of procedure. Specifically, those procedures require the operator to complete a step after the task goal has been achieved. It may seem plausible that these types of errors are simply the result of some stochastic function of global error rate and are really no more common than other errors of omission; however, this is not the case (e.g., Byrne & Bovair, 1997).
PCEs have a unique set of characteristics that set them apart from other procedural errors and make them a difficult problem to address. For example, even experts make these types of errors (Blandford, 2000). Furthermore, PCEs occur significantly more often than errors at any other step of a procedure (Byrne & Bovair, 1997; Li et al., 2008). Human factors experts have leveraged an understanding of these unique characteristics of PCEs, and the fact that PCEs occur at the last step of a procedural task, to develop a number of interventions intended to reduce PCE rates.
A number of approaches have been proffered to address the PCE problem, none of which completely eliminated the errors. In fact, some interventions failed to reduce PCEs at all. For example, retraining individuals who commit the error (i.e., “blame-and-train” techniques; Byrne & Davis, 2006) was not found to reduce PCEs, which makes sense given the fact that even experts continue to make these types of errors. Similarly, motivational factors, such as reprimands and praise and reinstruction, did not significantly reduce PCE rates relative to a baseline condition (Byrne & Davis, 2006). Additionally, providing a static cue throughout the trial reminding participants to complete the PC step did not lead to a significant reduction in PCEs (Lee, 1992). More promising results were observed after the implementation of interventions designed to protect an operator from task load (Byrne & Bovair, 1997) and interruptions (Li et al., 2008) external to his or her primary task. Although these approaches significantly reduced PCEs, the errors were not completely eliminated. Possibly the most effective approaches revolve around redesigning the technology used to complete the task, creating just-in-time reminders, and redesigning the procedural steps (Byrne & Davis, 2006).
Of course, the ideal solution to the PCE problem is to design it out. For example, reordering procedural steps of a task to suspend task completion until after the PC step is satisfied (e.g., an ATM requiring users to retrieve their ATM card before cash is dispensed) can eliminate PCEs altogether (Blandford, 2000). However, in many instances, a complete redesign of technology would be cost-prohibitive. Fortunately, through the use of an appropriate experimental paradigm, and an understanding of the cognitive constructs associated with forgetting, it is possible to test a theoretically motivated behavioral intervention designed to eliminate PCEs.
Across a number of theories of the PCE, virtually all of them share some common characteristics. One of these characteristics is that task steps before the PC step are cognitively more related with each other than they are with the PC step. Concepts such as “closure” (Thimbleby, 1990; Ziegarnik, 1927), “killoff” (Polson, Lewis, Rieman, & Wharton, 1992), and goal (Byrne & Bovair, 1997), all serve to capture the notion that the PC step is separate from the more tightly associated steps of the primary task. The tenuousness of the connection between task and PC step is also reflected in the contention that the connection is more vulnerable to load than are other associations. Indeed, PCEs would tend to increase with increases in load (e.g., working memory, workload), including interruptions occurring immediately before the PC step.
To illustrate, imagine that you want to make five copies of an original letter. Because of the technology of photocopiers, the step of retrieving the original exists after the five copies are made. Thus, the goal of having five copies is satisfied before the PC step of retrieving the original. At the completion of the goal, the goal may decay, be purged, or be killed off, leaving you with a sense of closure. If you were interrupted as the fifth sheet appeared, the probability that you would leave the original would increase. Because the goal and the task steps are not strongly associated with the PC step, the operator may go on to the next task in the day instead of completing the PC step.
Existing theoretical explanations for the causes of PCEs can be used to develop behavioral interventions to help ensure that errors at the end of a task do not occur. Such interventions would be especially important in safety-critical systems where procedural slips can be life-threatening. For example, in the railroad industry, if a locomotive conductor changes the direction of a rail switch but fails to report this change, another train traveling on the same track could derail (Ranney & Raslear, 2013).
Train yards, which can be thought of as relay stations, are composed of a number of connected tracks that are used for breaking down complete trains into individual units (i.e., blocks) so that the blocks can be connected to other trains to be delivered to their final destination. Where two tracks converge, a track switch is used to change the direction in which a train will travel (see Figures 1 and 2). If trains “run through” improperly lined switches, the switches could be damaged, which increases the likelihood that another train traveling on the same tracks will derail (Ranney & Raslear, 2013).
Screenshot from the TrainMaster virtual environment displaying a locomotive lined to move toward the viewer. The image displays an improperly lined switch, which can occur if a conductor fails to line the switch for their movement. If the conductor were to move the locomotive through the intersection while the switch was lined in this position, it would force the switch point into line (i.e., a run-through switch), resulting in damage to the switch point. Subsequent trains traveling over the switch point are more likely to derail as a result of the damage.
Screenshot from the TrainMaster virtual environment displaying a locomotive lined to move toward the viewer. The image displays a properly lined switch because the locomotive will travel straight through the track intersection without “running through” the switch.
In order to safely operate trains in the train yard, conductors must remain aware of the alignment of track switches across the yard. Although it is often the case that experienced conductors can see the direction in which a switch is lined as they approach it, and therefore can realign the switch if needed, there are many blind curves in train yards that prevent direct line of sight with track switches. For this reason, track switch position can sometimes be difficult to determine. Furthermore, cognitive factors, such as where a conductor’s attention is focused when approaching a switch, could contribute to a failure to detect an improperly lined switch. Therefore, conductors communicate with yardmasters who are responsible for assigning train yard tasks and communicating track conditions to employees working in the train yard.
Although conductors are trained to report the position of track switches to the yardmaster after they have completed a main task of building or breaking down a train (i.e., a PC step), this final step is often forgotten. In discussions with conductors, we have received numerous reports of conductors forgetting to realign switches and forgetting to report the position of switches. In one interview (Durso et al., 2014) a conductor told of an instance in which he aligned a switch against himself to aid a conductor on a neighboring track, forgot he had realigned the switch, and then ran through the switch that he personally had altered 30 min earlier. If conductors forget to report the position of track switches to the yardmaster after completing a main task (i.e., a PCE), other conductors working on the same set of tracks may lose awareness of track switch positions. The actual run-through switch is not the PCE; however, the PCE of forgetting to communicate switch position can set the condition for the run-through switch. In general, a lack of communication in the train yard has been identified as a contributing factor of run-through switches (Ranney & Raslear, 2013). In the current study, we examine a feasible intervention for reducing PCEs in train yard operations and thus reducing the number of situations in which a run-through switch might occur.
In the case of the railroad, introducing new technology to facilitate remembering the critical steps or redesigning mechanical switches would be prohibitive due to various system-level factors, including, of course, fiscal ones. Therefore, we propose that a behavioral intervention designed to shift conductors’ attention back to the PC step at the time the error is likely to occur could help ameliorate the PCE problem. We accomplish the attention switch to the PC step by making a reasonable assumption that conductors begin thinking about their next task goal prior to completing the PC step.
Memory research has demonstrated that an effective way to remind individuals about items that they have forgotten is through the use of retrieval cues (Tulving & Pearlstone, 1966). In the case of train yard operations, it may be possible to utilize associative cuing from their next task—which the conductor will naturally consider—to direct attention back to the PC step of communicating the switch position for the previous task, thus increasing the probability that track switches are reported after the completion of each task. Our intervention accomplishes this by changing the way in which conductors receive their daily tasks.
Normally, conductors are given all of their assignments for the day at the beginning of the shift. For our intervention, participants were given only one task at a time, thus necessitating that they call the yardmaster near the end of their current task (i.e., before moving their train to the departure track) to obtain their next train yard assignment. One might worry that giving the conductors another task after the main task would only present another opportunity for a PCE. However, the fact that conductors know they are still on the job and require additional assignments places this request outside the normal space of PCEs. Thus, barring the very last switch of the conductor’s shift, the obligation to consider the next assignment could implicitly cue the PC step of the previous assignment.
Although conductors were forced to move on to the next task when PCEs were likely to occur, we assume that the act of calling the yardmaster to receive the next assignment acted as a retrieval cue, reminding the participant to also communicate the position of switches (i.e., the PC step) for the almost-complete current task. In other words, because the acts of calling in for the next assignment and the PC step of calling in the switch positions for the current task shared many characteristics (e.g., communicating to the yardmaster, considering switch positions required to complete the next task), the goal of completing the former shared a strong cognitive link with the goal of completing the latter. We predict that forcing workers to move on to the next task via a step that is cognitively linked to the PC step of the current task will significantly reduce PCEs.
Although PCEs occur at a higher rate than most errors of omission, errors are generally a rare occurrence and are therefore extremely difficult to capture in a natural environment. Therefore, researchers typically construct laboratory tasks designed to artificially inflate error rates, which brings into question how likely the observed results will replicate in the real environment. In the current study, we constructed a scaled-down version of a representative task carried out in a typical train yard. Fortunately, we observed more PCEs than one would see in the field for a variety of reasons, including that participants were less experienced than rail conductors and were new to the simulator and to the task. The relatively high PCE rate allowed us to reliably measure error rates without artificially eliciting errors (e.g., by interrupting participants immediately before the PC step with a “false-completion signal”; Byrne & Bovair, 1997; Li et al., 2008), while providing enough data points to analyze the effect of the intervention at reducing those rates.
Method
Participants
Thirty-three volunteers were recruited through flyers posted around the Georgia Institute of Technology (Georgia Tech) campus and through the Georgia Tech Sona Experiment Management System. Participants who completed the study received a total of $50.00 for their participation. Four participants did not complete the study and were not included in the analysis. One participant was accidentally given incorrect instructions on the 2nd day of trials and was excluded from the analysis. Thus, 28 participants completed the study.
Design
The experiment was a within-subjects design, with two levels of the independent variable. The independent variable was whether or not participants were required to request their next task from the yardmaster (intervention, yes; control, no). Conditions were counterbalanced across participants. The dependent variable was PCE rate, which was obtained by dividing the total number of opportunities to make a PCE in one session (i.e., up to four opportunities to make a PCE) by the total number of PCEs made in the session (i.e., zero to four PCEs).
Apparatus and Train Yard Tasks
The study took place in the Cognitive Ergonomics Laboratory at Georgia Tech in Atlanta. TrainMaster simulator software was controlled by an Alienware laptop computer with an NVIDIA GeForce GTX 780M Graphics card and an 18.4-in. monitor with a resolution of 1,920 × 1,080 pixels. A USB remote-control device was used to operate locomotives in the virtual environment; a joystick and fixed-function buttons were used to move in the virtual environment and to control various actions needed to perform tasks in the simulator. Instructional slides appeared on a 30-in. monitor driven by an Alienware desktop computer.
TrainMaster simulator and hardware
The software used for each trial was the TrainMaster Rail Operation Simulation program. This mid-fidelity virtual environment is used by railroad companies to simulate various tasks carried out by traditional locomotive engineers and conductors as well as remote-control operators (RCOs), who use remote-control devices to control movement of the locomotive (see Figure 3). The operator control unit (OCU) is used by RCOs in remote-controlled freight rail switchyard operations in place of a locomotive engineer operating the engine. Participants used a simulator analog of an OCU to operate the locomotive in the simulated train yard environment (see Figure 4).
TrainMaster mid-fidelity training environment for remote control operators (RCOs) of locomotives. Screenshot displays a train yard within the virtual environment. The front car on the track closest to the viewer is selected, and information specific to that car, as well as task-related information, is displayed below the train yard environment.
Simulator version of the operator control unit (OCU) used by remote control operators (RCOs) to control movement of locomotive in TrainMaster simulator.
Main task development
Two categories of simulator train yard tasks were developed for this study: (a) building a train out of blocks of train cars (i.e., “Building Task”) and (b) breaking down a train into individual blocks of train cars (i.e., “Breaking Down Task”). These task categories were chosen to represent the types of jobs performed in a train yard.
See Table 1 for the steps required to complete the Building Task. For the Breaking Down Task, participants deconstructed complete trains into two blocks of cars and deposited two blocks of train cars onto two classification tracks (see Figure 5).
List of Steps Required to Complete the Simulator Building Task
Note. An additional step (i.e., “Communicate the position of switches in the train yard”) was imposed on participants after the main task of building a train had been completed. This final step is the postcompletion step. OCU = operator control unit.
Track diagram of simulator train yard. Initial positions of blocks of cars are displayed on Tracks 303, 304, 305, and 306. Classification tracks (i.e., Tracks 303, 304, 305, and 306) are outlined by a dotted line at the top of the figure, and the departure track (i.e., Track 312) is outlined by a dotted line at the lower right of the figure. The location of the track switch connecting the classification tracks to the departure track is identified with a circle and label.
Four tasks were created for each category, for a total of eight tasks, with variations in specifics of the tasks across the four representative tasks. Participants referred to task assignment sheets (see Figure 6 for an example) to know which cars to separate to break down a block and a train car list to know where those cars were located within the respective block.
An example of a task assignment sheet used during practice and experimental trials. Participants referred to task assignment sheets to build and break down trains within the simulated train yard environment.
PC step and PCEs
As illustrated in Table 1, each of the two main tasks can be thought of as a series of subtask steps. The final subtask step for each of the main tasks was to communicate the final position of the switches (i.e., the PC step). In the field, RCOs are instructed to communicate to the yardmaster (who is in charge of the activated remote-control zone) the position in which they left each of the track switches. In this experiment, participants accomplished this step by saying out loud, “Reporting the position of track switches” when they felt that they had accomplished the train yard task. If participants failed to report the position of track switches at the end of a task, a PCE was recorded for that task. Therefore, at the end of every task, there was an opportunity for participants to make a PCE.
Procedure
Training
The first two sessions (i.e., first 2 days of 1-hr sessions) were training sessions (see Table 2 for a complete list of activities included in these training sessions).
The First 2 Days of the Experiment in Which Participants Were Trained on the Simulator and the Train Yard Tasks
Following simulator controls training, participants were trained on each of the two train yard task categories (i.e., building a train and breaking down a train) using a felt board replica of the train yard environment (including train tracks, blocks of train cars, locomotive, and track switches). The felt board served as a training aid, allowing experimenters to teach participants the overall concept of building and breaking down trains from a “bird’s-eye” view of the train yard, which is quite large. This procedure was important because the simulated train yard environment is viewed from a first-person perspective, and we wanted to ensure that participants gained an understanding of the overall layout of the yard before beginning experimental trials. Furthermore, because participants completed tasks on the felt board, these exercises enabled experimenters to verify that participants understood the train-related tasks before they completed these tasks using the simulator and OCU. This training was important because these task exercises were used to ensure that participants did not commit errors simply because they did not understand what they were expected to accomplish.
On the 2nd (final) day of training, participants completed task exercises within the simulator. Participants completed two to three train yard tasks using the simulator and control devices. All participants completed at least one task of building a train and one of breaking down a train. During training sessions, participants were not trained on the experimental manipulation of requesting their next task.
Feedback during training
Following each of the felt board exercises on Day 1 and the simulator exercises on Day 2, participants were given feedback regarding procedural errors related to the task. Specifically, if participants performed procedural steps out of order (e.g., if they uncoupled a block of cars before applying handbrakes), failed to initialize steps that require initialization (e.g., they moved the OCU speed control from stop to coupling speed without first pressing the vigilance switch), or failed to complete the PC step of reporting track switch positions at the end of the task, then the experimenter provided feedback on these errors. Participants were required to commit fewer than three errors in one task and to report the position of track switches (i.e., the PC step) on at least one of the previously described exercises to move on to trial sessions. It was important to ensure that participants reported track switches at least once during training so that, during trial sessions, failing to report track switches could not be due to participants’ not knowing that this step was necessary. All participants who attended both training sessions met the performance criteria during training and moved on to trial sessions.
Trial sessions
Trial sessions took place on Day 3 and Day 4 of the experiment. Participants completed two 1-hr trial sessions (in addition to 2 days of training) in 1 week. No more than one session was completed in 1 day. Four of the eight tasks were randomly selected for Day 1 trial sessions, and the order in which the tasks were presented on their respective days was determined randomly. Participants were not always able to complete all four tasks in a 1-hr session, so each session comprised two to four train yard tasks. If the 1-hr allotted session time ended while a participant was still completing a task, the experimenter stopped the session by entering the room and informing the participant that the session was over. If the participant was currently working on a task when the session time expired, the task was not scored.
Task assignments were given to participants on sheets of paper. Each participant was in the experimental condition for 1 day of trial sessions and the control condition for the other. To control for the possibility of sequence effects, the order in which participants received each condition was counterbalanced.
Experimental condition
In the experimental condition, the experimenter gave participants one task assignment sheet at a time. Specifically, after reading the experimental session instructions, the experimenter gave the participants the first task assignment sheet and informed them that they would be given one task assignment sheet at a time. The experimenter instructed participants to report completion of the task by saying out loud, “Reporting completion of task. Requesting next task.” Although the experimenter instructed participants to report the completion of a task (which includes completing the PC step) after they had completed it, many participants did so before completing the PC step. We discuss possible explanations for this observation in the Discussion section of this paper. After hearing the request, the experimenter entered the room and gave the next task assignment sheet to the participant.
Control condition
In the control condition, the experimenter gave participants all of their task assignment sheets at the beginning of the session. Specifically, the experimenter read the control session instructions, placed the stack of all four task assignment sheets on the table next to the participant, and informed participants that the task assignment sheets were all of the assigned tasks for the day. The experimenter instructed participants to complete the task assignments and bring the task assignment sheets to the experimenter after they had completed the tasks.
Experimenter role
In our experiment, the experimenter mirrored the role of the yardmaster in the field, as the actions carried out by the experimenter in our study are carried out by the yardmaster in the field.
Recording PC step communications
The experimenter sat directly outside of the experimental room, and the door was left open to ensure that all communications could be heard and recorded. Participants reported position of track switches (i.e., the PC step) by stating out loud, “Reporting position of track switches.” The ease with which conductors in the field report switches via a walkie-talkie radio was mirrored by having the experimenter nearby, although the complexity of the train yard is likely higher in the field. Experimenters recorded the total number of train yard tasks completed during the session and the number of times participants reported the position of track switches. It is important to note that the experimenter was not in the room when participants completed the PC step of stating, “Reporting the position of track switches.” For this reason, it is unlikely that the experimenter served as a memory cue for completing the PC step.
Results
Arcsine-transformed proportions of PCEs were analyzed using SPSS 21.0. Each reported statistic was tested at α = .05. Analysis focused on proportion of PCEs. Differences in the control and experimental conditions were examined using paired-samples t tests. Effect sizes are expressed in both original units and standardized Cohen’s d values. Unless otherwise stated, data from 28 participants were included in analysis.
The number of opportunities to make PCEs differed across participants and across trial sessions because participants completed two, three, or four train yard tasks per 1-hr trial session. However, the number of trials completed in the intervention (M = 3.50, SD = 0.75) and control (M = 3.43, SD = 0.69) conditions were similar, t(27) = 0.37, p = .711. See Table 3 for mean number of opportunities to make PCEs, mean number of PCEs made, and mean proportion of PCEs in original units for the control and intervention conditions. Also presented in Table 3 are mean proportions of PCEs in transformed data values for each condition.
Opportunities to Make Postcompletion Errors (PCEs), Number of PCEs Made, and Proportion of PCEs for the Control and Intervention Conditions Expressed in Original Units
Note. Proportion of PCEs for the control and intervention conditions after arcsine transformation.
PCE rate was substantially higher in the control (M = 0.69, SD = 0.43) than in the intervention (M = 0.24, SD = 0.41), t(27) = 5.37, p < .001, d = 1.06, 95% CI [.27, .62]. Furthermore, analysis on transformed data confirmed the significant difference in PCEs between the control and intervention conditions, t(27) = 5.37, p < .001, d = 1.06, 95% CI [.17, .67]. Thus, the intervention across all participants reduced PCE rates by 65%. When we excluded the seven participants who did not make an error in the control condition (and therefore did not need an intervention), the means were 0.92 (SD = 0.17) and 0.33 (SD = 0.45), for control and intervention respectively, a drop of 0.59 points and a percentage reduction similar to the overall group (64%).
Although conditions were counterbalanced across participants, data were subjected to a repeated-measures ANOVA to test for possible sequence effects. The effect of intervention order on PCE rate was not significant (p = .137), and neither was the Intervention Order × Intervention Condition interaction (p = .090). Despite the lack of an interaction, we tested whether the main effect manifested at each order. PCE rate was statistically lower in the intervention condition compared with the control whether the intervention occurred first (intervention, M = 0.07, vs. control, M = 0.66), t(13) = 4.73, p < .01, or second (intervention, M = 0.42, vs. control, M = 0.72), t(13) = 2.93, p < .05.
Discussion
Our hypothesis that requiring participants to call experimenters near the end of a task to request their next assignment would result in a reduction in PCEs was supported by the intervention reducing PCE rates by 65%. Because our intervention moved participants on to a subsequent task before the PC step of the current task was performed, one might think that the final step of tasks would easily be omitted. However, it was not.
We suggest that the significant decrease in PCE rates occurred because the additional step was related to the PC step of the current task in that participants used the same communication method (i.e., they reported to the experimenter) to complete this additional task as they had for the PC step of the current task. In this way, the new first step of a task (i.e., requesting the next task from the experimenter) served as a reminder to complete the final step of the current task (i.e., report the position of track switches). In other words, the additional step served as a retrieval cue to complete the possibly forgotten PC step.
Previous studies have identified task interruptions (Li et al., 2008) and high operator workload (Byrne & Bovair, 1997) as contributing to the PCE problem. However, in these previous studies, high PCE rates were observed even when task interruptions and additional tasks were removed, suggesting that there are other unidentified factors that contribute to the systematic PCE problem. We believe that PCEs were observed in the absence of secondary tasks and interruptions because users began thinking about their next task before completing the current task, contributing to their tendency to abandon any unsatisfied steps in the current task. Our intervention facilitates remembering the intention to complete the final step of a procedure by pointing conductors to the last step of the previous task via a retrieval cue.
One alternative explanation for the observed difference in PCE rates in the control and experimental conditions would be that participants in the control condition might have read all of the task assignments at the beginning of the session and prior to beginning their first task, which could result in overloading their working memory. In other words, the relatively high PCE rates for the control condition could have been due to the high workload that participants may have imposed on themselves by activating future goals in memory, which, as Byrne and Bovair (1997) would suggest, interferes with the goal of completing the current task. However, there are two reasons that high workload could not be the only factor contributing to high PCE rates.
First, if participants in the control condition experienced higher workload due to holding multiple upcoming tasks in working memory, it is likely that PCE rates would be the highest for the first task, when participants had three remaining tasks to complete, and would decrease the further along they were in the session, as there would be fewer future task assignments to interfere with the current task. However, we did not observe such a trend of decreasing PCE rates on later tasks. Second, according to Byrne and Bovair’s (1997) workload-based PCE explanation as it is currently presented, after the goal for the current task had dropped below threshold in a high-workload situation, it would not be possible to recover the uncompleted steps. However, this was clearly not the case in our study. Therefore, we do not believe that simply overloading participants in the control condition by giving them four task assignments at the same time is the most likely explanation.
Although forcing participants to request their next task before moving on with the study significantly reduced PCEs, it might seem like a logical next step to simply force participants to report the position of track switches instead. This modification could be thought of as a behavioral analog to functionality built into technology that prevents the user from moving forward without completing the PC step (Blandford, 2000). However, in our train yard operations example, this modification would require the yardmaster to ensure that conductors report track switches before giving conductors their next assignment, which would create another opportunity for error, this time on the part of the yardmaster. Because our intervention forces conductors to slow down and also serves as a reminder to complete the current task, it addresses the PCE problem at the level of the user and does not rely on the yardmaster, who is removed from the actual work of building and breaking down trains, to correct the problem.
As our results would suggest, retrieval cues can be used to ensure that valuable steps to promoting the safe operations of trains in a train yard, such as reporting the position of track switches, will be completed by conductors. Furthermore, even if contributory factors, such as high workload or task interruptions, are present at the time of the PC step, possibly contributing to the likelihood that the PC step will be forgotten, it is possible that the PC step is still likely to be recovered with our intervention, as conductors are cued to revisit the final step after they have completed the main task. Although previous theories would suggest that interruptions and high task load inevitably increase PCEs, if designed in the right way, interventions that leverage the operator’s ability to revisit previous tasks may be a good solution for ensuring that critical steps are remembered.
Our participants were undergraduate students, which always brings into question the transferability of results to the target population. However, although our sample was not taken from the target population, the fact that the effect of the intervention on PCE rates was robust to participants who performed poorly on the main train yard tasks is important because it speaks to the overall effectiveness of the intervention even though the main task may have been more difficult for some than for others.
We are aware that our intervention would be introduced into a complex system of systems. Taking a human–systems integration approach (Boehm-Davis, Durso, & Lee, in press) suggests that it is worth considering the trade-offs or side effects of the intervention on other elements of the system. For example, one possible trade-off in our case is that the workload of the yardmaster may increase. This trade-off may be made minimal with appropriate design and policy decisions. Alternatively, a decision would have to be made as to whether the increased safety of fewer run-through switches is worth the increased workload or increased manpower.
Although our intervention did not completely eliminate PCEs for every participant, the significant reduction in average PCE rate should greatly contribute to a decrease in run-through switches in general. We do not believe that failing to report the position of track switches is the sole cause of run-through switches; however, when this error occurs in conjunction with other problematic system-level factors, run-through switches are likely to occur. In other words, especially when other factors that contribute to run-through switches are present, our behavioral intervention can reduce the likelihood that a catastrophic event will occur. Because PCEs have been found to be a particularly difficult issue to address (Blandford, 2000), our intervention could go a long way in increasing the resilience of safety-critical systems, especially when implemented along with other systemwide preventative measures. Finally, not only is the intervention effective at reducing PCEs, it does not require any changes to technology, which would be prohibitive in train yard operations.
Key Points
Postcompletion errors (PCEs) are systematic, widespread, and persistent memory errors that people commit on a daily basis.
PCEs can be particularly problematic in safety-critical systems, where making these errors could be life-threatening.
Behavioral interventions are an effective option for significantly reducing PCEs when technology redesigns are prohibitive.
Footnotes
Acknowledgements
This work was supported in part by the Sigma Xi Scientific Research Society–GIAR (Grant No. G2013164171). The authors would like to thank CSX Transportation, and especially Kenneth Glover and Emma Mosier, for coordinating sessions with subject matter experts.
Joseph D. McDonald, MS, is a graduate student in the engineering psychology PhD program at the Georgia Institute of Technology. He received his BA from Southwestern University and his MS in engineering psychology from the Georgia Institute of Technology in 2014. His research focuses on understanding how organizational, team-level, and individual-level factors interact with technology to bring about system-level changes.
Francis T. (Frank) Durso is a professor in the engineering psychology program at the Georgia Institute of Technology. He received his BS from Carnegie-Mellon University and his PhD in learning/cognitive processes from the State University of New York at Stony Brook in 1980. He is a past president of the Human Factors and Ergonomics Society and a member of the NRC Board of Human Systems Integration. His current research interests focus on cognitive factors used to manage dynamic situations in domains such as aviation, rail, and health care.