Hangar Talk Survey

Abstract

Objective:

The current study developed an online hangar talk survey (HTS) to solicit narratives describing challenging scenarios that professional pilots encountered during the hours-building phase of their career.

Background:

The predicted pilot shortage will effectively reduce the minimum flying hours required for pilots to be hired at an airline, resulting in less opportunity to develop nontechnical skills naturalistically. To compensate, threat and error data from the hours-building phase of a pilot’s career are required to inform training development. Pilots often share stories of such experiences, colloquially termed “hangar talk.”

Method:

The HTS gathered 132 narrative descriptions of general aviation (GA) events from pilots along with the event’s impact and whether the pilots would react differently if the scenario were encountered again.

Results:

The distribution of threats reported by GA pilots was similar to that reported at the airline level. Logistic regression analysis revealed that decision-making errors were associated with recognition of the need to react differently in the future, and decision-making errors and proficiency errors were associated with greater perceived impact on skill development.

Conclusion:

The current HTS solicited an array of data similar to the findings of airline-based threat and error observations. Pilots perceive decision-making and proficiency errors as impactful on skill development.

Application:

An HTS can be used to gather naturalistic threat and error data and to create a database of operational stories that can be used to develop nontechnical training based on narrative thought.

Keywords

hours-building phase narrative thought nontechnical skills pilot training

Introduction

In North America, professional pilots are not eligible for an interview at a major airline until they have approximately 2,500 to 3,500 hr of flying experience. It can take 4 to 6 years of flying smaller aircraft in general aviation (GA) to accumulate this experience, and for the purposes of this article, GA refers to all flight operations that are not associated with airline or military activities. This stage of a pilot’s career, between the point of receiving a commercial license and being hired by a major airline, can be thought of as the “hours-building” phase. The North American airline industry works with the assumption that through the course of accumulating the flight hours required for an airline interview, a pilot has been naturalistically exposed to enough challenging scenarios to develop sufficient nontechnical skills to be a safe and effective airline pilot. Such nontechnical skills include communication, decision making, workload management, and situation awareness. GA flight operations present a range of challenges to inexperienced pilots, and this hours-building phase of a pilot’s career has historically been effective in developing these skills.

Airlines will face an acute hiring challenge in the near future, given the predicted global pilot shortage attributable to projected retirements and forecasted growth within the industry (International Civil Aviation Organization, 2011). Increased demand for pilots will eventually require major airlines to reduce their minimum hour requirements for hiring, which will necessarily decrease pilots’ naturalistic exposure to operational scenarios that build nontechnical skills. Significant research has been conducted to explore the types of threats and errors that pilots encounter after being hired in airline operations (Klinect, Wilhelm, & Helmreich, 1999; Thomas, 2004). However, little is known about the types of threats and errors pilots encounter during the hours-building phase of their career. Threats are conditions that have the potential to negatively affect the safety of a flight, whereas errors refer to a crew’s response, or lack of response, that diverges a flight from its expected or intended course (Thomas, 2004). With fewer opportunities to develop nontechnical skills naturalistically, it is important to examine the threats and errors pilots encounter within GA so that new instructional methods may be developed to augment the development of nontechnical skills.

Nontechnical Pilot Training

The aviation industry generally accepts that pilots require a combination of two types of skills: technical and nontechnical. Specific technical skills, also called hard skills, are those required to safely maneuver an aircraft within standard operations. Technical skills require the psychomotor attributes necessary to maneuver flight controls and the cognitive ability to apply operational and regulatory knowledge to flight-specific tasks. Generic nontechnical skills, also called soft skills, allow pilots to apply broad interpersonal and cognitive skills to unusual and complex situations (Thomas, 2004). Nontechnical skills are often regarded as supplementary safety skills beyond those required for standard operations. Technical skills are observable and therefore comparatively easy to train and evaluate, whereas nontechnical skills are often based on a pilot’s cognitive processing and are therefore unique to each aviator and more difficult to assess.

Historically, aviation training focused exclusively on the development and evaluation of technical pilot skills. The first nontechnical training program, initially called Cockpit Resource Management (CRM), was introduced in 1979 in response to several high-profile accidents caused by pilot error (Helmreich, Merritt, & Wilhelm, 1999). The late 1990s marked a move, led by the University of Texas Human Factors Research Project, toward threat and error management (TEM) training that was more operationally focused than was early CRM (Helmreich et al., 1999; Klinect, 2005; Li & Harris, 2006). The TEM approach is based on the premise that human error is a natural component of normal operations, shifting the goal of nontechnical training away from the elimination of error and toward the management of errors (Maurino, Reason, Johnston, & Lee, 1995). An important assumption of the TEM approach is that pilots encounter threats and commit errors on a daily basis. Although each threat and error has the potential to lead to an accident or incident, the vast majority are effectively managed by the flight crew.

The Line Operations Safety Audit (LOSA) was developed to gather data about the types of threats and errors normally encountered in airline operations (Helmreich, Klinect, & Wilhelm, 2001). LOSA uses cockpit observations of normal operations to provide proactive evidence of safety performance (Klinect, Murray, Merritt, & Helmreich, 2003). To conduct a LOSA, a trained observer sits in an aircraft jump seat, which is a third seat located behind and between the seats of the captain and first officer, and records threat and error data observed during the course of a normal flight (Klinect, 2005). LOSAs are a widely accepted method of collecting threat and error data in the aviation industry (International Civil Aviation Organization, 2002), and the results of a LOSA are often used to inform the instructional development of TEM training for the targeted improvement of nontechnical skills.

Although a LOSA is a valuable tool in collecting TEM data, there are drawbacks to this approach. The cost to train and compensate observers and the expertise required to analyze findings are beyond the capacity of many aviation operations. An alternative approach is a confidential survey, which can also be used to collect data on threats and errors (Helmreich, Wilhelm, Klinect, & Merritt, in press). The strategy of using a survey to collect threat and error data is similar to confidential voluntary nonpunitive incident reporting systems, such as the Aviation Safety Reporting System (ASRS) managed by NASA. ASRS provides aviation employees with a secure and anonymous method of submitting accounts of hazardous situations and unsafe occurrences (Kearns, 2009). Information submitted to ASRS is used to identify systemwide deficiencies and to alert persons who are able to correct them (ASRS, n.d.). However, in the course of normal operations during the hours-building phase of a pilot’s career, it is expected that pilots encounter challenging situations that exercise their nontechnical skills. Most of these events are regarded as thought provoking rather than unsafe. Examples of these events include encountering a heavy workload or unexpected bad weather and making a mistake while operating within GA. In aviation, such events are often shared among pilots while conversing in the hangar and are therefore referred to as hangar talk.

The Current Investigation

Anthropologists long ago identified that across cultures, stories are a popular method of conveying information (Vitz, 1990). Given this popularity, it seems a logical approach for educators to incorporate stories into training. The current investigation explored the educational potential of hangar talk stories as a method of informing TEM training.

Cognitive psychologists have suggested that humans are capable of two separate types of thought processes: paradigmatic and narrative (e.g., Bruner, 1986). Paradigmatic thought deals with logic, theory, arguments, analysis, and empirical discoveries. The majority of training, which presents facts and concepts, is meant to elicit this type of thought. By comparison, narrative thought deals with action, intention, and the human element. Training that presents new knowledge through the use of stories targets narrative thought. When information is presented in a story, it is given context, which allows learners to understand and make sense of new information by relating it to their own previous experiences (Hull, 1993). From this perspective, stories offer far more than entertainment, as they can facilitate narrative thought about instructional objectives. Sarbin (1986) suggested that narrative thought influences human thinking, imagination, and decision making, which are well aligned with the goals of aviation nontechnical training. However, before nontechnical training can be designed to elicit narrative thought, it is important to assess hangar talk data for comparison with data collected through LOSAs to determine whether the data collected through a hangar talk survey (HTS) is operationally similar.

In the current investigation, we developed a confidential online self-report HTS to collect narratives of significant flying events during the hours-building phase of a pilot’s career. Whereas LOSA observations focus on external threats, flight crew errors (including how the error was managed and the eventual outcome), and CRM (Klinect et al., 1999), the current study focused just on external threats and flight crew errors reported in the narratives. Examples of threats included unexpected poor weather or an aircraft system malfunction, and examples of flight crew errors included faulty decision making, intentional noncompliance with a regulation or instruction, and failure to use an appropriate procedure. Moreover, the HTS was developed with applicable LOSA principles: confidential and anonymous data collection, voluntary participation by pilots, data collection targeting safety, a secure and trusted data collection source, trained reviewers, and roundtables to clean data (Klinect et al., 2003). Therefore, the HTS allowed for the broad collection of narrative-based threat and error data across GA operations, which would not be feasible with traditional observational methods.

Method

Participants

Participants were pilots whose experience included having obtained a pilot’s license. Pilots submitted an estimation of their number of flight hours when encountering the scenario (M = 1,028, SD = 1,090) and their total hours of experience (M = 4,377, SD = 5,484). Participation was solicited through aviation organizations and social media utilities, resulting in 704 pilots accessing the survey. The total number of participants who completed the survey was 143, for an overall response rate of 20%. Of these, 11 surveys were not analyzed because they did not provide complete narrative descriptions of GA scenarios, resulting in a total of 132 valid responses. The study was approved by the Research Ethics Review Board at Seneca College, Markham, Ontario.

Materials

The HTS was an online questionnaire developed to collect narratives of scenarios that pilots encountered during the course of normal operations within GA. Data on associations between types of GA operation and threats, errors, and nontechnical skills were addressed in Kearns and Sutton (in press). The current analysis focuses on the distribution of threats and errors within the narratives as well as the impact of the event reported in the scenario on the pilot and the effect of the scenario on the pilot’s future decision making. The relevant survey items for the current investigation included one open-ended narrative question and two closed-ended multiple-choice questions. The survey was presented in English and took approximately 20 to 25 min to complete. The survey was distributed exclusively online, allowing participation from diverse geographic areas.

Before the survey questions were finalized, an early version of the HTS was completed by a focus group of 10 flight instructor pilots. The focus group took approximately 30 min to complete the survey and then discussed the survey as a group to identify issues associated with wording of questions, confidentiality, and motivation. On the basis of this feedback, an incentive was offered to participants (inclusion in a raffle for $500). The relevant survey items were as follows:

Based on your real-world flying experiences, after achieving a commercial pilot’s license but before being employed with a major airline, please think of a situation you have encountered that was outside the limits of standard performance and required you to think creatively to maintain safe flight. The scenario should describe a situation where you were challenged as a pilot and ultimately learned a valuable lesson.

Please rate how impactful this event was on the development of your piloting knowledge and skills, using the following scale: (1) almost no impact, (2) small impact, (3) minor impact, (4) moderate impact, (5) large impact, (6) very large impact.

With the expertise you now possess, would you react any differently if you encountered this situation again? Yes or No.

Scoring the HTS: Identification of Threats and Errors

Each completed survey contained a narrative description of a single scenario and was subjected to a data-cleaning process. Valid survey responses were distributed to three professional pilots who acted as reviewers. All pilot reviewers possessed a minimum of a commercial pilot’s license and flight instructor credentials and were working as full-time flight instructors at the time of the investigation. Reviewers were given written instructions on how to identify threats and errors in the narrative responses. Each reviewer worked independently to identify threats and errors in the survey responses. Within a single scenario, reviewers could identify a single threat, a single error, or a combination of threats and errors. Examples of HTS narratives, altered to protect privacy, are included in Table 1.

Table 1:

Example Hangar Talk Survey Narratives

Primary and Contributing Threats and Errors Identified by Reviewers	Narrative Response
Aircraft malfunction (threat, primary)	I was half an hour into a three hour trip flying solo cross country when I noticed that my attitude indicator was acting strangely. It was showing extreme bank angles when little bank was applied and was generally inaccurate. All my other instruments seemed to be working fine so I relied on them instead. However, I was worried that another instrument would go offline as well so I amended my flight plan and returned home.
Weather (threat, primary) Operational pressure (threat, contributing)	I was flying a single engine airplane in freezing drizzle. My boss strongly suggested that I do the flight, being young, keen, and easily replaceable I did it. He advised me to stay low so that the drizzle would be colder and hence not stick. Bad advice because I picked up a lot of ice very fast. I remembered that the air above was warmer and sure enough a 1000 foot climb put me in warm air where the ice started to sublimate off the airframe. Two lessons learned, being young and keen will not save your life and saying “No” when you know better is the best option. Also, review weather systems and have an escape route planned at all times.
Communication (error, primary) Weather (threat, contributing)	I flew for a single pilot charter company in southern [omitted] early in my career. It was a single engine, day-VFR [visual flight rules] operation which often required conducting flights in coastal [omitted] in marginal VFR weather. On one occasion, in bad visibility and ceiling over water, I was involved in a very near mid-air collision with a float plane. I had transmitted my position and intentions on 126.7 but not on the frequency of a float base nearby. This was a harrowing lesson in loss of situational awareness due to my lack of preparation in knowing the [communication] frequencies and habits of local float traffic and my own task saturation while navigating alone on the coast in bad weather. After that event it became clear that the worsening weather would not allow me to continue to my destination safely. My subsequent decision to divert to an alternate required me to fly at low level over the ocean in a single engine land aircraft. It was not a comfortable feeling. However, I landed safely at my alternate, waited out the weather and headed back to my point of origin.
Procedural (error, primary)	I was ferrying a brand new aircraft home from the factory in [omitted], VFR in March heading to [omitted] where the weather can change rapidly. The “fancy” avionics helped a lot for determining enroute weather and information on alternate airports. I did some training on the aircraft before leaving the factory and felt confident in my navigation, flight planning and weather reports and planned to stop as needed to wait out weather. One such stop was somewhere outside of [omitted]. I landed just before the foul weather came in and parked on the apron into the wind then went inside to find someone who could rent me hangar space for the night. Light snow was expected and I was hoping to avoid de-icing, etc, in the morning! While I was inside, the wind picked up, shifted, the air temperature dropped and light snow began to fall.
	I came out of the office 20 minutes later only to find the plane had spun around 130 degrees and the wing and tail just barely missed hitting a parked truck! After the initial shock and inspection for damage, I realized I hadn’t chocked the wheels nor had I applied the parking brake.
	Lesson re-emphasized to me . . . don’t ever leave your aircraft without properly securing it! Make sure you always install control locks, apply parking brake, tie down and/or chock the tires, close the doors, etc., even if you are leaving the airplane for a few minutes. The damage that could occur when you’re not around will cost much more than the 15 seconds it takes to properly secure your aircraft! Not only is this good airmanship, it shows your employer that you pay attention to details and you respect their equipment.

The length of the narratives varied from a few sentences to several paragraphs. Through the data-cleaning process, reviewers noted that shorter descriptions resulted in the identification of only a primary threat or error, whereas longer or more detailed descriptions allowed reviewers to also identify a contributing threat or error. This process resulted in every survey response being tagged with one or two threats or errors. Therefore, reviewers identified one primary threat or error (mandatory) and one contributing threat or error (if identifiable) for each response.

Once all survey responses had been coded by the three reviewers, it was determined whether or not a consensus had been reached by two of the three reviewers. If a consensus was not reached, reviewers worked with the researcher to identify the threats and errors in a data-cleaning roundtable.

Results

The survey generated 132 valid scenario narratives, although some pilots did not complete the follow-up questions addressing the impact of the event and whether they would behave differently in the future. Therefore, although all 132 scenarios are presented in the frequency data, only the 120 scenarios with complete follow-up responses were included in the logistic regression analysis of the impact of the event, and only 116 responses were included in the logistic regression analysis of whether the pilot would react differently in the future.

Descriptive Analysis

As presented in Table 2, a primary threat or error was identified in all 132 narratives. In addition, a contributing threat or error was identified in 97 narratives. As is customary with LOSA observations from an airline environment, threats and errors were separated for analysis purposes.

Table 2:

Frequency of Primary and Contributing Threats and Errors Reported by Pilots

	Primary		Contributing
Category	Frequency	%	Frequency	%
Threats	92	70	47	48
Errors	40	30	50	52
Total	132	100	97	100

The frequencies and percentages of specific threats reported in the scenarios are presented in Table 3. Primary threats were those identified as the main threat or error in the reported scenario. Contributing threats were identified as a secondary factor in the reported event, although as can be seen by the total number of contributing threats, not all scenarios provided sufficient detail to identify such a factor. The cumulative threats represent the combined total number of instances in which a specific threat was identified as a primary or contributing threat. The Airline column is presented for comparison purposes and shows the percentage of threats identified in an airline LOSA reported by Thomas (2004).

Table 3:

Threats Reported by Pilots in the Hangar Talk Survey and by Thomas (2004)

	Primary		Contributing		Cumulative		Airline^a
Name of Threat	Frequency	%	Frequency	%	Frequency	%	%
Weather	29	32	17	36	46	33	21
Aircraft malfunction	34	37	6	13	40	29	14
Operational pressure	5	5	6	13	11	8	12
ATC command	4	4	3	6	7	5	8
Student-pilot error during instruction	4	4	3	6	7	5	0
Traffic	4	4	2	4	6	4	8
Passenger event	4	4	1	2	5	4	5
Terrain	0	0	6	13	6	4	6
Airport condition	2	2	2	4	4	3	7
Ground handling event	2	2	1	2	3	2	6
Communication	1	1	0	0	1	1	4
Inappropriate SOP	1	1	0	0	1	1	0
Onboard fire	1	1	0	0	1	1	0
Pilot incapacitation	1	1	0	0	1	1	0
Total	92	100	47	99	139	99	91^b

Note. ATC = air traffic control; SOP = standard operating procedure.

Threats reported at the airline level (Thomas, 2004).

Only the top 90% of threats were reported.

The frequency and percentage of errors are presented in Table 4. As with threats, primary errors were those identified as the main error reported in the scenario, and contributing errors were secondary errors within the scenario (if sufficient detail was provided). The cumulative errors represent the combined total number of instances in which a specific error was identified as a primary or contributing error. The Airline column is again presented for comparison purposes, and represents the percentage of errors identified in the LOSA reported by Thomas (2004).

Table 4:

Errors Reported by Pilots in the Hangar Talk Survey and by Thomas (2004)

	Primary		Contributing		Cumulative		Airline^a
Error	Frequency	%	Frequency	%	Frequency	%	%
Decision making	15	38	26	52	41	46	10
Procedural	8	20	10	20	18	20	41
Proficiency	13	33	4	8	17	19	3
Intentional noncompliance	3	4	4	8	7	8	38
Communication	1	3	6	12	7	8	7
Total	40	98	50	100	90	100	99

Errors reported at the airline level (Thomas, 2004).

Logistic Regression

We conducted logistic regression analyses to determine whether specific threats or errors identified in the scenarios predicted pilots’ erception of (a) the impact of the event on the development of their piloting skills and (b) whether they would react differently if the scenario were encountered again.

Impact of event

Pilots rated their perceptions of the impact of the reported event on a 6-point Likert-type scale. For the regression analysis, we created a dichotomous variable by categorizing ratings as either low impact (for responses from 1 to 4) or high impact (for responses of 5 and 6). There were 53 events (44%) rated as low impact and 67 events (56%) rated as high impact.

With the logistic regression, we determined whether any specific threats or errors could predict the pilot’s perception of the impact of the event. As can be seen in Table 3, a number of threats were reported fewer than 10 times, including the threat categories of air traffic control command, student-pilot error during instruction, traffic, passenger event, terrain, airport condition, ground handling event, communication threat, inappropriate standard operating procedure, onboard fire, and pilot incapacitation. To conform to the assumptions of the logistic regression analysis, we used only threats with more than 10 cases reported, resulting in weather, aircraft malfunction, and operational pressure being tested as predictors in the regression. We filtered errors using the same criterion, resulting in the inclusion of decision-making error, procedural error, and proficiency error in the test of the model.

We conducted the logistic regression using a stepwise method and a .05 criterion for entry and removal from the model. The final model included decision error and proficiency error as significant predictors of pilots’ perceptions of the impact of an event. The positive beta values indicate that the presence of a decision or proficiency error was significantly associated with a higher perceived impact of the event. Values for the final regression model are shown in Table 5.

Table 5:

Logistic Regression Analysis for Pilots’ Perception of the Impact of a Reported Event

		95% Confidence Interval for Odds Ratio
Predictor	B(SE)	Lower	Odds Ratio	Upper
Constant	1.17 (0.39)
Decision error	0.79* (1.24)	1.88	4.83	12.45
Proficiency error	0.69** (0.35)	1.01	3.97	15.61

Note. Model χ²(2) = 15.92, p < .001.

p < .01. **p < .05.

Future response to situation

A total of 116 participants responded yes or no to whether they would react differently if encountering the reported scenario again. There were 48 no responses (41%) and 68 yes responses (59%). The logistic regression was conducted with the same stepwise method described earlier and included the same predictor variables: weather, aircraft malfunction, and operational pressure threats, and decision-making, procedural, and proficiency errors. The final model included decision-making error as a significant predictor of whether pilots would be likely to act differently when encountering the same scenario again. Values for the model are shown in Table 6. Note that the negative beta value for decision error indicates that pilots were more likely to judge an event that was attributed to decision error as something they would react differently to in the future versus something they would not react differently to (yes was coded as 1 and no was coded as 2 in the data analysis).

Table 6:

Logistic Regression Model for Pilots’ Judgment of the Appropriateness of Their Response to the Reported Event

		95% Confidence Interval for Odds Ratio
Predictor	B(SE)	Lower	Odds Ratio	Upper
Constant	0.73 (0.25)
Decision error	–0.73* (0.25)	0.09	0.23	0.62

Note. Model χ²(1) = 10.01, p < .01.

p < .01.

Discussion

Using an online survey, 132 pilots provided narrative descriptions of significant flying events encountered during the hours-building phase of their career. Then, three reviewers identified threats and errors reported in the scenarios. A majority of the threats reported were attributed to weather (33%) and aircraft malfunction (29%); a number of other threats were reported, each comprising less than 10% of the total threats. The errors reported were most often attributed to errors in decision making (46%), procedural errors (20%), and proficiency errors (19%). Logistic regression analyses revealed that decision-making and proficiency errors were associated with events that had a higher impact on the pilot’s perception of skill development. In addition, errors in decision making were associated with the belief that the pilot would react differently if a similar scenario were encountered in the future. Overall, although HTS data were based on narratives rather than observations, the distribution of the threats identified during the hours-building phase of a pilot’s career was similar to that identified in the professional airline environment (see Thomas, 2004).

Pilots perceived decision-making errors as an important component of learning during the hours-building phase of their careers, reporting them more often than other errors and more strongly associating them with skill development and the perception that they would alter their future behavior on the basis of the experience. Most models of aviation decision making include a component for recalling a past experience, either in assessing a situation or choosing between options or both (see O’Hare, 2003, for a review). Our data suggest that pilots recognize the impact of decision-making events and place increased value on the learning that results from errors encountered naturalistically. From a training standpoint, it seems that providing junior pilots with the opportunity to commit and learn from decision-making errors would be an important component of TEM training for this demographic.

Proficiency errors, although reported less often in our sample, were also associated with an increased perceived impact on skill development. This finding suggests that gaining awareness of their skill limitations is particularly salient for pilots in this career stage. Indeed, one reason GA pilots are at greater risk for accidents is that their level of confidence may be greater than their level of skill (Trollip & Jensen, 1991). Within TEM training in a flight simulator, it may be beneficial to expose pilots to very challenging flight scenarios so that they can further develop an understanding of the limits of their technical and nontechnical capabilities. When a pilot is at the airline level, proficiency errors are not the most frequently committed error, but they are the most difficult to manage (Helmreich et al., 2001), as reflected in the fact that proficiency errors are associated with 73% of GA accidents and 43% of air carrier accidents (Shappell et al., 2007). Therefore, targeted training that emphasizes a realistic appraisal of skills may result in increased safety even beyond this career phase.

Communication errors were reported at similar rates (about 8%) in the current survey and in the airline LOSA (Thomas, 2004). One might anticipate the percentage of communication errors to be significantly lower within GA than at the airline level, since the majority of HTS responses (76%) described single-pilot operations. However, even in single-pilot operations, pilots reported committing communication errors in their interactions with air traffic control, ground crew, passengers, and company personnel. This finding highlights the importance of practicing communication skills within TEM training.

The types of threats reported by pilots in the narratives were similar to those identified at the airline level in a LOSA (Thomas, 2004). The most commonly identified threats in the current study were weather, aircraft malfunction, and operational pressure, representing 70% of all identified threats. The same three threats, in the same order, were the most commonly identified at the airline level, representing 47% of identified threats (Thomas, 2004). This similarity supports the validity of an HTS for collecting examples of threats, as the current self-reports were well aligned with those observed by experts. It also supports the North American practice of relying on an hours-building phase in GA as preparation for an airline environment, as it appears that during the hours-building phase, pilots do gain real-world experience with the same types of threats encountered in airline flying.

Although HTS-identified GA threats and LOSA-identified airline threats were similar, the distribution of errors between the two data sources differed in some ways. For instance, intentional noncompliance was the most frequently identified type of error in the airline LOSA, representing 38% of identified errors, compared with only 8% in the HTS (Thomas, 2004). Helmreich (2000) suggested that the high percentage of intentional noncompliance errors among airline pilots could be attributable to a belief in personal invulnerability that leads to a disregard for rules. It is unknown whether airline pilots are more likely to demonstrate a disregard for rules compared with pilots during the hours-building phase of their career or whether the self-reporting nature of the HTS resulted in lower self-identification of this type of error. Future research is required to investigate why noncompliance errors differ in the two samples. In addition, pilots reported more proficiency errors in GA (19%) than the small number (3%) observed by Thomas (2004). It is perhaps not surprising that junior pilots, being less experienced and therefore less skilled, would commit more errors of this type.

The similar distribution of threats identified in HTS narratives and observed at the airline level (Thomas, 2004) supports the validity of HTS as an initial step in understanding the experiences of pilots during the hours-building phase. The self-report HTS identified similar threat and error data as were found with the use of LOSAs, but there are some important differences. LOSAs allow for nontechnical training to be operationally specific, a significant enhancement compared with early generations of crew resource management (Helmreich et al., 1999). However, a LOSA requires a trained observer to watch pilots during the course of normal flights and to track the threats they encounter and errors they commit. The cost and complexity of LOSAs have been a prohibitive factor in allowing smaller aviation companies, or those that operate aircraft without jump seats (to accommodate an observer pilot for a typical LOSA), to develop TEM training that is specific to their operations. The usefulness of this methodology is also limited by questions as to whether pilots behave naturally while they are being observed.

An HTS is an affordable alternative methodology for informing TEM training, as it is relatively easy to administer and analyze. TEM that is based on HTS data would not be substantially different from TEM based on a LOSA. A limitation of HTS data is that it is subjective and may therefore be influenced by personal bias or unwillingness to report errors. However, the benefit of an HTS is that it allows all aviation companies to gather operationally specific nontechnical data to inform TEM.

Overall, the HTS offers two important opportunities: (a) the ability to gather naturalistic threat and error data from the hours-building phase of a pilot’s career within GA and (b) the ability to gather narrative data with the potential to inform TEM training with the use of operational stories to target the development of nontechnical skills. Not only do stories provide context that can support learning, but they may also offer insight into a pilot’s cognitive and emotional processes that may not be observable through a LOSA. Future work is required to analyze the effectiveness of TEM training as a replacement for nontechnical skill development via operational experience during the hours-building phase of a pilot’s career. Furthermore, instructional techniques that involve the types of stories gathered by the current survey should be developed and their effectiveness assessed for such nontechnical skill development training.

Key Points

The number of hours pilots must accumulate during the hours-building phase of their career will decrease with the predicted pilot shortage, allowing less exposure to scenarios that build nontechnical skills.

A self-report survey, called a hangar talk survey (HTS), was developed for the gathering of narrative descriptions of threats and errors encountered naturalistically by pilots in general aviation.

HTS is an affordable and easily administered method of gathering data that can be used to inform the development of threat and error management instruction.

Footnotes

Acknowledgements

The authors wish to acknowledge and sincerely thank the staff of Seneca College who assisted with several aspects of this investigation, including the production of the online survey, recruitment, and development of the electronic database. This project was funded by a College and Community Innovation Program grant from the Natural Sciences and Engineering Research Council of Canada.

Suzanne K. Kearns is an assistant professor teaching commercial aviation management within the Dan Program in Management and Organizational Studies at the University of Western Ontario in London, Ontario, Canada. She received a PhD in education from Capella University in 2007.

Jennifer E. Sutton is an assistant professor of psychology at Brescia University College in London, Ontario, Canada. She earned a PhD in psychology from the University of Western Ontario in 2001.

References

Aviation Safety Reporting System. (n.d.). ASRS concept and mission. Retrieved from http://asrs.arc.nasa.gov/overview/briefing/br_1.html

Bruner

(1986). Actual minds, possible worlds. Cambridge, MA: Harvard University Press.

Helmreich

R. L.

(2000). Culture and error in space: Implications from analog environments. Aviation, Space, and Environmental Medicine, 71, 133–139.

Helmreich

R. L.

Klinect

J. R.

Wilhelm

J. A.

(2001). System safety and threat and error management: The line operations safety audit (LOSA). In Jensen

R. S.

(Ed.), Proceedings of the Eleventh International Symposium on Aviation Psychology (pp. 1–6). Columbus: Ohio State University.

Helmreich

R. L.

Merritt

A. C.

Wilhelm

J. A.

(1999). The evolution of crew resource management training in commercial aviation. International Journal of Aviation Psychology, 9, 19–32.

Helmreich

R. L.

Wilhelm

J. A.

Klinect

J. R.

Merritt

A. C.

(in press). Culture, error and crew resource management. In Salas

Bowers

C. A.

Edens

(Eds.), Applying resource management in organizations: A guide for professionals. Hillsdale, NJ: Lawrence Erlbaum.

Hull

(1993). Opening minds, opening doors: The rebirth of American education. Waco, TX: Center for Occupational Research and Development.

International Civil Aviation Organization. (2002). Line operations safety audit (LOSA). Montreal, Canada: Author.

International Civil Aviation Organization. (2011). Global and regional 20-year forecasts: Pilots. Maintenance personnel. Air traffic controllers. Montreal, Canada: Author.

10.

Kearns

(2009). Canadian aviation. Dubuque, IA: Kendall Hunt.

11.

Kearns

Sutton

J. E.

(in press). Exploring the experiences of pilots within Canadian general aviation flight operations. Collegiate Aviation Review.

12.

Klinect

J. R.

(2005). Line operations safety audit: A cockpit observation methodology for monitoring commercial airline safety performance (Unpublished doctoral disertation). University of Texas, Austin. Retrieved from https://www.lib.utexas.edu/etd/d/2005/klinectj57340/klinectj57340.pdf

13.

Klinect

Murray

Merritt

Helmreich

(2003). Line operations safety audit (LOSA): Definition and operating characteristics. In Proceedings of the 12th International Symposium on Aviation Psychology (pp. 663–668). Dayton: Ohio State University.

14.

Klinect

J. R.

Wilhelm

J. A.

Helmreich

R. L.

(1999). Threat and error management: Data from line operations safety audits. In Proceedings of the Tenth International Symposium on Aviation Psychology (pp. 683–688). Columbus: Ohio State University.

15.

W.-C.

Harris

(2006). Pilot error and its relationship with higher organizational levels: HFACS analysis of 523 accidents. Aviation, Space, and Environmental Medicine, 77, 1056–1061.

16.

Maurino

D. E.

Reason

J. T.

Johnston

Lee

R. B.

(1995). Beyond aviation human factors. Aldershot, UK: Ashgate.

17.

O’Hare

(2003). Aeronautical decision making: Metaphors, models, and methods. In Tsang

P. S.

Vidulich

M. A.

(Eds.), Principles and practice of aviation psychology: Human factors in transportation (pp. 201–237). Mahwah, NJ: Lawrence Erlbaum.

18.

Sarbin

T. R.

(1986). The narrative as root metaphor for psychology. In Lanfield

A. W.

(Ed.), Narrative psychology: The storied nature of human conduct (pp. 3–21). New York, NY: Praeger.

19.

Shappell

Detwiler

Holcomb

Hackworth

Boquet

Wiegmann

D. A.

(2007). Human error and commercial aviation accidents: An analysis using the human factors analysis and classification system. Human Factors, 49, 227–242.

20.

Thomas

M. J.

(2004). Predictors of threat and error management: Identification of core nontechnical skills and implications for training systems design. International Journal of Aviation Psychology, 14, 207–231.

21.

Trollip

S. R.

Jensen

R. S.

(1991). Human factors for general aviation. Englewood, CO: Jeppesen Sanderson.

22.

Vitz

P. C.

(1990). The use of stories in moral development: New psychological reasons for an old education method. American Psychologist, 45, 709–720.