Abstract
The use of driving simulators is increasing in clinical settings that provide driving evaluation and rehabilitation. To identify the driving simulator scenarios desired by certified driver rehabilitation specialists (CDRSs), we developed a questionnaire consisting of 22 driving scenario situations. A total of 164 CDRSs rated each situation in terms of its importance for inclusion in simulator-based driving. The four situations they identified as most critical were turning left across oncoming traffic, navigating four-way intersections with traffic lights or signs, driving in multiple lanes with traffic on both sides, and reacting to unexpected events that require emergency braking or aggressive maneuvers to prevent an accident. We conducted exploratory and confirmatory factor analyses to group the 22 driving scenario situations. The model with the best fit included 11 situations forming three factors: (1) Intersections, (2) Roadway and Traffic Conditions, and (3) Environmental Conditions. Future studies should include these factors in driving simulator scenarios and evaluate their clinical efficacy in driving evaluation and rehabilitation.
One of the major limitations of on-road driving evaluations is the lack of control over specific traffic and roadway conditions, making it all but impossible to standardize these evaluations (Kowalski & Tuokko, 2007). Closed-course circuits attempt to improve standardization, but they may have insufficient ecological validity because of restricted ability to build in many of the traffic conditions that are critical to accurately assess in evaluating driving skills (Kowalski & Tuokko, 2007). Driving simulators are therefore increasingly used in clinical research settings to investigate driving ability and factors affecting driving performance and to evaluate training program outcomes in a variety of patient populations. Although driving simulators have been used primarily in research settings to date (Fisher, Rizzo, Caird, & Lee, 2011), the use of driving simulators is also increasing in clinical settings that provide driving evaluation and rehabilitation (Korner-Bitensky, Bitensky, Sofer, Man-Son-Hing, & Gelinas, 2006).
Using simulators for driving evaluation and rehabilitation eliminates the risk of crash injuries inherent in on-road alternatives. Additional advantages of using a driving simulator include the following: A variety of real-life traffic and road conditions can be simulated; objective measures of driving-related skills are possible; methods for assessing driving performance are safe, efficient, and perhaps sensitive, especially in hazardous driving conditions; and driving conditions for assessing driving skills can be standardized and reproduced (Liu, Watson, & Miyazaki, 1999).
To maximize the efficacy of using driving simulators in clinical and research settings, the scenarios must involve appropriate driving (road and traffic) conditions. Individual groups of researchers have tended to design their own driving scenarios to suit the aims of their projects (e.g., Brooks et al., 2010; Cox, Taylor, & Kovatchev, 1999; Galski, Ehle, & Williams, 1997; Lee, Cameron, & Lee, 2003; Lew et al., 2005; Patomella, Tham, & Kottorp, 2006; Yuen et al., 2007). Driving simulator manufacturers (e.g., Systems Technology Inc., Hawthorne, CA; FAAC Incorporated, Ann Arbor, MI; DriveSafety, Murray, UT) offer computer engineering support to design various driving scenarios with built-in simple self-customization features to meet the investigators’ needs. However, little literature is available to guide the design and essential components of simulated driving scenarios (Fisher et al., 2011). In the current study, we sought to compile a list of situations important in developing scenarios for driving simulators used in clinical and research settings. We distributed a questionnaire to certified driver rehabilitation specialists (CDRSs) to collect their expert opinions on the situations that should be included in driving simulator scenarios.
Method
Participants and Procedures
A list of the CDRSs’ e-mail and physical mailing addresses was downloaded from the Association for Driver Rehabilitation Specialists membership directory Web site. The membership directory included 258 CDRSs practicing in the United States and Canada. We contacted all 258 CDRSs via e-mail with a cover letter explaining the purpose of the study and a link to our questionnaire through Survey Monkey, an online survey engine (www.surveymonkey.com). To increase the response rate, we sent out two follow-up mailings about 2–3 wk apart. We sent the first follow-up via a second e-mail and the second via regular mail in which we included a cover letter, the two-page questionnaire, and a preaddressed, return postage paid envelope. Data were collected between mid-September 2009 and the beginning of December 2009. The institutional review board of the Medical University of South Carolina approved the study.
Of the 253 delivered questionnaires (five regular mailings were returned by the postal service), we received 171 responses (online and via regular mail), for a response rate of 67.6%. Seven respondents did not rate the importance of the driving scenarios at all. As a result, the analytic sample consisted of 164 respondents with usable data.
Survey Item Development
The survey consisted of 22 items describing driving situations followed by an open-ended question for respondents to provide additional driving situations that should be included in simulator-based driving scenarios. The 22 items included a variety of driving situations related to vehicle maneuvering and environmental (driving or traffic) conditions during on-road driving. The instructions directed CDRSs to rate the extent of the importance (using a 5-point rating scale from 1 = not at all important to 5 = critically important) of including each of the 22 driving situations in simulator-based driving scenarios. We developed the survey items on the basis of a review of literature related to on-road and simulator driving examinations and evaluations (e.g., Brooks et al., 2010; Cox et al., 1999; Di Stefano & Macdonald, 2010; Galski et al., 1997; Hunt et al., 1997; Lee et al., 2003; Lew et al., 2005; Patomella et al., 2006; Yuen et al., 2007). The items were then reviewed for content validity by a panel consisting of a CDRS, an occupational therapist academician who had taught a course that included driving evaluation and rehabilitation content for 11 yr, and a psychologist (human factors) with 8 yr of experience using a driving simulator. The questionnaire also included questions regarding the number of years the respondent had been a driver rehabilitation specialist and patient diagnoses or conditions they commonly encountered in driver evaluation.
Data Analysis
We tabulated and analyzed the collected data using descriptive statistics, then conducted exploratory and confirmatory factor analyses to better understand how the 22 driving situations were grouped. Internal consistency reliability of the emerging factors was estimated using Cronbach’s α.
Sample Size Justification
Using a Monte Carlo simulation, MacCallum, Widaman, Zhang, and Hong (1999) found that the sample size for an exploratory factor analysis depends on the level of communality of the variables and the number of factors in the instrument. Communality is the proportion of variance of the items explained by the common factors. With a consistent, moderate level of communalities across the variables (e.g., >.50) and a moderately overdetermined factor (e.g., 5) with 3–7 items per factor, a sample size in the range of 100–200 can achieve a stable and precise estimate of population loadings (MacCallum et al., 1999). According to these guidelines, our sample size of 164 was sufficient to conduct the factor analysis.
Exploratory Factor Analysis
An initial maximum-likelihood oblimin-rotated exploratory factor analysis using squared multiple correlations as initial communality estimates resulted in five factors with eigenvalues ≥1 explaining over 96% of the estimated common variance; however, no simple factor structure emerged. Considering loadings greater than 0.32 in the factor pattern matrix as important in interpreting the variable-factor relationships (Tabachnick & Fidell, 2007), one item did not load on any factors, five items loaded on more than one factor, and the remaining 16 items loaded on single factors. To obtain a simpler structure, an iterative process was conducted in which items that did not load on single factors were removed before rerunning the extraction algorithm.
All statistical analyses were conducted using SAS STAT Version 9.2 software (SAS Institute Inc., Cary, NC). Comments with multiple topics from the open-ended question were separated into individual topics. A graduate student research assistant who was blind to the purpose of the study categorized all comments.
Results
The CDRSs’ experience ranged from 1 yr to 27 yr, with the average years of experience estimated at 10.1 (SD = 5.9). Although the certification process to become a CDRS was established in 1995, 23 (14.0%) respondents reported being in the business of driving evaluation and rehabilitation before 1995. The most common patient diagnoses or conditions in clients whom the CDRSs evaluated specifically for driver rehabilitation were stroke (98.2% of respondents reported) and degenerative neurological diseases (98.2%), followed by traumatic brain injury (97.0%), physical disabilities (94.5%), visual impairment (81.1%), and developmental disorders (76.8%). A number of CDRSs also mentioned mental illnesses and pain syndromes (e.g., fibromyalgia).
More than 70% of the respondents gave four driving situations a rating of 5 (critically important): turning left across oncoming traffic (86.0%), navigating four-way intersections with traffic lights or signs (77.4%), driving in multiple lanes with traffic on both sides (73.2%), and reacting to unexpected events that require emergency braking or aggressive maneuvers to prevent an accident (71.3%). Table 1 presents the percentage of respondents who rated each item as 4 or 5, as well as the mean responses and standard deviations for all questionnaire items.
Percentage of CDRS Respondents Who Rated Survey Items as Critical (N = 164)
Note. Item ratings of 4 or 5 indicate that the respondent considered the item important and critically important, respectively. CDRS = certified driver rehabilitation specialist; SD = standard deviation.
Results of the Exploratory Factor Analysis
After two iterations, the final solution that resulted in a simple factor structure included three factors with 11 items. These three factors were named Intersections, Roadway and Traffic Conditions, and Environmental Conditions. The factor Intersections consisted of two items: navigating four-way intersections with traffic lights or signs and using a median lane to make a left turn. An average of more than 90% of the respondents rated this factor as critically important (i.e., assigned a rating of either 4 or 5). Between 75% and 90% of respondents rated Roadway and Traffic Conditions as critically important; this factor included seven different roadway and driving items. Finally, between 55% and 70% of respondents rated Environmental Conditions as critically important; this factor consisted of two items, adjusting to challenging driving conditions and navigating hazardous road conditions.
Results of the Confirmatory Factor Analysis
This solution was then tested in a structural equation model fitted with asymptotically distribution-free methods. Figure 1 shows the path diagram summarizing the final confirmatory factor analysis model. All estimated model parameters were significant at the .05 level. Overall model fit was assessed with four measures: χ2 goodness-of-fit test, χ2(42, N * = 66) = 50.3, p = .178; root mean square error of approximation = .03; goodness-of-fit index = .95; and Bentler’s comparative fit index = .96. (N * is the number of unique variance/covariance parameters in the correlation matrix of the 11 items.) All measures indicated excellent model fit (Tabachnick & Fidell, 2007).
Final structural equation model, with standardized path coefficients from the confirmatory factor analysis identifying three factors.
Reliability
The 11-item solution had a coefficient α of .89. The coefficient α of each of the factors was .87 for Intersections (2 items), .89 for Roadway and Traffic Conditions (7 items), and .72 for Environmental Conditions (2 items). The coefficient alphas of the 11-item solution and each of the three factors were all above .70, suggesting adequate reliability for an instrument in the early stages of development (Nunnally & Bernstein, 1994).
Finally, we examined the comments provided by 91 of the respondents. After a blind rater categorized the comments, we reviewed and edited the categories and agreed on the organization. Some of the categories were consistent with the survey items; however, the categories were typically broader in scope (Table 2).
Categorization of CDRSs’ Comments on the Open-Ended Question
Note. CDRS = certified driver rehabilitation specialist.
New categories suggested by the CDRSs.
Discussion
Results of this survey indicated that more than 90% of respondents considered traffic conditions involving the potential intersection of two vehicles and roadways with signs or signals to be critically important to include in driving scenarios. These scenario items are consistent with the top five probable leading causes of crashes among older drivers (Wilson, 2006). The factor Intersection further confirms the importance of these scenario components. Even though environmental conditions such as slick roads and inclement weather account for 60% of the environment-related automobile crashes (National Highway Traffic Safety Administration, 2008), CDRSs are unlikely to take the risk of assessing a client’s driving skills under hazardous road or driving conditions and therefore typically do not include this factor in on-road driving evaluations. Instead, they advise clients to avoid such driving conditions. This practice may explain why fewer respondents rated items in the factor Environmental Conditions as critically important.
It is notable that of the initial pool of 22 items in the questionnaire, only 11 were retained in the final factor solution. It is common in any instrument development process that not all items in the initial pool are retained in the final solution. Preference is given to items that load on a single factor, which increases the likelihood of obtaining an interpretable factor structure.
Three of the highest rated items (i.e., turning left across oncoming traffic, navigating four-way intersections, and merging into highway traffic) refer to common situations in the natural on-road environment. Navigating these situations requires a large visual field, visual scanning, and speed maintenance, but not all driving simulators have the capacity to provide a wide enough horizontal field of view (e.g., >300°) to allow for the assessment of driver responses in these scenarios. In addition, some of the driving situations the respondents suggested, such as glare, may not be feasible to incorporate into simulator-based driving scenarios. Another suggested driving situation, backing up in a parking lot, is also a challenge in many simulators that do not use a full vehicle.
The driving scenarios described in the current study need to be tested in simulators used in clinical settings to determine their effectiveness. Further studies need to investigate the feasibility and acceptability of incorporating the driving scenarios described in the current study in simulators for clinical use as well as the validity of the driving scenarios in predicting driver responses in real-life, on-road driving performance.
Because the responses to this survey were completely anonymous, it was not possible to identify the characteristics of the nonrespondents. We do not know whether the nonrespondents would have identified the same driving simulator situations as important for inclusion in simulator-based driving.
Implications for Occupational Therapy Practice
The current study offers the following implications for occupational therapy practice:
Turning left, navigating intersections, driving in multiple lanes, and reacting to unexpected events should be included in simulator driving evaluations.
Intersections, roadway and traffic conditions, and environmental conditions are three recommended categories for driving simulator scenarios.
Conclusion
This study provides a preliminary foundation for driving scenario themes on which future research may build. The results can serve as a guide in the creation of future simulator scenarios for driver evaluation and rehabilitation.
Footnotes
Acknowledgments
The authors thank graduate students Cris De Luna, Jessica Emerson, Jordan Falkiewicz, Amanda Holman, Allison DuBois, Rebekkah Beeco, and Kenna Duckworth for their assistance in data collection and management and Laura Vogtle for her suggestions.