Abstract
We examined whether participants who failed to complete a simulated drive because of simulator sickness (dropouts) differed from those who completed the simulation (completers). Thirteen healthy older adult dropouts (mean age = 74.8 yr) and 12 comparable completers were compared on the following variables: on-road driving performance, the Useful Field of View® test, the Attention Network Test, and the Trail Making Test Part A. Results showed that completers scored more demerit points during the on-road drive than did dropouts. In addition, only 1 of 13 comparisons based on participants’ cognition was statistically significant. These results suggest that in healthy senior drivers, simulator sickness does not prevent examination of those who need it most (i.e., those with the poorest on-road driving performance) and that cognitive differences are not associated with dropping out because of simulator sickness.
Simulator sickness (SS), or simulator adaptation syndrome, is a type of motion sickness that occurs in simulators because of “an inability to simulate the motion environment accurately enough” (Johnson, 2005, p. 22). SS is polysymptomatic (Kennedy & Fowlkes, 1992) and includes symptoms of oculomotor disturbance (e.g., fatigue, headache, eyestrain, difficulty focusing, blurred vision), disorientation (e.g., dizziness, vertigo), and nausea (e.g., increased salivation, sweating, stomach awareness; Kennedy, Lane, Berbaum, & Lilienthal, 1993). It occurs in many types of simulated environments, including flight simulators, driving simulators, space simulators, and three-dimensional virtual environments viewed through head-mounted displays, although the profile of SS symptoms experienced differs across these environments (Kennedy & Fowlkes, 1992). Symptoms usually subside within 1 to 2 hr of ending the simulation (Johnson, 2005), although in rare cases, effects have been reported to continue for 48 hr (Barrett & Thornton, 1968).
The percentage of simulator users that SS affects varies widely depending on the simulator type and task. Kennedy and Fowlkes (1992) reported, “The average incidence of symptoms of sickness [from flight simulators] ranges from a low of less than 20% in the ‘best’ simulator to more than 60% in the ‘worst’” (p. 24). SS rates as high as 80%–95% have been reported in virtual environments (encompassing all human–computer interactive simulators; Stanney, Kingdon, & Kennedy, 2002; Stanney et al., 1998), whereas lower rates of approximately 10% have been found specifically in driving simulators (e.g., Freund & Green, 2006; Lee, Cameron, & Lee, 2003).
SS can be sufficiently severe that it prevents users from completing a simulation. In virtual environment research, 5% to 30% of users end participation early because of SS (Stanney et al., 1998). For example, 88% of participants in Stanney et al.’s (2002) study reported symptoms of SS after exposure to a three-dimensional virtual environment with a head-mounted display, and 12% ended participation prematurely. Similar dropout rates have been found in driving simulators. In Rinalducci, Mouloua, and Smither’s (2003) driving simulator study, 12% of drivers across three age groups (ages 19–34, ages 35–59, and age 60+) were unable to complete the drive, with older drivers appearing more susceptible to dropping out (1 dropout was age 19–34, 2 were ages 35–59, and 4 were ages 60+). Twenty-seven percent of Park, Allen, Fiorentino, Rosenthal, and Cook’s (2006) older drivers failed to complete 10 driving scenarios, and 21% of Rizzo, Sheffield, Stierman, and Dawson’s (2003) drivers (ages 35–89) dropped out. Freund and Green (2006) found that almost 11% of their older drivers (ages 60–99; referred for a driving evaluation) reported experiencing SS, and more than half (57%) of these participants were unable to complete the drive. This situation creates an issue for the use of driving simulators as assessment tools.
Simulator, task, and individual variables influence susceptibility to SS. Simulator factors affecting SS include position of the user relative to the design eye point (the position where the visual display is best viewed), optical distortion, display flicker, image resolution, graphics update rate, and refresh rate (Johnson, 2005; Kennedy & Fowlkes, 1992; Kolasinski, 1995; Mollenhauer, 2004). Generally, wider fields of view increase the likelihood of SS (Jeng-Weei Lin, Duh, Parker, Abi-Rached, & Furness, 2002; Johnson, 2005; Mollenhauer, 2004), although Allen et al. (2003) found that driving simulator configuration (which varied the field of view) was not related to SS in younger novice drivers. Another driving simulation study examining simulator set-up found that SS occurred more frequently in an out-of-car simulator set-up (consisting of a dashboard) than in an in-car set-up (consisting of the shell of a car; Burnett, Irune, & Mowforth, 2007).
Task variables that affect the incidence of SS include simulation duration; unnatural maneuvers, such as freezing the screen; maneuver intensity; head movement; and scene content (Jaeger & Mourant, 2001; Johnson, 2005; Kennedy & Fowlkes, 1992; Kennedy, Stanney, & Dunlap, 2000; Kolasinski, 1995; Mourant, Rengarajan, Cox, Lin, & Jaeger, 2007). Mourant et al. (2007) found that road surface (shaded or textured) did not affect the degree of SS in a driving simulator; however, SS was affected by the type of driving environment. SS scores were significantly higher in a city simulation (with many left and right turns) and in a simulation with numerous curves than in a country and a suburban simulation, which both had many straight roads. One explanation for this phenomenon is that turns increase the optic flow rate experienced by the driver and distort the textures of the optic flow (Mourant et al., 2007). Park et al. (2006) examined dropout rates as older drivers progressed through nine increasingly complex driving scenarios. The dropout rate was highest (11%) in the ninth scenario, a complex city scenario that involved high speed (45 mph), high traffic volume, and multiple turns at intersections (the prior eight scenarios had drop-out rates of <5%).
Turn predictability has also been found to affect SS. Jeng-Weei Lin, Parker, Lahav, and Furness (2005) found that removing visual cues (a path indicating the vehicle’s route) to decrease motion (turn) predictability in a driving simulator increased SS compared with driving conditions in which vehicle motion cues were provided. Vehicle velocity can also influence susceptibility to SS. Mourant and Thattacherry (2000) found that drivers experienced more SS symptoms in simulations in which they drove 60 mph than in simulations in which they drove 25 mph.
Individual factors influencing susceptibility to SS include gender and age. Some studies (e.g., Allen et al., 2003; Freund & Green, 2006; Jaeger & Mourant, 2001; Rizzo et al., 2003), but not all (e.g., Kolasinski, 1996; Kolasinski & Gilson, 1998; Mourant et al., 2007), have found women to be more likely to experience SS than men. Similar results have been found with increasing age. In a driving simulator study, Park et al. (2006) found that subjective reports of SS symptoms did not vary by age or gender, but rates of dropout caused by SS did; older participants (ages 70–90) had a higher dropout rate (37%) than did younger participants (ages 21–50; 14%), and women (32%) dropped out more than men did (22%). Older women had the highest dropout rate at 47% (younger men = 9%, younger women = 17%, older men = 30%). Other individual variables that increase susceptibility to SS include some medications, poor health, sleep deprivation, less experience in the simulated environment, more experience in the nonsimulated environment, and history of motion sickness (Crowley, 1987; Johnson, 2005; Kennedy & Fowlkes, 1992; Lerman et al., 1993).
Cognitive factors may also play a role. Although research with healthy and cognitively impaired participants has found no significant differences in SS incidence rates (e.g., Freund & Green’s, 2006, driving simulator study, and Kang et al.’s, 2008, study with a virtual shopping simulator), cognitive health has been associated with drop-out rates. Rizzo et al., 2003, examined cognitively healthy and cognitively impaired (by stroke or Alzheimer’s disease) drivers (ages 35–89) on a driving simulator. After controlling for age and gender, SS scores (summed ratings of self-reported intensity of SS symptoms) were not related to cognitive health, but the odds of dropping out were 2.4 times greater for cognitively impaired drivers than for cognitively healthy drivers. Clarifying the relationship between cognition and SS is important because driving simulators are often used for evaluating drivers with cognitive impairment (e.g., older drivers, drivers recovering from stroke, drivers with acquired brain injury).
Regardless of its cause, SS remains an issue for simulator users. SS can prevent users from completing the simulation and make the simulator experience unpleasant. SS is also an issue for researchers. Most people overcome SS through adaptation to the simulated environment (Johnson, 2005). This adaptation takes any number of sessions, but some people (3%–5%) never adapt and continue to experience SS (Johnson, 2005). Kennedy et al. (2000) found a linear negative relationship between repeated simulator exposure and SS, such that SS decreased with repeated exposure to the simulator. This relationship generalized across a range of different simulators. For researchers, however, waiting for participants to adapt is often not a practical option. Some researchers choose to exclude participants who report a history of susceptibility to motion sickness (e.g., Edwards, Creaser, Caird, Lamsdale, & Chisholm, 2003; Rizzo et al., 2003) or who show signs of SS after a brief time in the simulator (e.g., Weiler et al., 2000). Hence, it is not always possible to clearly determine the incidence of SS in research studies.
It is possible that participants who fail to complete a simulation because of SS differ in important ways from those who complete the simulation. Edwards et al. (2003) attempted to assess the performance of drivers at intersections, but 40% of older drivers and 14% of younger drivers dropped out because of SS. Edwards et al. (2003) reported, “The loss of older participants due to simulator sickness may affect the representativeness of the… sample. In the worst case, those that drop out due to simulator sickness may be more at risk for intersection collisions” (p. 37). This issue is an important consideration and may have implications for the use of simulators as clinical tools.
Hence, the current study arose from the concern that participants who do not complete driving simulations because of SS may be more at risk of on-road crashes. To investigate this possibility, we compared the on-road performance of completers and noncompleters, as well as their performance on cognitive assessments known to predict driving ability. After a review of the literature, we hypothesized that participants who were unable to complete a simulated drive because of symptoms of SS would show poorer on-road and cognitive performance than participants who completed the drive.
Method
Participants
This retrospective study was based on participants’ data from three studies (currently unpublished). All participants volunteered to participate in driving simulator research at Lakehead University (Thunder Bay, Ontario). Thirteen healthy senior participants (6 men, 7 women) from three studies failed to complete a simulated drive because of verbally reported symptoms of SS (the dropouts). The participants were ages 66–83 (mean [M] = 74.8 yr, standard deviation [SD] = 5.3 yr). A comparison group of completers was formed by selecting all participants who were in the same three studies, had completed the simulated drives, and were within the same age range. Twelve participants (3 men, 9 women), ages 67–82 (M = 74.8 yr, SD = 4.8 yr), formed the completers group.
Apparatus
On-Road Assessment.
The on-road assessment consisted of participants driving a city route in their own vehicle. The 7.5-mi route was a standard road test route for obtaining a driver’s license in the participants’ local city. Driving performance was evaluated by a trained driving examiner, who occupied the passenger seat of the vehicle and completed a standardized demerit point scoring sheet while the driver completed the circuit. Driving performance measures included the number of demerit points scored in each of the five following areas: starting, stopping, and backing; signal violation, right-of-way, and inattention; moving in the roadway; passing and speed; and turning. Demerit points in the five areas were summed, providing a total score (higher scores indicated poorer performance).
Driving Simulation.
The driving simulator (Figure 1) consisted of a driver’s seat, passenger seat, steering wheel with horn, foot pedals (brake and accelerator), indicators, and dashboard (including speedometer and RPM indicator). The driver’s view was presented across three 17-in. monitors. The monitors were arranged in a semicircle and positioned approximately 80 cm in front of the driver’s seat. The three monitors provided a 135° field of view at any one time. A rear view mirror was displayed on the central monitor.
The driving simulator.
The driving simulation courses were designed using STISIM Drive™ software (Systems Technology Incorporated, Hawthorne, CA). They were set under daylight driving conditions with fine weather. A 5.0-mi (approximately 15-min) orientation drive was used to familiarize participants with the simulator. The course included highway and residential sections as well as external environmental cues, such as traffic signs and signals.
The 7.5-mi (approximately 17-min) test drive was a reproduction of the on-road assessment route. The driving simulator software and a researcher collected driving performance data; however, those data were not analyzed because participants who dropped out failed to complete the course.
Neuropsychological Measurements.
The Useful Field of View® test (UFOV; Ball & Owsley, 1993; Edwards et al., 2005) was used to assess participants’ visual processing speed, divided attention, and selective attention. This test was PC based; stimuli were presented on a 17-in. monitor, and responses were made using a computer mouse. This version of the UFOV is both reliable and valid (Edwards et al., 2005). It took approximately 10 min to complete the test. The test was scored using each of the three UFOV subtest scores (UFOV–1, UFOV–2, and UFOV–3) and the sum of the three subtest scores (UFOV–total).
The Attention Network Test (ANT; Fan, McCandliss, Sommer, Raz, & Posner, 2002) was used to assess the efficiency of visual attention. This PC-based test was presented on a 17-in. monitor, and responses were entered on a keyboard. It took approximately 15 to 20 min to complete. This test provided efficiency scores for each of three attention networks: alerting, orienting, and executive control. Using data from correct trials only, the following summary measures of response time were also computed for each participant: median, mean, standard deviation, minimum, and maximum.
The Trail Making Test Part A (Trails A; Reitan, 1955, 1958) was used to measure visual search speed. This pen-and-paper test required participants to correctly identify numbers in sequence under time pressure. A completion time score was obtained. Trails A took <5 min to complete.
Procedure
All participants were from three studies. The exact procedure of each study differed, but the studies had several commonalities, the most important being that they all used the same simulated drives. Approval from the Lakehead University Human Research Ethics Board was obtained before commencing the studies, and informed consent was obtained from participants at the beginning of the studies.
Twenty-four of the 25 participants completed the on-road driving assessment. On a subsequent day, all participants completed the UFOV and ANT, and 21 of the 25 participants also completed Trails A.
After the cognitive tests, participants were seated in the driving simulator. They were asked to familiarize themselves with the pedals and steering wheel and to adjust the seat as required. Participants were informed that some people experience sickness when driving the simulator and that should they experience this sickness, they should immediately inform the researcher. They then drove the orientation drive to become familiar with the driving simulator and controls. When participants noted that they were feeling some sickness, the simulation was paused for approximately 2 min. Two participants did not feel ready to continue after 2 min; in those instances, the pause was extended to 3–4 min. The simulation was then resumed, but if participants again reported feeling unwell, the simulation was terminated. Some participants ended the orientation drive prematurely because of the severity of SS symptoms experienced.
Participants who completed the orientation drive subsequently drove the simulator test drive course. Again, some participants were forced to drop out because of sickness. All participants were thanked for their participation and debriefed at the end of their respective studies.
Data Analysis
Initial screening of the data revealed no influential points (Cook’s distance <1 in all cases; Tabachnick & Fidell, 2001). Because men and women were unequally distributed between the completer and dropout groups, data were analyzed using general linear models that included both group (completers and dropouts) and gender as explanatory variables (Tabachnick & Fidell, 2001). Thus, all reported group means are adjusted for gender. α was set at .05. Effect sizes were calculated using Cohen’s d (Cohen, 1988), with Hedges’ adjustment for sample size (Hedges & Olkin, 1985).
Results
Demographic Measures
Completers (3 men, 9 women) and dropouts (6 men, 7 women) did not differ significantly on gender (p = .411; from Fisher’s exact test) or age (completers M = 74.7 yr, SD = 5.5 yr; dropouts M = 74.7 yr, SD = 5.2 yr; F [1, 22] = 0.001, p = .979, d = 0.01).
On-Road Driving Performance Measures
Table 1 shows descriptive and inferential statistics for on-road driving variables for completers and dropouts. On average, completers scored significantly more demerit points than did dropouts for starting, stopping, and backing and for the total number of demerit points. No other significant differences were observed between completers and dropouts for the remaining on-road driving variables in Table 1.
Results for On-Road Driving Variables
Note. All differences between means were in a direction that favored dropouts. SD = standard deviation.
a n = 24, df = 1,21.
bWhere an effect size (d) of 0.20 = small, 0.50 = medium, and 0.80 = large (Cohen, 1988).
Cognitive Measures
Table 2 shows descriptive and inferential statistics for the UFOV variables for completers and dropouts. No significant differences were found between completers and dropouts for any of the UFOV variables shown in Table 2.
Results for Useful Field of View (UFOV) Variables
Note. All differences between means were in a direction that favored completers. SD = standard deviation.
a n = 25, df = 1,22.
bWhere an effect size (d) of 0.20 = small, 0.50 = medium, and 0.80 = large (Cohen, 1988).
Table 3 shows descriptive and inferential statistics for the ANT variables for completers and dropouts. Examination of mean error rates for the alerting, orienting, and executive control networks for completers and dropouts determined that no speed–accuracy trade-offs influenced the ANT efficiency scores. The orienting network mean was significantly lower for completers than for dropouts (where lower scores indicate poorer cognitive performance), with a trend toward a difference in the same direction for the executive control network mean (where lower scores indicate better cognitive performance). No significant differences were observed between completers and dropouts for the remaining ANT variables in Table 3.
Results for Attention Network Test (ANT) Variables
Note. Differences between means for ANT orienting network and ANT minimum response time were in a direction that favored dropouts. Differences between means for all remaining ANT variables were in a direction that favored completers. SD = standard deviation.
a n = 25, df = 1,22.
bWhere an effect size (d) of 0.20 = small, 0.50 = medium, and 0.80 = large (Cohen, 1988).
Trails A scores did not significantly differ between completers (M = 32.6, SD = 10.9 s) and dropouts (M = 33.2, SD = 10.5 s; F[1, 18] = 0.017, p = .899, d = 0.05).
Discussion
Contrary to our hypothesis, participants who were unable to complete a simulated drive because of symptoms of SS showed better on-road driving performance than did participants who completed the simulated drive. Completers scored more total demerit points than dropouts and more demerit points in the starting, stopping, and backing subsection of the on-road test. Moreover, although the differences were not statistically significant, completers consistently scored more demerit points than dropouts in all subsections of the test. These results show that the simulator dropouts in our sample of healthy senior drivers were not those who had the poorest on-road driving performance. This finding should alleviate the concern that SS prevents examination of those who need it most.
Of 13 cognitive measures compared between driving simulator dropouts and completers, only 1 was statistically significant. Given that the familywise α for a set of 13 contrasts is somewhere between .487 and .650, the results predominantly support the null hypothesis, suggesting that cognitive differences are not associated with dropout rates in simulations with healthy senior drivers. It is possible that any cognitive differences in this sample were too small to detect with the small sample size, and effect-size estimates did indicate potentially important differences. Seven of the 13 F tests comparing cognitive measures had medium to large effect sizes, suggesting that differences might be found with a larger sample. These seven F tests, however, showed no consistent pattern with regard to which group had better performance (completers performed better than dropouts on five of the seven tests, all with medium effect sizes, whereas the only large effect size was for a test on which dropouts performed better than completers). This finding suggests that a larger sample size may not alter the results supporting the null hypothesis.
The two cognitive measures with the lowest p values (ANT orienting network, p = .021, and ANT executive control network, p = .095) also showed no consistent pattern with regard to which group had better cognitive performance; the orienting network was more efficient for dropouts than for completers, whereas the executive control network was less efficient for dropouts than for completers. The apparently random nature of these results provides further support for the null hypothesis.
The results support some previous findings on the relationship between cognition and SS (Freund & Green, 2006; Kang et al., 2008) but do not support Rizzo et al.’s (2003) results, which showed that cognitively impaired drivers were at greater risk of dropping out on a simulator drive. The failure to find significant differences on more cognitive measures could be because the cohort was made up of healthy seniors rather than healthy and cognitively impaired seniors, as in the Rizzo et al. (2003) study.
Limitations of the Study
The current study has several limitations. As mentioned previously, the sample size may have been too small to detect differences. We anticipate that our sample size will increase as we continue to conduct driving simulator research, and we intend to reanalyze the data when our sample is larger. At this stage, however, the sample size should be considered when interpreting the results.
A second limitation of the study was our inability to have gender-matched control participants. Having gender-matched control participants would have been desirable, given the possible influence of gender on SS, and it would have been possible with a larger sample size. Including gender as an explanatory variable in our statistical analyses should have controlled for any effect of gender on our data, however.
SS was not measured or quantified in this study. The retrospective nature of the study precluded the inclusion of a measure such as the Simulator Sickness Questionnaire (Kennedy et al., 1993). Rizzo et al. (2003), however, found that drivers who dropped out were those who experienced more severe SS symptoms, and this pattern is likely to generalize across most samples, including our senior drivers. Moreover, SS may be an issue only when it prevents measurement of simulator performance. As such, the degree of SS experienced may not be the variable of interest; rather, the key variable may be simply whether a participant is able to complete a simulation.
Future Research
Future research in this area should examine rates of both SS and dropping out. Establishing the SS incidence rate and the degree of severity (indicated by the dropout rate) for a range of simulators and scenarios would be beneficial for simulator users. Documenting both SS and dropout rates might also assist in resolving such issues as whether cognitive health is associated with SS. For example, although not statistically significant, Freund and Green (2006) found a consistent pattern with cognitively healthy drivers having higher rates of SS and dropping out. In their sample of 284 older drivers (ages 60–99) referred for a driving evaluation, almost twice as many drivers (15.7%) who screened negative for dementia using the Mini-Mental State Examination (MMSE; Folstein, Folstein, & McHugh, 1975) experienced SS compared with drivers who screened positive (8.2%; p = .234). Similarly, 15.6% of drivers considered unimpaired with the Clock Drawing Test (Freedman et al., 1994) experienced SS, compared with 9.9% of drivers considered cognitively impaired (p = .285). Concerning dropout rates, 40% of drivers who screened positive for dementia with the MMSE were unable to complete the drive, whereas 52.9% of those who screened negative for dementia dropped out (p = .999). Freund and Green’s (2006) pattern of results suggests that cognitive health may increase susceptibility to SS and dropping out; however, Rizzo et al. (2003) results suggest that cognitive health decreases the risk of dropping out. Further research examining cognitive health and reporting both SS and dropout rates should assist in clarifying the role of cognition in SS.
It is also possible that the type of cognitive deficit influences susceptibility to SS. For example, people with Alzheimer’s disease (AD) show a restriction in their field of view that increases (i.e., becomes more restricted) as the disease advances, whereas people with other types of dementia do not exhibit this reduction (Steffes & Thralow, 1987). SS is more prevalent in simulators with larger fields of view (Jeng-Weei Lin et al., 2002; Johnson, 2005; Mollenhauer, 2004), suggesting that people with more advanced AD will be less likely to experience SS than people who are in earlier stages of the disease, who have other types of cognitive impairment, or who are cognitively healthy. Although few people with advanced AD are likely to be using driving simulators, this example highlights the importance of considering the type of cognitive impairment rather than simply combining all cognitively impaired people for comparison with healthy participants.
Further research is also needed on the effects of SS on simulator performance. Rizzo et al. (2003) compared completers and dropouts matched for age, gender, neurological impairment, and scenario driven and found no difference in their driving performance before the point of dropping out. Although this result is encouraging, further research is needed to determine whether it generalizes across cohorts, scenarios, and simulators.
Conclusion
Although simulators continue to advance technologically, it is likely that some seniors will continue to experience symptoms of SS and be unable to complete the drive. Driving simulators offer a safe alternative to on-road assessment of driving performance, but it is important to ensure they can assess the performance of drivers who need it most. The results of the current study suggest that we can continue using driving simulator assessment for seniors without undue concern that those who drop out because of SS are systematically poorer drivers.
Footnotes
Acknowledgments
Michel Bédard is a Canada Research Chair in Aging and Health (
); he acknowledges the support of the Canada Research Chair Program. The research was supported by grants from the National Sciences and Engineering Research Council of Canada, the Canadian Institutes of Health Research, the Canada Foundation for Innovation, AUTO21: Network of Centres of Excellence, and the Ontario Neurotrauma Foundation.