Validation of a Test of Road Law and Road Craft Knowledge With Older or Functionally Impaired Drivers

Abstract

OBJECTIVE. The number of functionally impaired drivers being assessed is increasing the urgency to develop a standardized off-road driver assessment battery. We examined the validity of the Road Law and Road Craft Test (RLRCT) and a version of the Melbourne Slide Test to determine whether they should be included in the Occupational Therapy Driver Off-Road Assessment (OT–DORA) battery, which is under development.

METHOD. We conducted a file audit of 118 data sets with individual item scores for the RLRCT and Melbourne Slide Test.

RESULTS. The RLRCT test with one item removed provides clinicians with a valid indication of clients’ off-road driving skills. The Melbourne Slide Test added no new information over that provided by the RLRCT and can be excluded from the battery.

CONCLUSION. The revised 14-item RLRCT should be included in the OT–DORA battery. Further research is required to develop the OT–DORA battery as a valid and reliable measure of off-road driver skill.

Keywords

aged Australia automobile driving geriatric assessment reproducibility of results

In most developed nations, it is well documented that our populations are aging (Hakamies Blomqvist, 1996; Michalik, 1990; Stanfield, 1996). Accordingly, the number of older drivers (defined as age 65 and older for the purposes of this article) is expected to rise dramatically over the next 2 decades (Fildes et al., 2000; Stav, 2004). Older drivers experience many age-related declines in their health status, increasing numbers of medical conditions, and an increasing need for medications, all of which can impair driving ability (Unsworth, Wells, Browning, Thomas, & Kendig, 2007). However, driving and transportation use have been identified by older adults as their most important instrumental activities of daily living (Fricke & Unsworth, 2001). Moreover, many younger clients with disabilities (such as cerebral palsy or autism spectrum disorder), who may in the past have relied on public transportation, are seeking greater independence through driving (Redepenning, 2006). The ability to drive promotes a sense of autonomy and self-worth and enables people with functional impairments to access health and retail services as well as maintain social contacts (McGregor, 2002; Yassuda, Wilson, & von Mering, 1997). Hence, there is a growing need for occupational therapists who have been specially trained as driver assessors to assist older and functionally impaired adults attain or retain their driver’s license (Pellerito, 2006).

Driver licensing recommendations are of considerable importance given the need to ensure that licensed drivers are fit to drive safely to prevent road trauma and because recommendations of license suspension or cancellation have major implications for a person’s lifestyle and the need for family and community supports (Ralston et al., 2001). Once medically cleared for driving, occupational therapists and other driver rehabilitation specialists are responsible for assessing older people with a health-related problem and all other functionally impaired people’s fitness to drive (e.g., people who have stroke, mental illness, or conditions such as multiple sclerosis). In the state of Victoria, Australia (population 5.2 million), where this research was conducted, driver assessor–trained occupational therapists are included in Section 27 (Regulation 229) of the 1988 regulations of the Road Safety Act 1986 (Victoria) as able to assess clients’ driving skills and advise licensing authorities on license status (Unsworth, 2007a). This is an internationally unique situation. More than 100 Victorian occupational therapists currently perform assessments with >3,500 older and functionally impaired drivers annually (personal communication, VicRoads, 2007).

In Victoria, an off-road assessment is followed by an on-road evaluation of client performance in a standard vehicle on regular roads (Unsworth, 2007a, 2007b). This situation is similar in the United States, where Redepenning (2006) and Stav (2004) have both provided outlines of off-road and on-road procedures. The off-road assessment screens clients for participation in the on-road assessment (Smith-Arena, Edelstein, & Rabadi, 2006) and provides occupational therapy driver assessors with information concerning the client’s skills that can be observed on road. The off-road assessment currently conducted in Victoria takes approximately 60 min to administer, followed by a 45- to 70-min on-road driving assessment with a driving instructor in a dual-controlled vehicle (Unsworth, 2007a; Unsworth, Lovell, Terrington, & Thomas, 2005). Combining all information from the off- and on-road assessments, the driver assessor occupational therapist in Victoria makes a recommendation to the licensing authority, which can be simplified into the following categories: fit to drive with no conditions, fit to drive with conditions, or not fit to drive. The conditions imposed may include driving in a specific zone (e.g., within a 5-mi radius of the client’s home) or daylight driving only.

A collection of standardized and nonstandardized tests have been included in the off-road assessment battery since 1986 in Victoria (Macdonald, 1992; Macdonald, Griffith, Gregory, & Jones, 1992). The off-road assessment battery used by most occupational therapists includes the following evaluation: collection of demographic details; assessment of the need for driving; and an assessment of vision, hearing, sensation, physical function (muscle range, strength, and tone), reaction time, and cognitive–perceptual status (Unsworth et al., 2005). Inclusion of these areas in off-road driver assessment is supported in the international literature (Korner-Bitensky, Sofer, Kaizer, Gelinas, & Talbot, 1994; McGwin, Chapman, & Owsley, 2000; Road Safety Committee, 2003). However, one of the problems with the current approach is the lack of consistency in the assessments chosen by occupational therapists. For example, although some occupational therapists may administer the Mini-Mental State Examination (MMSE; Folstein, Folstein, & McHugh, 1975), others may choose to use the Motor-Free Visual Perceptual Test (Colarusso & Hammill, 1972) or both. This means that clients have different assessment experiences, depending on who assesses them, with potentially different outcomes.

Moreover, because many of the assessments chosen are not standardized (i.e., have unknown reliability or validity and no normative data to reference test scores), the results obtained have limited value in guiding therapists when making licensing recommendations concerning fitness to drive. Anecdotally, it appears that nearly all Victorian occupational therapists currently include a nonstandardized assessment known as the Road Law and Road Craft Test (RLRCT; La Trobe University, 1990) and a version of the Melbourne Slide Test (developed by the Victorian Association of Occupational Therapists [now OT Australia–Victoria] Driving Special Interest Group, 1988, with slides added from a variety of unknown sources) as part of their off-road assessment of clients. These assessments are taught as part of the postgraduate certification to become a registered driver assessor, and most occupational therapists continue to follow this protocol on graduation (Unsworth, 2007a). Both assessments appear to measure a similar range of skills, such as knowledge of road laws (including signs) and road craft, and provide insights as to how clients problem solve the answers to the questions and their ability to sustain attention throughout the task and visual–perceptual skills in interpreting the diagrams and pictures included in the tests. We are in the final phases of developing a standardized assessment called the Occupational Therapy Driver Off-Road Assessment (OT–DORA) battery, which can be completed in approximately 60 min. Given the limited time frame for the battery, it is important to determine whether one or both tests need to be included.

We also searched the literature to determine whether any other standardized tests of road laws and road craft should be considered for inclusion in the OT–DORA battery. The Stroke Drivers Screening Assessment (SDSA) (Radford et al., 1999) and its Scandinavian version, the NorSDSA (Lundberg, Caneman, Samuelsson, Hakamies-Blomqvist, & Almkvist, 2003) were reviewed. However, this assessment would require many changes and restandardization to enable its use with an Australian population, and further research is required to support its use beyond clients with stroke (Selander, Johansson, Lundberg, & Falkmer, 2009). In the United States, Eby, Molnar, Shope, and Dellinger (2007) developed and pilot tested a comprehensive battery of off-road assessments costing approximately $900. The battery can be administered by any professional involved in assessment of drivers and was tested with a convenience sample of 38 drivers age 65 or older. However, although occupational therapists need an off-road assessment to obtain information about the driver’s potential strengths and limitations on road, this battery was developed to assist researchers to “help identify declines in driving performance or changes in crash risk that might be associated with changes in specific assessment areas” (p. 536). Given that the aim of this battery does not align with occupational therapy driver assessor assessment needs, the lack of data to support reliability of the battery, and the high purchase cost, this battery may not be suitable for use with occupational therapy driver assessors. Kay, Bundy, and Clemson (2008) examined the construct and predictive validity and reliability of the Visual Recognition Slide Test developed at the University of Sydney (VRST–USyd). The VRST–USyd was developed >12 years ago and is used by graduates of the University of Sydney driver rehabilitation course. This assessment consists of 15 slides (photographs) of the same rotary (roundabout) in which the number and position of vehicles and pedestrians vary. Rasch analysis indicated that the VRST–USyd had sound internal validity and internal consistency after the removal of the few items that did not discriminate between clients with different skill level. The main limitation of this assessment is that the test is limited to assessing road law and road craft knowledge related to a rotary and does not assess clients’ knowledge of other traffic situations. Nonetheless, it may be suitable in the future for inclusion in an occupational therapy off-road assessment battery.

In summary, the OT–DORA battery is currently under development for use by occupational therapists to evaluate the fitness to drive of older or functionally impaired adults. Although there are no universally accepted guidelines to help driver assessors choose assessments to include in such a battery (Hopewell, 2002), occupational therapy driver assessors in Victoria, Australia, have specific standards to adhere to in their practice. The Competency Standards for Occupational Therapy Driver Assessors (OT Australia–Victoria, 1998) were devised to incorporate the knowledge, skills, and attributes considered essential for quality assessment provided by specialist occupational therapists (see Table 1). These standards should be considered in association with those mandated by Austroads (2003) when determining what tests should be included in a new battery. In developing the battery, it was considered important to research the two tests of road law and road craft knowledge, the Road Law and Road Craft Test (La Trobe University, 1990) and the Melbourne Slide Test. Hence, the primary aim of this research was to examine the psychometric properties and validity of these two tests to determine whether one assessment or both should be included in the OT–DORA battery. Initially, the research examined the internal consistency reliability, construct validity, and predictive validity for the outcome of on-road driving performance (fit to drive with no conditions, fit to drive with conditions, not fit to drive) for these two assessments. We used the results of these analyses to select the assessment tool that demonstrated superior psychometric properties, and then we conducted further analyses to determine whether suitable cut points could be identified with this test to guide on-road testing. This approach allowed us to minimize reporting results for a test that would ultimately be dropped from the OT–DORA battery.

Table 1.

Cognitive Screening Standards Required for Off-Road Driver Assessment

Elements	Performance Criteria
Screens the client's communication skills, considering the communication requirements for the driving task	Communication skills (expressive, receptive, verbal, and written) are ascertained at interview and through observation Cues: Following verbal and written instructions Identifying road signs Referral information
Screens the client's cognitive and perceptual function, considering the cognitive and perceptual requirements for the driving task	(a) The following areas of cognitive and perceptual function are screened: Concentration Attention Thought processing Perception Praxis Memory and learning Planning and problem solving Insight Behavior
	(b) The client's cognitive and perceptual functions are screened at interview and through structured observation and the administration of relevant tests. Cues: Road Law and Road Craft Test Melbourne Slide Test Reaction time test Visual recognition slide test Computerized tests of cognitive function
	(c) Tests used to screen cognitive and perceptual function are selected using sound clinical reasoning. Cues: Diagnosis Referral information Observation Predisposing conditions

Method

Participants

Participants were clients who were consecutive admissions to have their driving assessed at Austin Health Heidelberg Repatriation Hospital between 2005 and 2007. These clients experienced a wide range of medical conditions that potentially affected their fitness to drive. Of the 296 records audited, a sample of 118 client files was included in this study on the basis of the extraction of individual item scores for both the Road Law and Road Craft Test and Melbourne Slide Test, rather than a total score only for each test. Of the 118 participants, 70% were men and 30% were women ages 21 to 94 (mean = 72.32, standard deviation = 14.86). Thirty-one percent of the sample were diagnosed with Alzheimer’s disease or dementia, 27% were diagnosed with neurological disorders, and 20% were diagnosed with cognitive impairments. After their on-road driving assessment, 57% of participants were classified as fit to drive with no conditions, 24% were classified as fit to drive with conditions, and 19% were classified as not fit to drive.

Measures

The variables of interest were the clients’ age, gender, and scores obtained on the RLRCT (La Trobe University, 1990) and the Melbourne Slide Test. Little has been documented about the development of these assessments. These tests have been used at La Trobe University over the past 20 years; however, the authorship is not documented. The Melbourne Slide Test consists of 13 images projected as slides depicting city and suburban road scenes in Australia, which are each accompanied by a question (maximum score is 32). Sample questions include “Describe this intersection” and “You are the driver of the white van turning left. You are facing a green light. Who do you give way to?” The RLRCT is a 15-item test of road laws and knowledge of what to do in common driving situations (maximum score is 38). Two sample questions from the assessment are included in Appendix A. Information on the outcome of the driver assessment was obtained from the client’s VicRoads Occupational Therapy Car Driver Evaluation form and recorded for the purposes of this research as fit to drive with no conditions, fit to drive with conditions, and not fit to drive. When sample sizes were insufficient to analyze these three groups, the fit to drive with no conditions and the fit to drive with conditions groups were combined and referred to as the fit to drive group. Two qualified occupational therapy driver assessors with >5 years of experience in administering driver assessments provided scores on the RLRCT and Melbourne Slide Test. The therapists followed the standard written protocol to administer and score both tests.

Procedure

Permission to collect these data was granted from the La Trobe University Human Ethics Committee and the Human Research Ethics Committee, Austin Health, in Melbourne, Victoria. The study used a retrospective case review methodology. A total of 118 deidentified client files from the hospital were audited, and data relating to age, gender, and reason for presenting for assessment were collected along with scores obtained from the RLRCT and Melbourne Slide Test and the outcome of the driver assessment (fit to drive with no conditions, fit to drive with conditions, and not fit to drive).

Analyses

A series of analyses was conducted to assess the psychometric properties and external validity of the RLRCT. Rasch analysis was performed using RUMM2020 software (Andrich, Lyne, Sheridan, & Luo, 2003) to assess the internal validity of the RLRCT. The Rasch model, which is an important tool in modern test theory, is increasingly being used in the health sciences to assess and improve the measurement properties of the tests used for research and clinical purposes. It provides a detailed assessment of all aspects of a scale’s functioning, including overall model fit, suitability of the response format used, fit of individual items, item bias, detection of misfitting people, dimensionality, and appropriate targeting of the scale for the sample. A detailed description and illustration of Rasch analysis procedures is provided in Pallant and Tennant (2007).

Several tests of fit were used in the current study, including overall summary tests of fit, as well as individual item and person tests. These tests are designed to determine whether responses deviate from Rasch model expectations. Thus, a summary χ² interaction fit statistic should be nonsignificant, as should the individual item χ² statistics after Bonferroni adjustment to the α level. Fit residual values were also used to evaluate fit to the Rasch model. The standard deviation of the summary residual statistic should not deviate too much from 1 (perfect fit), and certainly should not be above 1.5. Individual item residuals should fall within the range ± 2.5 (99% confidence interval [CI]). High positive fit residual values indicate misfit, whereas high negative fit residuals suggest item redundancy.

The Person Separation Index is equivalent to Cronbach’s α and provides an estimate of the internal consistency reliability, with values >0.7 considered adequate (Nunnally, 1978). Item bias can occur when different groups within the sample display different response patterns to a particular item, despite being equivalent in terms of the underlying characteristic being measured. To identify any possible item bias across gender and age, differential item functioning was assessed using analysis of variance with a Bonferonni adjusted α level.

Unidimensionality was tested using the approach suggested by Smith (2002). Person estimates, derived from subsets of items identified by high positive and negative loadings on the first principal component of the residuals, were tested for significant differences. Using a series of independent t tests, if >5% of these tests are significant (or specifically the lower bound of the binomial confidence interval is >5%), the scale is deemed to be multidimensional. To assess the appropriateness of the scale in terms of difficulty for the current sample, a person–item threshold plot was generated showing the relative difficulty levels of items in relation to the ability of people in the current sample. An item map was used to evaluate the spread of items across the full range of the underlying scale.

Rasch-calibrated RLRCT scores were then exported to SPSS Version 15 (SPSS Inc., Chicago) for further statistical analysis to assess the validity of the scale. Nonparametric statistics were used throughout because of the skewed distribution of scale scores. Spearman correlation coefficients were generated to assess the convergent validity by comparing scores on the RLRCT with another conceptually similar scale, the Melbourne Slide Test. The criterion validity was assessed by conducting a Kruskal-Wallis test to compare RLRCT scores for groups of clients who were found to be fit to drive with no conditions, fit to drive with conditions, and not fit to drive on their on-road test. The incremental predictive validity of the RLRCT, compared with the Melbourne Slide Test, was assessed using a logistic regression analysis with the outcome of the on-road test (fit to drive, not fit to drive) as the dependent variable. This test was used to compare the relative predictive ability of the two tests and to determine whether the Melbourne Slide Test offered any additional predictive power over and above that already available with the RLRCT.

Results

Internal Validity of the RLRCT Using Rasch Analysis

The overall fit of the model was adequate after Bonferonni adjustment to the α level (p = .04). Item 1, however, recorded a very high positive fit residual value (4.34), suggesting that it did not fit with the remaining items. After its removal from the scale, the overall model fit improved (p = .47), and all items showed good fit to the model with no significant χ² probability values, and all fit residual values within the range ±2.5 (see Table 2). The Person Separation Index value for the revised 14-item rescored RLRCT (RLRCT–14) was 0.84, indicating good internal consistency. No differential item functioning was detected for gender or age.

Table 2.

Individual Item Fit Statistics for 14-Item Road Law and Road Craft Test (Final Model)

Item	Location	Standard Error	Fit Residual	χ²	P
2	−1.42	0.32	0.71	0.08	0.77
3	−0.62	0.26	−0.30	0.28	0.59
4	0.88	0.21	1.73	0.10	0.75
5	1.43	0.13	−0.22	0.10	0.75
6	0.27	0.10	−1.21	0.66	0.42
7	1.12	0.12	−1.16	0.81	0.37
8	0.40	0.14	0.50	1.12	0.29
9	0.05	0.13	−1.24	0.29	0.59
10	−0.90	0.28	−0.51	1.33	0.25
11	−0.18	0.17	1.84	1.96	0.16
12	0.12	0.15	1.10	0.27	0.60
13	−0.15	0.16	−1.44	4.86	0.03
14	−0.97	0.20	0.37	1.61	0.20
15	−0.02	0.13	−0.12	0.28	0.60

Note. Fit residual df = 106.57; χ² df = 1.

The distribution of scores on the Person-Item Threshold plot indicated that the RLRCT–14 was reasonably well targeted for the sample (see Figure 1). There was a good spread of items representing the lower and middle range of scores. Although there were few very difficult items on the test that only people with excellent skills would get correct, the main focus of the RLRCT–14 is to identify clients with low levels of road law and road craft knowledge. In other words, the test is appropriately targeted at picking up clients who have weaknesses rather than identifying those with strengths in this area.

Figure 1.

Person–Item Threshold graph showing the distribution of 14-item Road Law and Road Craft Test scores (top section) and item thresholds (bottom section).

The unidimensionality of the RLRCT–14 was assessed by performing a principal-components analysis on the residuals and identifying the most disparate sets of items. Comparison of the person estimates using a series of t tests revealed 7.69% of people with significantly different scores on the two sets of items. This value fell within the CI surrounding the accepted value of 5% (95% CI = 4%–12%); therefore, the unidimensionality of the RLRCT–14 was supported.

Validation of the RLRCT–14

The convergent validity of the RLRCT–14 was explored by assessing its relationship with the Melbourne Slide Test, which measures a related construct. Consistent with expectations, the RLRCT–14 showed a moderate positive correlation with the Melbourne Slide Test (ρ = .56, p < .001).

To assess the known groups’ validity of the RLRCT–14, raw scores were compared for groups who were found fit to drive with no conditions, fit to drive with conditions, and not fit to drive after their on-road test (see Table 3). Again, because data were not normally distributed, nonparametric statistics were used. A Kruskal-Wallis test showed a statistically significant difference in RLRCT–14 scores among the three groups (χ²[2] = 10.5, p = .005). Post hoc Mann–Whitney U tests indicated that the not fit to drive group was statistically significantly different from the other two groups (fit to drive with no conditions [z = −3.23, p = .001, r = .34]; fit to drive with conditions [z = −2.10, p = .04, r = .30]). For both of these post hoc comparisons, the effect size was moderate according to Cohen’s (1988) criteria (0.1 = small effect, 0.3 = moderate effect, 0.5 = large effect).

Table 3.

Descriptive Statistics for the 14-Item Road Law and Road Craft Test (RLRCT–14) and Melbourne Slide Test

	Total Sample	Not Fit to Drive	Fit to Drive With Conditions	Fit to Drive Without Conditions
RLRCT-14
No. of participants	116	22	28	66
Mean	24.87	20.18	25.18	26.30
SD	7.95	8.61	7.22	7.53
Median	27	22	27	28.5
Minimum	0	0	10	1
Maximum	35	32	34	35
Cronbach's χ	.84
Melbourne Slide Test
No. of participants	113	20	27	66
Mean	27.94	26.62	27.61	28.47
SD	3.66	3.40	3.67	3.66
Median	29	28	28	29
Minimum	14	19	14	14
Maximum	32	31	32	32
Cronbach's χ	.79

Note. SD = standard deviation.

To compare the performance of the RLRCT–14 against the Melbourne Slide Test, a similar set of analyses was conducted. The difference in Melbourne Slide Test scores among these three groups only just reached statistical significance (χ²[2] = 7.05, p = .03). Post hoc Mann–Whitney U tests indicated that the only statistically significant difference was between the fit to drive with no conditions and the not fit to drive groups (z = −2.49, p = .01, r = .27). The effect size for this comparison was small.

Predictive Power of RLRCT–14 and the Melbourne Slide Test

To test the incremental predictive power of the RLRCT–14 and the Melbourne Slide Test, a binary logistic regression analysis was conducted using the result of the on-road test as the dependent variable and scores on the RLRCT–14 and Melbourne Slide Test as the predictors. Although highly correlated predictors such as the items on these two tests would not normally be included in a regression analysis used for model testing, the logistic regression reported here is appropriate because it is used to assess incremental validity. Given the small sample sizes, the data for the on-road tests were collapsed into fit to drive and not fit to drive, as described in the Method section. The model containing both tests explained between 7% and 11% of the variance; however, only the RLRCT–14 was significant in the final equation (odds ratio = 0.93, 95% CI = 0.88–0.99, p = .03). The Melbourne Slide Test did not contribute anything further to the prediction of the outcome of the on-road driving test over and above that provided by the RLRCT–14 (odds ratio = 0.95, 95% CI = 0.83–1.09, p = .45).

Identification of a Suitable Cut Point on the RLRCT–14 to Guide On-Road Testing

Although there was a significant difference in the mean of the three groups (fit to drive with no conditions, fit to drive with conditions, and not fit to drive), there was a substantial spread of scores across the full range for each of the three groups, making the identification of a suitable cut point on the RLRCT–14 that could suggest a point at which clients would not pass the on-road test (see Figure 2).

Figure 2.

Box plot showing distribution of scores on 14-item Road Law and Road Craft Test (RLRCT–14) for people found to be fit to drive with no conditions, fit to drive with conditions, and not fit to drive.

Table 4 presents the percentages of the participants who were fit to drive compared with participants who were unfit to drive at different levels of the RLRCT–14 scores. This pattern of results suggests that although high scores on the RLRCT–14 are associated with the increased likelihood of passing the on-road test, there are a substantial number of people with quite low scores that pass the on-road test. For example, 14 participants recorded a score of ≤15 on the RLRCT–14; however, 8 of these (57.1%) went on to successfully pass the on-road driving test. This result is explored further in the Discussion section.

Table 4.

Percentage of the Sample Who Were Fit to Drive vs. Unfit to Drive at Different Levels of the 14-Item Road Law and Road Craft Test (RLRCT-14) Scores

	Fit to Drive		Unfit to Drive		Total
RLRCT-14 Score	n	%	n	%	n	%
≤15	8	57.1	6	42.9	14	100.0
16-20	8	66.7	4	33.3	12	100.0
21-25	23	85.2	4	14.8	27	100.0
26-30	24	77.4	7	22.6	31	100.0
31 χ	33	97.1	1	2.9	34	100.0
Total	96	81.4	22	18.6	118	100.0

Note. Scores were divided into 5-point increments.

Discussion

The primary aim of this study was to assess the psychometric properties of the RLRCT and the Melbourne Slide Test for possible inclusion in the OT–DORA battery. Rasch analysis provided support for the internal validity of the RLRCT; however, it was necessary to remove one item to achieve good fit to the Rasch model. This multiple- choice item was worded in an ambiguous manner and asked the client (for a U.S. audience), “If turning right at an intersection not controlled by traffic signals, signs, or police, you must give way. …” Although the correct answer was believed to be “to the vehicle on your left,” other answers such as “to all vehicles and pedestrians” and “to vehicles turning left only” can also be interpreted as correct. The resulting 14-item scale (RLRCT–14) was found to be unidimensional, with good internal consistency and no evidence of differential item functioning or bias across gender or age. Validity testing found support for the convergent, known groups, and incremental predictive validity of the RLRCT–14. The RLRCT–14 correlated as predicted with a closely related scale—the Melbourne Slide Test. There were statistically significant differences in RLRCT–14 scores for people who passed versus those who failed the on-road test, and the effect size for these comparisons was moderate. The Melbourne Slide Test, however, showed only a small effect size, with the differences in scores among the groups only just reaching statistical significance. The relative superiority of the RLRCT–14 was also reflected in the results of the logistic regression comparing the incremental validity of the RLRCT–14 against that of the Melbourne Slide Test. Only the RLRCT–14 was significant in the final equation, indicating that the Melbourne Slide Test made no additional contribution to the prediction of the results of the on-road test, over and above that of the RLRCT–14. Hence, it is suggested that only the RLRCT–14 be included in the OT–DORA battery.

Although these results suggest that performance on the RLRCT–14 is related to the outcome of on-road testing, the nature of the relationship is such that the identification of suitable clinical cut points on the RLRCT–14 was difficult. In other words, it is difficult to identify a score on the RLRCT that can clearly be associated with a client failing the on-road driving test. On the whole, people who performed well on the RLRCT–14 were more likely to pass the on-road tests. The opposite, however, was not true. Some people with quite low scores on the RLRCT–14 were able to successfully pass the on-road test. For example, the audited client files contained a case of a 75-year-old man who had suffered a right parietal stroke and obtained a low score (18/38) on the RLRCT but subsequently passed his on-road driving assessment.

The scores presented in Table 4 provide clinicians with some guidance by which to judge the performance of individual clients. In the current sample, only 57% of drivers with a RLRCT–14 score of ≤15 successfully passed their on-road tests, rising to 67% for scores between 16 and 20 and to >85% for scores between 21 and 25. These results suggest that people who record scores <20 on the RLRCT–14 should be monitored carefully during their on-road test, given their increased likelihood of failing. However, the findings clearly indicate that the RLRCT–14 should not be used to decide whether to allow an individual to undergo on-road testing. Rather, test results can be used in combination with other off-road and on-road information to guide clinical decision making.

Limitations and Future Research

The limitations of this study include the relatively small sample size (although adequate for all analyses conducted) and the fact that data were all collected from one driver assessment service. Further research is required to examine client assessment results on the RLRCT from a variety of driver assessment services. In addition, interrater and test–retest reliability studies could be undertaken to examine consistency of RLRCT scores over time and between different driver assessors. Research is currently under way to examine the psychometric properties of other cognitive, physical, and sensory components of the OT–DORA battery. This research is consistent with the recommendations of Stav (2004) and Redepenning (2006) that occupational therapy driver assessors continue to develop and research standardized driver assessments. The OT–DORA battery has been developed with items specific to Australian traffic situations and driving on the left. However, with relatively minor modifications, the RLRCT (and the OT–DORA battery as a whole) could be adapted for use in the United States or the United Kingdom. We are investigating the possibility of conducting research to standardize parallel versions of the OT–DORA battery for these licensing jurisdictions.

Conclusion

Accurate assessment of clients who are and are not fit to drive contributes to the prevention of road trauma, making our roads a safer place for all users. Driver assessor occupational therapists conduct detailed off-road and on-road assessments to make optimal clinical decisions concerning client fitness to drive. The field of driver assessment and rehabilitation is growing rapidly (American Occupational Therapy Association, 2008), and research is urgently required to develop standardized assessments so that driver assessors can be confident that findings are valid, reliable, and referenced to age- and gender-based norms.

This research examined the psychometric properties and validity of the RLRCT and the Melbourne Slide Test to determine whether one or both should be included in the OT–DORA battery currently under development. The data from 118 clients who underwent testing using the RLRCT, Melbourne Slide Test, and on-road driving test were subjected to Rasch and standard statistical analyses to examine the validity of these tests. After removal of the first item on the RLRCT, this assessment was found to provide a valid indication of drivers’ skills. However, although clinicians can use the RLRCT with confidence to gain information concerning client skills and possible on-road errors to look for, this test cannot replace on-road testing.

The Melbourne Slide Test was not found to be as robust on validity testing when compared with the RLRCT, and the logistic regression analysis showed that it did not add any new information over that already provided by the RLRCT. Therefore, given the time constraints placed on driver assessor occupational therapists when undertaking an off-road evaluation, it is recommended that the RLRCT be included and the Melbourne Slide Test be dropped from the OT–DORA battery. Further research is required to develop the remaining cognitive, physical, and sensory assessments included in the OT–DORA battery as valid and reliable measures of off-road driver skill.

Footnotes

Acknowledgments

We gratefully acknowledge the grants that have supported this research, including a Faculty of Health Sciences Research Grant, La Trobe University, and a Royal Automobile Club of Victoria Sir Edmund Herring Memorial Scholarship received by Carolyn A. Unsworth.

Example Items From the Road Law and Road Craft Test

References

American Occupational Therapy Association. (2008). Occupational therapy practice framework: Domain and process. American Journal of Occupational Therapy, 62, 625–683.

Andrich

Lyne

Sheridan

, & Luo

(2003). RUMM2020. Perth, Western Australia: RUMM Laboratory.

Austroads. (2003). Assessing fitness to drive: Commercial and private vehicle drivers. Medical standards for licensing and clinical management guidelines (3rd ed.). Sydney, New South Wales, Australia: Author.

Cohen

(1988). Statistical power for the behavioral sciences. Hillsdale, NJ: Erlbaum.

Colarusso

R. P.

, & Hammill

D. D.

(1972). Motor-Free Visual Perceptual Test. Novato, CA: Academic Therapy Publications.

Eby

D. W.

Molnar

L. J.

Shope

J. T.

, & Dellinger

A. M.

(2007). Development and pilot testing of an assessment battery for older drivers. Journal of Safety Research, 38, 535–543. doi:10.1016/j.jsr.2007.07.004

Fildes

Pronk

Langford

Hull

Frith

, & Anderson

(2000). Model licence re-assessment procedure for older and disabled drivers. Clayton, Victoria, Australia: Monash University Accident Research Centre.

Folstein

M. F.

Folstein

S. E.

, & McHugh

P. R.

(1975). “Mini-mental state”: A practical method for grading the cognitive state of patients for the clinician. Journal of Psychiatric Research, 12, 189–198. doi:10.1016/0022-3956(75)90026-6

Fricke

, & Unsworth

(2001). Time use and importance of instrumental activities of daily living. Australian Occupational Therapy Journal, 48, 118–131.

10.

Hakamies Blomqvist

(1996). Research on older drivers: A review. Journal of International Association of Traffic and Safety Sciences, 20, 91–101.

11.

Hopewell

C. A.

(2002). Driving assessment issues for practicing clinicians. Journal of Head Trauma Rehabilitation, 17, 48–61. doi:10.1097/00001199-200202000-00007

12.

Kay

L. G.

Bundy

A. C.

, & Clemson

L. M.

(2008). Predicting fitness to drive using the visual recognition slide test (USyd). American Journal of Occupational Therapy, 62, 187–197.

13.

Korner-Bitensky

Sofer

Kaizer

Gelinas

, & Talbot

(1994). Assessing ability to drive following an acute neurological event: Are we on the right road. Canadian Journal of Occupational Therapy, 61, 141–148.

14.

La Trobe University. (1990). Road Law and Road Craft Test (RLRCT) with 15 items. Unpublished manuscript, La Trobe University.

15.

Lundberg

Caneman

Samuelsson

S. M.

Hakamies-Blomqvist

, & Almkvist

(2003). The assessment of fitness to drive after a stroke: The Nordic Stroke Driver Screening Assessment. Scandinavian Journal of Psychology, 44, 23–30. doi:10.1111/1467-9450.00317

16.

Macdonald

W. A.

(1992, December). Driver performance capacity evaluation: Conditions necessary for reliability. Proceedings of the 28th Annual Conference of the Ergonomics Society of Australia, Melbourne, Victoria.

17.

Macdonald

W. A.

Griffith

Gregory

, & Jones

(1992). Performance capacity assessment of head-injured drivers: An investigation of the validity of some off-road tests. Proceedings of the 28th Annual Conference of the Ergonomics Society of Australia, Melbourne, Victoria.

18.

Michalik

(1990). Development and evaluation of measures to reduce accident risk of elderly drivers. Journal of International Association of Traffic and Safety Sciences, 20, 25–31.

19.

McGregor

(2002). Driving over 65: Proceed with caution. Journal of Gerontological Nursing, 28, 22–26.

20.

McGwin

Chapman

, & Owsley

(2000). Visual risk factors for driving difficulty among older drivers. Accident Analysis and Prevention, 32, 735–744. doi:10.1016/S0001-4575(99)00123-2

21.

Nunnally

J. C.

(1978). Psychometric theory. New York: McGraw-Hill.

22.

OT Australia–Victoria. (1998). Competency standards for occupational therapy driver assessors. Melbourne, Victoria, Australia: Author.

23.

Pallant

J. F.

, & Tennant

(2007). An introduction to the Rasch measurement model: An example using the Hospital Anxiety and Depression Scale (HADS). British Journal of Clinical Psychology, 46, 1–18. doi:10.1348/014466506X96931

24.

Pellerito

(2006). Driver rehabilitation and community mobility. St Louis, MO: Elsevier/Mosby.

25.

Radford

Lincoln

Hawkins

Murray-Leslie

Melly

Lilley

, et al. (1999). Validation of the stroke drivers screening assessment (SDSA). Clinical Rehabilitation, 13, 531–532.

26.

Ralston

L. S.

Bell

S. L.

Mote

J. K.

Rainey

T. B.

Brayman

, & Shotwell

(2001). Giving up the car keys: Perceptions of well elders and families. Physical and Occupational Therapy in Geriatrics, 19, 59–70. doi:10.1300/J148v19n04_05

27.

Redepenning

(2006). Driver rehabilitation across age and disability: An occupational therapy guide. Bethesda, MD: AOTA Press.

28.

Road Safety Act. (1986). Section 27 (Regulation 229) of the 1988 regulations of the Road Safety Act of Victoria, Australia.

29.

Road Safety Committee, Parliament of Victoria. (2003). Road safety for older road users. Victoria, Australia: Author.

30.

Selander

Johansson

Lundberg

, & Falkmer

(2009). The Nordic Stroke Driver Screening Assessment as predictor for the outcome of an on-road test. Scandinavian Journal of Occupational Therapy, 5, 1–8. doi:10.1080/11038120802714898

31.

Smith

E. V.

Jr. (2002). Detecting and evaluating the impact of multidimensionality using item fit statistics and principal component analysis of residuals. Journal of Applied Measurement, 3, 205–231.

32.

Smith-Arena

Edelstein

, & Rabadi

M. H.

(2006). Predictors of a successful driver evaluation in stroke patients after discharge based on an acute rehabilitation hospital evaluation. American Journal of Physical Medicine and Rehabilitation, 85, 44–52.

33.

Stanfield

(1996). The aging of America. Demographics National Journal, 29, 1578–1583.

34.

Stav

(2004). Driving rehabilitation: A guide for assessment and intervention. San Antonio, TX: Psychological Corp.

35.

Unsworth

C. A.

(2007a). Development and current status of occupational therapy driver assessment and rehabilitation in Victoria, Australia. Australian Occupational Therapy Journal, 54, 153–156. doi:10.1111/j.1440-1630.2006.00617.x

36.

Unsworth

C. A.

(2007b). Using social judgment theory to study occupational therapists’ use of information when making driver licensing recommendations for older and functionally impaired adults. American Journal of Occupational Therapy, 61, 493–502.

37.

Unsworth

Lovell

Terrington

, & Thomas

(2005). Review of tests contributing to the occupational therapy off-road driver assessment. Australian Occupational Therapy Journal, 52, 57–74. doi:10.1111/j.1440-1630.2005.00456.x

38.

Unsworth

C. A.

Wells

Browning

Thomas

S. A.

, & Kendig

(2007). To continue, modify or relinquish driving: Findings from a longitudinal study of healthy ageing. Gerontology, 53, 423–431. doi:10.1159/000111489

39.

Victorian Association of Occupational Therapists, Driving Special Interest Group. (1988). Melbourne Slide Test. Unpublished media.

40.

Yassuda

M. S.

Wilson

J. J.

, & von Mering

(1997). Driving cessation: The perspective of senior drivers. Educational Gerontology, 23, 525–538. doi:10.1080/0360127970230603