Abstract
More than 35 million people worldwide are estimated to have dementia from a combination of genetic and environmental factors (Prince et al., 2013). On the basis of the Diagnostic and Statistical Manual of Mental Disorders (5th ed.; American Psychiatric Association, 2013), the diagnostic criteria for dementia include declines in memory and cognitive abilities that are so severe as to interfere with daily activities. Although the early signs and symptoms of dementia are diverse, the most common presentation is characterized by increasing difficulty in remembering new information, together with declining activities of daily living and neuropsychiatric symptoms or behavioral changes. Other early signs include changes in mental status, a progressive loss of spatial and temporal orientation, a gradual decline in problem-solving ability, decreased performance of routine daily tasks, and increasing difficulty communicating with others (Alzheimer’s Association, 2011). The enormous demand for long-term care and growing insurance expenses are costly for caregivers and society.
The Loewenstein Occupational Therapy Cognitive Assessment–Geriatric (LOTCA–G; Itzkovich et al., 1996) is a modified version of the LOTCA (Itzkovitch et al., 1990), a measure of cognitive function designed to meet the needs of adults older than age 70 yr. The LOTCA–G provides larger testing items with fewer details and a shortened version of subtests, allowing the total assessment time to be reduced and making the tool more suitable for people older than age 70 yr or clients with a slower response to cognitive tasks. The results of the LOTCA–G not only address a wide array of cognitive functions relevant to older adults but also can be used to monitor changes in cognitive function during an intervention. The construct validity and interrater reliability of this test are well established (Katz et al., 1995, 1997), and good discriminant validity for people with dementia has been demonstrated (Erez & Katz, 2004).
Minimal clinically important difference (MCID) is defined as “the smallest difference in score in the domain of interest which patients perceive as beneficial and which would mandate, in the absence of troublesome side effects and excessive cost, a change in the patient’s management” (Jaeschke et al., 1989, p. 408). MCID is one indicator used to address the question of clinical meaningfulness for various outcome measures (Beaton et al., 2002). Three common methods are used to calculate MCID depending on the psychometric properties of the instrument: (1) anchor based, (2) distribution based, and (3) researcher defined by the use of relevant literature (Wells et al., 2001). The anchor-based method is used to determine the MCID by an external criterion that is correlated with the target assessment. The distribution-based method calculates several statistical measures for estimating variability to determine the MCID from an obtained sample. Finally, in some instances, it is not possible to find a suitable external criterion or to collect sufficient data; in these cases, the MCID can be defined by experts through a review of the current relevant evidence. Currently, the LOTCA–G MCID for dementia has not been established. In this study, the anchor-based and distribution-based methods were applied to establish the LOTCA–G MCID in people with mild dementia.
Method
Participants
Data were obtained from 49 participants (age range = 61–92 yr; 10 men and 39 women), all with a clinical diagnosis of dementia as established by a Mini-Mental State Examination (MMSE; Folstein et al., 1975) score of 24 or less. All participants were enrolled in a larger study investigating the effects of nonpharmacological treatments—including a life review program, recreational therapy, horticultural therapy, and reminiscence therapy—for people with dementia. All participants were assessed for cognitive function before and after the treatment. The beginning and end dates of the enrollment were November 2010 and December 31, 2018. The ethics committees of the participating sites approved this study. All participants and their family members gave informed consent before entering the study.
Inclusion and Exclusion Criteria
The following five inclusion criteria were used: (1) a diagnosis of mild to moderate dementia by neurologists, (2) a clinical dementia rating of 0.5 to 2, (3) an MMSE score of 24 or less, (4) the ability to follow verbal instructions, and (5) the use of Taiwanese or Mandarin as the primary spoken language. The following exclusion criteria were used: other neurological conditions (e.g., stroke) and any health problem (e.g., delusions) or disturbing behaviors that could limit participation in the intervention study.
Instruments
Two commonly used cognitive measures were administered before and after the interventions. The MMSE was chosen as the gold standard test because the LOTCA was strongly and positively correlated with the MMSE (r = .93, p < .001; Wang et al., 2014). The MMSE includes five domains: Orientation, Registration, Attention and Calculation, Recall, and Language. It is easily administered and has been widely used for clinical evaluations. The total score ranges from 0 (worst) to 30 (best). The cutoff score for potential dementia is 24 and is adjusted to 17 for people with less than an eighth-grade education (Liu et al., 1994; Perneczky et al., 2006).
The second cognitive measure was the LOTCA–G. The LOTCA–G includes 23 subtests in seven cognitive areas: orientation, visual perception, spatial perception, praxis, visuomotor organization, thinking operations, and memory. Each question is scored from 1 (severe deficit) to 4 (average performance) on an ordinal scale, except for orientation, which ranges from 0 to 8 in total score. Possible scores range from 23 to 100, with a higher score representative of a higher level of cognitive function. It usually takes 30 to 45 min for a trained occupational therapy practitioner to administer the test. The construct validity, discriminant validity, and interrater reliability of this test have been well established (Erez & Katz, 2004; Katz et al., 1995, 1997).
Procedure
Each participant was evaluated twice, once before the intervention and again 2 to 3 mo after the intervention. During the intervention, each participant received one or two treatment sessions per week for 8 to 12 wk. Each session was between 60 and 90 min of active intervention time. Intervention approaches, which included a life review program, recreational therapy, horticultural therapy, and reminiscence therapy, are all cognitive stimulation activities with equivalent training hours. Raters who administered the LOTCA–G before and after the intervention were assessed with a written competency test by a senior certified occupational therapy practitioner to ensure rater competence.
Data Analysis
Anchor-based and distribution-based approaches were applied to estimate the MCID of the LOTCA–G scores.
Anchor-Based Approach
Previous studies have established the value of 1.4 as the MCID for the MMSE (Burback et al., 1999; Howard et al., 2011); thus, a change score (MMSE score after interventions − MMSE at baseline) greater than 1.4 is indicative of an improvement in cognitive function. Using this criterion, we separated the data on the participants with MMSE change scores greater than 1.4 and focused our examination on only these data for further analysis. The change score on the LOTCA–G (LOTCA–Gafter interventions − LOTCA–Gbaseline) was then calculated for this subpopulation, and the LOTCA–G MCID was established by determining the mean of the change score.
Distribution-Based Approach
The effect-size method was also used to calculate the LOTCA–G MCID. The values of 0.2, 0.5, and 0.8 were used to represent small, medium, and large effects, respectively (Cohen, 1988). For estimation of the MCID, the medium effect size was multiplied by the standard deviation of the baseline score (Norman et al., 2003).
Results
Anchor-Based Approach
Sixteen participants with dementia with MMSE improvements of 1.4 or greater from baseline to posttest were included for the anchor-based approach. The mean MMSE score at baseline for this group suggested that participants had mild to moderate dementia. The mean change in LOTCA–G score was 5.75 for this group, which was the first estimate of the LOTCA–G MCID. Details are provided in Table 1.
Participant Characteristics and Outcome Measure Scores
Note. LOTCA–G = Loewenstein Occupational Therapy Cognitive Assessment–Geriatric; MMSE = Mini-Mental State Examination.
Distribution-Based Approach
Forty-nine participants with dementia completed the assessment at baseline and posttest. The mean MMSE score for this group at the baseline assessment suggested that participants had mild to moderate dementia. Multiplying the small, medium, and large effect sizes by the standard deviation of the LOTCA–G at baseline results in established distribution-based MCIDs of 2.49, 6.23, and 9.97, respectively. Detailed results for each estimate are presented in Table 2.
Two Estimates of the MCID for the LOTCA–G
Note. LOTCA–G = Loewenstein Occupational Therapy Cognitive Assessment–Geriatric; MCID = minimal clinically important difference.
Discussion
To the best of our knowledge, this study is the first to report the LOTCA–G MCID for dementia. Given the clinical focus on cognitive interventions by occupational therapy practitioners, establishing a MCID for standardized cognitive assessment tools for the aging population is necessary to support evidence-based practice in rehabilitation clinics. Currently, no concrete solution exists for determining MCID because the definition and the magnitude of important change could vary depending on how change is measured (Bombardier et al., 2001). However, MCID estimation of well-established clinical tools is a necessity for effectively monitoring change in clinical populations after practitioner-mediated interventions. In this study, we used two different methods to calculate the LOTCA–G MCID for people with dementia. Our results established change scores of 5.75 and 6.23 as the MCID determined by the anchor-based and distribution-based approaches, respectively.
The observed slight discrepancy between anchor-based and distribution-based approaches is likely because of the nature of the methods used to measure the change beneficial to the participants. For the anchor-based approach, the selection of the external criterion will dramatically alter the result. In this study, the MMSE—which is the most widely used and well-established cognitive measurement for dementia—was used as the gold standard; however, studies have revealed differing MMSE MCID values for people with dementia (Burback et al., 1999; Courtney et al., 2004; Howard et al., 2011).
Burback et al. (1999) surveyed 162 physicians to calculate the MMSE MCID for people with dementia. After deducting the mean annual rate of cognitive decline in Alzheimer’s disease (1.4 points on the MMSE), as suggested by the U.S. Food and Drug Administration (FDA; 1989), an MCID of 1.6 points was established (Burback et al., 1999). Howard et al. (2011) used expert-based and distribution-based methods to calculate the MMSE MCID. The expert-based approach recommended 1 to 2 points as the MMSE MCID. However, after consideration of the FDA recommendations, the experts were unable to reach a unanimous agreement on a single value. Using the distribution-based method on a sample of 127 people with Alzheimer’s disease, Howard et al. reported 1.4 points as the MMSE MCID. This estimation used a 0.4 standard deviation change from baseline score. They argued that using the default choice of 0.5 standard deviations would lead to 1.7 points as the MMSE MCID, and it was too close to the upper limit of expert-based suggestion. In this study, a 1.4 point change was used as the MMSE MCID for the anchor-based estimation of the LOTCA–G MCID. This selection was made after carefully considering all of the evidence because selection of this value greatly alters the calculation of the MCID using the anchor-based approach.
The advantage of the distribution-based approach is in taking the variability of the obtained sample into consideration and thereby providing an objective method for estimation of the MCID. Because the variance in the sample alters the outcome, the level of divergence from the mean leads to different results. Our findings indicate that the LOTCA–G MCID established using a distribution-based approach is slightly higher than the result from the anchor-based approach. This difference may be because of the mild heterogeneity in the sample, which leads to a slightly large standard deviation. However, people with dementia typically present with diverse symptoms and occupational performance characteristics, and many factors (e.g., medication or inattention) interfere with the training during the intervention sessions. If we convert the MCID values relative to total score into percentages, the LOTCA–G MCID has a comparable percentage with the MMSE MCID. The MMSE MCID (1.4 points) is 5% of the total score, which is similar to the current finding for the LOTCA–G MCID, in which the average value of the LOTCA–G MCID (6 points) is 6% of the total score. Taken together, people with dementia must demonstrate at least a 5% change in total score to achieve a meaningful clinical change.
Limitations
Our study had several limitations. First, the sample size was small, and the distribution of gender was uneven for study participants. However, consensus was not reached on the necessary sample size, and gender distribution was not raised as an issue for estimating cognitive MCID in dementia. Second, we recruited only participants with mild to moderate dementia, and the potential application to people with severe dementia should be further explored. Finally, the LOTCA–G MCID was an estimation based on mean differences before and after the intervention; some people may experience greater or less benefits than the single value reported here. Future research should examine the LOTCA–G MCID scores with different anchors, such as the Montreal Cognitive Assessment (Nasreddine et al., 2005) and the Saint Louis University Mental Status Examination (Tariq et al., 2006), to better understand the extent to which the current findings are representative of the larger population.
Implications for Occupational Therapy Practice
The results of this study have the following implications for occupational therapy practice:
It is important to consider the LOTCA–G MCID as one of the indexes for determining the success of interventions in people with dementia.
Current results suggest that a change score beyond 5.75 points on the LOTCA–G may be a measure of meaningful change for people with mild to moderate dementia.
Clinical practitioners should interpret the LOTCA–G MCID cautiously because the estimation of MCID could vary by the methods and the obtained sample.
Conclusion
In this study, we used both anchor-based and distribution-based approaches to estimate the LOTCA–G MCID for people with dementia. The estimated values were 5.75 and 6.23, respectively. One potential application of the MCID is to use this value to establish evidence-based treatment goals for determining meaningful performance changes during interventions in clinical practice. Our findings showed similar estimates between two different methods, which suggested a reliable result; however, the estimation of the MCID could vary by the methods and the obtained sample. Therefore, clinical practitioners should interpret the current results cautiously because this is the first study to explore the LOTCA–G MCID. Larger cohort and different anchors should be used in future studies to confirm the current findings.
Footnotes
Acknowledgments
This project was supported in part by the Ministry of Science and Technology (Grant MOST108-2628-B-182-004, MOST 109-2628-B-182-002), Chang Gung Memorial Hospital (Grants CMRPD1H0181, CORPD1H0012, and BMRPC58), and the Healthy Aging Research Center at Chang Gung University (Grant EMRPD1K0431) in Taiwan.