Abstract
The results of this pilot study highlight the promising potential of the TELE-self-GEN intervention for improving everyday memory challenges for people with multiple sclerosis.
Multiple sclerosis (MS) is an autoimmune disease affecting the central nervous system that can result in cognitive impairments, including problems with memory, processing speed, and executive functioning, in approximately 65% of people with MS (Benedict et al., 2020). Cognitive rehabilitation interventions aim to improve cognitive functioning, enhance adaptive strategies, and promote independence in activities of daily living (ADLs). These interventions may include a restorative process training approach (i.e., repetitive practice of tasks designed to challenge and improve cognitive functions) or a compensatory approach (i.e., a strategy or technique that compensates for specific cognitive deficits or impairment; Goverover, 2018). To date, the restorative approach has resulted in limited effectiveness, particularly in enhancing ADLs and quality of life (QOL; Chiaravalloti et al., 2022; Goverover et al., 2023), whereas the compensatory approach shows improvement primarily within the trained tasks (Goverover et al., 2009, 2011). In the context of compensatory treatment aimed at improving learning and memory, researchers have incorporated strategies from cognitive psychology to improve performance. One such strategy, self-generated learning, has led to the generation effect, whereby information that is self-generated by the individual is better remembered than items that are passively read or heard (Slamecka & Graf, 1978). Although previous studies (Chiaravalloti & DeLuca, 2002; Goverover et al., 2008, 2014) have shown enhanced learning and memory for specific trained tasks through self-generation, the transfer and generalization of these benefits to untrained tasks have been limited. Furthermore, significant personal and environmental barriers limit access to cognitive rehabilitation services for people with MS. Personal barriers include MS symptoms such as fatigue, visual problems, mobility issues, and bladder–bowel dysfunction; environmental barriers encompass challenges such as limited transportation and distance from clinics (Broch et al., 2021; Chiu et al., 2017). These barriers can make it challenging for people with MS to travel to health care facilities and attend appointments to receive suggested care (Mayo et al., 2021). Telehealth treatments have emerged as a viable solution to address these barriers (Gerritzen et al., 2022).
Despite evidence supporting the benefits, satisfaction, and cost-effectiveness of telerehabilitation, only four studies on telehealth cognitive interventions for individuals with MS have been reported (Di Tella et al., 2020; Yeroushalmi et al., 2020). Di Tella et al.’s (2020) systematic review and meta-analysis reported a small treatment effect for cognitive performance outcomes using telehealth intervention. Of the four studies reported in the systematic review, two were completely asynchronous (no real-time interaction between the therapist and the participant), and two were mixed (synchronous in the hospital setting and asynchronous at home). This difference in synchronicity also informed the type of treatment provided during the interventions. The asynchronous intervention involved cognitive restorative training for specific cognitive functions (Pedullà et al., 2016), whereas the treatment that involved both synchronous and asynchronous methods was based on a combination of psychoeducation or compensation and specific cognitive training (Stuifbergen et al., 2018). The review concluded that there is a strong need to systematically examine the efficacy of telehealth services using synchronous and asynchronous treatment methods (Di Tella et al., 2020).
We designed the current strategy-based treatment study on the basis of principles presented by the multicontext approach (Toglia & Foster, 2021) to address the transfer and generalization of treatment gains to ADLs and QOL, as well as some of the personal and environmental barriers that hinder cognitive rehabilitation accessibility of individuals with MS. The TELE-self-GEN, a new synchronous teleintervention, focuses on the intricate cognitive domains related to learning and memory, including encoding, retrieval, and the crucial ability to transfer learned information to untrained tasks. By embedding the technique of self-generated learning within a metacognitive strategy training, TELE-self-GEN targets not only memory enhancement but also the application of these improved memory skills to everyday activities. The intervention guides participants through self-predictions, self-monitoring, and journaling, fostering a deep understanding of how these cognitive strategies can be practically applied in real-life situations (Goverover et al., 2018).
Although the efficacy of the in-person self-GEN intervention has been established, the utilization of this treatment via telehealth is unclear. Thus, the purpose of this proof-of-concept pilot study (Arain et al., 2010) was to examine the use of remotely delivered cognitive rehabilitation treatment of individuals with MS. Specifically, the objectives were to examine (1) participant acceptance, engagement, and adherence to the program; (2) whether participants will demonstrate improved learning and memory performance on objective and subjective measures of memory and strategy use; and (3) whether participants will report improvement in their functional status.
Method
Design
The study used a pretest–posttest design. All participants received the TELE-self-GEN treatment via a Zoom virtual video call and were required to meet with us a total of eight times, which included two evaluation sessions and six treatment sessions.
Participants
We recruited participants through an advertisement in support groups and referrals from neurologists. Participants had to be English-speaking adults ages 18 to 70 yr, diagnosed with MS, who had memory problems in everyday life (reported or documented by a neurologist) and had a computer or a device through which they could connect to Zoom with a stable internet connection. Respondents were excluded if they were participating in a cognitive intervention at the time of the study. Of the 14 people who responded, 4 declined because of the length of the activity or scheduling issues, resulting in 10 participants. Two participants were from New York City, and the other participants were from out of state or outside the city. The study was approved by the Human Research Protections Program at New York University. All participants provided verbal informed consent before testing. Their verbal consent was documented, and a copy of the consent form was sent to them by email.
Outcome Measures
A research assistant administered assessments at baseline and posttreatment evaluations. The Lowton and Brody Instrumental Activities of Daily Living (IADL) scale (Lawton & Brody, 1969) and the Patient Determined Disease Steps (PDDS; Learmonth et al., 2013) were administered only at baseline to characterize participants’ independence in daily life. The feasibility measure was administered only at the posttreatment evaluations. In addition, treatment gains were assessed within each treatment session using stimuli related to everyday memory (described below). The main outcome measures are described below.
Feasibility Measure
We used a questionnaire to assess feasibility, adapting the questions from a previously published feasibility questionnaire (Jaywant et al., 2020). The questionnaire included seven questions designed to assess treatment enjoyability, efficacy, and applicability to daily life. Each question was rated on a 5-point scale (ranging from 1 = not true to 5 = very true). In addition, two open-ended questions were included in this questionnaire asking participants what their favorite part of the treatment was and what they would change in the treatment.
Contextual Memory Test–2
The Contextual Memory Test–2 (CMT) is a web-based assessment measuring objective memory performance on immediate recall (IR) and delayed recall (DR) tasks and strategy use (Toglia, 2019). The CMT includes 20 pictures of objects related to a particular scene (including two versions: restaurant or morning). Participants are required to learn the pictures for 90 s and are asked to recall those objects immediately following learning (IR) and 15 min after the initial learning (DR). For this study, we used different sets of pictures for both the baseline and follow-up assessments to ensure that each assessment is a unique version. The dependent variables of this study were the number of objects remembered on the IR and DR and the strategy used. Norms for the CMT were established for adults from 18 to 86 yr of age. Parallel-form reliability ranged from 0.73 to 0.81, test–retest reliability ranged from 0.85 to 0.95, and concurrent validity ranged from 0.80 to 0.84 (Toglia, 1993, 2019).
Everyday Memory Rating
The Everyday Memory Rating (Toglia, 2019) is a questionnaire that is administered as a precursor to the CMT in which participants answer 10 questions related to the frequency at which they remember and keep track of everyday tasks. Each response ranges from 1 (never) to 5 (always). For this dependent variable, a lower score indicates a better everyday memory (Toglia, 1993).
Functional Behavior Profile
The Functional Behavior Profile (FBP; Baum & Edwards, 2000 ; Baum et al., 1993) is a 27-item self-report questionnaire that assesses three domains of functional status: task performance, problem-solving, and social interaction. Each item of the FBP is scored from 0 (the behavior never occurs) to 4 (the behavior always occurs). A higher score on the FBP indicates better functional behavior reported by participants. The FBP has good test–retest reliability and internal consistency (Baum & Edwards, 2000) and has been widely used with individuals with MS (e.g., Goverover et al., 2020).
Session-Based Outcome Measures
Memory Transfer
At the end of each treatment session, we tested participants on a new, thematically related set of items, distinct from the session content, to assess their ability to transfer and recall diverse information. For example, we asked them to learn the following for the various sessions: Session 1, a list of 15 unrelated written words; Session 2, a list of 10 written shopping items; Session 3, written names of 10 people with their corresponding face pictures; Session 4, five daily errands written on calendar; and Session 5, 12 written items related to the preparation of a pastry. For the final activity in Session 6, we asked participants to learn and remember items from a self-selected topic. Participants were given unrestricted time to learn these items and were asked to recall them within 5 min of learning. By asking participants to complete this activity, we could document their generalization of the learning strategy to a novel list of words or items. We converted all the correct responses into percentage scores. These memory tasks were adapted from research designed specifically to assess the transfer of memory gains (Cavallini et al., 2003, 2010).
Session Motivation and Interest Rating
At the end of each session, we assessed participants’ engagement and interest for each treatment session (Sessions 1–6) through a structured evaluation process. Using a 5-point response scale and with the therapist present, we asked participants to rate their level of motivation and interest for the day’s treatment activity, ranging from 1 (low motivation) to 5 (high motivation). In addition, we asked participants about their preference for further practice on similar tasks, using a scale from 1 (no need for additional practice) to 5 (a desire for more practice; Toglia & Foster, 2021). This two-question assessment provided insight into participants’ motivation, interest, and readiness for continued skill development.
The Intervention
We adapted the TELE-self-GEN treatment from an in-person format (Goverover et al., 2018) to an online platform by using PowerPoint slide presentations and video calls. The intervention consisted of six synchronized biweekly 1-hr treatment sessions. Treatment sessions were administered by the first author (a licensed occupational therapist; hereinafter termed the provider) via Zoom meetings while participants were at their homes and the provider was in her office.
Each of the six treatment sessions consisted of different stimuli (i.e., words within sentences, word pair associations, names and object locations, dates, making meals and managing finances, or a customized task) to be learned (see Figure 1). All six treatment sessions followed a similar format, divided into five parts (outlined in Figure 1). In Part 1, participants were introduced to the session task and were asked self-evaluation questions. In Part 2, participants were presented with the items to be learned in both the self-generated and the provided conditions, with IR and 15-min DR of both conditions. During the 15 min between IR and DR, the provider and participants discussed various topics unrelated to the study (e.g., how the participant’s day was going, upcoming weekend plans). In Part 3, the provider discussed the results of learning the two lists (presented in Part 2) with the participants and posed reflection questions. The provider also explained the strategy of self-generation and its importance. In Part 4, participants were presented with a new list of words to be remembered with the goal of applying the self-generation strategy to elicit strategy transfer and generalization. In Part 5, participants were led into a closing discussion, in which they answered self-evaluation questions posed by the provider, and completed a short journal summarizing the activity of the session, what they learned in treatment, and what they found helpful. The journal was reviewed at the beginning of the following treatment session.
Six treatment sessions and each session’s format.
Data Analysis
To evaluate feasibility and acceptability, we summarized participants’ responses to the posttreatment patient satisfaction questionnaire. In addition, we examined the number of participants who enrolled and completed the treatment to assess adherence and acceptability of the treatment. To explore clinical outcomes, because of the small sample size, we used the Wilcoxon signed rank nonparametric test to compare changes in outcome measures pre- and posttreatment (Woolson, 2008). A p < .05 was considered statistically significant.
Results
Participants
Participants included 2 men and 8 women between the ages of 34 and 68 yr (M = 51.1, SD = 12.4), with 12 to 18 yr of education (M = 15.2, SD = 2.2). Seven participants had a relapsing-remitting course of MS, and 3 participants had a secondary progressive course. Disease duration ranged from 2 to 29 yr (M = 13.1, SD = 9.2). The mean PDDS score was 2.4 (SD = 2.2; ranging from a score of 0 for normal functioning to 7 for use of wheelchair or scooter), indicating a moderate level of physical disability. According to the Lowton and Brody IADL scale, most participants were independent in their IADL, with scores ranging from 3 to 8 (M = 6.3, SD = 1.7). Three participants were employed full-time, 2 were employed part-time, 4 were on disability, and 1 was retired.
Feasibility
Of the 18 individuals with MS approached to participate in the study, 10 (55.5%) were eligible, consented, and enrolled in the study, all of whom successfully completed the entire intervention. Participants’ responses to questions about treatment feasibility and satisfaction are summarized in Table 1.
Participants’ Responses to Questions Regarding Treatment Feasibility and Satisfaction (N = 10)
Note. Likert scale: 1 = not at all true, 2 = not true, 3 = somewhat true, 4 = true, and 5 = very true.
The virtual treatment program was well-received, with most participants finding it engaging and enjoyable. Specifically, participants reported increased preparedness for learning, improved memory, and enhanced confidence in applying the acquired skills to daily tasks. Notably, the virtual format was highlighted as a particular strength. Overall, participants expressed high satisfaction with the TELE-self-GEN intervention, consistently rating it positively on all items (Likert scores above 3).
Participants described the treatment as a safe place to learn memory strategies and as a source of comfort and relief derived from the realization that their memory was not as bad as they thought it was. For example, one participant, a 67-yr-old woman, verbalized that “the treatment showed me that my memory is not as bad as I thought, and made me feel better about my memory.” Another participant stated, “I felt that I was able to perform the task. I was satisfied with my memory. . . . It improved my self-confidence.” All participants said that before participating in this intervention, they were using various ways to try to remember, such as repetition and notes. They stated that the treatment they received during this study increased their awareness of their strategy use and their attention while learning. It also taught them to associate new information with personally meaningful concepts to aid in memory retention. One participant said, “[The treatment] gave me good ways to improve my memory; I can actually see that. I’m getting better at remembering what people told me.”
Exploratory Analysis of Clinical Outcome
Memory
Table 2 presents the postintervention outcomes, revealing noteworthy improvements in both the IR and DR scores of the CMT. Effect sizes, calculated using Cohen’s d, were substantial, indicating the impact of CMT (IR scores, d = 1.18; DR scores, d = 0.87). Specifically, 9 out of 10 participants demonstrated enhanced IR scores, whereas 7 participants exhibited improvements in their DR scores postintervention. It is interesting that participants’ CMT strategy use remained consistent from pre- to postintervention. Although there was a marginal significance in the improvement of Everyday Memory Rating scores, a significant enhancement was observed in FBP total scores after the intervention (d = 0.6). Notably, the FBP problem-solving scores displayed significant improvement, and the task performance scores exhibited a positive trend. The social interaction scores remained relatively stable.
Pre- and Postintervention Comparisons of Outcome Measures (N = 10)
Note. CMT = Contextual Memory Test; IR = immediate recall; DR = delayed recall; FBP = Functional Behavior Profile.
Within-Treatment Outcome
Memory Performance Across Treatment Sessions (Transfer)
As illustrated in Figure 2, across the six sessions, there was a general increase in the percentage of correct response from Sessions 1 to 4. The percentage of correct responses roughly stabilized for Sessions 5 and 6.
Memory performance trends across treatment sessions (N = 10).
Overall, participants reported high motivation and interest in all treatment sessions, ranging from 4.2 to 4.5 out of 5. In answer to the question of whether they would need more practice on items presented in the session, participants’ responses ranged from 3.7 to 4.6 out of 5.
Discussion
This pilot proof-of-concept study evaluated the TELE-self-GEN, a remote adaptation of the in-person self-GEN treatment (Goverover et al., 2018), focusing on participant responsiveness, delivery characteristics, and treatment efficacy. The findings provide valuable preliminary data and insights regarding TELE-self-GEN’s potential for further comprehensive research study (Bowen et al., 2009). The feasibility of the intervention was supported by participants who rated the intervention as acceptable and engaging and conveyed subjective improvements in their memory and everyday life activities. Six participants noted that they would like additional treatment sessions, because they found the intervention engaging and derived benefits from the strategies they learned during treatment. Furthermore, 9 participants were satisfied with the intervention delivery method, which suggests that the telerehabilitation method was convenient or accessible for these participants. Most telecognitive rehabilitation studies on MS used a restorative approach for cognitive treatment that was asynchronous (Bossa et al., 2022) and showed positive results in cognitive performance (e.g., Bove et al., 2021). Taken together with the present results, telecognitive rehabilitation holds significant promise for individuals with MS and supports a larger trial to study effectiveness.
To further assess the efficacy of the TELE-self-GEN intervention method, we examined the outcomes of the intervention, such as improvements in memory, functional status, and patient-reported outcomes. Similar to the original study (Goverover et al., 2018), participants’ IR and DR memory performance improved, with effect sizes similar to those observed in the in-person treatment version. Participants reported that their everyday memory improved following the intervention.
An important aspect of treatment efficacy is to improve the patient’s overall functional status beyond the immediate context of the intervention (i.e., generalization; Lamargue et al., 2020). Preliminary evidence for such generalization was observed on the FBP, with significant improvement in everyday problem-solving and positive trends in everyday memory and general task performance. This FBP finding, coupled with a gradual improvement in objective recall performance across treatment sessions, suggests that the strategy learned was generalized to everyday life activities. In support of this finding, participants communicated that they were able to carry over the strategies they learned from the TELE-self-GEN treatment sessions to other activities. For example, one participant noted how she generalized the strategies learned to various activities in her life, such as remembering the names of new people she met, recalling items needed from the grocery store, and recalling the location of her belongings (e.g., wallet, keys, phone).
This proof-of-concept study had several limitations that should be addressed in future research. First, the sample size was small, which, while providing initial support, must be examined in a larger study. Second, this study did not include a measure of depressive and anxiety symptoms, which are known to affect memory performance and complaints. Future studies should incorporate assessments of these factors to control for their potential effects on memory performance. Third, we did not assess the long-term effect of the intervention, which should be evaluated in future work. Lastly, although not the focus of this study, the lack of a control group limited the ability to rule out the presence of a practice effect or other factors that may have contributed to the observed improvement in the study outcomes. Including a control group and utilizing an experimental study design would allow for a more robust assessment of the efficacy of the intervention.
Implications for Occupational Therapy Practice
This study has the following implications for occupational therapy practice: Introducing a telecognitive intervention specifically designed for individuals with MS with everyday memory impairments offers occupational therapists a promising approach to addressing their clients’ memory and cognitive challenges. Further study of the efficacy of the TELE-self-GEN is required to support its use in occupational therapy practice with these clients.
Conclusion
The results of this proof-of-concept pilot study highlight the promising potential of TELE-self-GEN as an effective intervention for individuals with MS facing everyday memory challenges. Participants not only reported subjective enhancements in memory and strategy utilization but also demonstrated objective improvements in memory performance, leading to enhanced functional status. These encouraging results serve as a foundation, paving the way for more extensive clinical trials of TELE-self-GEN.