Abstract
The results of this study indicate that patients undergoing memory rehabilitation may benefit from methods to enhance attentional engagement during mobile application skill learning when executive dysfunction is a considerable element of their cognitive profile.
Many neurological conditions affect memory functioning, including acquired brain injury (ABI; e.g., stroke, encephalitis, traumatic brain injury [TBI]) and mild cognitive impairment (MCI). Impairments affecting prospective memory, or the ability to remember to carry out intended actions in the future, are problematic in maintaining independent functional capacity and quality of life. The current best practice for memory rehabilitation is to apply a compensatory approach with external memory aids (Cicerone et al., 2019). Mobile applications in smartphones can provide alerting cues in the form of reminders to attend appointments or complete tasks at particular times. Numerous studies have demonstrated that technology-based memory aids are successful at improving functional memory (Charters et al., 2015; de Joode et al., 2010; Ferguson et al., 2015), and a meta-analysis demonstrated a large effect size (d = 1.27) in favor of technological compensation for everyday tasks requiring memory (Jamieson et al., 2014).
Smartphones are now widely used in developed countries, making them powerful and accessible tools for people with brain injury to adopt as prosthetic technology. A critical obstacle, however, is that for those with moderate to severe learning and memory impairments, acquiring the necessary application skills is a considerable barrier to implementing a compensation strategy. Explicit memory impairments disrupt learning, but step-based skills (e.g., mobile application use) can still be acquired with sufficient practice through the implicit procedural memory system (Glisky & Schacter, 1987; Squire & Zola, 1996). Moreover, efficiency of learning in amnesia is enhanced with the integration of errorless learning, or guiding responses to prevent mistakes before they might occur. Of consideration, it has been proposed that executive dysfunction may impede implicit learning (Butters et al., 1985).
More recently, and of relevance to memory compensation training, it has been shown that greater learning challenges are encountered when executive deficits are present, in addition to considerable memory impairment (Svoboda et al., 2012; Vasquez et al., 2022). Executive dysfunction may act as a barrier to learning (Sarazin et al., 2002), manifested as difficulty staying on task or decreased attentional engagement during training sessions. This hypothesis is congruent with the adaptive control of thoughts theory of procedural learning, which states that executive functioning is required to support the acquisition of new skills before they can be established as a fully automatic process (Anderson, 1999).
Attentional engagement is known to be a critical factor in the facilitation of cognitive intervention or strategy training; however, a gap exists in knowledge relevant to the learning of procedurally based skills. In previous studies investigating the role of attention in implicit learning, researchers focused on healthy adult samples and found inconsistent results using a variety of experimental tasks, with some relying on attention (Jiang & Chun, 2001; Stadler, 1995) and others showing only minimal interaction (Mulligan, 1998; Rausei et al., 2007). A meta-analysis examining divided attention manipulations of experimental repetition priming tasks revealed a small, but significant, negative effect on implicit memory (Spataro et al., 2011). In another study, Franklin et al. (2016) attempted to circumvent previous confounds; they used an experience sampling approach to assess lapses in attention (i.e., mind wandering) during an implicit learning serial reaction time task, revealing that greater task focus was associated with improved implicit sequence learning. It is currently unknown whether and how attentional functioning interacts with implicit learning among patients with memory impairment and in the context of acquiring a real-world functional skill in cognitive rehabilitation (i.e., smartphone memory aid use).
In this case series study, we investigated attentional engagement during the skill acquisition phase of a memory intervention program among a heterogeneous sample of people experiencing memory difficulties. We quantified attentional engagement using a single-channel dry electroencephalographic (EEG) headset and a novel EEG marker of sustained attention: the Brain Engagement Index (BEI), computed in real time as a marker for attention on task. Substantial clinical experience and validation research has been acquired with the BEI in various rehabilitation and clinical settings, including physical, psychiatric, and cognitive rehabilitation (Bartur et al., 2017, 2020; Gvion et al., 2021; Isserles et al., 2018; Karpin et al., 2022; Shahaf, Kuperman, et al., 2018; Shahaf, Nitzan, et al., 2018 ; Shahaf et al., 2017).
The main goal of this study was to explore the interaction between attentional engagement and skill learning and to assess its potential importance to compensatory memory rehabilitation. We sought to determine whether slow learner participants with memory impairment and executive dysfunction would require greater engagement to obtain skill learning in mobile application use, compared with fast learner participants with less severe focal memory impairments. Moreover, we expected attentional engagement to change over the course of learning a skill. For example, acquiring the skill set to use a given smartphone application (as a memory aid) progresses over trials, with the assumption that the procedural ability transitions from initially requiring conscious cognitive effort to becoming automatic (Anderson, 1999). Thus, we predicted that as the steps of using the application became habitual, demand on attentional resources would decrease while interacting with the application.
Method
Participants
Ten patients (M age = 56.2 yr, SD = 15.1) participated in the study, which was approved by the Baycrest Hospital research ethics board. Participants with ABI were recruited from a memory intervention program waitlist, consisting of community-referred patients with memory dysfunction. Etiologies included TBI (1 participant), brain cancer (2 participants), cavernoma hemorrhage (1 participant), anti-GABA-B encephalitis (1 participant), and the genetic cerebrovascular condition cerebral autosomal dominant arteriopathy with subcortical infarcts and leukoencephalopathy; 1 participant. Three participants with MCI were recruited after the completion of another cognitive rehabilitation program in Baycrest Hospital’s neuropsychology department that focused on internal strategies. Three participants were excluded from analyses: 1 participant with MCI (F5) discontinued the study, another participant with MCI (F6) had too few trials with a valid BEI signal, and 1 participant with ABI (F7) was trained on a journaling (rather than calendar) application. Finally, 1 healthy older adult was included in the study. Thus, data are presented for a total of 6 participants with memory impairment and 1 older adult participant representing typical age-related memory changes. All participants owned and operated a functioning smartphone and were able to wear the EEG headset comfortably for the duration of sessions.
Participants were categorized into two groups on the basis of the speed at which they progressed through training. The first group (fast learners, n = 4) reached our learning criterion for core application steps in three or fewer sessions and were considered to demonstrate typical training experiences. The 1 healthy older adult was included in this group on the basis of similarities in learning profile and BEI dynamics. The second group (slow learners, n = 3) reached our learning criterion for core application steps in seven or more sessions or did not reach it. Each participant is described at the case level in the Results section.
Memory Intervention
Memory rehabilitation was provided through the Memory Link program at Baycrest (Richards et al., 2020), which specializes in training patients with moderate to severe memory impairment to use digital devices (e.g., smartphones) as external memory aids. The goal of the program is to facilitate the acquisition of device application skills that can be applied to compensate for lasting memory difficulties, ultimately leading to improved functioning and independence.
The main component of the program involves the acquisition of procedural skills in using applications on a digital device through large amounts of practice (e.g., 10 training trials per twice-weekly sessions). Typically, calendar applications are taught because they can be used flexibly as a powerful memory aid. Phase 1 of the program involves participants learning how to operate the application, whereas Phase 2 focuses on applying these new skills in real-life situations. Because in this study we focused on engagement during skill learning (Phase 1), no generalization (Phase 2) data are included or discussed in this article. Evidence-based principles of errorless learning (Wilson et al., 1994) and vanishing cues (Glisky et al., 1986) are integrated to facilitate the acquisition of application skills. Clients were not permitted to engage in unsupervised practice on their own between sessions in accordance with the errorless learning protocol.
Successful training with the Memory Link method can be lengthy. Clients are seen for two 1-hr sessions per week until they complete the program, which typically takes between 12 and 16 wk. To create a reasonable and equivalent learning endpoint for comparative analysis, we examined only data for the core calendar event entry steps (title, time, location, notification, and notes) in this study. Given the time restrictions on data collection, non–Memory Link clients were provided training for up to 14 wk (28 sessions), which is the time required for the average client to fully complete Memory Link training. Clients who were referred to the program by typical means continued training beyond this period. For the healthy older adult participant, six training sessions were provided (excluding the cognitive assessment), which was more than enough to reach our learning criterion. This participant was compensated with a $100 gift card.
Brain Engagement Index
Described in detail elsewhere (Bartur et al., 2017), a dry, two-lead EEG headset (NeuroSky Inc., San Jose, CA) was used to capture prefrontal neural activity. The live signal was transferred via Bluetooth® connection to a nearby computer, which calculated the BEI in real time. The BEI has a range of 0 to 1, calibrated to be (1) low (<0.3) when a person is in a state of attention deficit or depression (Isserles et al., 2018; Shahaf, Nitzan, et al., 2018 ; Shahaf et al., 2017), (2) high (>0.7) when a person is anxious or stressed (Shahaf, Kuperman, et al., 2018), and (3) midrange (≥0.3 and ≤0.7) when a client is effectively engaged (Bartur et al., 2017).
BEI was integrated into the Memory Link protocol to have as little impact on intervention sessions as possible. At the start of each session, the mobile EEG headset was positioned on the participant’s head and connected wirelessly to a laptop computer via Bluetooth before running BrainMARC’s (Yokneam, Israel) BEI software. If signal strength was low, a set of troubleshooting procedures was introduced, including adjusting the headset on the participant, cleaning the headset or participant forehead, restarting the mobile EEG device, and so forth. After appropriate signal strength was confirmed (for each trial), a start marker was created in the BEI software, and the trial was initiated. At the completion of each trial, an end marker was created in the BEI software. Markers permitted the attribution of BEI data to EEG activity occurring during training trials while separating BEI data collected between active training trials.
Outcome Measures
Brain Engagement Index Ratio
BEI was measured continuously during each memory intervention session. In this analysis, we used the BEI ratio, which is the proportion of BEI points that were between 0.3 and 0.7 for a given trial or session. BEI values between 0.3 and 0.7 are considered to be in the optimal range for effective task engagement; thus, a higher BEI ratio is favorable. Only data with a signal quality >2/6 were used in calculations of BEI scores. This procedure was done to ensure that noisy data were removed and only reliable BEI measures were included in the analyses.
Memory Link Skill Learning: Percent Independent Completion
The percent independent completion (PIC) was the primary outcome measure used to determine training aptitude and learning, and it is an indication of the degree to which a trainee completes a trial or session independently of trainer support. A single trial consists of all the steps of entering information into the given application, and level of support is recorded for each step with a cueing hierarchy of 0 to 4, with higher values indicating more support required. PIC is calculated with the following formula: PIC = [1 − (average numeric value of cue/4)] × 100. A criterion level of 98 PIC is used to recognize that the steps have been learned.
Neuropsychological Assessment
A neuropsychological assessment was conducted on a separate day after smartphone application training had begun (Table 1). Three of the 7 participants declined the neuropsychological assessment (1 with a tumor, 1 with CADASIL, and 1 with anti-GABA-B encephalitis). The 1 participant with a tumor who declined allowed us to use data from a previous brief assessment, and so only partial data are presented. Three participants were determined to have more complex cognitive profiles consisting of memory impairment and executive dysfunction; they were also categorized as slow learners: 1 participant with TBI (S1), 1 participant with MCI (S2), and 1 participant with CADASIL (S3).
Neuropsychological Test Data
Note. Values are T scores with raw scores in parentheses. Note that neuropsychological evaluation was not completed for 2 participants: 1 with acquired brain injury (ABI; Participant S3) and 1 with mild cognitive impairment (Participant F3). One participant with ABI had partial data (Participant F2). Participant F3 had a previous neuropsychological evaluation but no raw test data available. The summary of results is reported in their case description. BVMT–R = Brief Visuospatial Memory Test–Revised; HVLT–R = Hopkins Verbal Learning Test–Revised; WAIS = Wechsler Adult Intelligence Scale; WAIS–III = Wechsler Adult Intelligence Scale–Third Edition; WCST = Wisconsin Card Sorting Test.
Not an accurate estimation of premorbid functioning; low score likely because of English as a second language.
Denotes exceptionally low scores.
Denotes below average scores.
Statistical Analysis
Confirming Skill Acquisition of Application Steps
A Wilcoxon signed-rank test was used to assess improvement of mean PIC from the first session to the session in which criterion was reached (i.e., 98 PIC). The effect size was estimated as the ratio between the mean difference and the standard deviation of the difference (standardized response mean) where values of 0.2, 0.5, and 0.8 represent small, moderate, and large effect size cutoffs, respectively (Husted et al., 2000). One participant (with CADASIL) was excluded from the analysis because they never reached criterion and were quite far from it. Although another participant with MCI did not reach the criterion level officially, their peak performance was very close (96.8 PIC), and they were included in the analysis.
Subgroup Comparisons (Percent Independent Completion and Brain Engagement Index Data)
Data visualization involved scatterplots with fitted simple linear regression lines for BEI ratio and PIC versus session to obtain estimates of the slope and intercept for each participant. We evaluated the difference in the slopes of PIC and BEI ratio between fast and slow learners using a Wilcoxon rank sum (Mann–Whitney U) test with estimated effect sizes calculated as the z statistic divided by the square root of the total sample size. Data processing and calculation of PIC and BEI ratios were conducted in RStudio, Version 1.4.1717. We generated the graphical output for this article using SAS software, Version 9.4.
Results
Fast Learners
Participant F1
A 56-yr-old English-speaking man with 24 yr of education had a corpus callosum cavernoma hemorrhage requiring surgical resection. He underwent right frontal and right posterior parietal craniotomies and cavernoma resections, which resulted in bilateral medial and orbital frontal infarcts, extending to the fornices and optic chiasm, and also bilateral infarcts in the peri-sylvian insular cortex. He was referred for memory intervention to address considerable functional memory concerns that developed postsurgery. Specifically, he was endorsing symptoms of decreased short- and long-term memory, increased distractibility, slowed reading, fatigue, and anxiety. A neuropsychological evaluation revealed a memory impairment relatively isolated to delayed recall of material. Application training was started 22 mo postinjury.
Participant F2
A 21-yr-old English-speaking man with 14 yr of education had a history of thalamic glioma from age 6 yr. Over the years, he had developed hydrocephalus and underwent multiple shunt insertions and revisions. Memory problems had been noticed since age 8 yr, and functional changes were noted to align with surgery dates. The most considerable memory alteration was documented after tumor debulking 10 yr prior. The tumor had been stable for the past 6 yr. He reported trouble remembering important dates, phone numbers, people’s names, and things to do during the day, often forgetting information within 5 to 10 min. He relied quite a bit on his mother to support his memory functioning day to day. A neuropsychological evaluation revealed deficits primarily in processing speed as well as verbal and visual learning and memory. Application training was initiated 10 yr postsurgery related to the onset of notable memory dysfunction.
Participant F3
A 50-yr-old English-speaking woman was previously diagnosed with anti-GABA-B encephalitis. She initially presented with a first-ever seizure, focal with secondary generalization, and additional seizures occurring in the weeks that followed, leading to confusion and poor memory. Her diagnosis was confirmed with a lumbar puncture. Subsequent MRIs revealed signal abnormalities in the bilateral anterior temporal pole and left medial temporal lobe swelling, consistent with a diagnosis of limbic encephalitis. She was treated medically with intravenous immunoglobulin therapy. This participant declined cognitive testing within the study, but she had completed a full neuropsychological evaluation approximately 1 yr before. Mild impairments were noted in fine motor speed (nondominant hand), psychomotor speed, verbal learning and memory of unstructured information (i.e., a word list), delayed recall of structured information (i.e., short stories), abstract visual reasoning, and abstract problem solving. Impairments in the moderate range were found in the areas of incidental visual memory and basic visual–perceptual skill. Application training was started approximately 2 yr after initial diagnosis.
Participant F4
A 68-yr-old English-speaking woman with 14 yr of education was recruited from the community specifically for this study. She endorsed no memory concerns or any other cognitive changes, and she had no functional decline. No history of neurological conditions was reported. No impairments were found on cognitive testing. The participant was included to represent a non–brain injured comparator, displaying only age-normative memory changes.
Slow Learners
Participant S1: Skill Learning Success
A 51-yr-old English-speaking man with 17 yr of education incurred severe TBI in a transportation accident. At the scene of the accident, his Glasgow Coma Scale score was 9. Neuroimaging revealed a bifrontal subdural hygroma, a subarachnoid hemorrhage, and minor midline shift to the right. A follow-up computerized tomography scan identified a right inferior frontal density and left frontotemporal parietal convexity. He spent about 6 wk in an acute care hospital before beginning inpatient rehabilitation at another institution. He was discharged home 9 mo after his injury. At the time he was enrolled in the memory intervention program, he was living alone, but he had many community and health care supports, including visits to his home. He reported several functional memory difficulties in daily life, including forgetting to complete daily tasks, to take medication, to do laundry, and even to eat. He also had difficulties with time management; for example, he would often be late to appointments. Objective neuropsychological evaluation revealed severe learning and memory impairments, as well as deficits in executive abilities, in the context of intact intellectual functioning. Application training was started approximately 14 mo after injury.
Participant S2: Skill Learning Prolonged
A 66-yr-old native Hebrew-speaking (English fluent) man with 14 yr of education was diagnosed with MCI, single domain with executive dysfunction. He had a history of depression, which at various points in time interfered with his ability to function. His mood at the time of study enrollment was much improved. Etiology of his condition was unclear, and neuroimaging was not available. At the time of diagnosis, he reported a 2-yr history of self-perceived forgetfulness. He endorsed several memory and attention difficulties in daily life, including forgetting to lock the door, forgetting where he placed items, and poor concentration. Objective neuropsychological evaluation revealed impairments across a variety of memory and executive functioning tasks in the context of average intellectual ability. Application training was started 30 mo after diagnosis.
Participant S3: Skill Learning Impeded
A 67-yr-old English-speaking man with 17 yr of education was diagnosed with the hereditary cerebrovascular condition CADASIL approximately 3 yr before study enrollment. This participant had two previous strokes and another more recent stroke in the left frontal region. At the time of study participation, he was living alone in a condo; however, he had family support in close proximity and a personal support worker at his home multiple times per week. Difficulties were reported with orientation, scheduling, wayfinding, arranging transportation, prospective memory, and new learning. Additionally, his family reported that he had poor organization and often forgot whether he took his medications and paid his bills. This participant declined a neuropsychological evaluation, and so no cognitive data are available; however, behavioral observation was highly indicative of considerable cognitive impairment, demonstrated in the areas of learning and memory as well as higher order executive abilities. Application training was initiated 7 mo after his most recent stroke.
Confirming Skill Acquisition of Application Steps
For fast learners (Participants F1–F4), skill learning progressed well. These participants reached the criterion level for the core application steps (Stage 1) in two to three sessions. Their baseline performance was relatively high, indicating that a low amount of support was required for them to complete application steps on the first training session. With regard to attentional engagement, a negative BEI ratio slope was observed across sessions (Figure 1). It appears as if fast learners who reached criterion in only two sessions had an even sharper slope than those who took three sessions to reach the learning criterion.
Mean percent independent completion (PIC) and Brain Engagement Index (BEI) ratio by session for the fast (F) learner group.
For slow learners (Participants S1–S3), although initial performance was lower than the fast learner group, the rate of learning was also slower but steady. Participant S1 reached the criterion level for the core application steps (Stage 1) in 68 trials (approximately seven sessions), whereas Participant S2 reached a peak of 96.78 PIC in five sessions. Participant S3 did not approach the learning criterion within the time frame of the study; he had a baseline PIC of 40.91 and reached a PIC of 66.02 after 10 sessions. Unlike the fast learners who consistently displayed negative BEI slopes, slow learners demonstrated BEI slopes of near zero (Figure 2). It is important to note that a high BEI ratio (>50%) was maintained across training sessions (Bartur et al., 2020). A sustained high BEI ratio suggests that attentional engagement was present during skill learning and that equally high levels were required throughout the process, regardless of learning progression.
Mean percent independent completion (PIC) and Brain Engagement Index (BEI) ratio by session for the slow (S) learner group.
Overall, a significant 12.95% mean increase in PIC was found from initial training to the session in which criterion was reached (SD of the difference = 6.42, Wilcoxon signed-rank p = .028, effect size = 2.02 [very large effect size]). Significantly less variability in PIC was found at the session in which criterion was reached (SD = 2.83) compared with the initial training session (SD = 9.73), which demonstrates that participants were able to consistently perform the necessary steps involved in using the application after it was learned (SD of the difference = 4.21, Wilcoxon signed-rank p = .028, effect size = −1.64 [very large effect size]). These data demonstrate that participants learned the steps needed to be able to consistently operate the application without support.
Subgroup Comparisons (Percent Independent Completion and Brain Engagement Index Data)
Differences were found in the mean PIC and BEI ratio slopes between the fast and slow learner groups (Table 2). PIC increased with subsequent sessions in both the fast learner group (M = 7.5, SD = 3.9, Mdn = 6.5) and slow learner group (M = 2.0, SD = 1.7, Mdn = 1.0), in which the slope of the fast learner group was determined to be greater than that of the slow learner group (Wilcoxon rank sum test p = .036, effect size = 0.68 [medium effect size]). The BEI ratio showed a steep decrease with subsequent sessions in the fast learner group (M = −12.0, SD = 9.4, Mdn = −10.0), which distinctly differed from the slow learner group, who showed minimal to no change (M = 0.3, SD = 0.6, Mdn = 0.0) with subsequent sessions (Wilcoxon rank sum p = .025, effect size = 0.74 [medium effect size]).
PIC and BEI Ratio Slopes Between Fast and Slow Learner Groups
Note. BEI = Brain Engagement Index; IQR = interquartile range; PIC = percent independent completion.
Discussion
Attentional engagement is a potentially important element in cognitive rehabilitation for people with memory impairment. Modern-day smartphone applications, such as calendars, offer convenient features for external memory compensation. Given that patients with memory impairment can require considerable training time to learn the use of such mobile applications, consistent engagement throughout skill learning may be essential. In this study, we examined attentional engagement throughout skill learning of mobile applications within an established evidence-based memory intervention program. Attentional engagement was measured through an EEG-based template matching algorithm (the BEI), which was easily integrated into the memory intervention protocol. We found that attentional engagement was evoked in the vast majority of sessions, regardless of participant profile.
Relationship Between Engagement and Learning
The exploratory examination of BEI ratio and PIC revealed an interaction between participant groups. Fast learners reached criterion quickly, which was also associated with an attentional decrease over training sessions. That is, the fast learner group displayed a negative association between BEI and PIC trend lines across sessions, indicating a greater attentional engagement early in training, which declined across sessions. These participants also tended to have more straightforward cognitive profiles, consisting of milder memory deficits and greater (more intact) executive abilities.
The slow learners maintained a stable and high attentional engagement during most sessions while also improving on the behavioral learning measure (PIC). All of the slow learners had flat or nearly flat slopes for BEI trend lines across sessions, yet they still demonstrated positive learning slopes for PIC. These participants (especially Participant S3) demonstrated considerable learning difficulties that were attributed to their more complex cognitive profiles, which included executive functioning impairments. They also had markedly different BEI dynamics exhibited by consistent attentional engagement throughout learning.
These data suggest that for patients with milder cognitive dysfunction and relatively isolated memory impairment, attentional effort may be preferentially required early on for skill learning. For these patients, learning occurs swiftly without any barriers to progression; thus, monitoring attentional engagement would likely not provide useful insights for improvement here. Alternatively, patients with more complex cognitive profiles may depend on consistent and effective implementation of attentional effort for skill learning to progress well. In this way, executive dysfunction (in addition to memory) may act as a barrier to learning. Moreover, attention may be integral to executing procedural steps for clients with complex cognitive profiles, even after the task or specific steps have been learned. Thus, finding ways to improve attentional engagement has the potential to enhance performance and learning gains in a subgroup of patients with memory impairment.
Trajectory of Attentional Engagement During Skill Acquisition
Training patients with memory impairment to use mobile applications with many steps benefits from procedural learning through extensive practice. The three phases of procedural learning include cognitive, associative, and autonomous (Anderson, 1999). According to this model, complex cognitive abilities are recruited to support the acquisition of new skills in the first two phases before they become automatic in the final phase. Hence, we hypothesized that as skill learning of a given mobile application progressed, attentional engagement would decrease because of the ability of the trainee to complete more steps automatically without thinking deeply about them. The present findings for fast learners provide support for this theory of cognitive procedural learning in an ecologically valid task (mobile application operation). Attentional engagement was observed to decrease across sessions as skill learning progressed for all of the fast learners, suggesting that as the steps of application use became more automatic, there was less demand for attentional resources to aid in performance. These findings are the first to provide EEG evidence in support of the cognitive procedural learning model.
Limitations
The main limitation to this study was the case-based approach to evaluation. The intervention program was very intensive, requiring participants to come in for 1-hr sessions twice per week for several weeks until learning was completed. The quantity of data collected per person was sufficiently large, but the number of participants recruited was limited by staffing, resources, and time. Another limitation was the lack of participant diversity with regard to level of cognitive impairment. The present results would benefit from a replication and, if possible, a larger participant sample that includes a wide variety of memory impairment severity levels.
Implications for Occupational Therapy Practice
The present findings demonstrate the importance of evaluating attentional engagement among patients with more cognitively complex profiles. Executive dysfunction may represent a considerable learning barrier above and beyond memory impairment, requiring increased attentional engagement on task to overcome. Thus, the results of this study have the following implications for occupational therapy practice: Clinicians may see greater rehabilitative success among patients with executive dysfunction if they ensure that active attentional engagement is optimized whenever possible. Applying techniques to bolster engagement during training may improve learning efficiency among patients with complex cognitive profiles. For example, implementing periodic auditory alerts that are general and nonspecific has been shown to boost attention on task in previous experimental paradigms (Manly et al., 2002). Alternatively, breaking learning sets down into smaller chunks with fewer steps may allow similar patients with complex cognitive profiles to maintain focus throughout a trial. Moreover, relying on a greater integration of role play during sessions may increase relevance and interest, leading to elevations in engagement, and research has shown benefits to active client participation (Hu et al., 2009; Lequerica et al., 2009; Pyöriä et al., 2007). Real-time monitoring of brain engagement may be another avenue of intervention to determine when attention drops off during learning, which would allow the trainer to adapt in the moment. Further developments in EEG monitoring technology have allowed for the automatic identification of barriers when attention is disengaged and recommendations to overcome them (Gvion & Shahaf, 2021). There may also be potential for EEG-based brain engagement measures to identify the point at which automatization has been reached, signaling readiness to move on to a higher training stage or real-world use. Although the participants with complex cognitive profiles (slow learners) in this study did not show decreased attentional engagement as learning progressed, it is possible that they were simply very delayed in reaching this state. Further research would be required to explore this hypothesis.
Conclusion
In this study, we demonstrated the utility of the BEI in the context of an evidence-based memory intervention program that trains patients to use mobile technology as compensatory aids. The results highlight the relevance of attentional engagement in implicit errorless learning of a real-world functional skill. Attentional engagement was shown to decrease throughout skill learning of a mobile application for patients classified as fast learners. The findings support the notion that in short learning periods that progress quickly, procedural tasks require less attentional engagement as skills are acquired; however, patients with complex cognitive profiles including executive dysfunction (slow learners) evinced consistent and high attentional engagement throughout learning, with a flat slope that did not change across sessions. These patients required extensive training and likely did not reach the point of automatization within the study. Before automaticity of specific steps is established, skill learning may be more dependent on attentional engagement (as measured by the BEI). It may be important to monitor attentional engagement when the client encounters barriers, allowing intervention adjustments to facilitate advancement.
Footnotes
Acknowledgments
We thank Alaina Thomas and Jaanuha Sritharan for help with data collection. This research was supported by funding from the Centre for Aging and Brain Health Innovation–I2P2 (Illness and Injury Prevention Program) Canada–Israel Collaboration Program. Brandon P. Vasquez runs the memory intervention program featured in this study, for which a clinical manual is available for purchase; however, no commercial or financial conflicts of interest exist. Goded Shahaf is a cofounder of BrainMARC, the company that created the electroencephalographic marker (the Brain Engagement Index) used in this research.