Abstract
BACKGROUND:
Electronic cognitive assessment tools present potential benefits for clinical practice; however, they warrant examination before use with clinical populations such as people with traumatic brain injury (TBI).
OBJECTIVE:
The primary study purpose was to compare results from a tablet-based, electronic cognitive assessment to two paper cognitive assessments when administered to adults with TBI. We also explored the effect of iPad comfort on performance.
METHODS:
We employed a quasi-experimental, correlational study design. Forty adults between 18 to 615 months post TBI completed the Standardized Touchscreen Assessment of Cognition (STAC), the Montreal Cognitive Assessment (MoCA), and the Cognitive Linguistic Quick Test (CLQT) in a systematically, counterbalanced order. We compared participants’ performance on these tools and examined the effect of iPad comfort.
RESULTS:
Three STAC subtests had a good relationship with CLQT subtests: orientation, generative naming category, and generative naming first letter. A good relationship was also identified between two STAC and two MoCA subtests: orientation and generative naming first letter. People who were very comfortable using the iPad performed statistically better on the STAC first letter fluency item than participants who were not comfortable.
CONCLUSIONS:
Moderate correlations suggest validity for some STAC items; however, modifications and further testing are needed.
Introduction
Many people with traumatic brain injury (TBI) report persistent cognitive impairments that affect their completion of activities of daily living (DeLisa, Gans, & Walsh, 2005; Goverover, Genova, Smith, Chiaravalloti, & Lengenfelder, 2017). Evaluation of cognitive abilities is critical for effective delivery of rehabilitation assessment and selection of intervention strategies. Many clinicians administer screening tools or short assessments to develop a broad understanding of areas of cognition requiring further investigation (de Guise et al., 2014; de Guise et al., 2013). Additionally, screening tools may be used for people with chronic TBI when receiving evaluations for new services or other conditions that require them to receive medical care when previous medical records are not available.
Potential clinical benefits exist related to the use of electronic cognitive assessment tools (Bauer et al., 2012; De Marco & Broshek, 2016; Wild, Howieson, Webbe, Seelye, & Kaye, 2008). Electronic cognitive assessment tools that can be self-administered may help clinicians manage workload demands and quickly document cognitive abilities (de Guise et al., 2013). Given the increased use of electronic devices in clinical services (Erickson, 2015; Hansen, Bjornsen, & DeVeney, 2017; Stark & Warburton, 2016), an electronic assessment tool may be a valuable clinical resource. These tools include software programs administered via computer and applications administered via electronic tablets. We refer to these tools as electronic assessments as a general term and as computerized programs or tablets when specificity is appropriate.
Comparison of electronic assessments to paper assessments
Some comparisons of electronic and paper cognitive assessments involve healthy, neurotypical adults. These investigations can be helpful at providing normative data and as a foundation for future comparisons in clinical populations. One such study compared healthy adults’ and athletes’ performance on CogSport, an electronic cognitive assessment developed to monitor cognitive recovery following concussion, and two paper neuropsychological assessment tools (Collie et al., 2003). The paper assessments, often used in studies aiming to measure cognitive effects of concussions from sports-related activities, were the Digit Symbol Substitution Test and the Trail Making Test-Part B. The CogSport measures of decision making speed and working memory demonstrated high correlations with the Digit Symbol Substitution Test (r = –0.86 and –0.72) for elite athletes. However, the CogSport measures had low correlations with the Trail Making Test-Part B performance. These results provided some preliminary support for using CogSport, however, the authors hypothesized that limitations in traditional cognitive tests post-concussion could have contributed to the poor correlations especially if Cogsport items were more sensitive (Collie, Darby, & Marnuff, 2001). Given the mixed results further investigation is needed to determine the diagnostic capabilities of Cogsport.
Extension of this research to clinical populations is critical to their eventual adoption in clinical practice. Therefore, researchers have evaluated electronic cognitive assessment tools in clinical populations, such as people with TBI. Some tools are designed to measure cognitive effects of mild TBIs or concussions and some are appropriate for people with all severities of TBI.
Makdissi and colleagues (2001) conducted a study and included neurotypical adults and people with brain injury. The researchers compared the CogState Simple Reaction Time Test to two paper tasks (i.e., Digit Symbol Substitution Test and Trail Making Test-Part B). The researchers selected the Simple Research Time Test because electronic assessments are believed to have an advantage over paper and pencil assessments in that they precisely measure reaction time. Reaction time is believed to be sensitive to mild changes in cognition following mild TBI. The researchers compared individual change scores from before and after concussions for six rugby players and seven players without concussions. Moderate and small correlations were identified within the control and concussion groups respectively. The authors concluded that this was because the electronic assessment of reaction time was more sensitive to cognitive changes than the paper assessments that only provided information about accuracy.
Keith, Stanislav, and Wesnes (1998) investigated the Cognitive Drug Research electronic assessment tool comparing scores achieved by 15 people with TBI to their scores on Intelligence Quotient Testing and the Mini-Mental State Exam. Three components were significantly correlated with the Mini-Mental State Exam: choice reaction time, spatial working memory, and word recognition with correlations ranging from 0.54 to 0.94 for these items. Of note, the remaining five components did not have significant correlations with the Mini-Mental State Exam, and no components correlated with the Intelligence Quotient Testing. At the time of this study, the Mini-Mental State Exam was one of the most widely administered cognitive screening tools thus demonstrating the potential value of the computerized tool with strong correlations. However, the authors note that some of the limitations of the Mini-Mental State Exam, despite its wide-spread use, may have affected results. These limitations include: memory testing that only measures delayed recall, lack of age or education adjustment, and inadequate standardization for administration. Therefore, when conducting studies that make these types of comparisons, it may be valuable to make comparisons among more than one tool given the limitations of some of the cognitive assessment tools.
Most comparisons that have been made in previous literature are between paper and cognitive assessments that are delivered via computer. Additional research is needed evaluating the use of touchscreen, computerized tablets designed to measure cognitive performance (e.g., Zhang, Red, Lin, Patel, & Sereno, 2013) These tools offer greater flexibility in terms of portability and cost; however, should be evaluated separately as it is possible that computerized assessments using a keyboard and mouse result in different performance than assessments that use touchscreen access. While initial research in the validation of computerized and tablet-based cognitive assessments demonstrates moderate correlations between electronic and paper versions, clinicians must choose carefully and additional research is needed on each new tool that is brought to market.
Standardized touchscreen assessment for cognition
Of particular interest for the current study was work completed by Wallace and colleagues (2017) that explored the performance by neurotypical adults on the Standardized Touchscreen Assessment of Cognition (STAC; Cognitive Innovations LLC, 2013), a criterion-referenced, tablet-based tool that assesses cognitive and linguistic functions. The researchers needed to compare the STAC to other paper tools that were not an exact match for the STAC because no paper version of the STAC exists. Comparisons to commonly used assessment tools with overlapping areas of measurement are needed to provide convergent validity. Therefore, the researchers explored the relationship between the STAC and the Cognitive Linguistic Quick Test (CLQT; Helm-Estabrooks, 2001) and Cognitive Assessment of Minnesota (CAM; Rustad et al., 1995). Results from 88 adults without brain injury suggested the strongest correlations were between the STAC and CLQT subtests measuring generative naming first letter and generative naming category and weaker correlations for the CAM subtests measuring visual working memory, auditory working memory, delayed visual memory, and delayed auditory memory. Finally, high levels of iPad comfort were associated with higher subtest scores. The results showed some correlations to paper assessments, and some relationship between STAC performance and iPad comfort. Extension of this research to clinical populations and to understand participant reported usability is critical to support applications in clinical practice.
Current study
In the current study, we aimed to investigate the STAC for individuals with TBI to explore its convergent validity with two commonly used measures. Additionally, because reported iPad comfort prior to administration was associated with higher test scores previously, we sought to examine this variable in people with TBI. We also asked participants to evaluate the usability of the STAC to help determine if people with TBI perceive the tool to be appropriate for administration. The specific research questions were: What is the relationship among the STAC subtests and the commonly administered paper subtests? Do STAC subtest scores differ based on participant’s reported iPad comfort prior to administration? How do participants with TBI describe the STAC’s usability?
Methods
Within this prospective study, we employed a quasi-experimental, correlational study design. The study was approved by the University Institution Review Board.
Participants
A convenience sample of forty people with chronic TBI were included in this study. We determined our target sample size by considering feasibility for this exploratory study in a clinical population and sample sizes reported in previous literature (Clark, Anderson, Nalder, Arshad, & Dawson, 1995; Zgaljardic et al., 2013). Between 2015 and 2016, participants were recruited through flyers distributed to Midwestern TBI service providers including outpatient rehabilitation facilities, support groups, and assisted living facilities. Inclusion criteria stated that participants must: self-report hospitalization of more than 24 hours following injury, self-report post-traumatic amnesia (PTA) of more than 24 hours, be between 18 and 65 years of age, be at least 6 months post injury, and speak American English. Exclusion criteria included: demonstration of behaviors below Rancho Los Amigos Scale of Cognitive Functioning-Revised (RLA) Level 5 (Hagan, 200), performance below 90% accuracy on a standardized confrontation naming task on the CLQT, performance below 100% on the Nine Hole Peg Test with dominant hand, and performance below 100% on a vision screening requiring them to report the direction of the letter “E” in a series using 16-point font. We sought to exclude participants with severe language, motor, or vision impairments that might affect their ability to participate to full STAC administration. Additionally, participants who reported a history of neurological impairments prior to TBI, drug abuse, or alcohol abuse were excluded.
Materials
Screening tools and descriptive measures
Screening tools were the Nine Hole Peg Test (Grice et al., 2003), a vision screening (letter E) held 40.64 centimeters from the person’s face, and the CLQT confrontation naming subtest. These tests were used to ensure that the participants would have the physical and mental capabilities to participate in the study. We used a demographic form to collect information about the person’s medical history (e.g., age and length of PTA). Amount of time each person experienced PTA was gathered as an approximate measure of severity because it can accurately portray long-term cognitive function following TBI (Ponsford, Gershon, & Dean, 2016). Data on technology use and other variables was also gathered on this form as previous investigations suggested the possibility of self-reported comfort with technology being associated with performance (Wallace et al., 2017). Specifically, information regarding cell phone and tablet usage, primary language, educational level, previous history of neurological impairments, and living situation was included on this form. The RLA (Hagen, 2000) was used to determine present level of cognitive functioning through observation during screening and discussions with participants and/or caregivers.
Formal assessment tools
Formal assessment tools included the STAC delivered via an iPad (version 8.3), the CLQT, and the Montreal Cognitive Assessment (MoCA, version 7.1).
2.2.2.1 Standardized Touchscreen Assessment of Cognition. The STAC consists of theoretically-based tasks to assess attention, memory, visual and auditory memory, and executive functioning (Cognitive Innovations LLC, 2016). The STAC is self-administered, which minimizes examiner bias in administration. The STAC is an iOS based application and is intended to be administered on an iPad. Although the STAC may be self-administered, we supervised administration during the current study.
Automatically generated results include a four-page report with the description of each item, correct answer, participant’s response, response time, raw score, and when appropriate, percentile. The participant indicated his or her level of iPad comfort prior to completing the assessment via a scale (1 = not comfortable, 2 = somewhat comfortable, 3 = very comfortable). There is no total score available for the STAC. According to the developer, most subtest scores can be compared to the normative sample as a percentile. This percentile is calculated based on a normal distribution, using either the raw score alone or the raw score with time (Hagan, 2000). The available normative data came from 190 individuals without reported cognitive impairments (Cognitive Innovations LLC, 2013). Currently, there are no data available related to the performance of people with TBI on the STAC. In the current study, only raw data and scores, rather than percentiles were used in the analyses.
The maximum score for each cognitive domain varied based on the number of responses required from the participant. For example, in the orientation cognitive domain, the maximum score is six with one point per question asked (e.g., what is your last name?). For generative naming, STAC identifies accurate responses as any word that meets the criteria (e.g., belonging to the category animal), is spelled correctly, and is unique. For the letter cancellation and trail-making items, images allow clinicians to visualize errors.
2.2.2.2 Cognitive Linguistic Quick Test and Montreal Cognitive Assessment. We selected two commonly used cognitive assessments as comparison tools, the CLQT and the MoCA. The CLQT assesses performance in five cognitive areas: attention, memory, executive functions, language and visuospatial skills. The MoCA is a reliable and valid assessment tool, and has been found to have sensitivity and specificity when used to assess people with TBI (Wong et al., 2013). The MoCA includes eight sections: attention, memory, visuospatial construction, word retrieval, and executive functions. We selected the MoCA to replace the CAM which was used in previous studies because the correlations were relatively weak and the MoCA is a frequently used clinical tool.
2.2.2.3 Usability questionnaire. After administration, we used a modified Post Study System Usability Questionnaire (Lewis, 1993) to determine each participant’s perception of ease using the iPad for assessment. The modified questionnaire had seven questions with a seven-point, scaled response format including two anchors (i.e., strongly agree and strongly disagree).
Procedures
The single session occurred at a university clinic or in participants’ homes and lasted no more than 2 hours. The participants and/or their guardians completed a consent or assent form and demographic form. Next, we conducted the screening tasks outlined in the Materials section. Then, we administered the STAC, CLQT, and MoCA in a systematically, counterbalanced order. We followed the instructions from each assessment tool manual. After administration of all three assessments, we asked questions from the Usability Questionnaire. Information about participants’ performance on assessments was not consistently available during administration of the other assessments because of the counterbalanced order and need to score individual items offline or after administration of the tool. Following administration, we scored each assessment tool and entered the scores into IBM’s SPSS Statistics Software Program version 24.
Data analysis
We scored the STAC, CLQT, and MoCA according to their manuals. Matched subtests were determined based on previous research (Wallace, 2017) and task similarity. A list of eight comparisons between the STAC and CLQT, and eight comparisons between the STAC and MoCA were created (See Table 1 for a list of matched subtests). For example, we compared participants’ scores on the confrontation naming (CLQT), naming (MoCA), and picture naming (STAC) subtests. For some MoCA and CLQT subtests we used raw scores or subscores as they best reflected participants’ performance. For example, on the MoCA Verbal Fluency Subtest, the participant receives a score of 1 if he or she names at least 11 animals. To compute correlations, we used the raw score or number of animals named by each participant.
Eleven matched subtests
Eleven matched subtests
STAC = Standardized Tablet Assessment of Cognition; CLQT = Cognitive Linguistic Quick Test; MoCA = Montreal Cognitive Assessment.
We determined Pearson correlations between the related subtests using SPSS software. Finally, we used the following ranges to determine the value of the correlations: above 0.75 was a good to excellent relationship; 0.50 to 0.75 was a moderate to good relationship; 0.25 to 0.50 was a fair to moderate relationship and less than 0.25 showed little or no relationship (Portney & Watkins, 2000). We focused on cutoffs for strength of association instead of the traditional statistical significance because our goal was not to test if the correlations were different from zero which is the focus of statistical significance testing of correlation estimates. Additionally, despite the inherent challenges to conducting multiple comparisons, we were interested in investigating correlations for all 11 areas because of the previous studies that identified similar correlations within these areas.
Next, we identified six areas to further analyze related to participants’ iPad comfort level. We selected areas for follow up comfort rating analysis based on a cut off correlation value of 0.40 with either the MoCA or the CLQT based on previous results from Wallace and colleagues (2017). These areas included orientation, confrontation naming, generative naming category, generative naming first letter, immediate auditory memory, and cognitive flexibility. We reviewed the means, medians, and standard deviations. Then, to examine differences between iPad comfort groups, we conducted one-way ANOVAs within the Generalized Linear Model framework. Finally, we totaled the percentage of participant agreement with statements from the Usability Questionnaire.
Participants
The 26 male and 14 female participants ranged in age from 22 to 65 years (M = 45.2; SD = 10.6). Participants’ mean years of education was 13.42 (Range = 8 to 16; SD = 2.02) and their time post-injury ranged from 18 to 615 months (M = 224.3; SD = 173.41). Participants reported between 2 and 730 days of PTA (M = 140.52; SD = 193.97). Ten people were unable to report length of PTA, but did have information about the length of their coma. Reported length of coma ranged from 0 to 185 days (M = 45.46; SD = 51.51). Participants at the time of participation demonstrated characteristics consistent with RLA Levels 6 through 10 (M = 7.85; SD = 1.29). Twenty-three participants (57.5%) were at or above RLA Level 8. Thirty participants (75%) reported motor-vehicle accidents as the cause of their TBI. Ten participants (25%) lived independently and 16 participants (40%) lived in assisted living. All participants passed motor, language, and vision screenings. Prior to completing the assessment, 20 participants (50%) reported using a tablet and 19 (47.5%) participants reported using a touch-screen cellphone. Demographic information is available in Appendix A.
We had complete data sets from all participants for the STAC, MoCA, and CLQT. Two participants were missing demographic information related to time post-onset and years of education; we computed the above means and standard deviations without these data.
Comparisons among tests
The means and standard deviations for selected STAC, MoCA, and CLQT tasks appear in Table 2.
Means and standard deviations for selected tasks on the STAC, MoCA, and CLQT
Means and standard deviations for selected tasks on the STAC, MoCA, and CLQT
*Raw data rather than score used for calculations. STAC = Standardized Tablet Assessment of Cognition; CLQT = Cognitive Linguistic Quick Test; MoCA = Montreal Cognitive Assessment.
The correlations between the two standardized tests and the STAC test appear in Table 3. The MoCA and STAC had two areas with correlations greater than 0.5, while the CLQT and MoCA had three areas greater than 0.5. Both correlated well in the areas of orientation and generative naming first letter. The MoCA and STAC had one item with little to no relationship and CLQT and STAC had two items that demonstrated little to no relationship.
Correlations within selected areas of measurement for the STAC, MoCA, and CLQT
*Raw data rather than score used for calculations. Pearson correlations and with a sample size of 40 (38 df) and no correction for multiple tests values of.31 and above would be considered statistically significant at 0.05 two tailed. STAC = Standardized Tablet Assessment of Cognition; CLQT = Cognitive Linguistic Quick Test; MoCA = Montreal Cognitive Assessment.
Prior to administration of the full STAC, twelve participants reported that they were comfortable using an iPad, 20 participants reported being somewhat comfortable using an iPad, and 8 participants reported being uncomfortable using an iPad. iPad comfort reported by participants had little to no relationship with RLA level (r = –0.19).
The means and standard deviations of score across the three levels of iPad comfort are available in Table 5. Typically, mean scores were lower for participants who reported being uncomfortable with an iPad as compared to participants who reported being very comfortable or somewhat comfortable with an iPad. Only for Language Fluency Letters was there a statistical result of F(2,37) = 3.3, MSE = 16.46, p = 0.048 for the omnibus F-test. A Scheffe post-hoc comparison indicated the statistically significant F-test was driven by a difference between groups very comfortable and not at all comfortable (Mean difference = 4.75, SE = 1.48). The effect size for the omnibus ANOVA for Language Fluency Letters using omega-squared was 0.10, indicating 10% variance explained. A simple R-Squared effect size was 0.15, or 15% explained variance (Table 4).
Medians, means, standard deviations for STAC subtests across levels of iPad comfort
Medians, means, standard deviations for STAC subtests across levels of iPad comfort
*Raw data rather than score used for calculations. STAC = Standardized Tablet Assessment of Cognition.
Post-Assessment user satisfaction questionnaire means, standard deviations, and ranges
1 = Strongly agree; 7 = Strongly disagree.
Twenty-four participants (60%) strongly agreed with the statement “I felt comfortable doing this assessment using the iPad.” Similarly, twenty-four participants (60%) strongly agreed to use an iPad for a cognitive assessment in the future. Forty-five percent of participants (n = 18) reported a preference for using the iPad compared to the paper tests while 20% of participants (n = 8) preferred using the paper tests. Fourteen participants (35%) reported no preference. Means, standard deviations, and ranges for each item are available in Table 5.
Discussion
The primary purpose of this study was to investigate the convergent validity of the STAC subtests by comparing them to subtests from common paper assessments of cognition in a population of people with TBI. Secondary purposes included exploration of the effects of iPad comfort on participant performance and participant-reported usability of the STAC. The results showed that the STAC subtests had fair to good relationships with some similar paper subtests. Given this primary finding, the STAC or other electronic cognitive assessment may be appropriate, but require modifications prior to use with clinical populations. Additionally, participants’ self-reported iPad comfort level was not correlated to performance on the STAC or their observed cognitive status. Finally, participants’ preference for electronic versus paper assessment tools varied, with the greatest number of participants preferring the iPad.
Relationship between STAC and cognitive assessments
Previous studies have found weak to moderate correlations when comparing electronic tests to paper assessment tools. Other authors have proposed that the weak correlations may be due to the limitations in some of the paper assessment tools that affect their sensitivity (Collie, Darby, & Maruff, 2001; Keith, Stanislav, & Wesnes, 1998). That is, the electronic tests are sometimes hypothesized to be more sensitive to subtle cognitive differences than paper assessment tools (e.g., measures of reaction time are more precise). While this may have been the case in this study, it was beyond the scope of the current and previous studies to explore the reason for the poor correlations.
Given the current results, combined with results from Wallace and colleagues (2017), some of the STAC subtests appear to be appropriate to administer to adults with TBI. Specifically, verbal fluency and confrontation subtests appear to be most correlated to traditional paper assessments. However, other STAC items may benefit from modifications as evidenced by their lower correlations, review of participants’ errors, and observations made during administration.
A general recommendation is for the test to provide additional time to process instructions and to provide repetition of instructions. Additionally, instructions should be presented in visual and auditory modalities to support attention and language processing. Furthermore, during the trail making test, we observed that the participants with fine motor impairments, who passed our screening, still struggled to perform the swiping action on the touchscreen. One participant reported light sensitivity post-injury which may have affected her performance on the iPad and another participant experienced a seizure after the test administration. Finally, some participants reported a desire to have an opportunity to self-correct errors which could help provide an additional measure of insight and self-awareness. These recommendations are similar to those made by Wallace and colleagues (2017). Additionally, Keith, Stanislav, and Wesnes (1998) identified modifications needed to the computerized Cognitive Drug Research such as opportunities for breaks and slower stimulus presentation, to improve its use with people with TBI.
iPad comfort
There were no significant differences across the different levels of iPad comfort on selected tests with the exception of generative naming using first letter on the STAC. In contrast, Wallace and colleagues (2017) found that reported iPad comfort resulted in performance differences across multiple areas. Specifically, participants with high levels of iPad comfort performed significantly better than participants who reported no iPad comfort for the STAC Word Fluency (Animals), Word Fluency (r words), and the STAC Memory subtests. Of note, both Wallace and colleagues (2017) and the current study, identified an effect of iPad comfort on STAC Word Fluency (first letter). This may be due to the mode of response such that on the STAC, individuals type responses and on the CLQT and MoCA responses are provided verbally. The lack of significant results for the current study could be due to poor awareness and memory abilities of people with TBI who might not have accurately reported their comfort or familiarity with this type of technology. While clinicians should still ask people with TBI about their comfort level using a tablet before an assessment, the clinician may also want to gather information from a caregiver about the frequency of use and familiarity with technology.
Another possible explanation for the lack of significant differences across levels of iPad comfort is variability in performance. Variability in performance on cognitive assessments has been seen in previous studies for people with concussion (Makdissi et al., 1998) and moderate to severe TBI (Till, Colella, Verwegen, & Green, 2008). The variability present in this sample combined with the smaller sample size of the subgroups could have limited detection of a difference based on iPad comfort. Further studies could examine people with similar cognitive profiles following TBI to determine if differences in iPad comfort affect their performance.
Usability
While most participants responded favorability to questions about satisfaction with the tablet-based assessment, 20% of participants preferred the paper assessment and about a third of participants had no preferred assessment modality. Understanding participant preference is an important aspect of providing patient-centered care (Lopresti, Mihailidis, & Kirsch, 2004; Pegg et al., 2005). Preference may influence results of cognitive testing and could be used by clinicians to help determine appropriate methods of cognitive assessment when deciding among comparable tools. However, preference may not necessarily be reflective of validation so clinicians should carefully consider multiple factors when making these decisions.
Of note, the participant preference questionnaire was modified for use in the current study and therefore reliability and validity of the tool has not been determined (Lewis, 1993). Additionally, research regarding how best to measure participants’ satisfaction and experience with different cognitive assessment tools is needed. Future studies may also evaluate clinician satisfaction and perceived usability.
Limitations
The study results are limited by the relatively small sample size given the variable performance of the participants. The amount of variability in a heterogenic sample of 40 participants may have limited the strength of correlations or differences identified in performance based on iPad comfort. The relatively small sample size and reliance on participants’ self-selection to complete the study also limits the generalizability of the results. Additionally, factors such as age, time post-onset, and severity of cognitive deficits which were variable in this sample could not be explored further because of the sample size. These factors should be explored in future research. Additionally, for most participants, we relied on self- or family-report for medical history including loss of consciousness and length of PTA. Given that many participants were multiple years post-injury, it is possible that this information was not remembered accurately.
Participants also self-reported their level of iPad comfort as well as touch screen tablet and cellphone use. Given that memory impairments and reduced self-awareness are common after TBI, participants may not have provided accurate descriptions of their abilities (Vakil, 2005). Future studies may also include a measure of observed iPad skills.
In the current study as well as previous studies, we have compared the STAC to other assessment tools designed to measure the same general areas of cognition. This is necessary, in part, because no paper version of the STAC exists. Future research could explore questions related to parallel versions of tablet-based cognitive assessments.
Future research
After modifications are made to STAC, future research should include evaluating test-retest reliability and comparing these results to paper assessments (MacDonald & Duerson, 2015). This is particularly important for some cognitive assessments that are given repeatedly to determine changes in cognitive function following an injury or during the recovery period (De Marco & Broshek, 2016; Lovell, Collins, Podell, Powell, & Maroon, 2000; Schatz & Ferris, 2013). Similarly, researchers should investigate the predictive validity of scores on tablet-based assessment relative to functional measures and participation-focused outcomes. Further investigation of the potential effect that iPad comfort may have on participants’ performance is needed. Finally, establishing validity of STAC in other populations (e.g., mild TBI, dementia), would further its clinical utility.
Conclusion
Overall, the study findings suggest that tablet-based, touch screen assessment of cognitive abilities using the STAC may provide valid results particularly for areas of verbal fluency and orientation; however, many other areas require modifications. While iPad comfort did not appear to have a significant effect on performance, clinicians may consider this factor when interpreting STAC results for some participants, particularly for typing tasks like verbal fluency.
Conflict of interest
None to report.
Funding
This study was funded by the Duquesne University Faculty Development Fund (Wallace, PI). Dr. Wallace, Dr. Donoso Brown, and Dr. Schreiber are employees of Duquesne University which provided funding support for this project.
Supplemental material
Study procedures are available at this site: https://dsc.duq.edu/slp-faculty-scholarship/1/.
Footnotes
Appendix
Appendix A. Participant demographics
| Participant | Age | Gender | Months Post Onset | Years of Education | Current RLA Level | Living Situation | Use Tablet | Use Touch-Screen Phone | iPad Comfort |
| 1 | 38 | F | 250 | 14 | 9 | Alone | Yes | No | 1 |
| 2 | 35 | F | 49 | 16 | 10 | Family | Yes | Yes | 2 |
| 3 | 51 | M | 23 | 15 | 10 | Alone | Yes | Yes | 2 |
| 4 | 22 | F | 62 | 12 | 6 | Family | Yes | No | 1 |
| 5 | 47 | M | 47 | 11 | 8 | Family | Yes | Yes | 2 |
| 6 | 38 | M | 105 | 16 | 8 | Family | Yes | Yes | 1 |
| 7 | 61 | M | 468 | 13.5 | 8 | Family | No | Yes | 2 |
| 8 | 53 | F | 175 | 14 | 8 | Alone | No | No | 1 |
| 9 | 36 | F | 74 | 16 | 10 | Family | Yes | Yes | 2 |
| 10 | 22 | F | 136 | 14 | 6 | AL | Yes | Yes | 1 |
| 11 | 53 | F | 66 | 16 | 10 | Alone | No | Yes | 1 |
| 12 | 40 | M | 241 | 10.5 | 8 | Family | No | No | 3 |
| 13 | 29 | M | 85.5 | 14 | 6 | Family | No | No | 2 |
| 14 | 51 | M | 615 | 14 | 7 | Family | No | No | 2 |
| 15 | 49 | M | 335 | 12 | 8 | Alone | Yes | Yes | 2 |
| 16 | 65 | M | 121 | 16 | 7 | AL | No | No | 3 |
| 17 | 47 | M | 72 | 14 | 9 | Alone | Yes | Yes | 2 |
| 18 | 32 | F | 18 | 14 | 10 | Family | Yes | Yes | 2 |
| 19 | 54 | F | 429 | 12 | 7 | Alone | No | Yes | 3 |
| 20 | 34 | F | 68 | 16 | 7 | Family | Yes | Yes | 1 |
| 21 | 41 | F | 304 | 12 | 10 | Alone | No | No | 1 |
| 22 | 54 | M | 28 | 16 | 10 | Alone | Yes | Yes | 1 |
| 23 | 57 | M | 93 | 14 | 9 | Alone | Yes | No | 1 |
| 24 | 42 | M | 162 | 15.5 | 8 | AL | No | No | 3 |
| 25 | 50 | M | 130 | 14 | 8 | AL | No | No | 3 |
| 26 | 35 | M | 237 | 8 | 8 | AL | No | Yes | 2 |
| 27 | 43 | M | 455 | 12 | 8 | AL | Yes | Yes | 2 |
| 28 | 55 | M | 477 | 12 | 8 | AL | Yes | No | 2 |
| 29 | 60 | M | 520 | 11 | 8 | AL | No | No | 3 |
| 30 | 43 | M | 329 | 10 | 6 | AL | Yes | No | 2 |
| 31 | 60 | M | 515 | 12 | 7 | AL | Yes | No | 1 |
| 32 | 49 | F | 292 | 15 | 6 | AL | No | No | 3 |
| 33 | 41 | M | 97 | 14 | 7 | AL | Yes | Yes | 2 |
| 34 | 46 | M | 298 | 12 | 7 | AL | No | No | 3 |
| 35 | 44 | M | 296 | 12 | 7 | AL | No | No | 2 |
| 36 | 55 | M | * | 12 | 7 | AL | No | No | 2 |
| 37 | 51 | F | * | 15 | 7 | AL | No | No | 2 |
| 38 | 54 | M | 519 | * | 7 | AL | No | No | 2 |
| 39 | 43 | F | 276 | 11 | 7 | Family | No | Yes | 1 |
| 40 | 27 | M | 56 | 16 | 8 | Family | Yes | Yes | 2 |
| Mean | 45.18 | 283.66 | 13.5 | 7.88 | 1.9 | ||||
| SD | 10.46 | 189.19 | 2.5 | 1.25 | 0.71 |
*Missing Data AL = Assisted Living; RLA = Rancho Los Amigo Scale of Cognitive Functioning, SD = Standard Deviation.
Acknowledgments
First, the authors thank the study participants for their time and contributions to this project. The authors also thank Dr. Andrea Fairman for her assistance with early study conceptualization and institutional review board submission. Additionally, the authors thank Heather Coles and Simon Carson for their guidance in this area of research. The authors are grateful for the invaluable recruitment assistance provided by Dr. Jeffery Snell and his colleagues at Quality Living, Inc. Lauren Matthews provided assistance with data management and editing.