Evaluating Brain-Computer Interface Performance in an ALS Population: Checkerboard and Color Paradigms

Introduction

Neuromuscular disabilities such as amyotrophic lateral sclerosis (ALS) or brain stem stroke can leave a person with no physical ability to communicate. Commercially available augmentative and alternative communication (AAC) devices may be an option for those without speech due to ALS but these devices require some residual motor control. Advanced stages of ALS can result in the loss of all motor control except eye movement, and at that point the person is considered locked-in. Once a person has reached a locked-in state, commercially available AAC devices may no longer be effective.

A noninvasive brain-computer interface (BCI) provides a method of access that does not require physical movement; instead it relies on detection of electroencephalography (EEG) response. ¹ Furthermore, several studies have demonstrated the efficacy of a noninvasive BCI in a home environment with ALS patients.^2-4 Invasive methods of BCI (eg, electrocorticography and single unit recording) have also been shown to be effective ⁵ ; however, these methods require surgery, are more costly, mostly tested in epileptic patients, and are difficult to setup and support by an in-home caregiver. EEG-based BCI is relatively easy to set up because the electrodes are placed on the scalp (no surgery required) and training for the user only takes a few minutes. ⁶ At this time a noninvasive BCI best suits the needs and abilities of the user, caregiver, and environment found within the home of an ALS patient.

One of the biggest challenges to BCI is the low signal-to-noise ratio of event-related potentials (ERPs) in the EEG. ¹ BCI researchers attempt to resolve this through 2 approaches: (1) signal processing^7-12 and paradigm design^13-17; the current article will focus on the latter. Paradigm design should take into consideration all aspects of the visual presentation of the BCI from the spelling grid design to the methods by which the visual ERPs are elicited. The number of parameters that must be set by the paradigm is considerable; however, visual attention research may provide guidance for selecting optimized parameter values to maximize BCI performance and paradigm development. Enhancing visual attention and using visual processing components of stimulus properties (eg, color and motion) has been demonstrated to increase component amplitude and improve signal-to-noise ratio.^18-20

Visual attention principles can be used as a guide for multiple design decisions in ERP-based BCI paradigm development. One of the design decisions for BCI spellers is choosing which groups of characters will be flashed together. Characters are typically grouped in order to maximize speller speed with one of the more commonly used paradigms relying on groups based on rows and columns. ²¹ The efficacy of the row/column paradigm is reduced by 2 frequently occurring types of errors: adjacent flash distraction and reduced ERP amplitude due to consecutive flashes of the same item. Visual attention principles suggest that these errors may be related to the flanker task distraction and the attentional blink, respectively.^22,23 Taking these principles into consideration, the design of a checkerboard paradigm controls adjacent item flashes and prevents consecutive flashes of the same item. The checkerboard paradigm has demonstrated a minimization of errors observed with the row/column paradigm and significant improvement of performance. ²⁴

Other attempts to improve performance include varying the number of items in a flash group; Guger et al ⁶ compared a single character flash with the traditional row/column paradigm, and Pan et al ²⁵ examined single character flash and a regional based flash. These studies showed less accurate performance in the single character condition. Going beyond flash groups, BCI researchers have implemented different types of stimuli to enhance visual attention and the subsequent ERPs. To enhance the N200 (N2) ERP component, Hong et al ¹⁹ used motion onset instead of flashing items for character selection. The color processing component was augmented by Takano et al ²⁰ by implementing a blue-to-green item flash that outperformed the traditional gray-to-white item flash. ²⁰ Facial recognition has also shown to significantly improve BCI performance by using the N170 component in healthy participants and those with neurodegenerative diseases.^26-30 Other researchers have implemented a nontraditional matrix (Hex-o-spell) and examined the use of covert attention. ³¹ Each of these studies provides valuable information as to how a BCI paradigm can be designed to take advantage of visual attention principles.

Ryan et al ³² expanded on the success of the checkerboard paradigm and the blue-to-green item flash paradigm, with 2 types of color paradigms and compared them to a standard condition that presented gray letter that changed to white. The first color condition, gray-to-color, changes gray matrix items to 1 of 9 unique colors that differ from all adjacent item colors. In the second condition, color intensification, each item was assigned a different color from the adjacent items and the item then intensified its assigned color (eg, red-to-bright red). The traditional gray-to-white paradigm was used as control for the 2 color conditions. The gray-to-color condition resulted in an enhanced color processing ERP component and statistically improved performance as compared to the gray-to-white and the color intensification conditions. Moreover, 53% of participants preferred the gray-to-color condition, 25% preferred gray-to-white, and 22% preferred the color intensification condition.

While the results in demonstrated the potential for checkerboard and color-based paradigms, the experiments were conducted in a non-disabled population, as are the majority of BCI studies, including the studies mentioned above (exempting 3 participants with ALS in Townsend et al ¹⁴ ). Furthermore, the majority of participants are college age and the average age of ALS onset is between 40 and 60 years of age. ³³ For BCI research to reach its full potential it must be examined within the target population and in the intended environment.

The current study examines the BCI performance of participants with ALS spelling in a row/column with gray-to-white item flash (RCW) paradigm, a checkerboard with gray-to-white flash (CBW) paradigm, and a checkerboard with gray-to-color flash (CBC) paradigm. The CBC paradigm implemented the same nine unique colors used in. ³² We hypothesized that the CBW and CBC conditions would result in higher accuracy and information transfer rate than the RCW condition. We also hypothesized that that participants will prefer the CBC paradigm to the CBW and RCW paradigms.

Methods

Eleven participants diagnosed with ALS were recruited for this study (for demographic information, see Table 1. Participants were selected with the requirement of a diagnosis of ALS (el Escorial definite). All participants were referred by ALS support groups or ALS clinics. Eight of the 11 participants were tested in their home. The remaining 3 participants (ie, participants 6, 7, and 9) were tested at the Duke University ALS clinic. The study was approved by the East Tennessee State University Institutional Review Board and the Duke University Institutional Review Board.

OPEN IN VIEWER

Table 1.

Sex, Age at Participation of Study, Years Postdiagnosis of ALS, ALSFRS-r Score at Time of Participation, and BCI Experience in Number of Sessions Including the Current Study.

Subject	Sex	Age, y	Years Postdiagnosis	ALSFRS-r	BCI Experience
1	Male	62	14	0	8
2	Male	37	7	2	7
3	Female	51	2	15	5
4	Female	62	7	2	6
5	Female	52	5	15	5
6	Male	40	7	0	10
7	Male	58	5	1	3
8	Male	50	25	0	6
9	Female	56	4	27	5
10	Male	55	2	36	4
11	Male	56	1	34	6

Abbreviations: ALS, amyotrophic lateral sclerosis; ALSFRS-r, ALS Functional Rating scale–Revised; BCI, brain-computer interface.

The current study compared 3 conditions (see Figure 1). Each participant completed all 3 conditions in a pseudo-randomized order (Latin square). The RCW condition flashes entire rows or columns of items in random order. The CBW condition flashes groups of items in a quasi-random order. The RCW and CBW conditions had the same flash stimulus properties, the items were gray when not being presented (stimulus off) and then changed to white when presented (stimulus on). The CBC condition used the same presentation method of the CBW and consisted of items represented in gray (stimulus off) that changed to a color that was different for the 8 surrounding items (stimulus on). Henceforth, the color changes in each of the 3 conditions will be referred to as “flash(es)” for consistency with prior research. The duration of the flash (stimulus on) was 187.5 ms and time between flashes (stimulus off) was 62.5 ms, consistent across conditions.

OPEN IN VIEWER

Figure 1.

The 3 conditions examined: Color (CBC), Checkerboard (CBW), and Row/Column (RCW).

Stimulus presentation and online processing was conducted with BCI2000. ³⁴ Each participant sat approximately 1 m from the computer monitor that displayed the 6 × 6 matrix. EEG data were recorded from sixteen electrodes (see Figure 2) and referenced to the right mastoid. Impedances were reduced below 20.0 kohm before recording. Kappenman and Luck ³⁵ found minimal impedance-related attenuation of the P300 when a 0.5-Hz high-pass filter was used. The participants were instructed to attend to the item they wanted to select (ie, target) by either counting or repeating the letter silently in their head each time the target flashed and to ignore the nontarget items.

OPEN IN VIEWER

Figure 2.

The 16 channels of EEG data collected. Classification channels are presented in gray.

The EEG was amplified by a g.tec (Guger Technologies) 16-channel amplifier (amplification to ±2 V before ADC; high-pass and low-pass filters set to 0.5 Hz and 30 Hz, respectively; digitization rate of 256 Hz). Only 8 electrodes were used for classification (Fz, Cz, P3, Pz, P4, Po7, PO8, and Oz, highlighted in gray in Figure 2). ³⁶ A random word generator was used to obtain 5 six-letter words (ie, 30 character selections). The EEG data from the 30 item selections served as training data for a stepwise linear discriminate analysis (SWLDA) solution, which was then used for online response classification and feedback for an additional 30 selections. ⁷ EEG was downsampled to 20 Hz before SWLDA was performed. For calibration and online testing, the number of stimulus presentations was held constant at seven sequences per item selection during calibration and online testing phases of the study. A sequence is defined as 2 flashes of every item in the matrix. It should be noted that the CBW (and CBC) presentation constraints require more flashes to complete a sequence. In the RCW paradigm entire rows and columns are presented for each flash therefore, it requires 12 flashes for 1 sequence. The constraints of the CBW paradigm only allow items that are not adjacent to flash resulting in 18 flashes to present one sequence. This results in the CBW (and CBC) paradigms requiring an additional 3000 ms of items flashing per selection.

Participants were given a survey to assess their opinion of which condition facilitated their attention to the target, and in which condition the nontargets produced the most distraction.

Results

A repeated-measures ANOVA was performed on the accuracy outcome. The mean accuracy in the CBC condition was higher than the CBW and RCW conditions (89.7%, 84.3%, and 78.7%, respectively); however, there was no statistical difference between the conditions, F(2, 20) = 3.2, P = .062. The mean ITR was higher in RCW condition than in the CBC and CBW conditions (9.28, 7.68, and 6.99, respectively); however, the only significant difference was between RCW and CBW, F(2, 20) = 5.016, P = .04, pairwise comparison, P = .024. The CBW (and CBC) flash presentation constraints require more flashes to complete a sequence (see Methods section). Thus, even though the observed accuracy in the RCW condition was 11% lower than CBC condition, the IRT in the RCW condition was higher than the CBW and CBC conditions.

An additional 2-way ANOVA was performed to analyze ITR of each condition by each of the 7 sequences. The main effect of condition was not significant, F(2, 120) = 1.09, P = .357; however, the main effect of sequence and the interaction of condition and sequence were significant, F(6, 120) = 3.63, P = .004 and F(12, 120) = 3.60, P < .001, respectively. A post hoc Holm-Sidak analysis found the only significant difference of conditions was within the first sequence. In the first sequence the CBC condition had significantly higher ITR than CBW and RCW, P = .042 and P = .001, respectively (see Figure 4).

A chi-square goodness-of-fit test of the survey results revealed that the frequency of condition preference was significantly different than the frequency expected, χ²(2) = 8.91, P < .05. Of the 11 participants, 8 preferred the CBC and 3 preferred CBW condition.

Discussion

The CBC condition combines the visual attention driven constraints of the CBW paradigm with the attention enhancing aspect of a salient color change. These elements facilitate one another in a single paradigm. The statistical analysis did not yield significant differences between these conditions. Nonetheless, the data showed an 11% increase in accuracy, which is exemplified by the preference of the checkerboard paradigms.

The survey provided a very clear difference in personal preference. CBC and CBW were preferred eleven to zero over the RCW paradigm. Participants commented that in the RCW condition nontargets were more distracting than the nontargets in the CBW or CBC conditions. The participants that preferred the CBC condition commented that the colors facilitated their attention to the target. Those who preferred the CBW condition commented that the flashing colors of the CBC non-targets were distracting. These results suggest that personal preference should be considered when a BCI system is used in-home.

The personal preference likely reflected the CBC and CBW’s ease of use over RCW resulting from the visual attention problems associated with the RCW. Although the attention problems associated with the flanker effect and attentional blink did not evolve into statistical differences in performance, these attention issues did affect preference.^22,23 Individuals with ALS may require the extended use of a BCI on a daily basis; therefore, personal preference should be strongly considered when a system is implemented in-home.

The increased ITR of the RCW over the CBW and CBC paradigms is a result of more items being presented per flash in the RCW paradigm. A previous study, with a higher number of participants, has shown that when the number of sequences are optimized for the participant there is a significant increase in ITR of the CBW over RCW. ²⁴ In the current study, the performance curves (see Figures 3 and 4) revealed higher initial accuracy and ITR in the CBC condition for all participants. Collecting seven sequences reduced final accuracy differences between the 3 paradigms and RCW showed higher ITR at later sequences. The sequence performance curves suggest that a dynamic classifier would improve the online performance of CBC over RCW.

OPEN IN VIEWER

Figure 3.

(a) Post–data collection, an offline analysis of percent correct scored by number of sequences in the average for each participant was analyzed and presented as a performance curve. (b) Percent correct for each condition averaged across participants. The CBC (gray-to-color) condition resulted in consistently higher accuracy than the RCW (row/column) condition.

OPEN IN VIEWER

Figure 4.

(a) Post–data collection, an offline analysis of information transfer rate (ITR) by sequence for each participant was analyzed and presented as a performance curve. (b) ITR for each condition averaged across participants.

The dynamic offline simulation revealed an increase in ITR over the static sequence used online. The dynamic classifier reduced the necessary sequences (eg, flashes) to make a selection. The increase in speed came at the cost of reduced accuracy. This is a common trade off when trying to improve BCI performance. The dynamic classifier stopping threshold of 0.9 could be increased at the cost of additional flash presentations, requiring more time. There are several dynamic classifiers and it is possible that other dynamic classifiers may provide better performance than that found in this study.^39-41

The waveform analysis yielded no significant differences between conditions. A previous study in nondisabled users examining similar conditions revealed a color-processing component (ie, positive peak window from 113 to 310 ms) in the CBC condition that was statistically higher in amplitude than in the CBW condition. ³² Participants did perceive a color change by confirming that the characters flashed in the CBC condition varied in color. It is possible that the progression of ALS could have affected the visual sensory processing tracks of the stimuli ⁴² ; however, there are competing theories about how these tracks are affected by ALS. ⁴³

The results of this study suggest the need for a BCI with color stimuli and dynamic stopping for an ALS population. There is evidence of increased performance and enhanced personal preference with the system examined in this study. These improvements are incremental and crucial to the development of a reliable BCI system.

Conclusions

The participants with ALS in this study all preferred a version of the checkerboard paradigm over the RCW and showed a strong preference for the color paradigm as compared to the two achromatic conditions. The offline dynamic stopping paradigm significantly increased ITR. In addition, the color paradigm produced a modest, although statistically insignificant, improvement in performance. Given these findings, BCI paradigms that use color stimuli should be considered for individuals who have ALS.