Abstract
Objective
To explore how task and stimuli influence response accuracy in assessing functional communication in adults emerging from a Prolonged Disorder of Consciousness (PDOC).
Setting
Specialist post-acute brain injury assessment centre.
Subjects
Twelve adults (7 male; 5 female) emerging from PDOC, recruited consecutively between June 2021 and August 2023.
Methods
A prospective study exploring the impact of task and stimuli on response accuracy. Yes/no questions and spoken word-to-referent (colour/object) matching tasks were presented. Stimuli were controlled for psycholinguistic variables to reduce linguistic confounds in the assessment of consciousness. Accuracy was compared to a standard measure (visually based situational yes/no questions from the Coma Recovery Scale-Revised).
Results
Responses were most accurate for word-to-referent matching tasks compared to yes/no questions (t(23) = −6.49, P < 0.001, d = 1.33), with greater accuracy to colour than object stimuli (t(23) = 2.79, P = 0.01, d = 0.57). Participants also responded with greater accuracy to word-to-referent matching and yes/no questions involving colours and objects compared to the Coma Recovery Scale-Revised situational yes/no questions.
Conclusion
Task influences accuracy of responses in assessing return of consciousness. The advantage of colours over objects can be attributed to simpler visual processing and higher lexical frequency of these words. The current standard situational yes/no tasks from the Coma Recovery Scale-Revised resulted in lowest scores and should not be relied upon as the only measure of functional communication.
Introduction
A severe brain injury can lead to a disorder of consciousness which is considered prolonged if persisting for more than four weeks (Prolonged Disorder of Consciousness [PDOC]). Patients may improve along a continuum of consciousness through the stages of Coma, Vegetative State and Minimally Conscious State (MCS −/+) before emerging. 1
The term ‘consciousness’ encompasses a range of meanings in everyday use, including being conscious/awake versus unconscious, or being conscious/cognisant versus a lack of awareness. These everyday meanings encompass low-level behaviours, such as wakefulness, to those indicating a high-level meta-cognitive state. Vigilance (the state of readiness to respond) and attention (the focussing of mental resources onto specific information) are considered prerequisites for consciousness 2 which is only evidenced through ‘verbal reportability’. 3 When enough brain regions are activated by incoming sensory information, they synchronise into a large-scale network of activation extending into wider areas of cortex. This ‘global neuronal workspace’ is subjectively experienced as a conscious state,2,4,5 and once information from any domain is conscious, it can be reported due to connections with language areas. 3 A disorder of consciousness can be interpreted as an interruption of this global neuronal network and hence the verbal reportability of information.
The categorisation of emergence from a disorder of consciousness is arbitrary and is based on expert clinical opinion with different authors, working parties and guidelines presenting varying definitions.6–8 Emergence from MCS was originally defined as the demonstration of reliable functional communication and/or the functional use of two different objects. 6 To facilitate consistent practice when assessing emergence, tasks and scoring criteria were specified. Functional object use was defined as the appropriate use of two objects on two consecutive evaluations, for example, bringing a comb to the head. Functional communication was defined as a person answering six situational yes/no orientation questions accurately on two consecutive occasions without any errors, for example, ‘Am I pointing to the ceiling?’. 6 Reliability and accuracy of response are considered the defining features of emergence, with the 2020 Royal College of Physicians (RCP) PDOC guidelines also emphasising durability of response across a range of contexts and time.
Three formal PDOC assessments which include assessment of communication are recommended for use in the UK: the Coma Recovery Scale-Revised, 9 Wessex Head Injury Matrix 10 and the Sensory Modality and Rehabilitation Technique. 11 In cases where only one is used, the RCP guidelines recommend the Coma Recovery Scale-Revised as it has the strongest evidence for validity 12 and is more widely used internationally. It also includes situational yes/no questions recommended as an assessment of communication to indicate emergence. 6
Determining the boundary between MCS and emergence is challenging as the current criteria focus on a narrow range of behaviours and require 100% accuracy. 13 Given the severity of brain injury associated with PDOC, patients who emerge are likely to have significant language and non-language cognitive impairments that may prevent them achieving consistent accuracy.1,14–17 Individuals with aphasia are particularly at risk of being misdiagnosed as being in PDOC.13,18,19
The RCP guidelines (2013/2020) included an additional task in the evaluation of emergence. They specified a discrimination task of spoken word-to-picture/object matching in the presence of a distractor, with 100% accuracy across trials to meet the criterion for emergence. An individual is shown two items and asked, for example, ‘show me the cup’. Spoken word-to colour/object matching involves auditory perception; phonological analysis; lexical activation; and activation of the corresponding semantic representation. These stages occur rapidly and often in parallel, allowing for efficient comprehension and recognition of spoken words and their corresponding referent. In addition to auditory comprehension the task requires visual colour/object recognition but eliminates the need to generate a linguistic response (such as ‘yes/no’). This task is part of the auditory function scale of the Coma Recovery Scale-Revised (object-related command) used to indicate MCS, but in the UK, it is considered to indicate the return of functional communication 1 as it requires linguistic processing, the generation of a response and reflects skills which underpin the development of low tech Augmentative and Alternative Communication (AAC). Furthermore, Golden et al. 20 have proposed that consistent command following should be considered an indication of emergence as it is observed alongside the return of yes/no responses and functional object use.
The findings from a UK-wide survey on emergence from PDOC in a post-acute setting 21 highlighted a range of issues reported by clinicians when assessing functional communication. Approximately one-third (30.4%) of respondents used tasks other than those recommended, and almost half (46.4%) reported a lack of confidence in the current tasks. Most respondents (78.6%) indicated that they worked with patients who they considered to have emerged, but who were unable to demonstrate emergence based on the current criteria. Almost a third of respondents (30.6%) indicated they were unsure of principles that might be employed to select stimuli that are well-controlled for factors that influence linguistic processing, such as lexical frequency, age of acquisition and imageability, as well as ensuring semantic, visual and phonological distinctiveness. Respondents reported using objects, pictures, colours and biographical stimuli; however, the words labelling these constructs may differ markedly in their psycholinguistic characteristics.
When assessing linguistic impairments in conditions such as post-stroke aphasia, it is standard to employ stimuli that are carefully controlled in terms of these psycholinguistic variables which influence speed and/or accuracy of processing in both neurotypical and language-impaired participants. Typically, early acquired (age of acquisition), high imageability (mapping to a rich sensory-motor representation) and higher frequency words are processed more rapidly and are more resilient to disruption.22–27
In addition to processing demands linked to linguistic factors, tasks involving visual stimuli require careful design when used in assessment of individuals with severe brain injury. Typical impairments include visual field loss, neglect, double vision, difficulties with eye movements, perceptual dysfunction and visual agnosia. 28 Up to 65% of patients in PDOC may have visual problems. 29 Objects are recognised by their colour, shape, surface features and orientation, as well as tactile and kinaesthetic properties.28,30 Therefore, reducing demands on visual processing, for example by presenting blocks of colour could improve accuracy of response.
We report an experiment that explores how task and stimuli impact on response accuracy when assessing functional communication in patients emerging from PDOC.
Research question
How do task and stimuli impact response accuracy in patients emerging from a PDOC?
Hypotheses
Participants will respond with greater accuracy to spoken word-to-referent (object/colour) matching tasks compared to yes/no questions.
Participants will respond with greater accuracy to colour stimuli than objects in both yes/no and spoken word-to-object matching tasks.
Participants will respond with greater accuracy to yes/no questions with both colours and objects compared to Coma Recovery Scale-Revised visual situational questions.
Methods
The Standards for Reporting Diagnostic Accuracy Studies guidance and checklist were used to guide the reporting of this study. 31 The study was approved by the London Harrow NHS Research Ethics Committee (ID: 275560). Due to their significant level of cognitive and communication impairment, participants could not give informed consent and a consultee process was followed (Mental Capacity Act, 2005). 32
Participants
A consecutive sample of 16 adults demonstrating communicative behaviours associated with emergence from PDOC (attempting yes/no responses or indicating a choice from two items) was recruited over a 40-month period from a specialist neurorehabilitation centre and nursing home between June 2021 and August 2023. All suitable participants were identified by their treating team and a consultee approached to provide an opinion on what the individual's wishes would be if they were able to consent for themself. Table 1 displays the study inclusion and exclusion criteria.
Study inclusion and exclusion criteria.
Stimuli
Coloured acrylic tiles (10 cm × 10 cm × 5 mm), everyday objects (e.g. cup, sock) as well as the Coma Recovery Scale-Revised visual situational yes/no questions were tested. Colour and object stimuli were chosen based on psycholinguistic principles: they could be labelled with words of early age of acquisition, high lexical frequency, high familiarity and were also concrete and highly imageable. Items were selected from the Glasgow Norms, 33 a database that provides ratings of words on nine psycholinguistic dimensions (including concreteness, imageability and familiarity). Frequency of each word was extracted from the 100-million-word online British National Corpus. 34 Visual and psycholinguistic factors overlap as colours are labelled with adjectives that generally have an early age of acquisition and are of high imageability and high frequency. 35
Stimulus sets were similar in age of acquisition, familiarity and imageability but objects were more concrete than colours, and colour words were of much higher frequency than object words (see Appendix 1 for details of psycholinguistic properties of stimuli).
In the word-to-referent matching tasks, target items were paired with a distractor item. For the object task, distractors were semantically, phonologically and visually distinct from the target to minimise language/visual confounds (e.g. sock vs. pen). For the colour task, targets and their distractors were selected to avoid discrimination problems that occur with colour blindness. Accuracy to the experimental tasks was compared to the current standard of the two visually based situational yes/no questions from the Coma Recovery Scale-Revised: Am I touching my nose right now? /Am I touching my ear right now?.
Tasks
For the yes/no object/colour trials, items were held up in front of the participant (e.g. Is this a cup? Is this yellow?). The participant was required to indicate their response using any verbal or non-verbal means. For the spoken word-to-referent matching tasks, the participant was presented with two response choices (target and distractor). The participant was asked to indicate the referent (e.g. show me blue in the presence of yellow and blue tiles; show me the sock in the presence of a cup and a sock) using the participant's optimal response method (e.g. eye pointing, reaching).
Data was collected across a series of sessions. The planned protocol involved presentation of 10 test items for each task, split into two sets of five (A & B) across five sessions. The same test items were repeated across all sessions. During testing, many participants were unable to tolerate completing both sets A and B in a single session, and data was collected in up to 10 sessions. The order of tasks for each session was randomised using an online randomiser (https://www.randomizer.org), and the same randomised order was used for all participants. Each of the two visually based situational yes/no question from the Coma Recovery Scale-Revised (Am I touching my nose right now?/ Am I touching my ear right now?) was presented five times across the 10 trials with equal numbers of yes/no trials. This resulted in 50 data points for each task. To maximise understanding of each task, the assessor modelled accurate performance on two non-experimental items at the start of each test block. Full data collection protocol and scoring sheets are available from the corresponding author.
Test items in the word-referent matching tasks were presented with counterbalancing of targets on left/right side of space. Each question or instruction was presented up to two times. The assessor waited 10 seconds for a response before repeating the command or moving onto the next test item.
Data analysis
Sessions were video recorded to permit analysis of reliability of response scoring and to identify if any non-linguistic cueing of responses occurred. Two Logitech BRIO Ultra 4K webcams linked to a MacBook Air with OBS Studio 26.1.2 (mac) video recording software were used. Separate cameras recorded the assessor and participant. Responses for 25% of participants were re-scored by a researcher who was blind to the original scoring to establish interrater reliability. They also identified any cueing to the target response inadvertently provided by the assessor. Inter-rater reliability was determined using Cohen's Kappa.
To incorporate incomplete data sets into analysis, scores for each task were adjusted to a percentage score. For example, a score of 30/50 was adjusted to 60%, a score of 10/40 to 25%. Participants had a 50% chance of success on each trial. To determine whether participant performance across all 50 trials was better than chance binomial probability thresholds were calculated. Accuracy (%) scores between 45 and 55 were within the range expected by chance (P > 0.05) (i.e. indiscriminate guessing), while accuracy (%) scores of 56 or higher were unlikely to occur by chance (P < 0.05), indicating above-chance performance. Statistical analyses were carried out using OriginPro2022b software.
Results
Of the 16 participants enrolled in the study, one significantly improved between recruitment and data collection; one improved and indicated with gesture that they did not want to participate; and two medically deteriorated early in the study and became unresponsive so were withdrawn without providing data. Ten complete sets of data were collected, and two participants were able to provide partial data (participant 11 was withdrawn due to medical deterioration, and participant 12 due to increasing distress). Information on participants is presented in Table 2.
Participant characteristics.
To assess inter-rater reliability, a total of 750 observations for correct/incorrect responses were coded, with no missing data. Cohen's Kappa coefficient (k) was 0.842 with a 95% confidence interval of 0.803 to 0.881, suggesting a strong level of agreement between independent raters. 36 Regarding the possibility of unconscious target cueing, of the 750 observations, 1.33% showed evidence of target cueing. This included prosodic emphasis on the target word, body positioning or language indicating the target. Target cues mostly occurred during yes/no tasks. However, participants only performed accurately 30% of the time after an unintentional target cue, indicating minimal impact on data fidelity.
Table 3 and Figures 1 and 2 display individual and group performance by task. Most trials scored as incorrect were due to non-responses rather than errors. The situational yes/no task yielded lower variability amongst participants compared to other tasks, indicated by the standard deviation. However, this task was different to the object/colour tasks as the same questions were repeated more often, reflecting how they are presented in the Coma Recovery Scale-Revised. The variance in scores for the remaining tasks was greater and similar across tasks (Figure 1).

Boxplot showing percentage scores by task. n = 12, with mean (crosses) and median (line). Bars indicate highest and lowest scores.

Percentage scores across tasks by participant. Horizontal line indicates the binomial probability threshold; a score of 56% or more indicates performance significantly better than chance (P < 0.05).
Percentage scores by task, n = 2.
aSD: standard deviation.
Performance was variable across and within participants (Figure 2). Word-to-object/colour matching scores were higher than yes/no tasks for most participants (n = 10). Word-to-colour matching yielded the highest score for most participants (n = 9), with three participants scoring higher on word-to-object matching.
Situational yes/no questions yielded the lowest score for most participants (n = 9). However, three participants demonstrated a different pattern of performance. Participant 2 had similar performance for situational yes/no questions and yes/no objects and slightly lower performance for yes/no colours. Participant 3 performed better on situational yes/no questions compared to yes/no colours and objects. Participant 8 performed the same for situational yes/no questions and word-to-colour matching, but higher for situational yes/no questions compared to yes/no colours, yes/no objects and word-to-object matching.
Six participants performed significantly better than chance on the matching tasks (Figure 2). With respect to colour versus object stimuli, scores were typically higher on tasks with colours than objects. The situational yes/no task produced the lowest mean score, with all but one participant (P4) performing significantly below chance level (<45%), suggesting particular difficulty with this task. The mean score for yes/no objects was also below chance with only one participant (P4) performing better than chance, as indicated in Figure 2.
Total scores for all tasks were normally distributed as assessed by Shapiro-Wilk's tests (all P > 0.05). Mauchly's test indicated the assumption of sphericity had been met, χ2 (9) = 16.29, P = 0.06. As parametric assumptions were met a one-way repeated measures ANOVA with within-subjects factor of Task (yes/no colours, word-to-colour match, yes/no objects, word-to-object match, situational yes/no) was conducted. There was a statistically significant effect of Task, F(1,11) = 137.62, P < 0.0001, η²p = 0.66. Partial eta-squared indicated Task accounted for 66% of the variance in participant performance which is a large effect. Post-hoc comparisons using Tukey's HSD (Honest Significant Difference) test for multiple comparisons was performed to explore differences in group means between tasks (Table 4). Mean scores were significantly different between seven of the 10 pairs of tasks.
Comparison of means using Tukey Honest Significant Difference (HSD) tests.
*Tasks with significantly different mean scores at **P < 0.05 and ***P < 0.001.
To examine differences between task type and stimuli type (colours, objects), data was collapsed across categories, and two-paired t-tests were conducted.
Spoken word-to-referent matching tasks (colours and objects) (n = 24, M = 62.29, SD = 19.18) yielded significantly higher scores than yes/no tasks (colours and objects) (n = 24, M = 37.04, SD = 18.98), and this difference was large, t(23) = −6.49, P < 0.001, d = 1.33.
Colour stimuli tasks (n = 24, M = 53.08, SD = 23.33) yielded significantly higher scores than object stimuli tasks (n = 24, M = 46.25, SD = 22.20), and this difference was moderate, t(23) = 2.79, P = 0.01, d = 0.57.
Discussion
The influence of psycholinguistic variables on performance in language tasks is well-established in other populations, including neurotypical speakers and people with post-stroke aphasia and dementia.37–39 In this study, we applied these variables to the assessment of emergence from PDOC. All tasks required linguistic processing, but additional processing demands varied across tasks. Scores were higher on the spoken word-to-referent matching tasks compared to yes/no tasks, with mean scores for both objects and colours significantly higher than situational yes/no questions. Furthermore, mean scores for yes/no decisions on colours compared to situational questions were also significantly higher. With regards to colour or object stimuli, participants generally scored more highly with colour stimuli, despite colour words belonging to the same semantic category and the additional tactile information provided by objects.
There are two possible explanations for the colour advantage which may operate in tandem. One involves simpler visual perceptual properties. When shown colour tiles, participants only had to attend to the single attribute of colour, with no need to attend to shape, contour or orientation as required for objects. The second is psycholinguistic properties, and specifically the higher lexical frequency of colour names in comparison to object names.
Findings indicated that it may be task rather than stimuli that has more influence on accuracy of response. Results supported the hypothesis that spoken word-to referent matching tasks would elicit more accurate response than yes/no tasks. In contrast to yes/no questions, responses to spoken word-to-referent matching only need a simple motor response (e.g. eye movement or point) to a single key word (e.g. blue or yellow, cup or sock). A yes/no response requires remembering a question while choosing between options and formulating a response. Most participants provided a non-verbal response. For many, this was a gesture used naturally in conversation (nod/thumbs up). However, for others, a yes/no response was novel and needed to be remembered and executed, such as a thumbs up for ‘yes’ and moving thumb side to side for ‘no’ or pointing to written yes/no cards using a finger or eyes.
The results also supported the hypothesis that participants would respond with greater accuracy to yes/no questions with colours and objects compared to the more complex Coma Recovery Scale-Revised situational questions (‘Am I touching my ear right now?’ ‘Am I touching my nose right now?’). The situational questions place higher demands on linguistic processing and therefore run the risk of confounding consciousness assessment with issues of language capacity.40,41 These probes load language processing in a number of ways. First, the probe question is longer and thus contains more lexical information for processing compared to ‘is it blue?’, is it a cup?’. The inclusion of ‘right now’ is both superfluous, adding to the processing load and, at the same time, creates ambiguity. In the case of ‘Am I touching my ear right now?’, ‘right now’, is intended as a temporal adverbial phrase but might be interpreted as a spatial adverbial phrase (right ear). Second, the situational questions place greater demand for grammatical processing, not only in length of the word string but also through inclusion of fine morphological contrasts such as the person reference of pronouns (I vs. you) and articles (my vs. your).
Clinical implications
Findings from this study indicate that the tasks and stimuli used to assess the return of functional communication in patients emerging from PDOC should be carefully considered. Stimuli chosen to assess for communication in patients emerging from PDOC should be based on psycholinguistic principles. This will ensure that tasks are minimally confounded by language or cognitive deficits, to reduce contamination of assessment of consciousness with other cognitive impairments. This will support clinician confidence in using tasks to identify returning language skills and patient reporting of conscious experience.
The situational yes/no tasks from the Coma Recovery Scale-Revised resulted in the lowest scores compared to other tasks and stimuli. They should not be relied on as a sole measure of functional communication if patients display low scores. Spoken word-to-referent matching tasks can be used in preference to yes/no questions, and in most instances, colours score more highly than objects. Word-to-referent matching tasks reduce linguistic and other cognitive demands and have the additional benefit of developing prerequisite skills required to use low-tech AAC to express a need or preference.
Limitations and further research
Limitations regarding this study reflect the difficulty in research involving a complex small population. The study was also disrupted by the COVID-19 pandemic as planned recruitment was originally 20–30. The study was, however, adequately powered as shown by effect sizes, but there is a need to replicate findings to test for generalisability of results. Furthermore, recruitment criteria excluded emerging patients with severe visual impairments, and recommendations for use of colours versus objects might not hold for these individuals.
Although the results of this study lead to the recommendation for use of colours in assessment, the selection of colours needs careful consideration of issues of colour blindness and the need to avoid commonly confused colour distinctions (e.g. red vs. green). Also noteworthy is the difference across languages in colour terms in both the number of colour words and the location of colour word boundaries. 42 For example, English and many languages spoken in industrialised societies include 11–12 basic colour terms, while languages spoken in some non-industrialised cultures use as few as two or three colour terms.43,44 This may make it difficult to generalise findings to all cultures.
This study compared colours, objects and the Coma Recovery Scale-Revised situational Y/N questions. Survey respondents 21 reported the use of other stimuli such as familiar photographs. The stimuli in this study were chosen to allow for the psycholinguistic properties to be controlled, but further research to compare response accuracy with personally relevant stimuli is required.
Family and caregivers’ contribution to the assessment of emergence is not well defined. There is some evidence that involving family members in carrying out assessment can improve responses of patients in PDOC.45,46 However, one issue overlooked in studies is the potential impact of unwitting cueing by untrained assessors. This study was originally designed to compare responses when tasks were presented by a family member versus a researcher. However, due to the COVID-19 pandemic restrictions, family visits were not possible. Therefore, further research is required to investigate the impact of familiarity of the assessor in determining emergence.
This study focused on assessment of emergence via language tasks based on one of the current emergence criteria.1,6 However, patients with unidentified aphasia may find such tasks difficult. Aubinet et al. 47 are developing the Brief Evaluation of Receptive Aphasia tool to identify residual language abilities following severe brain injury. However, although consciousness and self-awareness are considered to be intrinsically linked with language,2,4,5 it is possible to demonstrate a range of higher-level cognitive behaviours in the absence of language. Patients with global aphasia have demonstrated cognitive abilities in domains that do not rely on language processing, such as arithmetic, logic problems, spatial navigation and Theory of Mind (ToM – the ability to attribute mental states to others).48,49 If emerging patients can demonstrate awareness using non-verbal ToM tasks, which underpin social interaction, this could be considered the beginning of the return of functional communication, but such areas are not routinely assessed. Therefore, further research is also needed to investigate the return of other higher-level cognitive functions and how this may indicate the return of conscious awareness in patients emerging from PDOC.
Clinical messages
Emerging patients respond with greater accuracy to spoken word-to-referent tasks compared to yes/no questions, and to colours more accurately than objects. Patients scored lowest on the Coma Recovery Scale-Revised visually based situational questions. These should not be used as a sole measure to determine the return of functional communication.
Supplemental Material
sj-docx-1-cre-10.1177_02692155251403995 - Supplemental material for Assessing functional communication in patients emerging from a disorder of consciousness: Impact of task and stimuli on response accuracy
Supplemental material, sj-docx-1-cre-10.1177_02692155251403995 for Assessing functional communication in patients emerging from a disorder of consciousness: Impact of task and stimuli on response accuracy by Amy Pundole, Sophie Duport, Victoria Fleming, Suzanne Beeke and Rosemary Varley in Clinical Rehabilitation
Footnotes
Acknowledgements
The authors are grateful to the patients and their families for supporting participation in this study. The authors would also like to thank clinicians at the Royal Hospital for Neuro-disability who supported the project.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Amy Pundole has received funding from the Linder Foundation as part of her PhD fellowship with the Royal Hospital for Neuro-disability.
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Supplemental material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
