The scope of Mixed Reality in Aphasia Rehabilitation (MiRAR)

Introduction

Aphasia is a language disorder that results from a brain injury (e.g. stroke or traumatic brain injury, to name a few). Almost one-third of individuals experience aphasia secondary to stroke. ¹ Aphasia has a severe impact on an individual’s language and communication. Communication is an essential human act that supports successful participation in life activities, like social gatherings. Deficits in communication can negatively impact interpersonal relationships and social networks, leading to a poor quality of life (QOL) in persons with aphasia (PWA). ² Speech-language pathologists assist these individuals in regaining their skills and train strategies to overcome communication and participation restrictions in everyday life. Intervention in aphasia aims to maintain effective participation in everyday life. ³

Aphasia management has three major approaches: the neoassociationist or syndromic approach, the cognitive neuropsychological or impairment-based approach, and the social or consequences-based approach. ⁴ The neoassociationist approach focuses on the patient’s symptoms by targeting the superficial components of language (phonemic, lexical, or syntactic) without focussing on the underlying impairment (e.g. Melody Intonation Therapy: ⁴ ). The cognitive neuropsychological approach aims to improve the cognitive subcomponents essential for language processing (e.g. mapping therapy). Both approaches target only the linguistic aspect, which is insufficient to improve PWA participation in social settings. The social approach is widely accepted and has led to a shift toward these approaches that enhance the individual’s functional ability in daily life. ⁵

Speech-language pathologists (SLPs) generally provide therapy within structured clinical settings (e.g. hospitals and health centres). Such settings restrict clinicians from providing functional communication rehabilitation in an effective manner that reflects the real-world environment. The significant constraints on providing training include time, staffing, workloads, and service delivery. ³ Communication is a multidimensional process influenced by various environmental factors (e.g. setting, skills, and attitudes of communication partners) and personal factors (such as anxiety, fatigue, motivation, education, and culture). Hence, it can be challenging to address these factors in clinical settings. ⁶ However, new technological advancements, such as Extended Reality (XR), may alleviate some of these issues.

Extended Reality (XR) refers to the interaction between humans and machines in a combined real and virtual environment generated by computer technology and wearable devices. There are three types of XR: Augmented, Virtual, and Mixed Realities. ⁷ In Augmented Reality (AR), the users’ perception of Reality is enhanced by providing additional computer-generated virtual objects/artefacts. ⁸ AR has three essential functions: it combines real and virtual environments, provides real-time interaction, and places virtual objects in real-world environments. ⁸ For example, AR can produce a 3D model of a box on top of a real table. Virtual Reality (VR), on the other hand, is a computer-generated 3D environment comprising digitalised experiences that can simulate or differ entirely from the real world. ⁹ Depending upon the immersion, the VR is classified into Non-immersive (e.g. video games), Semi-immersive (e.g. EVA-Park – aphasia rehabilitation: ¹⁰ ), and Immersive types (e.g. virtual gaming zone). Unlike AR and VR, Mixed Reality (MR) is the most advanced Extended Reality technology, which merges real and virtual worlds to create a new environment and visualisation, where objects in physical and digital environments coexist and interact in real-time. ¹¹ MR produces unprecedented advancements in fields like education, entertainment, engineering, and healthcare.

Virtual Reality is the most used among all the Extended Realities, ¹² followed by Augmented Reality^13,14 in aphasia rehabilitation for functional and linguistic outcomes. Here, we highlight the advantages of MR compared to other Extended Reality technologies for the intervention of functional communication skills in PwA. With this purpose, we intend to evaluate each XR technology (VR, AR, and MR) in terms of its suitability for implementing real-life contexts in conventional, clinician-assisted rehabilitation environments.

Augmented reality in aphasia rehabilitation

AR devices can produce highly immersive 3D characters that interact with both real and virtual objects. ¹⁵ This technology helps people to experience virtual augmentation within the real world. AR in aphasia rehabilitation can supplement home language therapy. For instance, the home interior can be enhanced with additional information, which increases relevance while training on naming objects in natural settings. ¹⁴ An interactive AR platform with a basic user interface for home-based programs was developed using dynamic language therapy. ¹⁴ This program aims to enhance object naming and complement traditional therapy. Another program, the ‘Aphasia Rehabilitation Agent’ (ARA), targets phonological deficiencies in PwA. The ARA program is a portable, home-based device that delivers individualised therapy without the assistance of a therapist. ¹³

The two programs mentioned above highlight the benefits of AR in aphasia rehabilitation, such as being readily available, intensive, and cost-effective, as well as permitting repeated practice and reducing the need for recurrent visits to therapy centres.^13,14 However, these AR programs in aphasia rehabilitation do not address the process of social interactions, such as grammar constructs, spoken comprehension, and turn-taking during conversations. ¹³ Importantly, AR-driven aphasia therapy cannot replace the traditional face-to-face therapy provided by the speech therapist. ¹⁴ These programs mainly focus on training the linguistic impairment of PwA. ¹⁴

Recently, there has been a growing awareness of functional communication approaches in aphasia rehabilitation. ⁵ In functional communication approaches, such as script training, there is a need to simulate a completely interactive virtual environment with the PwA, replicating real-world interactions. The AR supports only live interaction with virtual objects in the real world. It cannot produce a 3D environment like VR. For instance, while training on a functional communication scenario, ‘Restaurant’, the PwA should feel that they are in a restaurant and can interact with the 3D server/waiter (i.e. an avatar). Although AR can produce 3D objects such as tables, menus, or plates in real-world restaurants, it limits the creation of the entire context in which training functional communication is possible. Thus, the standalone AR may not suit the requirements for implementing a functional communication intervention for PwA.

Virtual reality in aphasia rehabilitation

Virtual Reality offers a three-dimensional environment on a computer for one or many users. Multi-user virtual environments might be exceptionally well-suited to handle communication in context. ¹² Many different learning contexts have leveraged the capability to create accurate, simulated experiences. For instance, virtual simulations of surgical procedures could be used to instruct medical professionals. ¹⁶ Systematic reviews on the use of VR in aphasia rehabilitation have been published since 2020.^12,17–19 The highlights of VR are that it has the potential to raise therapy dosage, optimise sensory stimulation, and enhance the ecological validity of aphasia intervention. ¹⁹

VR technology is not superior to traditional therapy in terms of interventional outcomes. For instance, Grechuta and colleagues ²⁰ assessed the effect of conventional versus virtual reality therapy on 17 participants (traditional therapy = 8, Virtual Reality = 9) with chronic non-fluent aphasia. Both groups received 8 weeks of training in naming, repetition, spelling, and articulation. The outcome measures were the Boston Diagnostic Aphasia Examination, Communicative Activity Log analysis, and the vocabulary test analysis. Results indicate that both groups had similar outcomes in speech production, auditory comprehension, communicative effectiveness in everyday life, and lexical access. ²⁰ A recent systematic review opined on the lack of immersive VR in Aphasia rehabilitation. ¹² For example, the EVA-Park ¹⁰ uses a point-and-click interface that lacks complete immersion in the virtual environment. ⁷ Fully immersive VR technology provides a ‘feeling of presence’ and induces real-world-like situations, but head-mounted devices (HMDs) are necessary for such an experience. However, users of traditional HMDs may experience symptoms of simulation sickness, such as fatigue and nausea, which prevent simultaneous interaction with the environment in real-time. ²¹ While visual dissociation from the real world is necessary for certain activities, such as gaming, this dissociation could potentially restrict its use in specific clinical populations (e.g. PwA), where the SLP’s concurrent involvement is essential.

During complete immersion, specific clinical populations (e.g. individuals with stroke) may encounter concomitant problems like nausea, which in turn could limit the usage of such devices for prolonged practice. It may be in this regard that some non-immersive VR systems (like EVA-Park) are clinically beneficial as they are less likely to cause usage-related issues. Further, as a recent systematic review suggests, ¹² why use such an expensive technology if there is no difference in therapy outcomes compared to traditional therapy? Hence, as mentioned above, VR technology has certain potential limitations that could hinder its practical application in aphasia rehabilitation. In this context, we maintain that the recent developments in the ‘Extended Reality’ (XR) technology that allows users to interact with the virtual and real worlds may simultaneously be well-suited for the functional communicational rehabilitation of PwA. For instance, combining Virtual and Augmented realities (VR + AR) – that is, a Mixed Reality (MR) – could be advantageous over complete immersion (i.e. VR in immersive environments).

As discussed earlier, Mixed Reality (MR) combines Virtual and Augmented Reality to harness the benefits of the two technologies and deliver functional communication training for PwA. MR technology can provide the function of 3D object interaction in AR and the virtual environment of VR in real-time, offering immersive experiences. Hence, this technology can eliminate the potential barriers in providing social communication training, such as script training, to promote functional communication in PwA. Preliminary attempts have been made at the Aphasia Rehabilitation House in Florida, USA. ²² Another program, the Mixed Reality in Aphasia Rehabilitation (MiRAR; see Figure 1) was developed in Manipal, India.^23,24OPEN IN VIEWER

The Aphasia House is an intensive rehabilitation program centred on treating PWA at the impairment level in an immersive environment. ²² This program involves recreating a home environment rich in memories and relationships that foster social interaction and language use through conversational stories. On the other hand, MiRAR implements the Life Participation Approach to Aphasia (LPAA) therapy using script training, which is deployed on Microsoft HoloLens.^23,24 The MiRAR program is developed with 20 social scenarios, such as restaurants and banks, to target functional communication in PWA. This program has been successfully developed and is now under validation in PWA.^23,24 The preliminary clinical outcome of this program was recently (2025) presented at the Science of Aphasia conference in Copenhagen, Denmark. ²⁵ The study compared the performance of PWA on script training delivered through the traditional method and the MiRAR program. The results showed that both groups improved compared to their baseline performance. However, when comparing the group performance, the MiRAR group obtained higher scores in the script-related words. Thus, the preliminary finding indicated that the MiRAR program has an added advantage in delivering functional communication training, which can be further explored using an extensive clinical trial. More importantly, the MiRAR is deployed in a semioccluded HMD (Microsoft HoloLens), which does not induce simulation sickness, and has been reported as a safe tool in aphasia rehabilitation. ²⁶

Conclusion

In this paper, we reviewed and appraised various extended reality (XR) technologies and their suitability for providing functional communication training to persons with aphasia. VR technology is still in its infancy and is currently limited to non-immersive VR applications in aphasia rehabilitation; the implications of immersive VR are complex. AR technology potentially limits the creation of contexts for training individuals with aphasia. The combination of virtual and augmented realities – the Mixed Reality – may offer more realistic, yet controlled contexts that support safe clinician-assisted functional communication training in clinical environments for persons with aphasia.