Abstract
Transportation accessibility remains a critical challenge for visually impaired individuals, constraining their autonomy and societal participation. Although autonomous vehicles (AVs) hold transformative potential for enhancing mobility, prevailing human-machine interfaces (HMIs) frequently neglect the unique interaction requirements of this population. This study investigates the efficacy of a multimodal HMI explicitly designed to facilitate autonomous ridesharing interactions for visually impaired users. Employing a between-subjects experimental design, we evaluated user trust and satisfaction across six core ridesharing functions under three distinct conditions: (1) visually impaired participants with multimodal (audio-visual) feedback, (2) non-visually impaired participants, and (3) visually impaired participants without audio feedback (N = 24). Our findings demonstrate that audio-enhanced multimodal interfaces bridge the accessibility gap, enabling visually impaired users to attain trust and satisfaction levels statistically comparable to those of non-visually impaired users. Furthermore, the absence of audio feedback significantly degraded navigational confidence, vehicle identification accuracy, and overall user experience (p < .05). These results theoretically validate the significance of auditory cues in AV HMIs, while empirically confirming design principles for universal accessibility. By providing actionable guidelines for inclusive interface design, this work advances equitable mobility solutions and underscores the imperative of user-centered autonomy in next-generation transportation systems.
Introduction
Transportation represents a fundamental challenge for visually impaired (VI) individuals, with over 25 million Americans experiencing transportation-limiting disabilities due to sensory, cognitive, or motor impairments (Fink et al., 2023). This lack of accessible transportation significantly impacts their independence, workforce participation, and overall quality of life (Flaxman et al., 2021). Fully autonomous vehicles (FAVs) hold immense potential to transform mobility by offering safe, independent travel options for VI individuals. Yet, current human-machine interfaces (HMIs) in these vehicles depend heavily on visual cues, rendering them inaccessible to this population.
Compounding the issue, current policies and interface designs often fail to address the specific accessibility needs articulated by blind and visually impaired people, preventing the realization of this technology’s life-changing mobility benefits (Fink et al., 2021). Despite these challenges, recent advances in inclusive AV interface design demonstrate that effective user interfaces can be developed through targeted evaluation methodologies tailored specifically for persons with visual impairments (Angeleska et al., 2022).
Key research indicates that multimodal feedback systems—integrating auditory, tactile, and visual cues—significantly enhance usability for VI users, yet most existing navigation systems lack necessary flexibility, forcing users to adapt to the system rather than accommodating individual needs (Dourado & Pedrino, 2023). This gap is particularly critical as emerging evidence shows that different explanatory techniques and modality combinations substantially impact cognitive load, situation awareness, trust, and overall user experience in automated systems (Kaufman et al., 2024).
In summary, this study applies human factors methodologies to develop and evaluate a multimodal HMI inclusive interface for autonomous vehicles aimed at enhancing independence and overall experience of VI users. The research focuses on identifying essential features and modalities preferred by VI individuals, developing a prototype that integrates auditory and visual feedback, and conducting comprehensive usability testing to assess the system’s effectiveness. By advancing accessible HMI solutions, this work contributes to the broader goal of making autonomous transportation more inclusive for VI users.
Methods
This study employs a structured three-phase methodology to systematically analyze and optimize travel-related tasks. The initial phase involves a functional decomposition of the complete trip cycle into six critical operational components: (1) identity verification, (2) trip confirmation, (3) standard driving operations, (4) unexpected event resolution, (5) destination arrival protocols, and (6) termination procedures. Building upon this framework, a comprehensive hierarchical task analysis was implemented to delineate fundamental elements and decision points within each functional domain. Subsequently, these analytical insights informed the development of an optimized human-machine interface grounded in user-centered design principles. The final validation phase employed a controlled experimental paradigm to quantitatively assess the efficacy and usability of the proposed interface prototype. The study was approved by the University of Michigan Institutional Review Board (HUM00263286).
Task Analysis
Task1: Pre-drive preparation
Identity Verification
Upon boarding, the autonomous vehicle’s voice module prompts the passenger to fasten their seatbelt for safety. Then, audio and visual cues guide them to confirm their identity by speaking their four-digit ride code (shown in Figure 1).
Confirm ID process.
Trip Confirmation
The system presents destination and prompts passenger for verbal confirmation. Passenger responds with “yes” or provides correct address. If recognition fails, system offers customer service connection.
Task2: During the Ride
During Routine Driving Tasks
The autonomous driving system uses audio-visual cues to inform passengers about upcoming events and decisions during routine driving. For example, as the vehicle nears an intersection, it announces, “Approaching intersection. Stopping for red light in 3 s” Once stopped, it confirms, “Stopped at red light,” and the HMI shows the estimated red light duration. The system briefly describes intersection details, like traffic and pedestrian activity. Before the light turns green, it states, “Light turning green soon. Checking intersection safety. Preparing to move.” After crossing, it confirms, “Intersection cleared,” and updates the estimated arrival time audibly and visually (shown in Figure 2).
Traffic lights procedure.
Handling Unexpected Events
In the event of potential traffic issues—such as sudden obstructions, accidents, or hazards—the HMI switches to the special mode designed to handle these situations by informing passengers of real-time dangers and evasive actions (shown in Figure 3).
Emergency avoidance procedures.
Task3: Reaching Destination
Destination Notification
The system will provide an audible alert 3 min prior to reaching the destination.
Exit Interaction
Upon arriving at the destination, the system will provide disembarking guidance, detailing a safe exit procedure.
Prototype Design
Based on the task analysis results and human-centered design principles, we developed a multimodal in-vehicle HMI prototype. The design incorporates key human factor principles including:
Attention Management (e.g., salience compatibility, multimodal approach).
Perception Optimization (e.g., minimizing information access costs, gain redundancy, proactive updates).
Memory Support (e.g., predictive aids, consistency, knowledge in the world).
The prototype provides real-time feedback on the vehicle’s intentions, communicates the surrounding environment, anticipates driving decisions, and facilitates smooth mode transitions during unexpected events.
Technical Approach
A laptop connects to an external monitor mounted on the back of the front seat, serving as the visual interface to display route status, pre-arrival alerts, and environmental descriptions. Surrounding the screen are physical buttons in yellow (#F4B300) and purple (#B87AFF), each tied to predefined functions. An external speaker on the rear seat enables voice interaction. On the software side, MATLAB is used to develop codes controlling audio playback and touchscreen content, ensuring synchronized or asynchronous real-time display and audio feedback for dynamic multimodal presentation. Lastly, Audio content for the voice module is pre-recorded using OpenAI’s web-based advanced voice generation (Voice: Maple).
Experiment Design
A between-subjects study with 24 participants (12 males, 12 females; Mean age = 27.58 years, SD = 3.07) was conducted to evaluate the prototype’s usability. Post-hoc power analysis revealed adequate power (0.50–0.81) for detecting medium to large effects (ε² = 0.192–0.322).
Independent Variable
Participants were randomly assigned to three interface accessibility conditions (N = 8 participants per condition).
Visual Impairment Simulation
To simulate visual impairment conditions, we utilized the Cambridge Visual Impairment Simulator Software (Inclusive Design Toolkit, University of Cambridge) to adjust screen displays for normal-sighted participants. The simulated condition applied a moderate blur setting, reducing visual acuity to 0.1 to 0.3 (Snellen chart), consistent with WHO-defined moderate low vision. To ensure experimental consistency, all participants in the visual impairment condition experienced the same moderate blur setting. This approach was chosen because blur effectively replicates common visual challenges while allowing standardized implementation. The selected level represents visual impairments that significantly impact interface usability while still enabling interaction with properly designed elements (shown in Figure 4).
Visual impairment simulation (Normal vs. Visually Impaired).
Experiment Process
Experiment procedure.
Data Analysis
Data were collected through an online questionnaire including demographic information, 1 to 5 Likert-scale questions on trust and satisfaction with the HMI prototype, and open-ended interviews. The data were processed by software SPSS (ver. 26) and MATLAB (ver. 2024a).
For the quantitative data, due to the non-normal distribution of the Likert-scale questions, a nonparametric Independent-Samples Kruskal-Wallis Test was employed to compare group differences across eight dimensions:
Qualitative data from open-ended interviews, focusing on audio-visual feedback preferences, perceived effectiveness, and improvement suggestions were thematically coded and analyzed to complement the quantitative findings and provide deeper insights into user experience with our HMI prototype.
Results
Trust and Satisfaction
The Kruskal-Wallis test revealed significant differences across the three groups in three aspects (shown in Figure 6):
No significant differences were found between VI and NVI groups across all measures (8 aspects), suggesting that the multimodal interface effectively supported visually impaired users when audio feedback was available.
Ratings on key dimensions.
Figure 7 highlights the effectiveness of our multimodal interface for visually impaired users in autonomous ridesharing. With audio feedback, the VI group achieved performance levels comparable to the NVI group across all eight aspects, demonstrating that the interface successfully bridges accessibility gaps. However, the VIX group showed notable declines, particularly in navigation and rideshare identification, underscoring audio feedback’s critical role.
Evaluation across eight dimensions.
Subjective User Feedback
The open-ended interview revealed several key findings:
94 % participants in the NVI and VI groups (15 out of 16) reported positive experiences, highlighting “clear communication,” “specific directions,” and the effective integration of audio-visual feedback.
Seventy-five percentage participants in the VIX group (6 out of 8) expressed concerns about the interface limitations, primarily focusing on difficulty receiving timely information without audio feedback.
Conclusion
Audio feedback played a critical role in enhancing the experience of visually impaired users. Participants who received multimodal feedback demonstrated higher confidence in key tasks, including correct rideshare, route navigation, and overall satisfaction. In contrast, those without audio cues exhibited decreased confidence and increased frustration, particularly during complex interaction tasks such as location positioning and handling unexpected incidents.
The findings further highlight the effectiveness of multimodal coordinated feedback based on human factors design principles in developing accessible interfaces. By reducing cognitive load and incorporating redundant audio-visual information, the multimodal system successfully addressed the specific needs of visually impaired users, enabling a seamless and inclusive ride-hailing experience. However, the controlled experimental environment differs from real-world AV experiences, and using simulated visual impairment with normal-sighted participants limits generalizability to diverse VI populations. Future research should test with actual VI users in real road conditions. This pilot study was designed as preliminary research to establish proof-of-concept for the multimodal interface design before conducting larger-scale studies with diverse visual impairment populations.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
