Human Factors Principles Enhance the Design of a Machine Learning-Based Lower-Limb Exercise System for Older Adults with Knee Pain

Abstract

Knee pain in older adults significantly reduces their physical function and quality of life. While lower-limb exercises are effective for treating this health issue, individuals’ access to supervised programs is often limited. To address this, our research team developed a machine learning-based lower-limb exercise system that offers video demonstrations of exercises, provides real-time movement feedback, and tracks performance and progress. Initial evaluation of the system revealed usability barriers. Consequently, we applied human factors principles to improve the system design and subsequently evaluated the usability and acceptance of the enhanced system. Ten adults (aged 60–73) with knee pain used the system to perform lower-limb exercises. Their opinions on usability and acceptance were gathered. We found that the enhanced system was perceived as both usable and acceptable. These findings underscore the necessity of integrating human factors principles into the design of such digital health applications, as this approach allows developers to create intuitive user interfaces that align with users’ preferences and expectations.

Keywords

knee pain lower-limb exercise human factors usability acceptance

Introduction

Knee pain is a prevalent health problem among older adults worldwide. Physical exercise is a recommended non-pharmacological treatment (Porcheret et al., 2007). Challenges with access and cost of therapist-led exercise programs have led to the use of digital technologies for remote exercise training. However, many digital programs lack real-time supervision and progress tracking, which is crucial for effective knee health management.

Our research team developed a machine learning-based lower-limb exercise training system for older adults with knee pain (Chen et al., 2024; Chen & Or, 2021, 2023). The system offers video demonstrations of exercises, provides real-time posture analysis and feedback based on machine learning techniques, and tracks performance and progress. In a 3-week implementation study (Or et al., 2024), 60 participants used the system to perform lower-limb exercise training. During this period we identified usability issues through direct observation and participants’ responses. Usability issues can hinder user acceptance and potentially diminish a system’s benefits. It is essential to address these issues before promoting the wide adoption of a system in the community. Based on the authors’ prior research methodology (Chen et al., 2025; Liu et al., 2022, 2023; Or et al., 2014; Tao & Or, 2012), usability and acceptance evaluations were implemented in this study. Also, human factors principles were applied to enhance system design (Nielsen, 1994a, 1994b; Or & Chan, 2024; Or et al., 2022).

Therefore, this study aimed to (1) summarize the usability issues in our lower-limb exercise training system, then apply human factors principles to enhance its design; and (2) assess the usability and user acceptance of the enhanced system among aging individuals with knee pain. Based on the findings, we briefly compared the enhanced version of the system with the earlier version. We also discussed the study limitations and implications for future research.

Enhanced System Design

We implemented our system on a computer equipped with a camera and the necessary peripherals (e.g., monitor, mouse, keyboard). The lower-limb exercise training system guides users through video demonstrations. Throughout the exercise session, the camera captures the users’ exercise movements. These movements are analyzed by machine learning-based pose estimation techniques. The system provides real-time feedback to inform users about incorrect postures and movements. After the exercise session, users can check their exercise duration and performance in a training log. For further technical details, see our previous articles (Chen & Or, 2021; Or et al., 2024).

While implementing the preliminary version of the system (Or et al., 2024), we observed usability issues when the participants logged in to the system, calibrated the camera, and performed exercises. For each usability issue, we followed human factors principles (Nielsen, 1994a, 1994b) to improve the system design (Table 1). Figure 1 illustrates our enhanced system.

Table 1.

Usability Issues, Human Factors Principles Followed, and Improvements Made to the System.

Usability issue	Human factors principle(Nielsen, 1994a, 1994b)	Improvement
Logging in to the system
- Users were unsure whether to click on the input box before entering their username.	- Efficiency of use: Minimizing the number of actions required to complete a task, thus enhancing overall efficiency.	- The default cursor was placed directly in the input box so that users could start typing immediately.
- Users were unaware that their phone number served as their username.	- Help and documentation. Providing clear and concise instructions to help users understand what to do.	- An instruction was added next to the input box stating: “Enter your phone number to log in.”
Calibrating the camera
- Users were unaware that they needed to click a “start” button to initiate the calibration process.	- Visibility of the system status: Informing users of the system status to help them understand what to do next.	- A voice instruction was added: “Now you can start calibration”; a blinking effect was added to highlight the “start” button.
- When users completed the calibration, they were unsure whether to click the “back to the home page” button or proceed to the exercises.	- Efficiency of use: Minimizing the number of actions required to complete a task, thus enhancing overall efficiency.	- The “back to the home page” button was removed; users were automatically directed from the “calibration completed” page to the “perform exercises” page.
Performing lower-limb exercises
- Users were unaware that they could switch between the tabs in the sidebar to choose different exercise categories, such as strengthening and stretching exercises.	- Minimalist design: Focusing on the essentials in content and visual design; removing unnecessary elements.	- All exercises were presented on one page so that users could easily see and access them all without needing to navigate the tabs in the sidebar.
- Users were unclear about the order in which to perform the exercises.	- Consistency and standards: Following existing guidelines to ensure external consistency.	- Exercises were organized and presented following the order in the clinical guideline.

Figure 1.

Demonstration of the enhanced system.

Methods

Study Design and Participants

Ten individuals with knee pain tested our enhanced system during a 45-minute session of lower-limb exercise training in our study laboratory. After a brief introduction to the system’s features, the participants were asked to explore the system independently and freely. During this period, no detailed instructions were provided. The participants (i) were aged 55 years or over, (ii) self-reported experiencing knee pain for at least one month during the previous 12 months, (iii) were able to speak and read Chinese, and (iv) were able to provide written informed consent. Exclusion criteria included having inflammatory arthritis, having a knee injury within the previous 6 months, having a history of knee replacement surgery, waiting for knee surgery, having physical therapy for knee problems within the previous 3 months, and being unable to participate in exercise training due to physical impairment or medical comorbidities. To ensure unbiased evaluation, we selected different participants from those who participated in our previous studies.

Outcomes

We assessed the usability and user acceptance of the enhanced system using questionnaires. We assessed usability using the 10-item System Usability Scale (SUS) (Brooke, 1996). The participants rated each item on a 5-point Likert scale, from 1 (strongly disagree) to 5 (strongly agree). The raw scores were converted to a standardized SUS score (possible range: 0 to 100), with higher scores indicating better usability. We assessed user acceptance across four dimensions adapted from previous studies (Chen et al., 2022; Davis, 1989; Or & Karsh, 2006; Xie & Or, 2020). These dimensions were: perceived usefulness (four items), perceived ease of use (four items), attitude toward the system (three items), and behavioral intention to use the system (two items). The participants rated each item on a seven-point Likert scale, ranging from 1 (very strongly disagree) to 7 (very strongly agree), with higher scores indicating better acceptance.

Procedures

Eligible participants visited our laboratory, where a research assistant introduced the study details, obtained written informed consent, and collected demographic and clinical information using a questionnaire. The participants then used the enhanced system to perform lower-limb exercises for 45 minutes. Afterward, they completed a questionnaire to indicate their opinions on the system’s usability and acceptance for managing knee pain. Each participant received a HK$150 incentive after the experiment.

Statistical Analysis

Descriptive statistics were calculated for the sample characteristics, and for the usability and user acceptance scores of the enhanced system.

Results

Sample Characteristics

The participants comprised seven females and three males, aged 60 to 73 years (mean = 66.1 years, standard deviation [SD] = 4.1). Nine participants reported experiencing knee pain for at least three months within the previous 12 months. Knee pain intensity over the previous 12 months, measured on an 11-point scale (0 = no pain, 10 = worst possible pain), had a mean score of 4.9 (SD = 1.9) across all participants.

Usability of the Enhanced System

The enhanced system achieved a mean SUS score of 78.5 out of 100 (SD = 7.19, 95% confidence interval [CI]: [73.4, 83.6], range: 70 to 90, median = 76.3). All of the participants’ SUS scores exceeded 68, indicating above-average usability (Brooke, 2013). According to adjective ratings by Bangor et al. (2009), the mean SUS score of 78.5 fell between “Good” (score: 71.4) and “Excellent” (score: 85.5). Notably, all but one participant scored above the “Good” threshold.

User Acceptance of the Enhanced System

The participants expressed positive opinions across all of the four acceptance aspects (Table 2).

Table 2.

Descriptive Statistics of User Acceptance.

	Mean	SD	95% CI	Range	Median
Perceived usefulness	5.63	0.84	(5.02, 6.23)	5.00−7.00	5.00
Perceived ease of use	6.08	0.66	(5.61, 6.54)	5.00−7.00	6.00
Attitude	5.67	0.74	(5.14, 6.19)	4.67−7.00	5.83
Behavioral intention	5.40	0.84	(4.80, 6.00)	4.00−7.00	5.00

Discussion

This study demonstrated the positive impact of applying human factors principles to the design of human–computer systems, exemplified by a machine learning-based lower-limb exercise training system. Aging individuals with knee pain perceived the enhanced system to be both usable and acceptable.

Before this study, we developed and implemented a preliminary version of the system (Or et al., 2024), and assessed its usability and acceptance using the same questionnaire as was used in this study. Compared with the preliminary version, the enhanced system used in this study achieved higher mean scores for both usability and perceived ease of use. These improvements can be attributed to design modifications that addressed specific usability issues, particularly the clarity of instructions and ease of navigation. For instance, clear instructions helped the participants understand the system status and how to interact with its features effectively. Additionally, the use of blinking effects made the buttons visually distinct, enabling the participants to focus on the essential elements and use the system as intended. Moreover, reducing unnecessary clicks simplified system operation. This was particularly beneficial for users with limited computer skills or those unfamiliar with the system.

However, the mean scores for other aspects of user acceptance (perceived usefulness, attitude, and behavioral intention) did not surpass those of the preliminary version of the system. One potential explanation for this is the ceiling effect, as these aspects had already achieved high scores (over 5.6 out of 7 points), leaving limited scope for further improvement. Another contributing factor could be that the design enhancements focused on addressing usability issues to improve ease of use, with less attention on other aspects of user acceptance. Future improvements should expand the design focus to create a system that is not only easy to use but also highly useful, fosters positive attitudes, and encourages a strong intention to use it.

This study has several limitations. First, the sample size was relatively small, which limits the generalizability of the findings regarding the system’s usability and user acceptance. Future studies should include larger samples to confirm these results. Second, we did not conduct direct pre–post comparisons of usability and user acceptance between the different versions of the system as different participant cohorts were involved in the earlier and later studies.

Our study has implications that suggest directions for future research. First, whereas we assessed system usability using subjective measures, future research could benefit from incorporating objective measures. Examples of objective measures include eye-tracking data or task completion time for deeper insights into the identification of usability issues than this study (Wang & Or, 2020). Second, as perceptions may change, a longitudinal study is needed to evaluate whether the system’s usability and acceptance persist over time. Third, beyond usability and user acceptance, future research should explore the health benefits of the enhanced system.

Conclusions

This study assessed the usability of the enhanced design of a machine learning-based lower-limb exercise training system and proposed design enhancements. End users expressed positive feedback on the usability and acceptance of the enhanced system. These findings underscore the importance of applying human factors principles to improve system design. If users perceive the system developed in this study as easy to use, they may be more willing to engage in exercise training that benefits their knee health. The implications of this work extend beyond individual exercise programs. A human-centered approach throughout the design, development, implementation, and improvement enables the creation of systems that meet the needs of end users. Future research should apply human factors principles to promote the development of usable and accessible healthcare innovations for vulnerable populations.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was partly supported by a General Research Fund grant from the Hong Kong Research Grants Council (Project No. 17200323).

ORCID iDs

Tianrong Chen

Hao Liu

References

Bangor

Kortum

Miller

(2009). Determining what individual SUS scores mean: Adding an adjective rating scale. Journal of Usability Studies, 4(3), 114–123.

Brooke

(1996). SUS: A ‘quick and dirty’ usability scale. Taylor & Francis.

Brooke

(2013). SUS: A retrospective. Journal of Usability Studies, 8(2), 29–40.

Chen

C. K.

Yeung

E. H. K.

Chen

(2025). Perceptions of patients with stroke regarding an immersive virtual reality–based exercise system for upper limb rehabilitation: Questionnaire and interview study. JMIR Serious Games, 13(1), e49847.

Chen

C. K.

(2024). Heuristic evaluation and end-user testing of a machine learning-based lower-limb exercise training system for management of knee pain in individuals aged 55 years or over. International Journal of Industrial Ergonomics, 102, 103607.

Chen

C. K.

F. P.

(2022). Path analysis of the roles of age, self-efficacy, and TAM constructs in the acceptance of performing upper limb exercises through immersive virtual reality games. International Journal of Industrial Ergonomics, 91, 103360.

Chen

C. K.

(2021). Development and pilot test of a machine learning-based knee exercise system with video demonstration, real-time feedback, and exercise performance score. In Proceedings of the human factors and ergonomics society annual meeting. Sage Publications.

Chen

C. K.

(2023). Perceptions of a machine learning-based lower-limb exercise training system among older adults with knee pain. Digital Health, 9, 20552076231186069.

Davis

F. D.

(1989). Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 13(3), 319–340.

10.

Liu

Lou

(2023). Perceptions of and intention to use wearable and assistive devices among older adults. In Proceedings of the 14th International Conference on Applied Human Factors and Ergonomics (AHFE 2023) and the Affiliated Conferences, San Francisco, CA, USA. AHFE International.

11.

Liu

C. K.

Cheung

Chan

Tiwari

Tan

(2022). A longitudinal examination of tablet self-management technology acceptance by patients with chronic diseases: Integrating perceived hand function, perceived visual function, and perceived home space adequacy with the TAM and TPB. Applied Ergonomics, 100, 103667.

12.

Nielsen

(1994a). Heuristic evaluation. In Nielsen

Mack

R. L.

(Eds.), Usability inspection methods (pp. 25–62). John Wiley & Sons.

13.

Nielsen

(1994b). Usability engineering. Morgan Kaufmann.

14.

Karsh

B. T.

(2006). The patient technology acceptance model (PTAM) for homecare patients with chronic illness. In Proceedings of the human factors and ergonomics society annual meeting. Sage Publications.

15.

Wong

Tong

Sek

(2014). Private primary care physicians’ perspectives on factors affecting the adoption of electronic medical records: A qualitative pre-implementation study. Work, 48(4), 529–538.

16.

C. K.

Chan

A. H.

(2024). Inspection methods for usability evaluation. In Stephanidis

Salvendy

(Eds.), User experience methods and tools in human-computer interaction (pp. 17–-192). CRC Press.

17.

C. K.

Chen

Yam

L. Y. C.

Wong

E. L. Y.

Yeoh

E. K.

Cheung

M. T.

(2024). Technological surrogate physiotherapy to improve knee health through exercise: Human-computer interaction to build trust and acceptance notwithstanding pain. International Journal of Human–Computer Interaction, 1–12.

18.

C. K.

Holden

R. J.

Valdez

R. S.

(2022). Human factors engineering and user-centered design for mobile health technology: Enhancing effectiveness, efficiency, and satisfaction. In Duffy

V. G.

Ziefle

Rau

Tseng

M. M.

(Eds.), Human-automation interaction: mobile computing (pp. 97–118). Springer.

19.

Porcheret

Jordan

Croft

(2007). Treatment of knee pain in older adults in primary care: Development of an evidence-based model of care. Rheumatology, 46(4), 638–648.

20.

Tao

(2012). A paper prototype usability study of a chronic disease self-management system for older adults. In 2012 IEEE International Conference on Industrial Engineering and Engineering Management (pp. 2072–2076). IEEE.

21.

Wang

C. K.

(2020). Effects of text enhancement, identical prescription-package names, visual cues, and verbal provocation on visual searches of look-alike drug names: A simulation and eye-tracking study. Human Factors, 62(7), 1102–1116.

22.

Xie

C. K.

(2020). Acceptance of mHealth by elderly adults: A path analysis. In Proceedings of the human factors and ergonomics society annual meeting. Sage Publications.