Abstract
Owusu et al. recently reported LiteFallNet as a lightweight model for real-time fall detection and concluded that its speed, accuracy, and transparency make it suitable for deployment in smart homes, eldercare facilities, and wearable health technologies. 1 The main issue, however, is not the architecture itself, but the interpretation of the evaluation protocol when deployment and userlevel generalization are discussed. In the Methods, the authors state that each original fall instance in FallAllD was augmented twice by Gaussian jittering before partitioning, increasing the fall class from 1722 to 5166 samples; the augmented dataset was then split 60%/20%/20% using stratified instance-level partitioning, and the test set was described as “completely blind.” 1 The same strategy was also applied to UMAFall, and the two datasets comprise only 15 and 17 participants, respectively. 1 A blind test set is not automatically an independent one. When augmented copies derived from the same original event are allowed to enter different partitions, the model may be tested on samples that are not identical to the training data but remain closely related to it. Likewise, when splitting is record-wise rather than subject-wise, participant-specific movement signatures can appear in both training and test sets. Preserving activity-level coverage may be useful for architecture development, but it does not remove the need for subject-wise evaluation when claims are made about deployment or user-level generalization.
This matters because the article makes a deployment claim, not only a modeling claim. 1 In digital health machine learning, record-wise splitting is a recognized source of identity confounding, and subject-wise evaluation is a widely recommended safeguard against it. 2 Recent IMU work has shown how large the effect can be: random splitting yielded 86.08% test accuracy, whereas subjectwise separation reduced it to 62.31% in the same task. 3 For the same reason, bootstrap confidence intervals do not address the central problem here, because resampling a non-independent test set cannot restore independence. The performance reported by Owusu et al. should therefore be read as performance under an augmentation-first, instance-level protocol, not, by itself, as evidence of generalization to unseen users. We suggest a re-analysis in which subjects are partitioned before any augmentation, augmentation is applied only within the training set, and final evaluation is conducted on untouched participants excluded from model development. Cross-dataset evaluation can be informative, but it is not a substitute for user-level independence within the evaluation protocol. If LiteFallNet remains competitive under those conditions, the case for real-world deployment would be substantially stronger.
