Multi-sensor fusion for differentiating swallows between healthy adults and patients with post-stroke dysphagia

Abstract

Objective

The aim of this study was to develop a non-invasive method using multi-sensor fusion to discriminate between abnormal swallows in patients with post-stroke dysphagia and normal swallows in healthy individuals.

Methods

Acceleration signals, nasal airflow signals, and sound signals were obtained from 108 healthy adults and 108 post-stroke dysphagia patients. Each swallowing signal was segmented according to videofluoroscopic swallowing study (VFSS), followed by features extraction and selection. Support Vector Machine, Decision Tree, K-Nearest Neighbor, Naïve Bayes, and Logistic Regression models were employed to discriminate between normal swallows in healthy individuals and abnormal swallows in post-stroke dysphagia patients.

Results

Overall, classification models utilizing signals from multi-sensor demonstrated superior performance when compared to signals from single-sensor and dual-sensor. Among the five models, the Support Vector Machine model (accuracy: 91.39±3.03%; specificity: 89.53±4.09%; sensitivity: 92.99±2.84%; F1-score: 91.86±2.92%) and Logistic Regression model (accuracy: 91.00±2.71%; specificity: 89.88±3.77%; sensitivity: 91.97±2.70%; F1-score: 91.45±2.55%) have better overall performance for classification.

Conclusions

The multi-sensor fusion has shown promising ability in differentiating swallows in healthy adults from those in patients with post-stroke dysphagia. The findings may provide an important foundation for the use of non-invasive multi-sensor fusion methods to identify dysphagia.

Keywords

stroke dysphagia swallowing non-invasive multi-sensor fusion

Introduction

Dysphagia is a common consequence of various neurological disorders, especially stroke.^1,2 The prevalence of post-stroke dysphagia ranges from 8.1% to 80%, depending on the stroke phase, assessment methods, and diagnostic criteria used.³ Complications associated with dysphagia include dehydration, malnutrition, and aspiration pneumonia, which may lead to prolonged hospitalization and even increased mortality.⁴ Additionally, long-term dysphagia has been related to the adverse psychological health outcomes.⁴ Therefore, early identification of dysphagia is critical for patients as it facilitates timely and targeted interventions to reduce the risk of these associated complications.⁵

Bedside assessments and instrumental evaluations are commonly used for screening dysphagia. Although bedside assessment is convenient, it often relies on subjective interpretation and may fail to detect subtle signs of dysphagia. Fiberoptic endoscopic evaluation of swallowing (FEES) is an important method used to assess dysphagia, but it is invasive and may not be advisable for patients who cannot tolerate the examination. Videofluoroscopic swallowing study (VFSS) is the gold standard for assessing dysphagia,⁶ but it has drawbacks such as radiation exposure and the inability to be performed at the bedside,⁷ and not suitable for patients in intensive critical unit. These limitations may pose challenges for some patients, particularly those necessitating regular dysphagia assessments. Furthermore, none of the methods mentioned above facilitate the dynamic monitoring of swallowing function under actual feeding situations in daily life. Therefore, there is an urgent need for the development of a non-invasive, real-time, and effective method to identify dysphagia.

Several non-invasive sensors have been investigated for dysphagia assessment.^8,9 For example, accelerometer has been used to capture signals of laryngeal motion during swallowing. Lee et al.¹⁰ showed that the magnitude of the acceleration signal correlated with the degree of laryngeal elevation, which is important for the safety and efficacy of swallowing. Furthermore, acoustic characteristics of swallowing sounds recorded through the microphone can also help to assess dysphagia.^11,12 However, relying on the signals captured by a single sensor for dysphagia assessment may not be sufficient because swallowing is a complex physiological activity. Therefore, it has been suggested that multi-sensors might be considered as a way to improve the accuracy of dysphagia detection.

Since the pharynx is the shared pathway for both bolus entering the esophagus and air entering the trachea, the coordination between respiratory and swallowing is crucial in preventing aspiration during swallowing process. Furthermore, it has been shown that patients with dysphagia have impaired respiratory and swallowing coordination.¹³ Therefore, recording respiratory and swallowing signals simultaneously using multiple sensors may help to better assess dysphagia. In view of this, Yagi et al.⁸proposed a swallow monitoring system that combined the nasal cannula-type flow sensor with the piezoelectric sensor to obtain respiratory flow, laryngeal motion, and swallowing sound signals to detect and assess swallowing activity. Following this, Inoue et al.¹⁴ evaluated the ability of the system to differentiate between normal swallowing in healthy individuals and swallowing in patients with dysphagia, and found that multi-sensors were better at differentiating than single sensors.

Although multi-sensor fusion has shown promise for dysphagia assessment, the existing literature remains limited and methodologically constrained. The few available studies often enrolled patients with heterogeneous etiologies rather than focusing on a well-defined disease population, employed small and imbalanced sample sizes, and did not consistently use VFSS to guide swallowing event segmentation. These gaps indicate the need for further investigation. Therefore, the present study aimed to evaluate the effectiveness of a multi-sensor system in discriminating between normal swallows in healthy adults and abnormal swallows in patients with post-stroke dysphagia.

Discriminating between normal swallows in healthy individuals and swallows in patients with dysphagia is important for the development of non-invasive assessment methods for dysphagia.¹⁵ To reduce heterogeneity caused by different etiologies, the present study focused exclusively on post-stroke dysphagia. Moreover, we employed a relatively large and balanced sample size and used VFSS to guide swallowing event segmentation in patients. Nasal airflow, acceleration, and sound signals are non-invasive and easily obtainable. Each of these signals has been used in previous studies to assess swallowing function.^9,15 Therefore, in the present study, we used these three signals to investigate the effectiveness of a multi-sensor system for screening abnormal swallows in post-stroke dysphagia patients by differentiating them from normal swallows in healthy individuals.

Methods

Participants

Two groups of subjects were included in this study: Group (i) consisted of 108 healthy adults recruited from the community with a median age of 51 years (range 20-76 years) and no history of dysphagia or any medical condition that might impact swallowing and respiratory function. Group (ii) included 108 stroke patients with a median age of 61 years (range 27-80 years) who were referred to VFSS following clinical assessment and ultimately diagnosed with dysphagia. The main exclusion criteria included cognitive dysfunction, tracheotomy, severe lung infection, dysphagia caused by other diseases, and unsuitable for VFSS. These patients were inpatients in the Department of Rehabilitation. We conducted this study from May 2023 to April 2024. Of note, the present study is not a new clinical trial but rather an extension of a previously conducted trial. Ethical approval for this prospective study was obtained (no. [2022]02-192-01), which applies to both the original clinical trial and the present extension study. The trial registration number provided (ChiCTR2300068908) refers to that prior study. The study was conducted in accordance with the principles of the Declaration of Helsinki. All participants signed informed consent prior to this study.

Sample size

The sample size was justified by the fact that it is comparable to or larger than that used in previous multi-sensor fusion studies,¹⁴ which successfully achieved discriminative accuracy >80% with a smaller patient group. Thus, our sample size of 108 per group is considered sufficient for reliable model development.

Data collection

The swallowing monitoring system simultaneously recorded the nasal airflow signals, triaxial acceleration signals, and sound signals during swallowing process. Nasal airflow signals were captured using a nasal cannula placed at the nares, connected to a gas flow sensor (AFM3000-200, Guangzhou Aosong Electronics Co., Ltd., China) with a sampling rate of 100 Hz. The triaxial accelerometer (ADXL357, Analog Devices, Shenzhen Vite Intelligent Technology Co., Ltd.) was placed at the level of the cricoid cartilage with medical adhesive tape.¹⁶ The sampling rate of the accelerometer was 2 kHz. The sensitive axes of the accelerometer corresponded to the superior-inferior (S-I), anterior-posterior (A-P), and medial-lateral (M-L) anatomical directions. The contact microphone (CM-01B, China Medical System Biotechnology Co., Ltd.) affixed with adhesive tape to the right below the triaxial accelerometer¹⁶ had a sampling rate of 48 kHz. Figure 1 illustrates the placement of nasal cannula, triaxial accelerometer, and contact microphone. Figure 2 shows the examples of simultaneously acquired acceleration signals, nasal airflow signals, and sound signals.

Figure 1.

Placement of nasal cannula, triaxial accelerometer, and contact microphone during the acquisition of swallowing signals.

Figure 2.

Example of simultaneously collected (Aa) acceleration signals, (Ab) nasal airflow signals, and (Ac) swallowing sound signals. Example of the four types of swallowing breathing patterns, including (a) exhale-swallow-inhale, (b) exhale-swallow-exhale, (c) inhale-swallow-exhale, and (d) inhale-swallow-inhale. In the section of acceleration signals, “X” for superior-inferior (S-I), “Y” for medial-lateral (M-L), and “Z” for anterior-posterior (A-P).

VFSS images were acquired using the Lanmage dynamic digital radiography machine (Athena Plus 7500; Shenzhen Lanmage Medical Technology Co., Ltd.; Shenzhen, China). Lateral images were recorded at a rate of 30 frames per second. Patients were seated upright and instructed to swallow different viscosities (thin, low-, med-, and high-viscosity) of 40% barium sulfate suspension. The low-, med-, and high viscosity suspensions were thickened with Softia-S thickener (NUTRI Co., Ltd., Japan). The textures of the thin, low, medium, and high viscosity suspensions met the criteria of level 0, level 1, level 2, and level 3 of the International Dysphagia Diet Standardisation Initiative (IDDSI) framework (https://iddsi.org/Framework), respectively. Assessment began with medium viscosity and then progressed to low and thin viscosity before finally testing with high viscosity. The bolus of each viscosity was administered to the oral cavity of patients using a syringe in the following order: 3 ml, 5 ml, and 10 ml. However, the type of stimulus delivered would be adjusted according to the condition of each participant. Swallowing signals and VFSS images were acquired simultaneously during the examination. Specifically, swallowing signals were collected and transmitted in real-time to monitoring software on the host computer, while simultaneous real-time acquisition of VFSS videos was conducted on the host computer. The monitoring software on the host computer synchronously triggers and records all collected data in real-time, and periodically sends calibration signals to ensure temporal synchronization of the swallowing signals and VFSS images.

Healthy participants were seated upright and instructed to swallow 3 ml, 5 ml, and 10 ml of each viscosity (medium, low, thin, and high viscosity fluid without barium), with a 30-second interval between each swallow. The bolus was administered into their oral cavity in the same way, i.e., by syringe. Visual images of normal swallows were recorded using a digital video recorder, and this visual image was acquired synchronously with nasal airflow signals, acceleration signals, and sound signals.

Data segmentation

The manual segmentation of swallowing events was performed by two well-trained speech-language pathologists to identify the onset and offset of swallowing events. The onset of swallowing was defined as the frame in which the bolus head passed the inferior margin of the shadow of the ramus of the mandible, and the offset of swallowing was defined as the frame in which the hyoid returned to its resting position.⁹

VFSS could not be performed in healthy subjects due to radiation exposure; therefore, segmentation for healthy participants was based on visual images, acceleration signals, and swallowing sound signals, an approach well-validated in the literature.^17,18 The onset of swallow was determined according to the start of laryngeal motion associated with pharyngeal activity (visual images and acceleration signals). The offset of swallow was determined by the cessation of swallowing sounds (sound signals) concurrent with no laryngeal motion (visual images and acceleration signals).^17,18

In total, we obtained 1273 segmented swallows from healthy participants and 1391 segmented swallows from stroke patients. Intra-class correlation coefficients (ICCs)¹⁹ were used to evaluate the intra- and inter-rater reliability in swallowing segmentation. 15% of swallows were randomly selected for analysis. For both the patient and healthy control groups, the intra-rater ICCs were 0.94 and 0.93 for the two raters, respectively, and the inter-rater ICC was 0.92, indicating excellent consistency.

Data preprocessing

The recorded signals underwent the processing steps to enhance its quality. The sampling frequency of sound signals was downsampled to 16 kHz. We used tenth-order Meyer wavelet and soft thresholding method to remove noise of sound signals. Here, the threshold value was calculated based on $\sqrt[σ]{2 \log N}$ , where N represents the number of samples, and σ denotes the estimated standard deviation of noise obtained through downsampling of wavelet coefficients. We then divided the signal data into components based on frequency range so that each component can be analyzed using a scale that matches its resolution. The fourth-order least squares splines was utilized for the removal of motion artifacts and low-frequency noise components in the acceleration signals. For nasal airflow signals, we applied a fourth-order Butterworth low-pass filter to preprocess the nasal airflow signal, aiming to remove the interference of high-frequency noise. Additionally, the denoised nasal airflow signals underwent the sliding smoothing process to facilitate the extraction of swallowing apnea duration (SAD) and respiratory swallowing pattern in subsequent processes.

Feature extraction

For each segmented swallow, features were extracted from the pre-processed signals. Specifically, the features of the sound signal were extracted by pyAudioAnalysis²⁰ library. During the feature extraction process, we applied a 50 ms Hamming window for data framing, with a 25 ms step size to ensure continuity between frames. We used the tsfresh²¹ library for feature extraction from acceleration signals. From the nasal airflow signals, two important features were extracted: SAD and respiratory swallowing pattern. For each segmented swallowing, the duration of airflow signal value of zero was defined as SAD; positive airflow signal values before and after the swallowing apnea indicate expiratory, while negative values indicate inspiratory, and the respiratory swallowing pattern was determined accordingly. Extracting these features can help to identify the coordination between respiratory and swallowing in participants. Figure 2 shows example of the four types of respiratory swallowing patterns: exhale-swallow-inhale, exhale-swallow-exhale, inhale-swallow-exhale, and inhale-swallow-inhale.

Feature selection and models development

Feature selection was performed using mutual information, which measures the statistical dependency between each feature and the target class label. For each candidate feature, a mutual information score was calculated with respect to the classification label, and features were ranked in descending order according to this score. Because higher mutual information indicates stronger dependency between a feature and the class label, the top-ranked features were retained for model development. To determine the number of selected features, we evaluated multiple top-k feature subsets during model development. This selection was performed within the training procedure rather than on the full dataset, so that the held-out test set was not involved in feature ranking or model optimization. The value of k was selected according to cross-validation performance on the training set, while also considering model stability and parsimony. Based on this procedure, 20 sound signal features and 10 acceleration signal features were retained for the final models.

Several classical machine learning classifiers were then developed, including including Support Vector Machine (SVM), Decision Tree, K-Nearest Neighbor (KNN), Naïve Bayes, and Logistic Regression. Hyperparameters were optimized using grid search with cross-validation on the training set. The dataset was split into training and testing sets using stratified sampling to preserve the class distribution. The test set was used only for final model evaluation. This workflow reduced the risk of overfitting and information leakage by ensuring that feature selection and parameter tuning were conducted independently of the final test evaluation.

In this study, we employed 10-fold cross-validation and averaged the results across the ten folds. We calculated accuracy, specificity, sensitivity, and the F1-score to evaluate the performance of the five models. To systematically evaluate the contribution of each sensor modality, we analyzed model performance for single-sensor, dual-sensor, and three-sensor approaches, respectively. This was designed to quantify the value of each signal type and to assess whether multi-sensor fusion offers advantages over single-sensor approaches.

Results

Classification performance of models based on signals from a single sensor

Based on the features extracted from the acceleration signals, the classification performance of Logistic Regression was overall superior to that of the other models, with accuracy, specificity, sensitivity, and F1-score values of 84.35±3.35%, 84.91±5.86%, 83.66±2.33%, and 84.78±3.24%, respectively (Table 1). Based on the features extracted from the sound signals, the classification performance of SVM was overall superior to that of the other models, with accuracy, specificity, sensitivity, and F1-score values of 77.75±2.65%, 74.81±5.66%, 80.69±4.04%, and 79.06±2.06%, respectively (Table 2).

Table 1.

Classification performance of models based on acceleration signals.

Classifier	Accuracy (%)	Specificity (%)	Sensitivity (%)	F1-score (%)
SVM	83.90±3.80	86.88±6.10	80.96±2.81	83.99±3.71
Decision Tree	78.24±3.30	77.14±5.08	79.07±2.02	79.08±3.20
KNN	83.11±3.86	85.23±6.00	80.99±2.99	83.33±3.71
Naïve Bayes	81.99±2.72	92.32±4.20	72.20±3.56	80.61±3.21
Logistic Regression	84.35±3.35	84.91±5.86	83.66±2.33	84.78±3.24

Table 2.

Classification performance of models based on sound signals.

Classifier	Accuracy (%)	Specificity (%)	Sensitivity (%)	F1-score (%)
SVM	77.75±2.65	74.81±5.66	80.69±4.04	79.06±2.06
Decision Tree	69.49±2.16	66.49±4.94	72.67±4.24	71.21±1.97
KNN	74.86±2.87	74.77±6.38	75.30±4.04	75.73±1.93
Naïve Bayes	68.80±3.03	81.92±7.69	56.59±5.15	65.29±3.16
Logistic Regression	76.22±3.91	75.26±5.74	77.21±4.33	77.24±2.73

Two features extracted from the nasal airflow signals were analyzed. The results showed that the percentages of the four types of respiratory swallowing patterns, i.e., exhale-swallow-exhale, exhale-swallow-inhale, inhale-swallow-exhale, and inhale-swallow-inhale in patients, were 43.50%, 21.64%, 18.03%, and 16.83%, respectively. The percentages of the corresponding four types of respiratory swallowing patterns in healthy subjects were 84.34%, 7.25%, 7.09%, and 1.32%, respectively. In addition, post-stroke dysphagia patients (median (P₂₅, P₇₅): 0.81 (0.60, 1.21) S) have significantly longer SAD compared to healthy subjects (median (P₂₅, P₇₅): 0.64 (0.54, 0.78) S) (P < 0.001) (Figure 3).

Figure 3.

The percentage distribution of four types of swallowing breathing patterns in patients and healthy participants (a). “E-S-E” for exhale-swallow-exhale; “E-S-I” for exhale-swallow-inhale; “I-S-E” for inhale-swallow-exhale; “I-S-I” for inhale-swallow-inhale. (b) Demonstrates that the swallowing apnea duration of patients is significantly longer than that of healthy participants (P < 0.001). The swallowing apnea duration is measured in seconds.

Classification performance of models based on signals from two sensors

Overall, the models that utilized both acceleration and sound signal features demonstrated superior performance compared to the corresponding models that relied solely on either accelerometer or sound signal features. The classification performance of Logistic Regression was overall superior to that of the other models, with accuracy, specificity, sensitivity, and F1-score values of 89.86±1.97%, 88.84±4.03%, 90.68±2.21%, and 90.30±1.94%, respectively (Table 3).

Table 3.

Classification performance of models based on both acceleration and sound signals.

Classifier	Accuracy (%)	Specificity (%)	Sensitivity (%)	F1-score (%)
SVM	89.71±2.68	88.46±4.17	90.74±2.71	90.15±2.82
Decision Tree	84.14±2.85	82.49±4.60	85.59±2.00	84.84±3.04
KNN	88.15±2.25	89.38±3.37	86.82±3.01	88.33±2.65
Naïve Bayes	83.85±2.47	92.13±3.27	75.99±3.63	82.95±3.20
Logistic Regression	89.86±1.97	88.84±4.03	90.68±2.21	90.30±1.94

The performance of the model, which initially relied solely on acceleration signal features, was improved by incorporating nasal airflow signal features. The classification performance of Logistic Regression was overall superior to that of the other models, with accuracy, specificity, sensitivity, and F1-score values of 87.03±3.66%, 87.29±5.13%, 86.76±2.67%, and 87.47±3.42%, respectively (Table 4).

Table 4.

Classification performance of models based on both acceleration and nasal airflow signals.

Classifier	Accuracy (%)	Specificity (%)	Sensitivity (%)	F1-score (%)
SVM	86.69±4.00	86.10±5.84	87.14±2.72	87.22±3.81
Decision Tree	81.90±3.26	81.23±5.44	82.54±2.36	82.57±3.32
KNN	85.57±3.49	86.86±5.62	84.40±2.24	85.92±3.22
Naïve Bayes	82.90±2.49	92.78±3.90	73.50±3.43	81.67±2.95
Logistic Regression	87.03±3.66	87.29±5.13	86.76±2.67	87.47±3.42

The performance of the models, which initially relied solely on sound signal features, was improved after incorporating nasal airflow signal features. The SVM model demonstrated relatively better classification performance among the five models, achieving an accuracy of 83.09±2.29%, specificity of 82.53±3.51%, sensitivity of 83.77±4.83%, and F1-score of 83.70±2.58% (Table 5).

Table 5.

Classification performance of models based on both sound and nasal airflow signals.

Classifier	Accuracy (%)	Specificity (%)	Sensitivity (%)	F1-score (%)
SVM	83.09±2.29	82.53±3.51	83.77±4.83	83.70±2.58
Decision Tree	73.76±2.51	72.43±4.67	75.15±3.90	74.85±2.40
KNN	79.28±2.13	82.84±4.74	76.29±4.65	79.27±2.02
Naïve Bayes	75.86±2.97	83.06±7.32	69.12±5.84	74.80±3.22
Logistic Regression	81.58±3.00	81.61±4.16	81.70±4.91	82.17±2.97

Classification performance of models based on signals from three sensors

Compared to models based on single or dual sensor signal feature extraction, the models that utilizes feature extraction from three sensors exhibited superior classification performance. Among the five models, both Logistic Regression and SVM demonstrated promising classification performance (Table 6).

Table 6.

Classification performance of models based on signals from three sensors.

Classifier	Accuracy (%)	Specificity (%)	Sensitivity (%)	F1-score (%)
SVM	91.39±3.03	89.53±4.09	92.99±2.84	91.86±2.92
Decision Tree	85.12±3.18	84.18±4.63	85.84±2.46	85.78±2.74
KNN	89.01±2.19	90.77±2.86	87.29±2.45	89.20±2.24
Naïve Bayes	84.88±2.31	92.49±3.03	77.64±3.45	84.15±2.94
Logistic Regression	91.00±2.71	89.88±3.77	91.97±2.70	91.45±2.55

Discussion

In this study, we used a multi-sensor monitoring system to concurrently capture nasal airflow, acceleration, and sound signals, finding it effective for differentiating swallowing between healthy adults and post-stroke dysphagia patients. Overall, the classification performance of models based on signals from all three sensors was superior to that of models relying on signals from a single sensor or two sensors. The Logistic Regression and SVM models exhibited notable classification performance among the five models.

Previous studies have primarily investigated patients with dysphagia caused by various conditions, including Parkinson’s disease, amyotrophic lateral sclerosis, stroke, and nasopharyngeal carcinoma. Additionally, the sample sizes in these studies were relatively small. In contrast, our study specifically focused on patients with post-stroke dysphagia and included a significantly larger sample size. Compared with previous multi-sensor studies, Inoue et al.¹⁴reported 82.4% sensitivity and 86.0% specificity on an imbalanced dataset (140 healthy vs. 52 patients). By contrast, our SVM model achieved 92.99±2.84% sensitivity and 89.53±4.09% specificity on a larger, balanced post-stroke cohort (108 healthy vs. 108 patients) with VFSS-guided segmentation, suggesting improved discriminative ability.

However, individual differences in clinical characteristics such as lesion location, time since stroke onset, and stroke severity exist among patients with post-stroke dysphagia. These factors may influence swallowing physiology and contribute to variability in multi-sensor signals. For example, lesion location may affect sensor signals differently: brainstem strokes may impair laryngeal elevation, pharyngeal transit, and respiratory-swallow coordination,^22–24 and may potentially affect acceleration and nasal airflow signals. Disease severity may also influence signal patterns—mild dysphagia may produce subtle changes that overlap with healthy signals, potentially reducing sensitivity. Consequently, model performance and generalizability may vary across patient subgroups. Future studies with larger sample sizes are warranted to perform subgroup analyses and further investigate the impact of these clinical variables on dysphagia assessment using multi-sensor fusion approaches.

Prior to combining multi-sensor signals, this study initially assessed the effectiveness of signals captured by a single sensor and the integration of signals obtained from two sensors in identify post-stroke dysphagia. Our findings indicated that the models utilizing acceleration signals show satisfactory classification performance with high accuracy, sensitivity, and specificity. Research on dysphagic swallowing has suggested that penetration or aspiration may occur due to delayed or absent hyolaryngeal motion.²⁵ Acceleration signals can provide vibrational information that is associated with the motion of the hyoid bone and larynx.¹⁸ Moreover, differences in hyolaryngeal motion may lead to changes in acceleration signals, and the features resulting from such changes can be used to differentiate swallows in healthy individuals from swallows in patients with dysphagia.¹⁸ Therefore, it can be assumed that the acceleration signal of laryngeal motion recorded during swallowing in patients were different from that of healthy individuals, and this difference may be the rationale for the ability of acceleration signals to effectively discriminate between abnormal and normal swallowing in this study. Furthermore, better classification performance was achieved by combining acceleration signals and sound signals, suggesting that the swallowing sound signals may contain features that were not captured by the accelerometer. Therefore, swallowing sound signals are also essential for dysphagia assessment.

The coordination between respiratory and swallowing is vital for normal swallowing function, and its malfunction can result in serious consequences.⁸ A previous study demonstrated that respiratory and swallowing are not coordinated in patients with dysphagia.¹³ Therefore, simultaneously capturing respiratory and swallowing signals can help in identifying dysphagia.²⁶ Consistent with the above, our results showed that the classification performance of the models generally improved when acceleration and sound signals were combined with nasal airflow signals. Normal swallowing activity usually occurs during the expiratory phase of the respiratory cycle, at which time respiratory is interrupted, and respiratory resumes with exhalation when swallowing is complete. The results of this study showed that patients had significantly higher percentage of exhale-swallow-exhale patterns than that of healthy individuals. In addition, the SAD of patients were significantly longer than that of healthy adults. The prolonged SAD has been hypothesized as a manifestation of the compensatory mechanism of airway protection, whereby the patient extended airway closure time during swallowing in order to prevent aspiration.²⁷ In our study, differences in respiratory-swallowing coordination between patients and healthy individuals may contribute to the improvement in model classification performance with the addition of nasal airflow signals. The findings may facilitate early intervention in patients who do not show obvious symptoms of swallowing disorders but have issues with respiratory and swallowing coordination, which may contribute to a reduced risk of related complicationsDysphagia may manifest as impaired laryngeal elevation, disrupted respiratory-swallowing coordination, or altered swallowing sound characteristics. These impairments can occur independently; a patient with normal laryngeal elevation may still exhibit unsafe respiratory patterns, while another with delayed laryngeal elevation may have intact coordination. Therefore, a single sensor may not fully capture all clinically relevant information, whereas integration of multiple sensors could provide a more comprehensive assessment across the heterogeneous presentations of post-stroke dysphagia. This may explain the superior performance of multi-sensor fusion observed in this study.

Multi-sensor fusion has achieved outstanding classification performance in various biomedical applications.²⁸ Recently, this approach has also been explored for dysphagia assessment. For example, Inoue et al.¹⁴ combined nasal airflow, acceleration, and acoustic signals to differentiate between normal and abnormal swallowing. However, the sample size of normal swallows obtained from healthy participants in their research was notably larger than that of patients with dysphagia. Consequently, the imbalance of data has contributed to the lower sensitivity and F1-score of the classification results. In contrast to the previous study, the present study achieved a balance between normal and abnormal swallowing sample sizes. Furthermore, our results demonstrated that the classification models achieve satisfactory performance in terms of accuracy, sensitivity, specificity, and F1-score by incorporating signals from three sensors. Physiologically, acceleration signals reflect vibrations associated with hyolaryngeal motion, while sound signals capture acoustic events generated by bolus flow through the aerodigestive tract; the two signals thus provide complementary information about the same swallowing events.²⁹ Nasal airflow signals capture respiratory-swallow coordination, including swallowing apnea duration and phase patterns, which are key indicators of airway protection.⁹ It can be explained that multi-sensor fusion enables a more thorough and precise comprehension of observed phenomena through the simultaneous analysis of multiple signals,²⁸ and that multi-sensor fusion helps to reduce uncertainty and enhance robustness, which ultimately improving the accuracy and reliability of the classification model.⁹ Therefore, the observations of this study shed light on the potential of multimodal data integration in the assessment of dysphagia.

Long wait times for VFSS are prevalent in communities with limited access to healthcare services and under-resourced radiology departments. Utilizing a non-invasive tool to pre-screen individuals who need to be referred for VFSS assessment can significantly shorten the wait time. Additionally, the non-invasive method can facilitate real-time monitoring of swallowing function in daily life, and can provide immediate feedback for healthcare professionals to make informed decisions about treatment strategies or rehabilitation interventions. This preliminary study provides support for the potential of multi-sensor fusion method in assessing dysphagia. For future work, we plan to extend the model to differentiate swallowing in patients with post-stroke dysphagia with different PAS scores, thus enabling this non-invasive method to more accurately assess the severity of dysphagia. In addition, we anticipate that our findings can spark additional exploration in this field, and ultimately lead to the development of non-invasive methods for assessing dysphagia.

Limitations

There are some limitations that exist in this study. Firstly, no VFSS was performed on healthy participants due to radiation exposure concerns. Therefore, normal swallowing was segmented based on laryngeal motion information and sound signals. This approach may introduce slight discrepancies in the segmented swallows between the two groups. However, during swallowing, the trajectory of the hyoid bone is upward and forward and then returns to its resting position, and the trajectory of the laryngeal prominence is closely related to that of the hyoid bone. Therefore, it is reasonable to determine the onset and end point of swallowing based on the movement of the laryngeal prominence.⁸ Notably, high intra- and inter-rater reliability (ICC > 0.90) was achieved for swallowing segmentation, indicating consistent identification of swallow events. Secondly, the stimulus swallowed by the patients contained the barium sulfate suspension, whereas that of the healthy subjects did not, which may have had an effect on the swallowing signal. Thirdly, the healthy participants were relatively younger than the stroke patients. We did not perform statistical adjustment or age-matching in the present study, as the age ranges of the two groups overlapped considerably (20-76 vs. 27-80 years) and the sample size was limited for matching. Nevertheless, we acknowledge that age remains a potential confounder, and future studies should consider age-matched designs or statistical adjustment. Fourth, our study specifically focused on patients with post-stroke dysphagia, excluding those with dysphagia caused by other diseases, such as Parkinson’s disease, amyotrophic lateral sclerosis, and nasopharyngeal carcinoma. Therefore, caution should be exercised in generalizing the findings of this study to other patient populations. Furthermore, we did not examine the impact of clinical heterogeneity among stroke patients—such as lesion location, stroke severity, or time since onset—on model performance or generalizability. These factors may alter swallowing physiology and sensor signals, potentially affecting the robustness of our results across different patient subgroups. Future studies with larger, well-characterized cohorts are warranted to address this limitation.

Conclusions

The multi-sensor fusion method has shown promising performance in discriminating between normal swallows in healthy adults and swallows in patients with post-stroke dysphagia. The findings of this study provide an important foundation for the use of non-invasive multi-sensor fusion to assess dysphagia. Further research is warranted to validate and optimize the use of multi-sensor fusion methods in larger cohorts and diverse clinical settings.

Footnotes

Acknowledgments

The authors acknowledge all the researchers who contributed to this study.

ORCID iDs

Lian Wang

Jia Qiao

Ethical considerations

We conducted this study from May 2023 to April 2024. Ethical approval for this prospective study was obtained (no. [2022]02-192-01). This study was registered in the Chinese Clinical Trial Registry (no. ChiCTR2300068908).

Author contributions

Lian Wang conceived the study concept, developed the methodology, carried out the experiments, conducted formal data analysis, and wrote the original draft. Nannan Cui and Xiaozhen Li contributed to statistical analysis and result interpretation. Jia Qiao and Zhenhai Wei contributed to data curation and software implementation. Yanxia Liu and Zulin Dou assisted in sresult interpretation and manuscript revision. Xiaomei Wei provided overall supervision, acquired funding, administered the project, and critically reviewed and edited the manuscript. All authors reviewed and approved the final version of the manuscript.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the National Natural Science Foundation of China (82272617) and Guangzhou Science and Technology Programme Key R&D Programmes (2023B03J1234).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

The data support the findings of this study were shown in the paper, further inquiries can contact to the corresponding author.*

References

Daniels

Anderson

Willson

. Valid items for screening dysphagia risk in patients with stroke: a systematic review. Stroke 2012; 43(3): 892–897. https://doi.org/10.1161/STROKEAHA.111.640946

Martino

Foley

Bhogal

, et al. Dysphagia after stroke: incidence, diagnosis, and pulmonary complications. Stroke 2005; 36(12): 2756–2763. https://doi.org/10.1161/01.STR.0000190056.76543.eb

Banda

Chu

Kang

, et al. Prevalence of dysphagia and risk of pneumonia and mortality in acute stroke patients: a meta-analysis. BMC Geriatr 2022; 22(1): 420. https://doi.org/10.1186/s12877-022-02960-5

Jones

Colletti

Ding

. Post-stroke Dysphagia: Recent Insights and Unanswered Questions. Curr Neurol Neurosci Rep 2020; 20(12): 61. https://doi.org/10.1007/s11910-020-01081-z

Smith

Kent

Bulsara

, et al. Effect of dysphagia screening strategies on clinical outcomes after stroke: a systematic review for the 2018 guidelines for the early management of patients with acute ischemic stroke. Stroke 2018; 49(3): e123–e128. https://doi.org/10.1161/STR.0000000000000159

Jaghbeer

Sutt

Bergstrom

. Dysphagia management and cervical auscultation: reliability and validity against FEES. Dysphagia 2023; 38(1): 305–314. https://doi.org/10.1007/s00455-022-10468-8

Wang

Shieh

Weng

, et al. Non-invasive assessment determine the swallowing and respiration dysfunction in early Parkinson's disease. Parkinsonism Relat Disord 2017; 42: 22–27. https://doi.org/10.1016/j.parkreldis.2017.05.024

Yagi

Nagami

Lin

, et al. A noninvasive swallowing measurement system using a combination of respiratory flow, swallowing sound, and laryngeal motion. Med Biol Eng Comput 2017; 55(6): 1001–1017. https://doi.org/10.1007/s11517-016-1561-2

Lee

Steele

Chau

. Classification of healthy and abnormal swallows based on accelerometry and nasal airflow signals. Artif Intell Med 2011; 52(1): 17–25. https://doi.org/10.1016/j.artmed.2011.03.002

10.

Lee

Blain

Casas

, et al. A radial basis classifier for the automatic detection of aspiration in children with dysphagia. J Neuroeng Rehabil 2006; 3: 14. https://doi.org/10.1186/1743-0003-3-14

11.

Dudik

Jestrovic

Luan

, et al. A comparative analysis of swallowing accelerometry and sounds during saliva swallows. Biomed Eng Online 2015; 14: 3. https://doi.org/10.1186/1475-925X-14-3

12.

Lazareck

Moussavi

. Classification of normal and dysphagic swallows by acoustical means. IEEE Trans Biomed Eng 2004; 51(12): 2103–2112. https://doi.org/10.1109/TBME.2004.836504

13.

Hopkins-Rossabi

Armeson

Zecker

, et al. Respiratory-swallow coordination and swallowing impairment in head and neck cancer. Head Neck 2021; 43(5): 1398–1408. https://doi.org/10.1002/hed.26609

14.

Inoue

Yoshioka

Yagi

, et al. Using machine learning and a combination of respiratory flow, laryngeal motion, and swallowing sounds to classify safe and unsafe swallowing. IEEE Trans Biomed Eng 2018; 65(11): 2529–2541. https://doi.org/10.1109/tbme.2018.2807487

15.

Dudik

Kurosu

Coyle

, et al. Dysphagia and its effects on swallowing sounds and vibrations in adults. Biomed Eng Online 2018; 17(1): 69. https://doi.org/10.1186/s12938-018-0501-9

16.

Donohue

Khalifa

Perera

, et al. A preliminary investigation of whether hrca signals can differentiate between swallows from healthy people and swallows from people with neurodegenerative diseases. Dysphagia 2021; 36(4): 635–643.

17.

Frakking

Chang

Carty

, et al. Using an automated speech recognition approach to differentiate between normal and aspirating swallowing sounds recorded from digital cervical auscultation in children. Dysphagia 2022; 37(6): 1482–1492. https://doi.org/10.1007/s00455-022-10410-y

18.

Merey

Kushki

Sejdic

, et al. Quantitative classification of pediatric swallowing through accelerometry. J Neuroeng Rehabil 2012; 9: 34. https://doi.org/10.1186/1743-0003-9-34

19.

Rousson

. Assessing inter-rater reliability when the raters are fixed: two concepts and two estimates. Biom J 2011; 53(3): 477–490. https://doi.org/10.1002/bimj.201000066

20.

Giannakopoulos

. pyAudioAnalysis: An open-source python library for audio signal analysis. PLoS One 2015; 10(12): e0144610. https://doi.org/10.1371/journal.pone.0144610

21.

Christ

Braun

Neuffer

, et al. Time series feature extraction on basis of scalable hypothesis tests (tsfresh–a python package). Neurocomputing 2018; 307: 72–77. https://doi.org/10.1016/j.neucom.2018.03.067

22.

Park

Choi

Takatoh

, et al. Brainstem control of vocalization and its coordination with respiration. Science 2024; 383(6687): eadi8081. https://doi.org/10.1126/science.adi8081

23.

Huai

Hou

Guan

, et al. Botulinum toxin a injection using esophageal balloon radiography combined with ct guidance for the treatment of cricopharyngeal dysphagia. Dysphagia 2020; 35(4): 630–635. https://doi.org/10.1007/s00455-019-10070-5

24.

Toor

Sun

Kumar

, et al. Neurons in the intermediate reticular nucleus coordinate postinspiratory activity, swallowing, and respiratory-sympathetic coupling in the rat. J Neurosci 2019; 39(49): 9757–9766. https://doi.org/10.1523/JNEUROSCI.0502-19.2019

25.

Troche

Okun

Rosenbek

, et al. Aspiration and swallowing in Parkinson disease and rehabilitation with EMST: a randomized trial. Neurology 2010; 75(21): 1912–1919. https://doi.org/10.1212/WNL.0b013e3181fef115

26.

Cosentino

Todisco

Giudice

, et al. Assessment and treatment of neurogenic dysphagia in stroke and Parkinson's disease. Curr Opin Neurol 2022; 35(6): 741–752. https://doi.org/10.1097/WCO.0000000000001117

27.

Todd

Butler

Plonk

, et al. Effects of chemesthetic stimuli mixtures with barium on swallowing apnea duration. Laryngoscope 2012; 122(10): 2248–2251. https://doi.org/10.1002/lary.23511

28.

Hernandez

Carrault

Mora

, et al. Multisensor fusion for atrial and ventricular activity detection in coronary care monitoring. IEEE Trans Biomed Eng 1999; 46(10): 1186–1190. https://doi.org/10.1109/10.790494

29.

Dudik

Coyle

Sejdic

. Dysphagia screening: contributions of cervical auscultation signals and modern signal-processing techniques. IEEE Trans Hum Mach Syst 2015; 45(4): 465–477. https://doi.org/10.1109/THMS.2015.2408615