Development of a Short-Form Stroke Impact Scale Using a Machine Learning Algorithm for Patients at the Subacute Stage

Abstract

Importance: Several short forms of the Stroke Impact Scale Version 3.0 (SIS 3.0) have been proposed in order to decrease its administration time of about 20 min. However, none of the short-form scores are comparable to those of the original measure.

Objective: To develop a short-form SIS 3.0 using a machine learning algorithm (ML–SIS).

Design: We developed the ML–SIS in three stages. First, we calculated the frequencies of items having the highest contribution to predicting the original domain scores across 50 deep neural networks. Second, we iteratively selected the items showing the highest frequency until the coefficient of determination (R ²) of each domain was ≥.90. Third, we examined the comparability and concurrent and convergent validity of the ML–SIS.

Setting: Hospitals.

Participants: We extracted complete data for 1,010 patients from an existing data set.

Results: Twenty-eight items were selected for the ML–SIS. High average R ²s (.90–.96) and small average residuals (mean absolute errors and root-mean-square errors = 0.49–2.84) indicate good comparability. High correlations (rs = .95–.98) between the eight domain scores of the ML–SIS and the SIS 3.0 indicate sufficient concurrent validity. Similar interdomain correlations between the two measures indicate satisfactory convergent validity.

Conclusions and Relevance: The ML–SIS uses about half of the items in the SIS 3.0, has an estimated administration time of 10 min, and provides valid scores comparable to those of the original measure. Thus, the ML–SIS may be an efficient alternative to the SIS 3.0.

What This Article Adds: The ML–SIS, a short form of the SIS 3.0 developed using a machine learning algorithm, shows good potential to be an efficient and informative measure for clinical settings, providing scores that are valid and comparable to those of the original measure.

This article details the development of a short-form Stroke Impact Scale Version 3.0 (SIS 3.0) using a machine learning algorithm (ML–SIS) and concludes that the measure has good potential to be efficient and informative for clinical settings and provides scores that are valid and comparable to those of the original measure.

The Stroke Impact Scale (SIS) is a disease-specific quality of life measure (Duncan et al., 1999), and it is a widely used outcome measure for clinical trials with patients with stroke (Karimi & Brazier, 2016). The most recent version, the SIS 3.0, contains 59 items covering eight domains of life affected by stroke: Strength, Memory and Thinking, Emotion, Communication, Activities of Daily Living, Balance, Hand Function, and Social Participation (Duncan, Bode, et al., 2003). However, administering the SIS 3.0 is time consuming, requiring about 20 min (MacIsaac et al., 2016; Mohammad et al., 2014), which hampers its utility in time-pressed clinical settings. A short-form SIS with sound psychometric properties would help resolve this limitation.

Four short forms of the SIS 3.0 have been developed (see Table A.1 in the Supplemental Appendix, available online with this article at https://research.aota.org/ajot):

Duncan, Lai, et al. (2003) developed a 16-item version that provides a single total score representing physical function.

Jenkinson et al. (2013) developed an eight-item version with one item for each domain that provides a single total score reflecting overall level of health-related quality of life.

MacIsaac et al. (2016) developed an eight-item version almost identical to Jenkinson et al.’s that provides a total score. The only difference between this version and Jenkinson et al.’s version is the items selected for the Mobility domain.

Mohammad et al. (2014) developed a 16-item version that provides eight domain scores based on two items in each domain.

Table 1.

Participant Characteristics (N = 1,010)

Characteristic	n (%)
Age, yr, M (SD)	67.9 (13.4)
Gender, male	611 (60.5)
Stroke type
Ischemic	804 (79.6)
Hemorrhagic	184 (18.2)
Other (e.g., stroke mimic)	22 (2.2)
Self-reported race
White	785 (77.7)
Black or African American	145 (14.4)
Asian	71 (7.0)
Other	9 (0.9)
NIHSS score, M (SD)	1.9 (3.0)

Note. Because we resampled the patients in the developing and validating data in each of the 50 repetitions, patients’ characteristics are not presented separately for each repetition. NIHSS = National Institutes of Health Stroke Scale.

In general, these short forms have two weaknesses. First, three of them provide only a single total score; the fourth, by Mohammad et al. (2014), provides eight domain scores. Given that the SIS 3.0 is a multidimensional construct (Duncan, Bode, et al., 2003; Duncan et al., 1999), the output of a single score is minimally informative. Second, the scaling of scores on these short forms is different from that of the original SIS 3.0, so the scores are not comparable. Accordingly, the interpretability and utility of these short forms are limited.

Machine learning (ML) refers to a core methodology of artificial intelligence, which is powerful in data prediction and classification (Abbas et al., 2018; Thabtah et al., 2018; Wall et al., 2012). In the rehabilitation fields, ML has shown great potential for shortening assessments without sacrificing substantial information or psychometric properties (Lin et al., 2021; Sahdra et al., 2016; Wall et al., 2012). For example, Lin et al. (2021) found that a 10-item ML-based Fugl-Meyer assessment for patients with stroke provided scores comparable to those of the original 50-item measure with good test–retest reliability (coefficients ≥ .90) and concurrent validity (rs ≥ .90). Therefore, ML appears to be a promising tool for developing short-form measures with sound psychometric properties.

Comparability of scores is a particular strength of ML-based assessments. If the eight domain scores of the SIS 3.0 are defined as the targets to be predicted, the scaling of scores between the ML-based measure and the original measure will be the same, so the domain scores of the ML–SIS will be comparable to the original, ensuring interpretability. Such an advantage cannot be achieved by traditional statistical methods, which use majorly linear transformation (i.e., summation of item scores as domain scores). The scaling of domain scores varies depending on the number of items, and thus the domain scores of the short form are not comparable to those of the original. ML methods can be a solution for improving the interpretability of short-form measures.

In this study, we developed a short-form SIS 3.0 using an ML algorithm (ML–SIS) that provides domain scores comparable to those of the original measure. We also examined the concurrent and convergent validities of the ML–SIS. The research question was, Does a short form (<50% of the total number of items) developed using the ML method yield valid domain scores comparable to those of the original SIS 3.0?

Method

Participants

We extracted data from the Field Administration of Stroke Therapy–Magnesium Trial database (ClinicalTrials.gov NCT00059332; Saver et al., 2014). That trial included people who were suspected of having stroke (identified by the Los Angeles Prehospital Stroke Screen), were ages 45 to 95 yr, had symptoms of stroke that lasted >15 min, and initiated treatment within 2 hr of onset. The data were collected between January 2005 and December 2012, representing a total of 1,700 patients. These patients completed the SIS 3.0 90 days after stroke onset. For this study, we extracted only patient-reported and complete data. We did not seek ethics approval for this study because the data we used were free of personal identifiers.

Procedure

We adopted a three-stage analysis to develop and validate the ML–SIS. In Stage 1, we calculated the contribution of the SIS items to the prediction of the original eight domain scores (Gevrey et al., 2003; Lin et al., 2021) by dividing the sum of the absolute values of the weights of each item by the sum of those of all items (Gevrey et al., 2003; Tozlu et al., 2020). The weights between the input and information-shared layers (see the Data Analysis section) were used because the item scores were multiplied by these weights and passed into the models for the prediction. A total of 50 deep neural network (DNN) models were trained using the patients’ responses to the 59 SIS items (Figure 1). The frequencies of items showing the highest contributions across 50 models were used to select items for the ML–SIS.

Figure 1.

Deep neural network model used for the ML–SIS.

In Stage 2, we developed a scoring model for the ML–SIS. Another 50 DNN models were trained to predict the original domain scores using the selected items. The items were selected on the basis of the frequencies of items showing the highest item contributions across the 50 models alone. The number of items in each domain increased iteratively from one item to all items. We used the coefficient of determination (R ²) to validate the predictive power of the model and selected items in the validating sets. The inclusion of items continued until the average R ² of the domain was ≥.90. To control for variations in the results, the training and validating processes were repeated 50 times.

In Stage 3, we examined the comparability of scores and psychometric properties of the ML–SIS using the scores provided by the 50 models in Stage 2. We compared the scores on the ML–SIS to those on the original SIS 3.0 and examined the concurrent and convergent validity of the ML–SIS.

Measures

The SIS 3.0 contains 59 items divided into eight domains of life that are of concern to patients affected by stroke (Duncan, Bode, et al., 2003). All items are scored using a 5-point Likert-type scale measuring patients’ satisfaction, self-perceived difficulty, and agreement with a statement (Chou et al., 2015). Higher scores indicate better quality of life (Karimi & Brazier, 2016). The SIS 3.0 has generally good test–retest reliability (intraclass correlation coefficients [ICCs] = .78–.96, except for the Social Participation domain [.67]), convergent validity (moderate correlations with quality of life measures), predictive validity (moderate correlations with related measures with a 3-wk interval), and responsiveness (effect sizes = .22–.52, except for the Emotion [.06] and Social Participation [.04] domains; Chou et al., 2015; Lin et al., 2010).

The National Institutes of Health Stroke Scale (NIHSS; Brott et al., 1989) was used to evaluate patients’ stroke severity. The NIHSS contains 11 items with a total score range of 0 to 42; higher scores indicate more severe stroke. Good psychometric properties, including test–retest reliability (κ = .66–.77), interrater reliability (ICCs ≥.92), and factorial validity (one-factor solution using factor analysis), have been shown in patients with stroke (Dewey et al., 1999; Goldstein & Samsa, 1997; Zandieh et al., 2012).

Data Analysis

The extracted data were randomly divided into a developing set and a validating set with a ratio of 4:1 to ensure that the sample size in the validating set would be ≥200 patients, which is the minimum for a large sample size for psychometric studies (Kyriazos, 2018). We used the developing set to develop the ML–SIS in Stages 1 and 2 and the validating set to validate the generalizability of the predictive power and the concurrent and convergent validity in Stages 2 and 3. For the data used in the 50 models, the pairs of developing set and validating set were generated using different random seeds, so the constituents of these sets were different across the 50 repetitions and could control the variations caused by sampling. The repetition of 50 epochs was selected because it has been reported to provide stable results (Li et al., 2019).

The DNN model contains three parts (Figure 1): input layer, information-shared layer, and output layer. The input layer was used to input item scores of the SIS 3.0 and the ML–SIS (59 neurons for Stage 1 and the equal numbers to the selected items for Stage 2). The information-shared layer was used to extract and analyze the information provided by the input layer and generated the results to the output layer. The output layer consisted of the eight domain scores of the ML–SIS (one neuron for each parallel model). The information-shared layer comprised three layers with 64 neurons in each layer.

We selected the three-layer network because it can solve most prediction probes (Goodfellow et al., 2017). The selection of 64 neurons in each layer is consistent with recommendations that the number of neurons be (1) larger than that of the input layer (maximum of 59 items, approximated using the power of 2; i.e., 2⁶ = 64 neurons) and (2) consistent in all layers (Bengio, 2012). All neurons in the DNN model were fully connected with the rectified linear unit as the activation function, except that of the output layer, which was linear. To optimize the predictive power, each domain score was predicted by an independent three-layer DNN model with the same input layer. Accordingly, the ML–SIS was the combination of the eight parallel models (i.e., each parallel model predicted one domain score only).

The weights in the models were updated and trained in 600 epochs. In each epoch, the weights of the DNN models were fine-tuned to minimize the differences (mean squared error loss) in scores between the predicted eight domain scores of the ML–SIS and the eight domain scores of the original SIS 3.0 with the adaptive moment estimation optimizer (Abadi et al., 2016). To avoid overtraining, the early stopping method was used if the differences between the input and output layers did not increase for 50 epochs (Prechelt, 1998). To enhance the differences in the contribution of items, the L1 penalty was added to the input layer with a λ of 0.5 (Suárez-Álvarez et al., 2018). The analyses of the DNN models were performed in TensorFlow (Abadi et al., 2016).

The comparability of scores was examined with root-mean-square error (RMSE) and mean absolute error (MAE; Chai & Draxler, 2014). RMSE is sensitive to large errors and can be useful for detecting severe mispredictions (Chai & Draxler, 2014). RMSE was calculated as the square root of the average of the squared differences between the predicted domain scores and the original domain scores. MAE, used because of its better interpretability (representing the average magnitude of errors; Chai & Draxler, 2014), was calculated as the average of the absolute differences between the predicted domain scores and those of the original domain scores. The values of both indices range from zero to infinity, with smaller values indicating better prediction. We adopted a commonly used criterion, 10% of the domain scores, for acceptable random measurement error (Smidt et al., 2002).

For concurrent validity, Pearson’s correlations (rs) between the predicted eight domain scores of the ML–SIS and those of the original SIS 3.0 were calculated; we considered r ≥ .75 to indicate good concurrent validity (Van Deusen & Brunt, 1997). Regarding convergent validity, the interdomain correlations between the eight domain scores of the ML–SIS were compared to those of the SIS 3.0; we considered similar interdomain correlations between the two measures to indicate good convergent validity.

Results

We extracted SIS data for 1,289 patients. Data for the 1,010 patients with complete SIS scores were randomly assigned to the developing set (n = 808) and testing sets (n = 202). The patients’ average age was 67.9 yr (SD = 13.4; Table 1). The majority of patients were male (60.5%), White (77.7%), and diagnosed with ischemic stroke (79.6%). The average stroke severity (NIHSS score) was mild (M = 1.9, SD = 3.0).

The 28 items selected for the ML–SIS were those most frequently selected within the 50 models (see Table A.2 in the Supplemental Appendix, available online with this article at https://research.aota.org/ajot). The number of items in the Balance domain was reduced the most (by 77.8%); the number in the Emotion domain was reduced the least (by 11.1%).

Table 2.

Number of Items, Mean Scores, Comparability, and Concurrent Validity of the ML–SIS

Index	Domain
Index	Strength	Memory & Thinking	Emotion	Communication	ADLs	Balance	Hand Function	Participation
No. of items
ML–SIS	2	4	8	4	3	2	2	3
SIS 3.0	4	7	9	7	10	9	5	8
Scores, M (SD)
ML–SIS	14.6 (0.5)	17.4 (0.5)	12.2 (0.4)	18.6 (0.5)	15.8 (0.4)	15.0 (0.3)	13.7 (0.4)	14.5 (0.5)
SIS 3.0	14.6 (0.5)	17.4 (0.5)	12.1 (0.3)	18.5 (0.5)	15.8 (0.4)	15.0 (0.3)	13.8 (0.4)	14.5 (0.5)
Comparability
Average R ² [95% CI]	.96 [.95, .98]	.94 [.92, .96]	.92 [.87, .95]	.90 [.83, .94]	.93 [.90, .95]	.93 [.91, .95]	.94 [.91, .96]	.92 [.89, .94]
RMSE [95% CI]	0.87 [0.62, 1.05]	1.23 [1.02, 1.46]	1.22 [0.96, 1.60]	1.19 [0.97, 1.39]	2.84 [2.36, 3.4]	2.63 [2.24, 2.99]	1.80 [1.47, 2.17]	2.68 [2.18, 3.05]
RMSE%	4.4	3.5	2.7	3.4	5.7	5.8	7.2	6.7
MAE [95% CI]	0.49 [0.31, 0.65]	0.72 [0.59, 0.88]	0.72 [0.54, 0.94]	0.68 [0.53, 0.86]	1.85 [1.52, 2.21]	1.72 [1.40, 1.95]	1.05 [0.83, 1.25]	1.81 [1.43, 2.10]
MAE%	2.5	2.1	1.6	1.9	3.7	3.8	4.2	4.5
Concurrent validity
Pearson’s r [95% CI]	.98 [.98, .99]	.97 [.96, .98]	.96 [.95, .98]	.95 [.91, .97]	.96 [.95, .97]	.97 [.95, .98]	.97 [.96, .98]	.96 [.94, .97]

Note. The 95% CI was calculated based on the results of the 50 repetitions in Stage 2. RMSE% and MAE% were calculated by dividing the means of RMSE and MAE by the corresponding domain scores (e.g., 20 for the Strength domain with 4 items). ADL = Activities of Daily Living domain; CI = confidence interval; MAE = mean absolute error; ML–SIS = short-form SIS 3.0 using a machine learning algorithm; R ² = coefficient of determination; RMSE = root-mean-square error; SIS 3.0 = Stroke Impact Scale Version 3.0.

The ML–SIS provided similar scores (differences in average domain scores of <0.1), higher R ²s (average R ²s = .90–.96), and smaller residuals (average RMSE = 0.87–2.84, RMSE% = 2.7%–7.2%; average MAE = 0.49–1.85, MAE% = 1.6%–4.5%) compared with the original SIS 3.0 across the 50 repetitions (Table 2). The correlations between scores on the ML–SIS and on the original SIS 3.0 were high (rs = .95–.98; Table 2). Moreover, the interdomain correlations between the domain scores of the ML–SIS and the original SIS 3.0 were very similar (Table 3).

Table 3.

Interdomain Correlations of the ML–SIS and the SIS 3.0

Domain	Pearson’s r
Domain	Strength	Memory and Thinking	Emotion	Communication	ADLs	Balance	Hand Function
Memory and Thinking
ML–SIS	.21–.45	—
SIS 3.0	.22–.46	—
Emotion
ML–SIS	.22–.46	.28–.59	—
SIS 3.0	.20–.44	.22–.55	—
Communication
ML–SIS	.16–.44	.50–.79	.20–.67	—
SIS 3.0	.16–.43	.43–.76	.12–.63	—
ADLs
ML–SIS	.73–.85	.28–.53	.23–.54	.20–.52	—
SIS 3.0	.66–.80	.23–.51	.20–.47	.14–.49	—
Balance
ML–SIS	.73–.84	.23–.49	.21–.50	.18–.47	.88–.94	—
SIS 3.0	.66–.80	.19–.51	.18–.48	.14–.47	.84–.91	—
Hand Function
ML–SIS	.77–.89	.16–.38	.21–.42	.15–.42	.84–.91	.73–.84	—
SIS 3.0	.76–.87	.13–.40	.19–.44	.08–.41	.77–.88	.65–.81	—
Participation
ML–SIS	.70–.82	.27–.49	.30–.54	.23–.51	.74–.87	.70–.85	.72–.86
SIS 3.0	.65–.80	.25–.52	.26–.49	.18–.45	.67–.85	.68–.81	.66–.80

Note. Coefficient ranges were calculated on the basis of the results of the 50 repetitions in Stage 2. ADLs = Activities of Daily Living domain; ML–SIS = short-form SIS 3.0 using a machine learning algorithm; SIS 3.0 = Stroke Impact Scale Version 3.0.

Discussion

The ML–SIS reduced by over half the number of items in the original SIS 3.0. ML–SIS scores for our sample were very close to their scores on the original SIS 3.0, with residuals smaller than 10% of the corresponding domain scores. These findings indicate that the ML–SIS can improve the efficiency of assessments targeting health-related quality of life in patients with stroke, and they support the utility of ML methods for the development of short-form measures.

The ML–SIS provided scores highly correlated with those of the original SIS 3.0, indicating sufficient concurrent validity. Furthermore, the interdomain correlations of the ML–SIS were very similar to those of the original SIS 3.0, indicating good convergent validity. These findings demonstrate that the ML–SIS can provide valid assessments of health-related quality of life for patients with stroke.

The ML–SIS has two advantages. First, it provides the eight domain scores of the original SIS 3.0, whereas most other short forms provide a single total score only. Thus, the ML–SIS can be as informative as the SIS 3.0 in identifying patients’ needs in specific domains. Second, scores on the ML–SIS have the same scaling as those of the SIS 3.0, indicating that the scores can be interchangeable. Accordingly, the ML–SIS may be used as an alternative to the SIS 3.0 in time-pressed clinical settings. Moreover, interchangeable scores can improve interpretability because scores on the ML–SIS can be compared to those found in previous studies using the SIS 3.0. These advantages strengthen the utility of the ML–SIS in both clinical and research settings.

The ML–SIS development process was less effective in reducing the number of items in the Emotion domain, possibly because of the diversity of the content of those items. For example, the ML–SIS includes three reverse-worded items, which may provide different information than the other six items assessing negative emotions (Suárez-Álvarez et al., 2018). Another example is that two items were related to patients’ interpersonal relationships (e.g., “feel that there is nobody you are close to”), which could not be predicted by items assessing purely emotional status (e.g., “feel sad”). Therefore, the ML–SIS may need more Emotion items to improve the predictive power. However, although the reduction of Emotion items was limited, the total number of items of the ML–SIS remained small. Thus, the ML–SIS can still be an efficient option.

Regarding accessibility, we developed an online system for administering the ML–SIS. As with other online self-report questionnaires, our system has an administrative interface to present the item content and response options, collect patients’ responses, and provide immediate results when the test is completed. The online system has better utility for two reasons. First, all 50 models used in Stage 2 are included; therefore, average scores and corresponding 95% confidence intervals can be provided to improve the robustness and interpretability of results. Second, because the DNN model is hosted on the server of the online system, the score calculations are performed on the server, avoiding additional burden on users’ devices. Thus, the online system requires no additional equipment and improves the feasibility of administration. Prospective users can contact the authors to access the online system.

Limitations

This study has at least three limitations. First, participants completed the SIS 3.0 90 days poststroke, so the results cannot be generalized to patients in the acute and chronic phases. Second, the data were analyzed using simulations of secondary data; in particular, the same data were used to calculate the correlations between the ML–SIS and the original SIS 3.0. Accordingly, the performance of the ML–SIS may have been overestimated and different from that obtained if the ML–SIS and the SIS 3.0 were administered separately. Third, the ML–SIS requires complete data with no missing values, potentially constraining its utility. Further cross-validation of our results in independent samples with different stroke chronicity (i.e., acute and chronic stages) is needed. Researchers can consider using alternative structures for analysis (e.g., number of layers and neurons in information-shared layers) and other ML algorithms (e.g., extreme gradient boosting).

Implications for Occupational Therapy Practice

This study has the following implications for occupational therapy practice:

▪ The ML–SIS can be an efficient alternative to the SIS 3.0 and may be used in routine assessments (e.g., weekly) to reduce the burden of administration and provide scores as informative as those of the full-length SIS 3.0.

▪ The ML–SIS is an example of how machine learning algorithms can be helpful in occupational therapy practice.

Conclusion

The ML–SIS has substantially fewer items and can produce scores that are similar and comparable to the original SIS 3.0 with good validity. Therefore, the ML–SIS can be used as an efficient alternative to the SIS 3.0 to obtain informative and valid assessments of health-related quality of life.

Supplemental Material

Supplemental Appendix

Supplementary material for Development of a Short-Form Stroke Impact Scale Using a Machine Learning Algorithm for Patients at the Subacute Stage

Supplementary material, sj-pdf-1-aot-10.5014_ajot.2022.049136.pdf for Development of a Short-Form Stroke Impact Scale Using a Machine Learning Algorithm for Patients at the Subacute Stage by Shih-Chieh Lee, Inga Wang, Gong-Hong Lin, Pei-Chi Li, Ya-Chen Lee, Chia-Yeh Chou, Chien-Yu Huang and Ching-Lin Hsieh in The American Journal of Occupational Therapy

Footnotes

Acknowledgments

Ya-Chen Lee and Ching-Lin Hsieh equally contributed to this work and both serve as corresponding authors. This work was supported by the Ministry of Science and Technology (MOST 110-2628-B-038-013, MOST 109-2314-B-038-147, and MOST 109-2314-B-002-110-MY3).

References

Abadi

Barham

Chen

Davis

Dean

, . . . Isard

(2016). TensorFlow: A system for large-scale machine learning. In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (pp. 265–283). Advanced Computing Systems Association.

Abbas

Garberson

Glover

, & Wall

D. P.

(2018). Machine learning approach for early detection of autism by combining questionnaire and home video screening. Journal of the American Medical Informatics Association , 25 , 1000–1007. https://doi.org/10.1093/jamia/ocy039

Bengio

(2012). Practical recommendations for gradient-based training of deep architectures. In Montavon

Orr

, & Müller

K. R.

(Eds.), Neural networks: Tricks of the trade (pp. 437–478). Springer. https://doi.org/10.1007/978-3-642-35289-8_26

Brott

T. G.

Adams

H. P.

Olinger

C. P.

Marler

J. R.

Barsan

W. G.

Biller

, . . . Walker

(1989). Measurements of acute cerebral infarction: A clinical examination scale. Stroke , 20 , 864–870. https://doi.org/10.1161/01.STR.20.7.864

Chai

, & Draxler

R. R.

(2014). Root mean square error (RMSE) or mean absolute error (MAE)? Arguments against avoiding RMSE in the literature. Geoscientific Model Development , 7 , 1247–1250. https://doi.org/10.5194/gmd-7-1247-2014

Chou

C. Y.

Y. C.

, & Chiang

T. R.

(2015). Psychometric comparisons of four disease-specific health-related quality of life measures for stroke survivors. Clinical Rehabilitation , 29 , 816–829. https://doi.org/10.1177/0269215514555137

Dewey

H. M.

Donnan

G. A.

Freeman

E. J.

Sharples

C. M.

Macdonell

R. A.

McNeil

J. J.

, & Thrift

A. G.

(1999). Interrater reliability of the National Institutes of Health Stroke Scale: Rating by neurologists and nurses in a community-based stroke incidence study. Cerebrovascular Diseases , 9 , 323–327. https://doi.org/10.1159/000016006

Duncan

P. W.

Bode

R. K.

Lai

S. M.

, & Perera

; Glycine Antagonist in Neuroprotection Americans Investigators. (2003). Rasch analysis of a new stroke-specific outcome scale: The Stroke Impact Scale. Archives of Physical Medicine and Rehabilitation , 84 , 950–963. https://doi.org/10.1016/S0003-9993(03)00035-2

Duncan

P. W.

Lai

S. M.

Bode

R. K.

Perera

, & DeRosa

(2003). Stroke Impact Scale–16: A brief assessment of physical function. Neurology , 60 , 291–296. https://doi.org/10.1212/01.WNL.0000041493.65665.D6

10.

Duncan

P. W.

Wallace

Lai

S. M.

Johnson

Embretson

, & Laster

L. J.

(1999). The Stroke Impact Scale Version 2.0: Evaluation of reliability, validity, and sensitivity to change. Stroke , 30 , 2131–2140. https://doi.org/10.1161/01.STR.30.10.2131

11.

Gevrey

Dimopoulos

, & Lek

(2003). Review and comparison of methods to study the contribution of variables in artificial neural network models. Ecological Modelling , 160 , 249–264. https://doi.org/10.1016/S0304-3800(02)00257-0

12.

Goldstein

L. B.

, & Samsa

G. P.

(1997). Reliability of the National Institutes of Health Stroke Scale: Extension to non-neurologists in the context of a clinical trial. Stroke , 28 , 307–310. https://doi.org/10.1161/01.STR.28.2.307

13.

Goodfellow

Bengio

, & Courville

(2017). Deep learning. MIT Press.

14.

Jenkinson

Fitzpatrick

Crocker

, & Peters

(2013). The Stroke Impact Scale: Validation in a UK setting and development of a SIS short form and SIS index. Stroke , 44 , 2532–2535. https://doi.org/10.1161/STROKEAHA.113.001847

15.

Karimi

, & Brazier

(2016). Health, health-related quality of life, and quality of life: What is the difference? PharmacoEconomics , 34 , 645–649. https://doi.org/10.1007/s40273-016-0389-9

16.

Kyriazos

T. A.

(2018). Applied psychometrics: Sample size and sample power considerations in factor analysis (EFA, CFA) and SEM in general. Psychology , 9 , 2207–2230. https://doi.org/10.4236/psych.2018.98126

17.

Yin

Tian

, & Sun

(2019). Performance of repeated cross validation for machine learning models in building energy analysis. In Wang

Zhu

Wang

Shen

, & Liu

(Eds.), Proceedings of the 11th International Symposium on Heating, Ventilation and Air Conditioning (pp. 523–531). Springer.

18.

Lin

G. H.

Huang

C. Y.

Lee

S. C.

Chen

K. L.

Lien

J. J.

Chen

M. H.

, . . . Hsieh

C. L.

(2021). A 10-item Fugl-Meyer motor scale based on machine learning. Physical Therapy , 101(4), pzab036. https://doi.org/10.1093/ptj/pzab036

19.

Lin

K. C.

C. Y.

Hsieh

Y. W.

Chen

C. L.

, & Lee

P. C.

(2010). Psychometric comparisons of the Stroke Impact Scale 3.0 and Stroke-Specific Quality of Life Scale. Quality of Life Research , 19 , 435–443. https://doi.org/10.1007/s11136-010-9597-5

20.

MacIsaac

Ali

Peters

English

Rodgers

Jenkinson

, . . . Quinn

T. J.

; VISTA Collaboration. (2016). Derivation and validation of a modified short form of the Stroke Impact Scale. Journal of the American Heart Association , 5 , e003108. https://doi.org/10.1161/JAHA.115.003108

21.

Mohammad

A. H.

Al-Sadat

Siew Yim

, & Chinna

(2014). Reliability and validity of the Nigerian (Hausa) version of the Stroke Impact Scale (SIS) 3.0 index. BioMed Research International , 2014 , 302097. https://doi.org/10.1155/2014/302097

22.

Prechelt

(1998). Early stopping—But when? In Orr

G. B.

& Müller

K.-R.

(Eds.), Neural networks: Tricks of the trade (pp. 55–69). Springer. https://doi.org/10.1007/3-540-49430-8_3

23.

Sahdra

B. K.

Ciarrochi

Parker

, & Scrucca

(2016). Using genetic algorithms in a large nationally representative American sample to abbreviate the Multidimensional Experiential Avoidance Questionnaire. Frontiers in Psychology , 7 , 189. https://doi.org/10.3389/fpsyg.2016.00189

24.

Saver

J. L.

Starkman

Eckstein

Stratton

Pratt

Hamilton

, . . . Sanossian

; FAST-MAG Investigators and Coordinators. (2014). Methodology of the Field Administration of Stroke Therapy–Magnesium (FAST-MAG) Phase 3 trial: Part 1—Rationale and general methods. International Journal of Stroke , 9 , 215–219. https://doi.org/10.1111/ijs.12243

25.

Smidt

van der Windt

D. A.

Assendelft

W. J.

Mourits

A. J.

Devillé

W. L.

de Winter

A. F.

, & Bouter

L. M.

(2002). Interobserver reproducibility of the assessment of severity of complaints, grip strength, and pressure pain threshold in patients with lateral epicondylitis. Archives of Physical Medicine and Rehabilitation , 83 , 1145–1150. https://doi.org/10.1053/apmr.2002.33728

26.

Suárez-Álvarez

Pedrosa

Lozano

L. M.

García-Cueto

Cuesta

, & Muñiz

(2018). Using reversed items in Likert scales: A questionable practice. Psicothema , 30 , 149–158. https://doi.org/10.7334/psicothema2018.33

27.

Thabtah

Kamalov

, & Rajab

(2018). A new computational intelligence approach to detect autistic features for autism screening. International Journal of Medical Informatics , 117 , 112–124. https://doi.org/10.1016/j.ijmedinf.2018.06.009

28.

Tozlu

Edwards

Boes

Labar

Tsagaris

K. Z.

Silverstein

, . . . Kuceyeski

(2020). Machine learning methods predict individual upper-limb motor impairment following therapy in chronic stroke. Neurorehabilitation and Neural Repair , 34 , 428–439. https://doi.org/10.1177/1545968320909796

29.

Van Deusen

, & Brunt

(1997). Assessment in occupational therapy and physical therapy. Saunders.

30.

Wall

D. P.

Dally

Luyster

Jung

J. Y.

, & Deluca

T. F.

(2012). Use of artificial intelligence to shorten the behavioral diagnosis of autism. PLoS One , 7 , e43855. https://doi.org/10.1371/journal.pone.0043855

31.

Zandieh

Kahaki

Z. Z.

Sadeghian

Pourashraf

Parviz

Ghaffarpour

, & Ghabaee

(2012). The underlying factor structure of National Institutes of Health Stroke Scale: An exploratory factor analysis. International Journal of Neuroscience , 122 , 140–144. https://doi.org/10.3109/00207454.2011.633721

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.16 MB