Modeling color fading ozonation of reactive-dyed cotton using the Extreme Learning Machine,Support Vector Regression and Random Forest

Abstract

Textile products with a faded effect achieved via ozonation are increasingly popular nowadays. In order to better understand and apply this process, the complex factors and effects of color fading ozonation are investigated via process modeling in terms of pH, temperature, water pick-up, time (of process) and original color (of textile) affecting the color performance (K/S, L*, a*, b* values) of reactive-dyed cotton using the Extreme Learning Machine (ELM), Support Vector Regression (SVR) and Random Forest (RF), respectively. It is found that the RF and SVR perform better than the ELM as the latter were very unstable in the case of predicting a certain single output. Both the RF and SVR are potentially applicable, but SVR would be more recommended to be used in the real application due to its balancer predicting performance and lower training time cost.

Keywords

modeling color fading ozonation Extreme Learning Machine Support Vector Regression Random Forest

Increasingly, textile products with a faded effect, worn look or vintage style are attracting a growing number of young customers' attention and have gained a considerable share of the fashion market.¹ The faded effect, however, generally is achieved by chemical treatments (hydrogen peroxide/chlorine bleaching, for example), which are not only highly water- and power-consuming, but also release a wide range of toxic substances to the environment.²

Ozone is an excellent gaseous oxidant with an environmentally friendly nature that can be rapidly decomposed into O₂ after its application without emitting additional pollution. It is able to react with a large number of organic and inorganic substances in water due to a series of intermediates or by-products, such as hydroxyl radicals (which react with no selectivity), may be generated in the reaction between ozone and water.³ More significantly, ozone could be applied directly in the form of gas without a water bath to color fade the target products (with water content), which can dramatically decrease the water consumption in the sector. Therefore, it is regarded as a perfect alternative to traditional oxidizing agents and bleaching agents.⁴ In recent years, studies regarding color fading dyed textiles using ozone, namely ozonation, instead of the conventional processes have been increasingly reported.^1,5–8

The decolorization of dyes in ozonation, in short, could be attributed to the simultaneous oxidation of direct ozone and (but more of) indirect free radicals (which are generated from the decomposition of ozone) with the unsaturated organic compounds and the chromophoric organic system, for example, the chromophore groups of azo. This has been investigated in our previous works with the mechanism, effects and alternative possibilities of ozonation in the application of fading dyed textiles.^9–11 Not only was the prominent role of ozone in the use of textile processing reported, but we also found that color fading ozonation of textiles is affected by many interdependent different factors, ranging from the properties of textile material to the setting of the color fading process. How these factors affect the color fading process separately has been reported, while to understand their overall impacts simultaneously, the complex and nonlinear relationship between the factors of material properties as well as the technical parameters of ozonation and color fading effects must be taken into consideration. To the best of our knowledge, the simultaneous effects of multiple factors on color fading ozonation of textiles have been barely systematically investigated. In recent years, researchers have tended to address similar issues via process modeling.^12–16 However, unlike the traditional works that are limited in using analytical models based on physical or chemical laws, we propose intelligent techniques that can learn from data, which are more practical and suitable to be applied in this study.

The Artificial Neural Network (ANN) is a widely used artificial intelligence approach in the textile sector.¹⁷ It is inspired by the bionic simulation of the human brain that interconnects numerous neurons in different hidden layers to process the complex information of a specific input–output relation.¹⁸ In particular, the Extreme Learning Machine (ELM) is a novel algorithm for single-hidden layer feedforward neural networks (SLFNs, the structure of which is illustrated in Figure 1), which randomly chooses the input weight matrix ( W ) and analytically determines the output weights ( β ) of SLFNs. The ELM tends to acquire a good generalization performance at extremely fast learning speed.¹⁹ It has been successfully applied to forecast sales behavior in fashion retailing, and was found to perform better than backpropagation neural networks-based methods.²⁰

The Support Vector Machine (SVM) is a popular machine learning tool for classification and regression based on statistic learning theory, first identified by Vladimir Vapnik²¹ and his colleagues in 1992. Support Vector Regression (SVR) is the most common application form of the SVM. A typical feature is that instead of minimizing the observed training error, SVR minimizes the generalized error bound so as to achieve generalized performance. In addition, it only relies on a subset of the training data as the cost function for building the model neglects any training data that is close (within a threshold ε) to the model prediction.^22,23 The excellent use of SVR has been made for predicting yarn properties,^24,25 PU-coated cotton fabric qualities²⁶ and wool knitwear pilling propensity,²⁷ which simultaneously have proved the potential of SVR in the application of textile process modeling.

The Random Forest (RF) is a predictive model composed of a weighted combination of multiple regression trees. It constructs each tree using a different bootstrap sample of the data, and different from decision tree splitting, in which each node uses the best split among all variables, the RF uses the best among a subset of predictors randomly chosen at that node.²⁸ In general, combining multiple regression trees increases predictive performance. It makes an accurate prediction by taking advantage of the interaction of variables and the evaluation of the significance of each variable.²⁹ Kwon et al.³⁰ successfully developed a surface defect detection method based on the RF to inspect the fabric surface. Venkatraman and Alsberg³¹ predicted the important photovoltaic properties of phenothiazine dyes using the RF, which paves the way for rapid screening of new potential dyes and computer-aided material design. To the best of our knowledge, none of the works have ever studied the ELM, SVR and RF simultaneously in an investigation regarding textile applications, and their applicability is not clear in this area.

Reactive-dyed cotton is one of the most important textile products in the sector.³² Therefore, this study aims at modeling color fading ozonation in order to predict the color properties of ozone faded reactive-dyed cotton using different artificial intelligent techniques ranging from the ELM and SVR to the RF with a corresponding optimization process to comparatively find their potential applicability in reactive-dyed cotton color fading ozonation process modeling.

Brief overview of the Extreme Learning Machine, Support Vector Regression and Random Forest

Extreme Learning Machine

The ELM is an algorithm of SLFNs that randomly chooses the input weight matrix ( W ) and analytically determines the output weights ( β ). According to Figure 1, we can take K hidden node SLFNs as an example, using the activation function f (x) = (f₁(x),f₂(x),…,f_k(x)) to learn N samples (X_i, Y_i), where X_i = [x_i₁, x_i₂,…,x_in]^T ∈ R_n and Y_i = [y_i₁, y_i₂,…,y_in]^T ∈ R_m. The ideal approximation of the SLFNs to these samples is zero error, which results in

\sum_{j = 1}^{N} ‖ {\overset{\land}{Y}}_{j} - Y_{j} ‖ = 0

(1)

where

\overset{\land}{Y}

is the actual output value of the SLFNs. Taking the weights W , β and bias b into consideration, we have

\sum_{i = 1}^{K} β \cdot f_{i} (W_{i} \cdot X_{j} + b_{i}) = Y_{j}, j = 1, \dots, N

(2)

where W_i = [w_i1, w_i2,…, w_im]^T and β_i = [β_i1, β_i2,…,β_im]^T, i = 1,…K are the weight vector for the inputs and activated nodes respectively. b_i is the threshold of the ith hidden node. The compact expression of Equation (2) in terms of vectorization could be

H β = Y

(3)

where

H (W_{1}, \dots, W_{j}, b_{j}, \dots, b_{j}, X_{1}, \dots, X_{i}) = f (W_{j} \cdot X_{i} + b_{j})

(i = 1,…,N and j = 1,…K) is the hidden layer output matrix of the neural network, where its jth column is the jth hidden node output in regard to the inputs of

X_{1}, \dots, X_{i}

, while β = [β₁, β₂,…,β_K]^T and Y = [Y₁,Y₂,…,Y_N] ^T are the matrixes of the output weights and targets, respectively.

As the input weights ( W ) are randomly chosen, as well as the biases (b) in the ELM algorithm, the output weights ( β ) that connect the hidden layer and output layer could be simply determined by finding the least-squares solution to the given linear system. According to Huang et al.,³³ the smallest norm least-squares solution of the linear system (3) among all the solutions is

\overset{\land}{β} = H^{†} Y

(4)

where H ^† is the Moore–Penrose generalized inverse of the matrix H .³⁴ In this paper, a multi-output ELM regression function developed by Huang's group was used in this study with an optimal trial of the varied activation functions (i.e. Sigmoid, Sine and Hardlim, given in Equations (5)–(7)) and the number of hidden nodes (from 1 to 200) in the use of the ELM

Sigmoid (x) = \frac{1}{1 + e^{- x}}

(5)

Sine (x) = sin (x)

(6)

Hardlim (x) = 1 if x \geq 0; = 0 otherwise

(7)

Support Vector Regression

Compared with neural networks, SVR assures more generalization on the foundation of structural risk minimization, and generally performs better with fewer training samples. When we have training data {(x_l, y_l),…,(x_l, y_l)} ∈ ℝⁿ × ℝ for the SVR model, the targeted function g(x) should be as flat as possible and has ɛ deviation in maximum from the actual targets y_i for all the training data in the form of

g (x) = 〈 w, x 〉 + b with w \in ℝ^{n}, b \in ℝ

(8)

where x represents the n-dimensional input vectors, w is the weight vector and b is the bias term. Flatness in (8) means small w, and for the method of achieving it is recommended to minimize the Euclidean norm, that is,

\frac{1}{2} ‖ w ‖^{2}

,³⁵ which turns out to be a convex optimization problem

\begin{matrix} minimize \frac{1}{2} ‖ w ‖^{2} \\ subjectto \begin{matrix} y_{i} - 〈 w, x_{i} 〉 - b \leq ɛ \\ 〈 w, x_{i} 〉 + b - y_{i} \leq ɛ \end{matrix} \end{matrix}

(9)

This is a feasible optimization problem when the function g(x) actually exists and approximates all pairs (x_i, y_i) with ɛ precision, and slack variables $ξ_{i}$ , $ξ_{i}^{*}$ (referring to the upper and lower constraints on the outputs of the system) are introduced to deal with the otherwise infeasible constraints²¹

\begin{matrix} minimize \frac{1}{2} ‖ w ‖^{2} + C \sum_{i = 1}^{l} (ξ_{i} + ξ_{i}^{*}) \\ subjectto \begin{matrix} y_{i} - 〈 w, x_{i} 〉 - b \leq ɛ + ξ_{i} \\ 〈 w, x_{i} 〉 + b - y_{i} \leq ɛ + ξ_{i}^{*} \\ ξ_{i}, ξ_{i}^{*} \geq 0 \end{matrix} \end{matrix}

(10)

where C is a constant greater than 0, where determines the trade-offs of

\frac{1}{2} ‖ w ‖^{2}

and the sum of permitted errors. It is found that dual formulation makes it easier to solve this optimization problem,³⁶ so a standard dualization method utilizing Lagrange multipliers has been proposed

\begin{matrix} L = \frac{1}{2} ‖ w ‖^{2} + C \sum_{i = 1}^{l} (ξ_{i} + ξ_{i}^{*}) - \sum_{i = 1}^{l} α_{i} (ɛ + ξ_{i} - 〈 w, x_{i} 〉 + b) \\ - \sum_{i = 1}^{l} α_{i}^{*} (ɛ + ξ_{i}^{*} + y_{i} - 〈 w, x_{i} 〉 - b) - \sum_{i = 1}^{l} (η_{i} ξ_{i} + η_{i}^{*} ξ_{i}^{*}) \end{matrix}

(11)

where L is the Lagrangian and

η_{i}

η_{i}^{*}

α_{i}^{*}

α_{i}^{*}

are the Lagrange multipliers, which have to satisfy the positivity constraints of

\geq 0

. The partial derivatives of L with respect to the variables (w, b,

ξ_{i}

ξ_{i}^{*}

) have to vanish for optimality

\partial b L = \sum_{i = 1}^{l} (α_{i}^{*} - α_{i}) = 0

(12)

\partial w L = w - \sum_{i = 1}^{l} (α_{i} - α_{i}^{*}) x_{i} = 0

(13)

\partial ξ_{i}^{(*)} L = C - α_{i}^{(*)} - η_{i}^{(*)} = 0

(14)

here

ξ_{i}^{(*)}

refers to

ξ_{i} and ξ_{i}^{*}

. Substituting (12)–(14) into (11), the dual optimization problem is given by

\begin{matrix} maximize \begin{matrix} - \frac{1}{2} \sum_{i, j = 1}^{l} (α_{i} - α_{i}^{*}) (α_{j} - α_{j}^{*}) 〈 x_{i}, x_{j} 〉 \\ - ɛ \sum_{i = 1}^{l} (α_{i} + α_{i}^{*}) + \sum_{i = 1}^{l} y_{i} (α_{i} - α_{i}^{*}) \end{matrix} subjectto \sum_{i = 1}^{l} (α_{i} - α_{i}^{*}) = 0 and α_{i}, α_{i}^{*} \in [0, C] \end{matrix}

(15)

As the dual variables $η_{i}$ , $η_{i}^{*}$ can be reformulated on the basis of (14) as $η_{i}^{(*)} = C - α_{i}^{(*)}$ , Equation (13) becomes

w = \sum_{i = 1}^{l} (α_{i} - α_{i}^{*}) x_{i}, thus g (x) = \sum_{i = 1}^{l} (α_{i} - α_{i}^{*}) 〈 x_{i}, x 〉 + b

(16)

This is a so-called Support Vector expansion. In the SVM training algorithm, the next necessary step is to make it nonlinear, which was suggested to be achieved by a mapping ∅ (x) from ℝⁿ to a higher dimensional feature space using the kernel function $K (x, x_{i}) = 〈 Ø (x_{i}), Ø (x) 〉$ ; therefore (16) becomes

\begin{matrix} w = \sum_{i = 1}^{l} (α_{i} - α_{i}^{*}) Ø (x_{i}), \\ g (x) = \sum_{i = 1}^{l} (α_{i} - α_{i}^{*}) k (x_{i}, x) + b \end{matrix}

(17)

It is different from the linear case, as w means the flatness is no longer explicitly given. In this nonlinear case, the optimization problem refers to finding the flattest function in feature space, rather than in input space. The standard SVR is

g (x) = \sum_{i = 1}^{N} (α_{i} - α_{i}^{*}) k (x_{i}, x) + b

(18)

where N (< the total number of input–output pairs) is the number of input data having nonzero values of

α_{i}^{(*)}

. The kernel function k (x_i, x) corresponds to a linear dot product of the nonlinear mapping. As we are disposing of a case of the process modeling containing multiple outputs, we applied a multi-output least-squares Support Vector Regression (MLS-SVR) toolbox developed by Xu et al.³⁷ in particular in this study with an optimal trail on the kernel functions of

Linear : K (x, x_{i}) = x^{T} x_{i} + C

(19)

Sigmoid : K (x, x_{i}) = tanh (α x^{T} x_{i} + C)

(20)

Polynomial : K (x, x_{i}) = 〈 x, x_{i} 〉 p

(21)

Radialbasisfunction (RBF) : K (x, x_{i}) = e^{(- \frac{‖ x - x_{i} ‖^{2}}{2 σ^{2}})}

(22)

\begin{matrix} Exponentialradialbasisfunction (ERBF) : \\ K (x, x_{i}) = e^{(- \frac{‖ x - x_{i} ‖}{2 σ^{2}})} \end{matrix}

(23)

There is also an optimization process (using leave-one-out, LOO) of the parameters of γ, λ and p in the toolbox from where γ ∈ {2⁻⁵, 2⁻³,…, 2¹⁵} and λ ∈ {2⁻¹⁰, 2⁻⁸,…, 2¹⁰} (two positive regularized parameters controlling the bias-variance trade-off), p ∈ {2⁻¹⁵,2⁻¹³,…, 2³}(= $\frac{1}{2 σ^{2}}$ is a parameter of the RBF that sets the spread of the kernel).³⁷

Random Forest

The RF is an ensemble-learning algorithm depending on the bagging method that combines multiple independently constructed decision tree predictors to classify or predict certain variables.²⁹ In the RF, successive trees do not rely on earlier trees; rather, they independently use a bootstrap sample of the dataset, and therefore a simple unweighted average over the collection of grown trees {h(x,Θ _k )} would be taken for prediction in the end

\bar{h} (X) = \frac{1}{K} \sum_{k = 1}^{K} h (x, Θ_{k})

(24)

where k = 1, … , K is the number of trees, x represents the observed input vector and Θ is an independent identically distributed random vector that the tree predictor takes as numerical values. The RF algorithm starts from randomly drawing ntree bootstrap samples from the original data with replacements. Then a certain number of regression trees are grown in accordance with the bootstrap samples. In each node of the regression tree, the number of the best split (mtry) randomly selected from all variables is considered for binary partitioning. The selection of the feature for node splitting from a random set of features decreases the correlation between different trees and, thus, the average prediction of multiple regression trees is expected to have lower variance than that of individual regression trees.³⁸ The regression tree hierarchically gives a specific restriction or condition and it grows from the root node to the leaf node by splitting the data into partitions or branches according to the lowest Gini index

I_{G} (t_{X (X_{i})}) = 1 - \sum_{J = 1}^{M} f (t_{X (X_{i})}, j)^{2}

(25)

where

f (t_{X (X_{i}),} j)

is the proportion of samples with the value x_i belonging to leaf j as node t.³⁹ In the present study, the Multivariate Random Forest (MRF) developed by Rahman et al.⁴⁰ was employed with an optimal topology of three parameters in terms of ntree (the number of trees in the forest), minleaf (the minimum number of samples in the leaf node) and mtry (the randomly selected features considered for a split in each regression tree node).

Experimental details

Material

Desized gray cotton fabrics (³/₁ twill; 325.7 g/m²; supplied by Shunfu, Hubei, China) was dyed by three bifunctional fluorotriazine azo reactive dyes, namely Reactive Blue FL-RN (RB-RN), Reactive Red FL-2BL (RR-2BL) and Reactive Yellow FL-2RN (RY-2RN) (provided by Color Root, Hubei, China; commercial quality, purity of dyes: 92%,), respectively. Sodium hydroxide and hydrogen chloride (analytical grade, supplied by Sinopharm Limited, China) were used in the ozonation.

Apparatus

The ozone employed in this work was generated by a corona discharge ozone generator, CF-G50 (Guolin, China) that was fed by pure and compressed dry oxygen (≥99.9%, 1 MPa, 12 L/min) from an oxygen cylinder. Ozone flowed to the reactor (made of glass; the structure is shown in Figure 2), and in each single color fading ozonation experiment, samples were distributed evenly on the sample desk (made of an air-permeable steel net). Ozone was imported with a gas flow of 2 L/min and a dosage of 137 ± 3 mg/L/min (tested by an ultraviolet (UV) meter NS-xmd614, Naishi, China) throughout the treatment. The exhaust from the reactor was collected and decomposed by a heater (≥230℃) before evacuating to the atmosphere.

Methods

Ozonation processes

Three dyed cottons in different colors were treated respectively by the color fading ozonation using the following steps: wetting the fabrics by deionized water (pH = 7, or using sodium hydroxide and hydrogen chloride, respectively, when a specific pH is required) to obtain a certain pick-up water content. After ozone treatment, the samples were rinsed with deionized water before naturally drying.

Ozonation at different pH values (1, 4, 7, 10, 13) and temperatures (0℃, 20℃, 40℃, 60℃, 80℃) with variable pick-ups (water content of the samples, 0%, 75%, 150%) for different treatment times (0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60 min) was investigated in the three dyed cotton fabrics (blue, red, yellow, 612 fabrics samples in total). Besides the pH, which was set up depending on the method mentioned above (using sodium hydroxide and hydrogen chloride, respectively, in the water pick-up step), the temperature of ozonation was controlled by a water bath around the reactor (including the inlet tubes), and the pick-up of samples was calculated by Equation (26).

Pick-up (%) = \frac{W_{s} - W_{0}}{W_{0}} \times 100 %

(26)

where w_s is the weight of the wet pick-up sample and w₀ is the weight of the sample before wetting.

Analytical details

On the basis of the Kubelka–Munk theory,⁴¹ it is known that the K/S value is able to indicate the color depth of samples, while the L* a* b* values (or CIELab), an international standard widely used for color measurements, are capable of illustrating the color variation of textile samples. Normally, the color of the final textile product agreeing with specific K/S and L* a* b* values is within the acceptable tolerance of the consumer.¹⁴ Consequently, these values (an average of four measurements from different parts and both sides of each sample, within a relative error of 0.3%), tested with a Datacolor 110 spectrophotometer (Datacolor, USA), were used to characterize the color variation of dyed textiles in the color fading ozonation.

Modeling structure

In the present study, the ozonation process model is expected to be capable of predicting (or outputting) the color qualities of ozone treated samples in terms of K/S and L*, a*, b* values by giving five variables including not only the specific color of treated fabric but also the process parameters of pH, temperature, pick-up and treatment time. In other words, the anticipated model of color fading ozonation of reactive-dyed cotton realizes the complex and unclear relation of color fading ozonation parameters and its effectiveness on reactive-dyed cotton fabric in certain respects.

For instance, the real samples used in the ozonation, particularly at pH 7, 20℃ with 150% pick-up over different times from 0–60 min can be observed in Figure 3. The corresponding K/S, L^*, a^*, b* values of the exhibited samples are listed in Table 1. It is clear that none of the treated samples has an obvious difference from the others in regard to color properties as treated by different ozonation processes, which on the other hand apparently revealed how complex the process parameters are in influencing the color of dyed cotton fabric in ozonation. Table 2illustrates the variation, including the minimum, maximum, average and standard deviation of the dataset we used in the process modeling.

Table 1.

K/S, L*, a*, b* values of the samples shown in Figure 3

	Time (min)
	0	5	10	15	20	25	30	40	50	60
RB-RN
K/S	8.12	3.70	2.17	1.63	1.24	1.17	0.89	0.85	0.66	8.12
L*	6.13	14.89	21.90	26.12	29.50	29.93	33.44	33.80	36.66	6.13
a*	−2.86	−4.47	−5.18	−5.68	−5.57	−5.27	−5.25	−5.19	−4.76	−2.86
b*	4.48	11.38	15.46	16.85	18.83	19.59	21.40	22.16	23.62	4.48
RR-2BL
K/S	1.10	0.95	0.70	0.66	0.52	0.57	0.51	0.39	0.39	1.16
L*	22.66	25.93	29.74	30.26	33.05	31.96	33.79	36.04	36.43	21.68
a*	−32.59	−37.32	−39.25	−40.16	−41.17	−41.82	−43.19	−42.44	−43.49	−32.39
b*	12.84	13.69	12.84	12.49	12.05	11.82	12.38	11.23	11.69	13.41
RY-2RN
K/S	6.60	3.70	2.90	2.20	1.81	1.58	1.16	1.11	0.87	6.60
L*	8.02	8.49	9.80	10.82	11.60	13.19	13.96	14.14	15.43	8.02
a*	20.41	15.35	12.99	11.47	9.59	8.62	7.31	6.52	5.56	20.41
b*	−17.99	−28.27	−32.09	−37.32	−40.95	−41.58	−47.94	−48.92	−52.39	−17.99

Table 2.

The maximum, minimum, average and standard deviation of ozonation parameters

Parameters	Minimum	Maximum	Average	Std. dev.
Color^a	0 (Blue)	1 (Yellow)	–	–
pH	1	13	7	3.463
Temperature (℃)	0	80	40	24.91
Pick-up (%)	0	150	75	58.78
Time (min)	0	60	30	20.77
K/S	0.10	22.82	7.18	7.94
L*	0.99	65.27	33.52	17.61
a*	−58.99	53.68	−1.81	32.75
b*	−90.53	88.04	3.88	42.34

As the color of fabric is not a continuous variable, it was represented in the bipolar form as 0 (blue), 0.5 (red) and 1 (yellow).

A total of 612 sets of data were collected in the experiment. The data was divided into two groups, namely training data and testing data. The distribution of 75% for training and 25% for testing is the general one used in the machine learning sector, which was taken into account as well in this study; therefore, 459 datasets (75%) was used to train the models, while the remaining 153 datasets (25%) were distributed to the testing data. The correlation of these factors (pH, temperature, pick-up and time only, as the original color of the fabric is not a continuous variable) to K/S, L*, a*, b* values was estimated by Spearman's rank correlation coefficients (based on Equation (27)) and are listed in Table 3. It is revealed that the pH and treatment time are slightly more relevant than temperature and pick-up, as they play more important roles in the ozonation

ρ = 1 - \frac{6 \sum_{i}^{2}}{n (n^{2} - 1)}

(27)

where n is the number of observations and d_i is the difference between the ranks of corresponding variables.

Table 3.

The Spearman's correlation coefficients of the data for modeling

	pH	Temperature	Pick-up	Time
K/S	0.0383	0.0648	−0.4862	−0.7913
L*	−0.0579	0.1146	−0.1557	−0.3137
a*	0.0248	0.1490	−0.4500	−0.7430
b*	−0.0302	−0.0155	−0.0184	−0.0461

K-fold cross-validation (k = 10) was used in the process modeling. This is a popular statistical approach for estimating predictive models. Taking k = 10 as an example as it was the value used in the modeling study, 459 training sets of data were divided randomly and equally into 10 disjoint folds, nine folds of which were split into the training subset, while the remaining one fold was used as the validating subset. This procedure was repeated 10 times with a varied training and testing dataset at each time to validate the trained models. In order to evaluate the performance of the models in validation, the mean square error (MSE) was used based on

MSE = \frac{1}{n} \sum_{i = 1}^{n} (e_{i} - p_{i})^{2}

(28)

where e_i is the real experimental results and p_i is the predicted output of the specific model. In addition, four statistical performance criteria, namely the mean absolute error (MAE), root mean square error (RMSE), correlation coefficient (R) and mean relative absolute error (MRAE), are used in this study for indicating the predictive performance of the obtained models

MAE = \frac{1}{n} \sum_{i = 1}^{n} | e_{i} - p_{i} |

(29)

RMSE = \sqrt{\frac{1}{n} \sum_{i = 1}^{n} (e_{i} - p_{i}) 2}

(30)

R (e, p) = \frac{\sum_{i = 1}^{n} (e_{i} - \bar{e}) (p_{i} - \bar{p})}{\sqrt{\sum_{i = 1}^{n} {(e_{i} - \bar{e})}^{2} \cdot \sum_{i = 1}^{n} {(p_{i} - \bar{p})}^{2}}}

(31)

MRAE = \frac{1}{n} \sum_{i = 1}^{n} \frac{| e_{i} - p_{i} |}{e_{i}}

(32)

The development and construction of the models were carried out using MATLAB R2015b software for the multi-output ELM and MLS-SVR, but R studio for the MRF on a laptop (Core i7-4710, 2.5 GHz, 16 GB RAM). All of the original data was regularized in the range of [0, 1] before using.

Results and discussion

Modeling training

ELM models

ELM models with hidden nodes from 1 to 200 activated by Sigmoid, Sine and Hardlim functions are investigated (the corresponding validation MSE is illustrated in Figure 4, with a detailed demonstration of the trained ELM models possessing nodes from 1 to140 in detail). The overfitting situation of the ELM activated by Sigmoid and Sine is easy observe and starts from those with nodes around 100. More specifically, it is noted that Sigmoid trained ELM models performed similarly to those trained by Sine, since the MSE of these models both dropped as well as minimized at those with around 50 nodes (MSE ≈ 0.052), followed by a dramatic enhancement. In contrast, the validation MSE of Hardlim activated models performed generally stably with the growing number of nodes in the ELM model, but strictly with a minimum of MSE ≈ 0.069 (larger than Sigmoid and Sine) at that with 97 nodes, which can be seen in Figure 4. Similar comparative results of the use of these activation functions in the ELM can be found also in the work of Singh and Balasundaram.⁴²

The use of activation functions in an ANN is to convert an input signal of the node to an output signal by mapping the nonlinear properties. It is very important for an ELM model to learn and achieve the complicated mapping of input and output data by activating the nodes with a certain activation function. A graph of the activation functions we used is given in Figure 5. It is noted that Sigmoid and Sine have much in common with their S-shaped curve and both are infinitely differentiable functions, which make them easy to understand and apply. However, on the other hand, it may also result in their similar proximity and disadvantage in the ELM models, as we can see their similar performance variation and the overfitting situation with the increasing nodes in Figure 4. Hardlim performed the least compared with Sigmoid and Sine in terms of their activated ELM models in this issue, probably is owing to their oversaturation.

SVR models

Multi-output SVR models with kernel functions of Linear, Sigmoid, Polynomial, RBF and ERBF were trained and developed using the MLS-SVR toolbox. The corresponding results of minimum validation MSE are 0.05678, 0.00932, 0.08613, 0.00493 and 0.0092, respectively (as demonstrated in Figure 6). It is worth noting that models trained with the Linear kernel and Polynomial kernel are found to perform far more poorly than the others. Performance of those with the Sigmoid kernel and ERBF kernel are very close at a quite low level, although their validation MSE is nearly two times that of the SVR model with the RBF kernel (which performed the most in comparison in this issue when its parameters are optimized to γ = 32768, λ = 9.7656e⁻⁴ and p = 0.125; for more information regarding the LOO optimization process used in the toolbox for these kernel parameters, see Xu et al.³⁷).

The kernel function is used to transform the data as input into the required form to facilitate a nonlinear decision surface to be a linear equation in higher dimensions where the computational power of the learning machine is heightened. The type of kernel function used would influence many of the characteristics of the SVR model. A wide range of kernels exist and it is difficult to explain their individual characteristics, but it is well known that the RBF kernel is recommended to be tried first in the SVR model due to the fact that it not only possesses certain similar parameters and behaviors of Linear and Sigmoid but also has fewer hyper parameters than Polynomial to avoid complexity in the model. The RBF is assumed to have computational advantages over other kernels, as it is easier and faster to compute the kernel values.⁴³ The lowest MSE it achieved in this case validates its preferential suitability to be employed in this study, which is because we have not too many features in the model but with comparatively large numbers of observations.

RF models

RF models with different mtry (from 1 to 5), minleaf (from 1 to 10) and ntree (from 1 to 100) are trained and developed, and the validation MSEs of these models are given in Figure 7 with a detailed demonstration of mtry = 1 and ntree ranging from 1 to 100, excluding those for which the validation MSE is higher than 0.026. In Figure 7, the number of mtry in each regression tree node is found that plays a very significant role in affecting the prediction accuracy of the color properties of models of ozone treated cotton fabrics. The falling curves of the MSE with the growing number of mtry may reveal that the five inputs we used to construct these RF models, that is, the (1) color of dyed cotton and (2) pH, (3) temperature, (4) pick-up and (5) treating time of ozonation process, have a very clear independent relation with each other. As a result, RF models with five randomly selected features generally lead the low validation MSE in this comparison. It is also found that ntree played another significant role in RF models, as the MSE of these models decreased dramatically when the number of trees increased in the forest from 1 to 30. In general, these models perform steadily when there are more than 30 regression trees in the forest construction no matter what the mtry or minleaf employed, but in order to save time and cost in the model training process, a 10-tree forest is sufficient and may be better recommended to be used in the color fading ozonation of dyed textile predicting model for further experiments. However, unlike mtry and ntree, the minimum number of samples in the leaf node, that is, minleaf, seems to be preferable to be less, although it is relatively uninfluential. Depending on the observation of the detailed depicted MSE plots of 1-mtry RF models in Figure 7, we can see that the average MSE of the achieved RF models was generally enhanced when the number of leaves increased from 1 to 10.

Prediction performance

The quality of a model is not only determined by its ability to learn from the data but also its ability to predict unseen data, which are the so-called learning capacity and generalization ability of a model. Models are seldom good in both of these capacities. According to the training results obtained above, we find that Sine and Sigmoid trained ELM models have very similar performance that both optimized at 50 nodes, while SVR with the RBF kernel function and the RF as mtry = 5, ntree = 10, minleaf = 2, in contrast, clearly precede the others in their training process. In order to further comparatively investigate the potential application of these three techniques without losing significant observations, the two ELM models were taken into account together with the RBF-SVR and the optimized RF (mtry = 5, ntree = 10, minleaf = 2) in this section.

To estimate and compare these optimized models, a prediction test using a testing dataset (which has not been used in the training and validation processes) is carried out. Table 4 presents a comparison of the prediction performance of the ELM, SVR and RF models. It is found that, in general, the ELM models using activation functions of Sigmoid (MSE = 0.0172) and Sine (MSE = 0.0173) do not have any great difference in regard to their prediction performance, but both of them are slightly poorer comparing with the SVR and RF models. However, ELM models are the fastest-trained ones in the comparison, which means that the ELM model is still worth applying in certain resource-limited cases, especially when limited training time is concerned. The most accurately predicted model we can see, according to the finding in Table 3, is the RF, as it leads to the least testing error with higher R (0.9847) and lower MSE (0.0036), MAE (0.0295), RMSE (0.0601) and MRAE (0.0062). However, it is also noted that training the RF model requires a much longer time than the others (21 s). As a result, it is worth taking the SVR models into account as they achieved the second lowest error (R = 0.9777, MSE = 0.0043, MAE = 0.0429, RMSE = 0.656, MRAE = 0.0109) with a more acceptable shorter training time (0.9360 s).

Table 4.

Prediction performance of the optimized models

Parameters	ELM (Sigmoid)	ELM (Sine)	SVR	RF
Training time(s)	0.0312	0.0936	0.9360	21
Average error (%)	0.1527	0.1549	0.1530	0.149
Maximum error (%)	0.3752	0.3700	0.3503	0.302
Minimum error (%)	0.0300	0.0029	0.0304	0.038
MSE	0.0172	0.0173	0.0043	0.0036
MAE	0.0894	0.0921	0.0429	0.0295
RMSE	0.1311	0.1315	0.0656	0.0601
R	0.9052	0.9063	0.9777	0.9847
MRAE	0.0197	0.0168	0.0109	0.0062

ELM: Extreme Learning Machine; SVR: Support Vector Regression; RF: Random Forest; MSE: mean square error; MAE: mean absolute error; RMSE: root mean square error; MRAE: mean relative absolute error.

Table 4 demonstrates the overall performance of the constructed models in terms of certain estimation evaluation indexes, but the detail of these predictions is neglected, as it is known that the constructed models possess four output, that is, K/S, L*, a*, b* values of reactive-dyed cotton fabrics treated with the color fading ozonation. How these predictive models work in detail with them is unclear. In order to reflect the real prediction performance (using testing data) of each trained model on predicting each single output separately, the predicted results range from output 1 (K/S value) to output 4 (b*) versus the real experimental data (targets 1–4) is illustrated in Figures 8(a)–(d), respectively.

In Figure 8, the predicted values of the models generally agree with the actual values, although the predictive errors varied at different levels for different models. We can see that the gap in prediction performance of the models is not that significant in Table 4 (taking the MSE as an example, Sigmoid activated ELM = 0.0172, Sine activated ELM = 0.0173, SVR = 0.0043, RF = 0.0036), while the distribution of errors in terms of each single output prediction is observed to have a larger gap in the real application. In Figures 8(a) and (c), certain predicted data of ELM models can be clearly seen that is far different from the real target data in a certain range, a situation that would result in a great mistake in certain prediction applications where good overall performance of the average of multiple outputs may hinder the discovery of a wrong prediction on a specific single output. According to the linear fitting correlation coefficients of predicted data versus real experimental data (demonstrated in Figure 6), listed in Table 5, the testing result obtained reveals that the SVR (R² = 0.9505) model and RF model (R² = 0.9555) are actually more stable and suitable than ELM models (R² = 0.8025 and 0.8007 for Sigmoid and Sine activated, respectively) in modeling color fading ozonation of dyed textiles, in terms of overall prediction performance, and more importantly predicting multiple outputs without deviation on a certain single output. This may be because of the features of the data we used concerning color fading ozonation of dyed textiles. On the other hand, it could also be attributed to a disadvantage of the ELM, in that it completely relies on increasing the number of nodes to promote the prediction performance, which makes it risky to apply for a complicated issue such as that in the present investigation. The results also reveal that both the SVR and RF can well deal with the interaction of variables and are comparatively more stable in multi-variable nonlinear modeling.

Table 5.

Correlation coefficients of the data in Figure 8

R ²	ELM (Sigmoid)	ELM (Sine)	SVR	RF
Target 1 – K/S	0.8474	0.8596	0.9683	0.9954
Target 2 – L^*	0.7944	0.7517	0.9442	0.8816
Target 3 – a^*	0.7903	0.8048	0.9380	0.9719
Target 4 – b^*	0.7778	0.7868	0.9513	0.9731
Average	0.8025	0.8007	0.9505	0.9555

ELM: Extreme Learning Machine; SVR: Support Vector Regression; RF: Random Forest.

Conclusion

In this study, we used three modeling techniques, that is, the ELM, SVR and RF, to model the ozonation process for predicting the color properties of treated dyed textiles. The potential applicability of these models in the use of process modeling in the related textile process was estimated.

Color fading of dyed textiles is a vital process in the textile industry to obtain certain stylish effects on the product; this process has been increasingly used in recent decades. Ozonation is a novel technology developed in recent years to be employed to achieve the color fading effect of textiles with high performance not only in respect of efficiency and quality but also in regards to environmental sustainability. For the purpose of better understanding and application of color fading ozonation of textiles at the industrial scale, the complexity and nonlinearity of the factors and impacts of color fading ozonation on reactive-dyed cotton were investigated by process modeling. The effects of ozonation in terms of pH, temperature, water pick-up, treatment time of process and dyed colors of fabrics on the color fading performance in terms of K/S, L*, a*, b* values of reactive-dyed cotton were modeled using the ELM, SVR and RF, respectively. The results show that both the SVR and RF are potential candidates for modeling the color fading ozonation process of dyed textiles, as their predicted results on the ozonation process had a good agreement with the actual output data entirely as well as individually. However, taking the training time and cost into consideration, the SVR model would be better recommended than the RF to be applied in real use. In contrast, the ELM models performed poorer in the prediction and were very unstable in terms of predicting certain individual outputs in multi-variable process modeling.

The perspective development of modeling color fading ozonation should be drawn toward the realization of garment application. It is a mission that is more challenging that can barely be realized on the basis of experiments about reactive-dyed-cottons and the trials of computational development conducted in the present work. Researchers should expend more effort to expand the application to garment sector in future works.

Footnotes

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The first author was supported by the China Scholarship Council (CSC, Project Number 201708420166) for this study.

References

Kan

Cheung

Chan

. A study of plasma-induced ozone treatment on the colour fading of dyed cotton. J Clean Prod 2016; 112: 3514–3524.

Paul R. Denim: manufacture, finishing and applications. Cambridge: Elsevier, 2015.

Hoigné J. The chemistry of ozone in water. In: Process technologies for water treatment. In Samuel Stucki (ed.), Boston, MA: Springer, 1988, pp.121–141.

Eren

Ozturk

. The evaluation of ozonation as an environmentally friendly alternative for cotton preparation. Text Res J 2011; 81: 512–519.

Mao

Guan

Luo

, et al. Investigation of a homogeneous activating ozonation method in the rinsing procedure of cotton fabric dyed with reactive dye. Color Technol 2011; 127: 256–267.

Yigit

Eren

. Ozone utilisation for discharge printing of reactive dyed cotton. Color Technol 2018; 134: 13–23.

Eren

Gümüs

Eren

. Colour stripping of reactive-dyed cotton by ozone treatment. Color Technol 2016; 132: 466–471.

Kıcık

Eren

. Application of ozone gas for the stripping of fabric ink-jet-printed with reactive dyes. Color Technol 2017; 133: 485–490.

He Z, Li M, Zuo D, et al. The effect of denim color fading ozonation on yarns. Ozone: Science & Engineering 2018; 40:5: 377–384.

10.

Zuo

, et al. Color fading of reactive-dyed cotton using UV-assisted ozonation. Ozone Sci Eng 2019; 41: 60–68.

11.

He Z, Li M, Zuo D, et al. Effects of color fading ozonation on the color yield of reactive-dyed cotton. Dyes and Pigments 2019; 164: 417–427.

12.

Abd Jelil

Zeng

Koehl

, et al. Modeling plasma surface modification of textile fabrics using artificial neural networks. Eng Appl Artif Intell 2013; 26: 1854–1864.

13.

Yick

, et al. Parametric design and process parameter optimization for bra cup molding via response surface methodology. Expert Syst Appl 2012; 39: 162–171.

14.

Balci

Oǧulata

. Prediction of the changes on the CIELab values of fabric after chemical finishing using artificial neural network and linear regression models. Fibers Polym 2009; 10: 384–393.

15.

Elemen

Akçakoca Kumbasar

Yapar

. Modeling the adsorption of textile dye on organoclay using an artificial neural network. Dye Pigment 2012; 95: 102–111.

16.

Zervent Ünal

. The prediction of seam strength of denim fabrics with mathematical equations. J Text Inst 2012; 103: 744–751.

17.

Sette

Van Langenhove

. An overview of soft computing in textiles. J Text Inst 2003; 94: 103–109.

18.

Tehran MA, and Maleki M. Artificial Neural Network Prosperities in Textile Applications, In: Suzuki Kenji (ed.), Artificial Neural Networks Industrial and Control Engineering Applications, 2011. 35–64. Rijeka, IntechOpen.

19.

Wu Y, Zhang Y, Liu X, Cai Z, and Cai Y. A multiobjective optimization-based sparse extreme learning machine algorithm. Neurocomputing 2018; 317: 88–100.

20.

Sun

Choi

, et al. Sales forecasting using extreme learning machine with applications in fashion retailing. Decis Support Syst 2008; 46: 413–421.

21.

Vapnik V. The nature of statistical learning theory. New York: Springer Science & Business Media, 2013.

22.

Basak

Pal

Patranabis

. Support vector regression. Neural Inf Proc Rev 2007; 11: 203–224.

23.

Smola AJ and Schölkopf B. A tutorial on support vector regression. Statistics and computing 2004; 14(3): 199–222.

24.

Ghosh

Chatterjee

. Prediction of cotton yarn properties using support vector machine. Fibers Polym 2010; 11: 84–88.

25.

Nurwaha

Wang

. Prediction of rotor spun yarn strength using support vector machines method. Fibers Polym 2011; 12: 546–549.

26.

Güneşoğlu

Yüceer

. A modeling study of micro-cracking processes of polyurethane coated cotton fabrics. Text Res J 2018; 88: 2766–2781.

27.

Yap

Wang

, et al. Prediction of wool knitwear pilling propensity using Support Vector Machines. Text Res J 2010; 80: 77–83.

28.

Liaw

Wiener

. Classification and regression by randomForest. R News 2002; 2: 18–22.

29.

Breiman, L. Random forest. Machine Learning 2001; 45: 5–32.

30.

Kwon

Won

Kang

. Fast defect detection for various types of surfaces using random forest with VOV features. Int J Precis Eng Manuf 2015; 16: 965–970.

31.

Venkatraman

Alsberg

. A quantitative structure-property relationship study of the photovoltaic performance of phenothiazine dyes. Dye Pigment 2015; 114: 69–77.

32.

Khatri

Peerzada

Mohsin

, et al. A review on developments in dyeing cotton fabrics with reactive dyes for reducing effluent pollution. J Clean Prod 2015; 87: 50–57.

33.

Huang

G-B

Zhu

Q-Y

Siew

C-K

. Extreme learning machine: theory and applications. Neurocomputing 2006; 70: 489–501.

34.

Rao CR and Mitra SK. Generalized inverse of a matrix and its applications. Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Theory of Statistics, 601–620, University of California Press, Berkeley, Calif., 1972. Available at: https://projecteuclid.org/euclid.bsmsp/1200514113.

35.

Smola

Scho

. The connection between regularization operators and support vector kernels. Neural Network 1998; 11: 637–649.

36.

Scholkopf B and Smola AJ. Learning with kernels: support vector machines, regularization, optimization, and beyond. London: MIT Press, 2001.

37.

Qiao

, et al. Multi-output least-squares support vector regression machines. Pattern Recognit Lett 2013; 34: 1078–1084.

38.

Liaw

Wiener

. Classification and regression by randomForest. R News 2002; 2: 18–22.

39.

Breiman L. Classification and regression trees. New York: Routledge, 2017.

40.

Rahman R, Otridge J and Pal R. Gene expression IntegratedMRF: random forest-based framework for integrating prediction from different data types. Bioinformatics 2017; 33: 1407–1410.

41.

Kubelka

. Ein Beitrag zur Optik der Farbanstriche (Contribution to the optic of paint). Zeitschrift fur Tech Phys 1931; 12: 593–601.

42.

Singh

Balasundaram

. Application of extreme learning machine method for time series analysis. Int J Intell Technol 2007; 2: 256–262.

43.

Smits GF and Jordaan EM. Improved SVM regression using mixtures of kernels. In Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No. 02CH37290) (Vol. 3, pp. 2785–2790), 2002, May, IEEE.