Uncertain time series is chronological sequence overtime where each period is described by an uncertain variable. In this paper, we investigate the smoothly clipped absolute deviation (SCAD) penalized estimation method to determine the unknown parameters in the uncertain autoregressive model, and the autoregressive model order can be simultaneously obtained for a pre-given thresholding parameter λ. Besides, an iterative algorithm based on local quadratic approximations for finding the penalized estimators is provided. Based on the fitted autoregressive model, the forecast value and the future value’s confidence interval are given. Besides, the sum of the squared error approach to select the optimal λ is discussed. Finally, some examples are used to validate the effectiveness of the proposed method by the comparative analysis.
Relationships between a sequential set of past data measured over time to forecast future values are investigated by time series forecasting. The classical time series forecasting model is autoregressive firstly proposed by Yule [11]. Following that, the improved autoregressive model, such as moving average [10], autoregressive moving average, and autoregressive integrated moving average [7], were investigated. These models have been widely used in various applications, as can be seen in economics, control system, medicine, astronomy, and many others.
In time series modeling applications, choosing the proper order of the model is a fundamental step. It has been of considerable interest by many researchers, such as Akaike [12, 13], Liang et al. [8], Fuchs [17], Sadabadi et al. [19], Al-Smadi and Al-Zaben [1], and Schwarz [9]. The commonly used methods include Akaike’s information criterion (AIC), Akaike’s final prediction error (FPE), and Bayesian information criterion (BIC). Parameter estimation is another important topic and has also received extensive attention in time series analysis. Several methods for estimating parameters of the model have been proposed by Reinsel et al. [6], Paarmann and Korenberg [18], Macdonald [14], Daris [15], and Cochrane and Orcutt [5], but some researchers prefer method of estimation based on least-squares because this method is relatively simple and has an excellent overall performance.
Although the classical time series forecasting methods are widely used and have made great success in the application, they primarily deal with the forecasting problems in which the observed values are real numbers. In practice, the observed data are sometimes incomplete, missing, or censored due to various factors’ influence. Among them, interval-valued data are encountered frequently in multiple situations such as the diastolic and systolic blood pressure in a given day, the variation in an electric current, the company’s daily stock price, and the inventory demand of enterprises in a specific period, etc. It should be noted that such data are human linguistic data that describe the uncertainty or variability of the observed values instead of generic numeric intervals. For this kind of imprecise observed values, describing it and constructing its linguistic model are essential. Therefore, Liu [2] proposed the uncertainty theory based on normality, duality, subadditivity, and product axioms and considered the imprecise observed values as the uncertain variables.
On this basis, Yang and Liu [20] introduced a concept of uncertain time series, proposed an uncertain autoregressive model, and presented the least-squares method for estimating the unknown parameters in the uncertain autoregressive model. Following that, Yang and Ni [21] investigated an uncertain moving average model to deal with uncertain time series problems and presented the least-squares estimation method. Afterward, noting that the least-squares estimation is sensitive to outliers’ presence, the least absolute deviations estimation method was proposed in [22]. However, it isn’t easy to find the optimal solution in the process of calculation. To this end, Chen and Yang [4] proposed the maximum likelihood method to determine the uncertain autoregressive model’s parameters. By this method, the unknown autoregressive parameters and the parameters of uncertainty distributions of the disturbance terms can both be obtained. Furthermore, Zhang et al. [24] presented a least absolute shrinkage and selection operator (LASSO) estimation of the uncertain autoregressive model. For the LASSO obtained by adding the penalty term, the solution typically has many of the coefficients equal to zero, some information about the corresponding observations might be overlooked. To overcome this shortcoming, Chen and Yang [3] gave the ridge method to compute the unknown parameters in the uncertain autoregressive model. Ridge estimation can shrink the large coefficients but does not reach zero, which indicates that there is no feature of variable selection.
To summarize, in the parameter estimation methods mentioned above, the researchers assume that the order of the uncertain autoregressive model is known in advance, and then the parameters were estimated according to this assumption. Practically, the order is unknown, so it has to be correctly found to estimate the model’s correct parameters. To this end, Liu and Yang [23] investigated the cross-validation method to confirm autoregressive model order. Zhang et al. [24] presented the final prediction error criterion to determine the order of the model. Further, this paper proposes a new SCAD estimation method for simultaneously determining the model order and unknown parameters of the uncertain autoregressive model for a pre-given λ, which can overcome the defects of the ordinary LASSO estimator and the ridge estimator. The rest of this paper is organized as follows. In Section 2, the SCAD estimation method to determine the order and unknown parameters in the uncertain autoregressive model are presented. The iterative algorithm is proposed for solving the minimizing penalized estimator. After that, the sum of the squared error approach for selecting an optimal λ is discussed in Section 3. Then the simulation studies and numerical comparisons are given in Section 4. Finally, some conclusions are presented in Section 5.
SCAD estimation
This section first reviews the uncertain autoregressive model and presents the SCAD estimation method to determine the order and unknown parameters in the autoregressive model. Then, we give the iterative algorithm for optimizing penalized likelihood functions. Finally, the forecast value and the confidence interval of the future value are given.
An uncertain time series is a sequence of imprecisely observed values that are characterized in terms of uncertain variables, i.e.,
where Xt are uncertain variables at times t, t = 1, 2, ⋯ , n, respectively.
Uncertain autoregressive model is commonly used to describe and predict uncertain time series. The actual equation for the uncertain autoregressive model is as follows,
where φ0, φ1, ⋯ , φp are unknown parameters, ɛt is an uncertain variable, and p is called the order of the uncertain autoregressive model. In uncertain time series analysis, the order of an uncertain autoregressive model is often unknown. This allows the possibility of choosing an under-fitting or over-fitting model, which leads to the reduction of prediction accuracy. Therefore, a key point using the uncertain autoregressive model to fit the data is selecting the order and estimating coefficients simultaneously. To this end, we shall propose a penalized least-squares estimation based on the SCAD penalty that shrinks some regression coefficients to zero and estimates nonzero coefficients to recover the actual model. A form of penalized least-squares is as follows.
Definition 1. Let {X1, X2, ⋯ , Xn} be imprecisely observed values characterized in terms of uncertain variables, which satisfy the uncertain autoregressive model (1). Then the SCAD estimation for (φ0, φ1, ⋯ , φp) is defined as follows,
where
is called the SCAD penalty function, where a > 2 and λ > 0 are two tune parameters.
Theorem 1.Let {X1, X2, ⋯ , Xn} be imprecisely observed values characterized in terms of independent uncertain variables with regular uncertain distributions Φ1, Φ2, ⋯ , Φn, respectively, which satisfy the uncertain autoregressive model (1). Then the SCAD estimation for (φ0, φ1, ⋯ , φp) is the optimal solution of the following problem
where
for i = 1, 2, ⋯ , p .
Proof. For each t, the uncertain variable
is increasing with respect to Xt and decreasing with respect to Xt-i when φi ≥ 0 or increasing with respect to Xt-i when φi < 0. According to the operational laws of uncertain variable, the inverse uncertainty distributions of the uncertain variable (6) is
where
for i = 1, 2, ⋯ , p and t = p + 1, p + 2, ⋯ , n. By the formula of the second moment, we obtain
Hence the solution of equation (4) is equivalent to that of equation (2). Then the theorem is verified. □
However, finding the estimator of coefficients that minimizes the objective function (4) faces some challenges because the penalty function is not differentiable at the origin. Following Fan and Li [16], we use an iterative algorithm based on a local quadratic approximation to calculate the SCAD estimator for a given λ. Suppose that the initial value is obtained by the following least-squares estimation
where
for i = 1, 2, ⋯ , p and t = p + 1, p + 2, ⋯ , n. If is very close to 0, then set . Otherwise they can be locally approximated by a quadratic function as
where [Pλ (|φj|)] ′ is first derivative of penalty function Pλ (|φj|) and
By using the Taylor expansion of Pλ (|φj|), we get
Therefore, its first and second differential can be represented as
The corresponding Newton Raphson algorithm is presented below.
Initialize parameters. Setting
where
for i = 1, 2, ⋯ , p and t = p + 1, p + 2, ⋯ , n.
For m = 1, 2, ⋯, set .
Iterate Step 2 until convergence and use to denote the final result.
In Step 2 at any iteration, if some is smaller than the cutoff value 10-3, we set . We want to note again that the proposed method’s iterative algorithm is also presented in Fig. 1.
The iterative algorithm of the proposed method.
Furthermore, suppose that ɛt have the same expected value and variance for all t, i.e.,
By the definition of Yang and Liu [20], the expected value of ɛt can be estimated as
and the variance can be estimated as
where is the t-th residual errors that is
In addition, the fitted model is
Then, based on X1, X2, ⋯ , Xn, the forecast uncertain variable of the next uncertain variable Xn+1 can be determined by
and the forecast value μn+1 of Xn+1 is defined by
If we assume that the uncertainty distribution of error term ɛn+1 is known, then the uncertainty distribution of can be obtained as According to the Yang and Liu [20], the α-confidence interval of Xn+1 is
where b is the minimum value such that
Selection of tuning parameter λ
In this section, we present the sum of the squared error approach to select the optimal λ.
As we can see, before we apply the algorithm mentioned above to compute the SCAD estimator, we need to choose the tuning parameter a and λ. The tuning parameter a = 3.7 suggested in Fan and Li [16] will be used in the whole paper. The tuning parameter λ controls the size of the penalty and the number of selected variables. As a result, choosing the proper value of the tuning parameter λ is an important part of the fitting. In this paper, we use the sum of the squared error method to select the optimal tuning parameter λ in the SCAD estimator. The details of the procedures are as follows. The full dataset n is divided into k subsets of approximately the same size, and training and test set are denoted by nv and (nv) c, respectively, for v = 1, 2, ⋯ , k. For each λ and v, we obtain the estimator of φj (j = 0, 1, ⋯ , n) using the training set nv. Then the tuning parameter λ can be selected by minimizing the sum of the squared error SSE (λ),
According to the equation (7), the SSE (λ) can be calculated as
where
for i = 1, 2, ⋯ , p .
Numerical examples
This section will use two numerical examples and a real data example to demonstrate the effectiveness of the proposed method. Furthermore, we make a comparative analysis with other methods.
Example
Example 1. In this example, a set of imprecisely observed values X1, X2, ⋯ , X26 are generated from the following uncertain autoregressive model
Here, we suppose that X1, X2, ⋯ , X26 are independent linear uncertain variables with linear uncertainty distributions Φ1, Φ2, ⋯ , Φ26, respectively, which are listed in Table 1. Let us show how to use the proposed method to determine the model order and estimate the unknown parameters. Assuming that the maximum lag order of the uncertain autoregressive model is p, according to the empirical value, we set p = 5, the corresponding fitted model is
Imprecisely observed data where represents linear uncertain variable
X1
X2
X3
X4
X5
X6
X7
X8
X9
X10
X11
X12
X13
X14
X15
X16
X17
X18
X19
X20
X21
X22
X23
X24
X25
X26
According to the Step 1 in Section 2, the least-squares estimation without penalty function can be obtained as
where
for i = 1, 2, ⋯ , 5 and t = 6, 7, ⋯ , 26. Then, we obtain the least-squares estimation as
According to the Newton Raphson algorithm in Section 2, taking the least-squares estimation as the initial value of SCAD estimation. Next, according to the sum of the squared error method, we divide the 26 data sets into 2 subsets of approximately the same size, the first 10 data sets, and the last 3 data sets are used as training and test set, respectively. Therefore, the optimal tuning parameter λ can be selected via minimizing
where
for i = 1, 2, ⋯ , 5 . Result for SSE (λ) with λ ∈ {1, 2, ⋯ , 10} is given in Table 2 and Fig. 2. As we can see, taking λ = 4 smallest we get the SSE (λ) among λ ∈ {1, 2, ⋯ , 10} . Therefore, we obtain the corresponding SCAD estimation with λ = 4 as
Hence the fitted uncertain autoregressive model is
The results show that the SCAD estimation is more close to true parameter value (φ0, φ1, φ2) = (15, 0.09, 0.84) than least-squares estimation, which enables estimate the correct parameters of the model and improves the precision of forecasting. In the following, we further use the SCAD method’s estimated values to calculate the prediction value and prediction interval of the next period. It follows from Equations (8) and (9) that
Therefore, the forecast value of X27 is
It is almost equal to the actual average value 248.6675 obtained by the model (14). For the confidence level α = 0.80, if we further suppose that the disturbance term ɛ27 is a normal uncertain variable with uncertain distribution , then we get the prediction interval for X27 is
Example 2.
In this example, we consider a set of historical stock prices in AMZN company.
a
Here, we use the linear uncertain variables to describe daily prices over the period from 02 January to 31 December in 2019 and thus n = 252, where at and bt represent the lowest and highest values in the t-th day, t = 1, 2, ⋯ , 252, respectively, which are shown in Fig. 3. To obtain the optimal uncertain autoregressive model for stock price prediction, we assume that the fitted model is
For a set of λ, it follows from the equation (13) and the Newton Raphson algorithm that
where
for i = 1, 2, ⋯ , 5 . Result for SSE (λ) with λ ∈ {1, 2, ⋯ , 10} is given in Table 3 and Fig. 4.
Sum of squared error SSE (λ) for Example 2
SSE (1)
SSE (2)
SSE (3)
SSE (4)
SSE (5)
4702.4820
4655.3256
4656.9152
4659.3069
4661.3951
SSE (6)
SSE (7)
SSE (8)
SSE (9)
SSE (10)
4664.5000
4667.3336
4670.1673
4673.7347
13134.9381
Part of the interval-valued time series of the AMZN stock prices.
Therefore, the tuning parameter λ = 2 is chosen for SCAD estimation and the fitted model as
By using Equations (8) and (9),
and
Therefore, the forecast value of X253, i.e., the stock price on January 2, 2020 is
For the confidence level α = 0.80, if we further suppose that the disturbance term ɛ253 is a normal uncertain variable with uncertain distribution , then we get the prediction interval for X253 is
To further assess the proposed method’s effectiveness, we calculate the stock price forecasts for January 2 to January 30, 2020, by using the least square method and SCAD method, respectively, and the results are shown in Table 4. As we can see, the SCAD estimation method’s performance is better than the least-squares estimation method. The possible reason is that the SCAD estimation method can shrink some nonsignificant coefficients to zero and retain the nonzero coefficients to establish a truly effective model and improve prediction accuracy.
Comparison of SCAD estimation and least-squares estimation method
Time
Actual average values**
SCAD estimation method
Least-squares estimation method
Predicted values
Absolute deviation
Predicted values
Absolute deviation
1/2/2020
1881.08
1861.01
20.07
1860.39
20.70
1/3/2020
1875.35
1850.64
24.71
1849.87
25.48
1/6/2020
1881.85
1856.14
25.71
1854.96
26.88
1/7/2020
1902.97
1853.96
49.00
1852.52
50.45
1/8/2020
1898.72
1855.19
43.53
1853.42
45.30
1/9/2020
1906.81
1854.99
51.82
1852.93
53.88
1/10/2020
1893.47
1855.35
38.12
1852.99
40.48
1/13/2020
1889.4
1855.50
33.90
1852.85
36.55
1/14/2020
1872.83
1855.72
17.11
1852.78
20.05
1/15/2020
1866.98
1855.91
11.06
1852.69
14.29
1/16/2020
1875.81
1856.11
19.70
1852.61
23.20
1/17/2020
1871.95
1856.30
15.65
1852.52
19.42
1/21/2020
1877.14
1856.48
20.65
1852.44
24.69
1/22/2020
1892.92
1856.67
36.25
1852.36
40.56
1/23/2020
1881.37
1856.85
24.52
1852.28
29.08
1/24/2020
1871.22
1857.03
14.19
1852.21
19.01
1/27/2020
1828.17
1857.20
29.03
1852.13
23.96
1/28/2020
1844.07
1857.38
13.31
1852.06
7.99
1/29/2020
1864.89
1857.55
7.34
1851.98
12.90
1/30/2020
1861.74
1857.71
4.03
1851.91
9.83
Total absolute deviation
499.70
544.70
**Average value of the highest and lowest price of the stock.
Comparative analysis
Example 3. In reference [3], researchers gave the ridge estimation method to compute the parameters of the uncertain autoregressive model. They compared the five estimation approaches least-squares estimation, least absolute deviations, Huber estimation, LASSO estimation, and ridge estimation. They concluded that the ridge estimation can better handle outliers with less bias and shrinks faster than the LASSO. However, ridge estimation can’t simultaneously determine the model order and unknown parameters of the uncertain autoregressive model. Hence, they first employed the fixed-origin cross-validation to determine the order of the model and then estimate the model’s unknown parameters. In the following, we employ the ridge method to determine the uncertain autoregressive model and make a comparative analysis with our proposed method by using the imprecise observation data generated by example 1.
First, we select the model order p by fixed origin cross-validation in [23]. Assume that the longest forecast term is L = 3, and the average testing error ATE23 (p) is
where
for i = 1, 2, ⋯ , p and t = 24, 25, 26, respectively. Result for ATE23 (k) with k ∈ {1, 2} and the corresponding parameter estimation are given in Table 5. As we can see, cross-validation suggests p = 1 and the corresponding fitted model is
While the SCAD estimation suggest p = 2 and the fitted model is
Average testing error and ridge estimation for Example 3
p
1
2
ATE23 (p)
29.0073
34.1286
parameterestimation
Thus, it is not necessarily possible to select the real model order according to the cross-validation method. On the other hand, the ridge estimation does shrinkage, but does not reach zero. The difference is that the SCAD estimation method can shrink some nonsignificant coefficients to zero and retain the nonzero coefficients to establish the truly effective model. The method proposed here can be applied to other uncertain statistical contexts.
Conclusion
In this paper, we presented the SCAD estimation method to determine the uncertain autoregressive model’s parameters. This method can get both the order and the unknown coefficients of the uncertain autoregressive model. Furthermore, we presented the sum of the squared error method to find the optimal tuning parameter λ. Finally, a simulation study and a real data example were conducted to verify the effectiveness of the proposed method, and a comparative analysis is analyzed to illustrate the advantage of our approach in the presence of selecting the order and estimating parameters in an uncertain autoregressive model.
Footnotes
Acknowledgments
The authors gratefully acknowledge the financial support provided by the Program for Young Excellent Talents in UIBE (No.18YQ06).
References
1.
Al-SmadiA. and Al-ZabenA., ARMA model order determination using edge detection: A new perspective, Circuits, Systems and Signal Processing24(6) (2005), 723–732.
ChenD. and YangX., Ridge estimation for uncertain autoregressive model with Imprecise observations, International Journal of Uncertainty, Fuzziness and Knowledge-based Systems29(1) (2021), 37–55.
4.
ChenD. and YangX., Maximum likelihood estimation for uncertain autoregressive model with application to carbon dioxide emissions, Journal of Intelligent & Fuzzy Systems40(1) (2021), 1391–1399.
5.
CochraneD. and OrcuttG., Applications of least squares regression to relationships containing autocorrelated errors, Journal of American Statistical Association44 (1949), 32–61.
6.
ReinselG.C., BasuS. and YapS.F., Maximum likelihood estimators in the multivariate autoregressive moving average model from a generalized least-squares viewpoint, Journal of Time Series Analysis13 (1992), 133–145.
7.
BoxG.E.P. and JenkinsG.M., Time series analysis: Forecasting and control, Journal of the American Statistical Association68(342) (1970), 199–201.
8.
LiangG., WilkesD.M. and CadzowJ.A., ARMA model order estimation based on the eigenvalues of the covariance matrix, IEEE Transactions on Signal Processing41(10) (1993), 3003–3009.
9.
SchwarzG., Estimating the dimension of a model, Annals of Statistics6 (1978), 461–464.
10.
WalkerG.T., On periodicity in series of related terms, Proceedings of the Royal Society of London Series A131 (1931), 518–532.
11.
YuleG.U., On a method of investigation periodicities in disturbed series with special reference to Wolfer’s sunspot numbers, Philosophical Transactions of the Royal Society of London226 (1927), 267–298.
12.
AkaikeH., A new look at the statistical model identification, IEEE Transactions on Automatic ControlAC-19(6) (1974), 716–723.
13.
AkaikeH., Statistical predictor identification, Annals of Institute of Statistical Mathematics, 1970, 203–217.
14.
McDonaldJ.B., Partially adaptive estimation of ARMA time series models, International Journal of Forecasting5(2) (1989), 217–230.
15.
DavisR.A., Gauss-Newton and M-estimation for ARMA processes with infinite variance, Stochastic Processes and their Applications63(1) (1976), 75–95.
16.
FanJ. and LiR., Variable selection via nonconcave penalized likelihood and its oracle properties, Journal of the American Statistical Association96(456) (2001), 1348–1360.
17.
FuchsJ.J., On estimating the order of an ARMA process, Automatica23(6) (1987), 779–782.
18.
PaarmannL.D. and KorenbergM.J., Estimation of the parameters of an ARMA signal model based on an orthogonal search, IEEE Transactions on Automatic Control37 (1992), 347–352.
19.
SadabadiM.S., ShafieeM. and KarrariM., Two-dimensional ARMA model order determination, ISA Transactions48 (2009), 247–253.
20.
YangX. and LiuB., Uncertain time series analysis with imprecise observations, Fuzzy Optimization and Decision Making18(3) (2019), 263–278.
21.
YangX. and NiY., Least squares estimation for uncertain moving average model, Communications in Statistics-Theory and Methods, DOI:10.1080/03610926.2020.1713373, 2020.
22.
YangX., ParkG. and HuY., Least absolute deviations estimation for uncertain autoregressive model, Soft Computing24(23) (2020), 18211–18217.
23.
LiuZ. and YangX., Cross validation for uncertain autoregressive model, Communications in Statistics-Simulation and Computation, DOI:10.1080/03610918.2020.1747077, 2020.
24.
ZhangZ.Q., YangX. and GaoJ.W., Uncertain autoregressive model via LASSO procedure, International Journal of Uncertainty, Fuzziness and Knowledge-based Systems28(6) (2020), 939–956.