Water outlet temperature prediction method of nuclear power plant based on echo state network with variable memory length

Abstract

As a new type of energy which is developing vigorously in China, nuclear energy has been widely concerned in all aspects. The circulating water system in the nuclear power plant takes water from seawater, cools the steam engine through the condenser, and then carries waste heat from the outlet to the sea. If the temperature of the outlet is too high, it will not only cause the temperature rise near the water surface of the atmosphere and the ground layer near the shore, but also affect the ecological environment inside the ocean. In this paper, a model based on the echo state network with variable memory length (VML-ESN) is proposed to predict outlet temperature of the nuclear power plant. It can get memory according to the different input autocorrelation characteristic length to adjust the status update equation. The simulation results show that compared with ESN, Leaky-ESN, and Twi-ESN, the proposed model has better prediction performance, with a MAPE of 3.42%. In addition, when the reservoir size is 40, the error of VML-ESN is smaller than that of other models.

Keywords

Nuclear power temperature echo state network prediction

1. Introduction

Nuclear energy is considered as a new variety of energy with the most development value and potential at present due to its advantages of durability, economy, safety and cleanliness. The sustained, healthy and stable development of nuclear energy has provided a strong guarantee for national security and economic development [1, 2, 3]. In China’s nuclear power layout, all nuclear power unit has been in operation and under construction as the coastal sites, the cooling water through the drain into the sea, the quantity of heat of cooling water is not only can cause atmospheric layer near surface and nearshore ground temperatures and many heat after entering receiving waters, and make the water temperature, may be a certain effect on the ecosystem of water body [4, 5]. Therefore, accurate prediction of the outlet temperature of nuclear power plants and timely response measures will help protect the marine ecological environment.

In industrial construction, a lot of research has been done on temperature prediction at home and abroad [6, 7, 8, 9]. In order to predict soil temperature, Li et al. [10] introduced a unique attention-aware long-short-term memory model (ILSTM Soil), which generally performed more accurately than other models. Azari et al. [11] tested six temperature prediction methods, including: Linear Regression (L.R.), K-Nearest Neighbor (KNN), Support Vector Machine (SVM), Artificial Neural Network (ANN), Random Forest (R.F.) and Adaptive Boosting (AdaBoost). The findings demonstrated that ANN performed better than the competition in both training and forecasting temperature values in the Memphis, TN climate. Chen et al. [12] developed a temperature prediction model to compensate incomplete information, and the results of trial showed that the proposed method can cause the error within 2.28%. Qi et al. [13] proposed a prediction method (CRA-ConvLSTM) combining temporal attention mechanism and spatial attention mechanism for sea surface temperature, and the experimental results showed that the model could achieve 0.19 ${{}^{\circ}}$ C and 99.43%, respectively, in terms of the root mean square error and prediction accuracy. Al-Abdullah et al. [14] proposed an ANN method to establish a suitable force and temperature model for cancellous bone. This model estimated the milling force and temperature as a function of feed speed and spindle speed, which was consistent with the experimental results. Li [15] took the abnormal series of monthly mean sea surface temperature in NINO comprehensive area as the input of LSTM neural network, and proposed a forecasting method of mean sea surface temperature in NINO comprehensive area based on LSTM neural network. Gundu and Simon [16] proposed a prediction model based on LSTM. Through statistical analysis of various network models, the network structure suitable for accurate prediction of solar energy and temperature was obtained. Barrios et al. [17] obtained the heat flux and convection coefficient in the milling process artificially, and used two different temperature prediction models based on finite volume and Gaussian method for inverse solution. One model simplified a certain instantaneous temperature of the workpiece to the average temperature, and the other model showed that the workpiece temperature varied with time and space. The results show that the two temperature prediction models can get the heat flux and convection coefficient in milling process well, and can also be used to predict the cutting temperature of workpiece and tool. Compared with the second model, the first one will make the calculated heat flux smaller and the convection coefficient larger. Christian and Patel [18] used k-means clustering and fuzzy knowledge-base to predict the future atmospheric temperature from various meteorological parameters, and established a summer fuzzy knowledge-base including pressure, relative humidity, wind speed and other information, and the prediction error could be effectively controlled at 6%. Lv et al. [19] proposed a hybrid prediction model combining first-principle method and data-driven model was established, and the uncertainty was learned by TS fuzzy modeling and neural network technology based on ELM to accurately predict ladle furnace thermal system. Venkadesh et al. [20] developed ANN model to predict air temperature by performing a genetic algorithm search to determine the optimal duration and prior data resolution for each weather variable. Most of these studies have focused on various industrial temperature forecasts, but few have been involved in nuclear power plant export temperature forecasts.

In summary, a model based on VML-ESN is proposed to obtain better water temperature outlet prediction performance of nuclear power plant. It can adjust the state update equation by obtaining memory length according to different input autocorrelation characteristics, and adopts reservoir instead of hidden layer, with strong short-term memory ability. The remaining part of this paper is organized as follows. In Section 2, the theory of VML-ESN is briefly introduced. Experiments and results for the proposed model in this paper are performed in Section 3. The conclusions of this paper are given in Section 4.

2. Models

2.1 Echo state network (ESN)

ESN [21, 22, 23] is a new type of recurrent neural network, the network structure as shown in Fig. 1. It employs the neurons are connected by a random sparse reservoir of $x(n)$ to replace the hidden layer, reservoir internal weighting matrix of $W$ , to the input $u(n)$ for high-dimensional, nonlinear, said output to $y(n)$ , $W_{in}$ from the input layer to the reservoir of weighting matrix. $W_{\textit{out}}$ is the weight matrix from the reservoir to the output layer, and the feedback weight matrix from the output layer to the reservoir is $W_{fb}$ . ESH only trains the output weights, which overcomes the problems of traditional recurrent neural network, such as complex training algorithm, slow convergence speed and easy to fall into local minimum [22].

Figure 1.

The structure of ESN.

The construction process of ESN includes initialization, training, and use (test). The steps are shown in Fig. 2. First, the initialization operation is performed. Similar to the traditional multilayer perceptron (MLP), the more nodes, the stronger the fitting ability. Since ESN fits the output result linearly only by adjusting the output weight (i.e. $W_{\textit{out}}$ in Fig. 1, we finally determine this $W_{\textit{out}}$ using the state information of the reservoir), the general ESN needs to be much larger than the node size of the general neural network. The next step is to randomly generate the connection matrix, which represents which neurons are connected to each other, as well as the direction and weight of the connection, which is actually the matrix representation of the digraph ( $W$ in Fig. 1). The following scaling matrix is actually a normalization operation. Sometimes we will directly use a scaling factor and multiply the scaling factor by the original randomly generated matrix. It is faster than using eigenvalues to scale, but the accuracy is also greatly lost. For these neural networks, we usually initialize the weights between 0 $\sim$ 1 (or $-$ 1 $\sim$ 1), and finally, randomly generate the input weights $W_{in}$ and the output weights $W_{\textit{out}}$ .

These parameters will affect the short-term memory of the network. The smaller the input weight is and the closer the spectral radius of the internal matrix is to 1, the longer the network short-term memory time is. But as well as enhancing memory, it also makes the network less able to model rapidly changing systems. The second step is to train. It is worth noting that this idling process is essentially initializing the state of the reservoir. The internal connection of the reservoir is random, and the noise of the state of the reservoir obtained from the initial input sequence will be relatively large. Therefore, some data will be used to initialize the state of the reservoir to reduce the impact of noise.

The discrete equation of ESN is as follows:

$\displaystyle x(n+1)=f(W_{in}u(n+1)+Wx(n)+W^{fb}y(n))$ (1) $\displaystyle y(n)=g(W_{\textit{out}}[x(n);u(n)])$ (2)

where $f$ denotes the S-type function (tanh function), $g$ denotes the activation function (identity function), and [;] denotes the connection of two vectors.

Figure 2.

Procedure for constructing an ESN.

2.2 VML-ESN

The water outlet temperature prediction method using VML-ESN [24] is presented in this study. It has the capacity for short-term memory and the ability to adaptively alter the state update equation according to the autocorrelation features of the input signal. Varying types of input signals have different autocorrelations. The reservoir state update equation is adjusted to take into account input signal autocorrelation in order to get the best performance for various input signals. As a result, the calculation for the reservoir status update for the ESN can be changed as follows:

$\displaystyle x(n+1)=\sum\limits_{i=1}^{p}{a_{i}x(n-i+1)}+a_{f}f(W_{in}u(n+1)+% Wx(n)+W^{fb}y(n))$ (3) $\displaystyle y(n)=g(W_{\textit{out}}[x(n);u(n)])$ (4)

where $p$ is the memory length associated with the input signal’s autocorrelation, $a_{i}$ is the corresponding coefficients of $x(n-i+1)$ [24]. The model that satisfies Eqs (3) and (4) is referred to as VML-ESN. It introduces parameter $a_{i}$ , which suggests autocorrelation of the water temperature, and employs a past state display to highlight the current input and the effects of numerous previous inputs on the current state.

When $a_{i}=0$ ( $i=1,2,\ldots,p$ ), the VML-ESN changes to the standard ESN. Otherwise, the VML-ESN turns into a Leaky-ESN. Leaky-ESN and ESN are thus two particular instances of VML-ESN. The reservoir units of Leaky-ESN are all identical leak credit units with a single unit delay, as is well known. VML-ESN reservoir units, however, are various leak integral units with numerous units of delay.

Since VML-ESN is a special ESN, it needs to meet the echo characteristics. A necessary condition for the VML-ESN to satisfy the echo state characteristic is given so as to guarantee that the model adheres to the echo state characteristic.

For the discrete VML-ESN model, the following conditions are satisfied:

$f$ is an $S$ function or a linear function (e.g. tanh function);

$g$ is a bounded function (e.g. identity function);

$\left|{\left|{a_{1}}\right|+\left|{a_{2}}\right|+\left|{a_{3}}\right|+\cdot% \cdot\cdot+\left|{a_{p}}\right|+\delta_{\max}}\right|<1$ , ( $\delta_{\max}$ is the largest singular value of weight matrix).

It indicates that the VML-ESN model has the echo state characteristic.

3. Experiments and results

3.1 Data sources

To verify the accuracy of the prediction model based on VML-ESN, this paper refers to the data from the data acquisition system established by two nuclear power plants for simulation verification. Two nuclear power plants are located near the sea, and they are 1000 meters apart. The original drainage channel of the nuclear power plants is 600 meters long, and the circulating cooling water is discharged into the sea in the east-south direction. The design of the open canal is applied to the design of the water intake and the drainage outlet, i.e., the drainage of the two nuclear power plants is combined into one drainage system. The drainage outlet of the original nuclear power plant is extended to the drainage outlet of the new nuclear power plant, and the drainage breakwater is about 1190 meters long and discharged in the direction of 60 degrees to the north by east. The amount of drainage from the two power stations is now 190 m ${}^{3}$ /s, with a temperature rising 1 $\sim$ 2 ${{}^{\circ}}$ C at the drainage outlet.

Figure 3.

Location of water outlet.

In this paper, under different weather conditions from June to September 2020, outlet temperature and seawater temperature were recorded every 30 minutes. Using this 1000 samples temperature data to train the VML-ESN prediction model. The temperature data are shown in Table 1.

Table 1

Temperature data

Time	Outlet temperature ( ${{}^{\circ}}$ C)	Seawater temperature ( ${{}^{\circ}}$ C)
7:00	27.7	26.47
7:30	28.1	26.67
8:00	28.5	27.50
⋮	⋮	⋮
16:30	26.1	25.09
17:00	27.4	26.3

3.2 Analysis of simulation results

This paper focuses on the simulation of the outlet temperature prediction model through Matlab by establishing a time series with a training sample length of 1000, an initial sample length of 100, a prediction sample length of 100, a network training with an input dimension of 5, an output dimension of 1, a spectral radius of 0.8, and a sparsity of 2%. Firstly, the reservoir size is 40. According to the requirement of test error and test time, the memory length is 2. Then the VML-ESN reservoir state equation is updated as:

$\displaystyle x(n+1)=a_{1}x(n)+a_{2}x(n-1)+f(W_{in}u(n+1)+Wx(n)+W^{fb}y(n))$ (5)

The reservoir parameters of the VML-ESN based outlet temperature prediction model are shown in Table 2.

Table 2

The reservoir parameters of the VML-ESN

Reservoir size	Spectral radius	Sparsity	Training sample length
40	0.8	2%	1000

To facilitate the observation of the advantages of VML-ESN, BPNN [25], LSTM [26], ESN [22], Leaky-ESN [27], and Twi-ESN [28] are introduced in this paper, and the outlet temperature prediction model is simulated with each of these ESN. NRMSE and MAPE are used as the prediction performance indexes, which are as follows:

$\displaystyle\textit{NRMSE}=\sqrt{\frac{\left\langle{||y(n)-d(n)||^{2}}\right% \rangle}{\left\langle{||d(n)-\left\langle{d(n)}\right\rangle||^{2}}\right% \rangle}}$ (6) $\displaystyle\textit{MAPE}=\frac{100}{L_{a}}\sum\limits_{j=1}^{L_{a}}{\frac{|y% (j)-d(j)|}{d(j)}}\%$ (7)

where $y(n)$ denotes the predicted sample, $d(n)$ denotes the actual sample, and $L_{a}$ denotes the sample length. The results of the prediction accuracy of the six methods are listed in Table 3.

Table 3

Comparison of the prediction accuracy of the six methods

Predictive models	Reservoir size (SP)	Train NRMSE	Test NRMSE	Test MAPE(%)
BPNN [25]	–	2.53423	2.24141	–
LSTM [26]	–	0.95882	0.89324	–
ESN [22]	40	0.28319	0.55843	3.50
Leaky-ESN [27]	40	0.29918	1.0352	6.00
Twi-ESN [28]	40	0.88435	0.66581	4.91
VML-ESN	40	0.24624	0.45572	3.42

Table 4

Evaluation indexes under different reservoir sizes

Reservoir size (SP)	Sample length	Evaluation indicators (test MAPE)
20	1000	0.0377
30	1000	0.0352
40	1000	0.0342
50	1000	0.0398
60	1000	0.0361
70	1000	0.0365
80	1000	0.0390
90	1000	0.0343

Table 5

Training errors of the four methods

Methods	SP $=$ 20	SP $=$ 30	SP $=$ 40	SP $=$ 50	SP $=$ 60	SP $=$ 70	SP $=$ 80	SP $=$ 90
ESN	0.055	0.041	0.028	0.027	0.026	0.026	0.025	0.026
Leaky-ESN	0.051	0.038	0.029	0.029	0.028	0.027	0.026	0.026
Twi-ESN	0.099	0.098	0.088	0.087	0.086	0.085	0.084	0.082
VML-ESN	0.035	0.034	0.024	0.024	0.024	0.025	0.025	0.025

Table 6

Test errors of the four methods

Methods	SP $=$ 20	SP $=$ 30	SP $=$ 40	SP $=$ 50	SP $=$ 60	SP $=$ 70	SP $=$ 80	SP $=$ 90
ESN	0.058	0.056	0.055	0.055	0.054	0.054	0.053	0.053
Leaky-ESN	0.108	0.105	0.103	0.099	0.098	0.095	0.094	0.095
Twi-ESN	0.068	0.067	0.066	0.065	0.064	0.064	0.063	0.062
VML-ESN	0.052	0.048	0.045	0.045	0.046	0.046	0.046	0.047

Figure 4.

Target and the predicted results of the four methods.

As can be seen from Table 3, the training error and testing error of VML-ESN are smaller than the other three echo state networks with the same reservoir size, and the testing error of VML-ESN is reduced by 1.22 times, 2.27 times and 1.46 times compared with the standard ESN, Leaky-ESN and Twi-ESN, respectively, and the prediction accuracy is also the highest with MAPE $=$ 3.42%, which is 1.02 times higher than the standard ESN, 1.75 times higher than the Leaky-ESN, and 1.43 times higher than the Twi-ESN.

Figure 5.

MAPE with different reservoir sizes.

Figure 6.

The train and test NEMSE for four methods.

In Fig. 4, it can be observed that the curve of VML-ESN match with the curve of the original data, and the relative error of VML-ESN is the smallest, so it can be indicated that the temperature prediction model based on VML-ESN is the one with the least deviation from the original data among this four methods, and in the actual measurement of nuclear power plants, VML-ESN can overcome the influence brought by environmental factors, and this method is better than the other three echo state networks and can predict the outlet temperature of nuclear power plants more accurately.

There are many factors that affect the accuracy of outlet temperature prediction model. Here we adopt the orthogonal experimental design method, which is a multi-factor and multi-level experimental design method. In this paper, the experiments of orthogonal simulation are carried out by choosing different reservoir size, and changing the size of the reservoir (such as making SP $=$ 20, 30, 40 … 90) to analyze the prediction performance of VML-ESN by the prediction performance index MAPE, and the results are shown in Table 4.

In Table 4 and Fig. 5, the mean absolute error (MAE) of the prediction model is the smallest when the size of the reservoir SP $=$ 40, and the MAPE of the prediction model is the largest when SP $=$ 80. However, there is no certain regularity in the selection of the reservoir size, and the trial-and-error method is usually used to give the appropriate reservoir size. In future, we will choose a suitable optimization method to optimize the size of the reservoir for VML-ESN to obtain a higher prediction accuracy.

The size of the reservoir is still varied and the prediction model is analyzed by comparing the NRMSE of four ESN models with different reservoir sizes.

With different reservoir sizes, it is easy to observe from Fig. 6 that the test error and train error of VML-ESN are much smaller than those of the other three ESN models when the reservoir size SP $=$ 40, so it can be shown that VML-ESN has better prediction results compared with the other three networks because of its relatively smaller reservoir size.

4. Conclusions

The nuclear power plant should design the circulating water system and carefully select the location of the intake and discharge outlets in advance in the pre-construction planning layout based on the safety and economy as well as environmental protection aspects, so that the economic and ecological benefits can be maximized. According to the characteristics of temperature change, an outlet temperature prediction model with VML-ESN is proposed in this paper. Compared with the ESN, the model increases the memory length and adaptively adjusts the standby reservoir state update equation. The training and test errors of VML-ESN are the smallest compared with the other three ESN models under the same reservoir size condition, and the prediction accuracy is also the highest, with an average NRMSE of 0.047.

References

Christoforidis

Katrakilidis

Karakotsios

Dimitriadis

. The dynamic links between nuclear energy and sustainable economic growth. Do institutions matter? Prog Nucl Energy. 2021; 139: 103866.

Mathew

. Nuclear energy: A pathway towards mitigation of global warming. Prog Nucl Energy. 2022; 143: 104080.

Ozcan

Ulucak

. An empirical investigation of nuclear energy consumption and carbon dioxide (CO2) emission in India: Bridging IPAT and EKC hypotheses. Nucl Engg Technol. 2021; 53(6): 2056–2065.

Wang

Jia

Ding

, et al. The ecological mechanisms of Acetes blooms as a threat to the security of cooling systems in coastal nuclear power plants. J Coastal Conserv. 2021; 25(6): 1–10.

Tai

Tian

Lei

. A water-heat combined supply system based on waste heat from a coastal nuclear power plant in northern China. Appl Therm Engg. 2022; 200: 117684.

Han

Zhao

Zhang

Xue

. Modeling for indoor temperature prediction based on time-delay and Elman neural network in air conditioning system. J Build Engg. 2021; 33: 101854.

Feigl

Lebiedzinski

Herrnegger

Schulz

. Machine-learning methods for stream water temperature prediction. Hydrol Earth Syst Sci. 2021; 25(5): 2951–2977.

Zhou

Zhang

Liu

Zhang

. Prediction of the ground temperature variations caused by the operation of GSHP system with ANN. Geothermics. 2021; 95: 102140.

Grochowalski

Jachymek

Andrzejczyk

Klajny

Widuch

Morkisz

, et al. Towards application of machine learning algorithms for prediction temperature distribution within CFB boiler based on specified operating conditions. Energy. 2021; 237: 121538.

10.

Zhu

Shangguan

Wang

. An attention-aware LSTM model for soil moisture and soil temperature prediction. Geoderma. 2022; 409: 115651.

11.

Azari

Hassan

Pierce

Ebrahimi

. Evaluation of machine learning methods application in temperature prediction. Environ Engg. 2022; 8: 1–12.

12.

Chen

Zhao

Zhang

Shen

Liu

. State of health estimation for lithium-ion batteries based on temperature prediction and gated recurrent unit neural network. J Power Sources. 2022; 521: 230892.

13.

Song

Huang

Geng

. Sea surface temperature prediction algorithm combined with residual spatial-temporal attention mechanism. J Graphics. 2022; 43(4): 677.

14.

Al-Abdullah

KIA

Abdi

Lim

Yassin

. Force and temperature modelling of bone milling using artificial neural networks. Meas. 2018; 116: 25–37.

15.

. Sea surface temperature prediction model based on long and short-term memory neural network. IOP Conf Ser Earth Enviro Sci. 2021; 658(1): 012040.

16.

Gundu

Simon

. Short term solar power and temperature forecast using recurrent neural networks. Neural Processing Lett. 2021; 53(6): 4407–4418.

17.

Barrios

ANS

Silva

JBC

Rodrigues

Coelho

Junior

Matsumoto

. Modeling heat transfer in die milling. Appl Therm Engg. 2014; 64(1-2): 108–116.

18.

Christian

Patel

. Potential of k-means clustering-based fuzzy logic for prediction of temperature in ambient atmosphere. Arabian J Sci Engg. 2015; 40(1): 227–234.

19.

Mao

Yuan

Jia

. Pruned bagging aggregated hybrid prediction models for forecasting the steel temperature in ladle furnace. Steel Res Int. 2014; 85(3): 405–414.

20.

Venkadesh

Hoogenboom

Potter

McClendon

. A genetic algorithm to refine input data selection for air temperature prediction using artificial neural networks. Appl Soft Comput. 2013; 13(5): 2253–2260.

21.

Aboelmaged

. Knowledge sharing through enterprise social network (ESN) systems: Motivational drivers and their impact on employees’ productivity. J Knowl Manage. 2018; 22(2): 362–383.

22.

Shen

Cottrell

. DeePr-ESN: A deep projection-encoding echo-state network. Inf Sci. 2020; 511: 152–171.

23.

Qin

. Effective passenger flow forecasting using STL and ESN based on two improvement strategies. Neurocomputing. 2019; 356: 244–256.

24.

Lun

Yao

. A new echo state network with variable memory length. Inf Sci. 2016; 370: 103–119.

25.

Wang

Chau

Chen

Liu

. A comparison of BPNN, GMDH, and ARIMA for monthly rainfall forecasting based on wavelet packet decomposition. Water. 2021; 13(20): 2871.

26.

Lindemann

Maschler

Sahlab

Weyrich

. A survey on anomaly detection for technical systems using LSTM networks. Comput Ind. 2021; 131: 103498.

27.

Lun

Yao

. The modified sufficient conditions for echo state property and parameter optimization of leaky integrator echo state network. Appl Soft Comput. 2019; 77: 750–760.

28.

Lun

Yao

. A novel model of leaky integrator echo state network for time-series prediction. Neurocomputing. 2015; 159: 58–66.