Integrating expert knowledge for Bayesian network structure learning based on intuitionistic fuzzy set and Genetic Algorithm

Abstract

The Bayesian network (BN) structure learning from the observational data has been proved to be a NP-hard problem. The expert knowledge is beneficial to determine the BN structure, especially when the data are scarce and the related variables are huge in the researched domain. In this paper, we propose a new BN structure learning method by integrating expert knowledge. On the one hand, to improve the performance of expert knowledge usage, the intuitionistic fuzzy set (IFS) is introduced to express and integrate the expert knowledge. The determination of BN priori structure is transformed into the group decision making problem. On the other hand, the improved Bayesian information criterion score function and the Genetic Algorithm search algorithm are used to obtain the most suitable structure under the constraints from the priori structure. Some experiments demonstrate the validity of proposed scheme and compare the performance with the existing research results. The obtained BN structure owns better performance. The more the quantity of expert knowledge is, the better the performance of BN structure learning would be. Finally, the proposed method is applied to the thickening process of gold hydrometallurgy to solve the practical problem.

Keywords

Bayesian network structure learning expert knowledge Genetic Algorithm intuitionistic fuzzy set

1. Introduction

Bayesian network (BN) is an effective intelligent method to represent the dependent relationship and joint probability distribution among variables in the form of a directed acyclic graph (DAG), which has been widely applied to all kinds of fields, such as, genomics [2], industry [13], risk analysis [4], manufacturing [29], ecology social sciences [26] and so on. BN is the uncertain knowledge representing and reasoning model, which consists of two components: structure and parameter. In this paper, we focus on the structure learning.

Automatic structure learning of BNs from observational data has been developed in a great deal of researches, which mainly include two types: constraint-based methods and score-based methods. The constraint-based methods identify the dependent and independent relationships among variables by conditional independence tests. The score-based methods use a search algorithm to find the structure with the highest score based on the score function. In this paper, we mainly focus on the latter one. For the score-based methods, first of all, the score function needs be determined. Various score functions have been proposed including Bayesian Dirichlet equivalence [2], minimum description length [21, 36] and Bayesian information criterion (BIC) [30]. Furthermore, a search algorithm needs to be specified to find the BN structure with the highest score. When the number of random variables increases in the problem domain, the number of possible structures will grow exponentially. When the sample size is small, it is more difficult to determine the structure. It has been proved to be a NP-hard problem for the determination of structure which best adapts to the observational data. A variety of search algorithms have been proposed, such as heuristic search techniques [1, 11], evolutionary algorithms [16, 19, 20]. Among the search algorithms, as the global optimization algorithms, the evolutionary algorithms own strengths to determine the optimal structure.

For the BN structure learning, the introduction of expert knowledge is beneficial to determine the relationships among the variables. Especially when the data are scarce and the related variables are huge in the researched domain, the introduction of expert knowledge provides an effective way to reduce the inherent uncertainty of structure. Under the restrictions from the expert knowledge, the search spaces of possible structures are reduced greatly. Only the unknown relationships need to be learned, and the best BN structure can be obtained as soon as possible. This is an excellent approach to boost the performance of structure learning methods. The learned results will fit the practical domain better and reduce the search cost. How to integrate the expert knowledge and the automatic learning algorithm is an important problem to solve.

In this paper, we propose a new method of BN structure learning by integrating expert knowledge and observational data based on the score-based method. This method includes two parts: the determination of BN priori structure based on the expert knowledge and the structure searching by the observational data under the constraints from the priori structure. For the former one, the intuitionistic fuzzy set (IFS) method is applied to express and integrate the opinions of group experts. The BN priori structure can be determined by the group decision making (GDM) based on IFS. For the latter one, the score-based method is used to search for the most suitable structure under the constraints from the priori structure. For the score-based method, the BIC score function and Genetic Algorithm (GA) are improved to satisfy the expert knowledge in this paper.

The contribution of this paper reflects on two aspects. On the one hand, in the BN structure learning framework, the expert knowledge is introduced to increase the performance of structure learning. To improve the performance of expert knowledge usage, the IFS is introduced to express and integrate the expert knowledge. The determination of BN priori structure is dealt with in the intuitionistic fuzzy environment, which is transformed into the GDM problem. On the other hand, the score-based method is used to search for the most suitable structure under the constraints from the priori structure. In this process, the BIC score function and the GA are improved to integrate the expert knowledge, which provides a new angle to improve the performance of BN structure learning based on GA.

The remaining sections of this paper are organized as follows. The related work is discussed in Section 2. Section 3 discusses the determination of BN priori structure. The corresponding decision scheme is analyzed in detail. In Section 4, the score-based method is used to search for the most suitable structure under the constraints from the priori structure. The BIC score function and GA are improved to integrate the expert knowledge. In Section 5, some experiments are provided to demonstrate the validity of proposed scheme and compare the performance with the existing research results. The influence of quantity of expert knowledge also is discussed. The proposed method is applied to the thickening process of gold hydrometallurgy to solve the practical problem. Finally, the conclusions are presented in Section 6.

2. Related work

Many researches have been proposed to integrate the expert knowledge and the automatic learning algorithm [6, 7, 12, 22, 25]. In the paper [12], the structural restrictions can be represented as the following forms: the existence or absence of arcs and the causal ordering restrictions. If the expert holds the view that the presence possibility of an edge is high, the probability of BN structure including this edge is high. The priori probability distributions of BN structure can be obtained from expert knowledge in a certain degree. The paper [25] proposes an interactive approach to integrate domain/expert knowledge. The information from the data learning provides the support for obtaining the domain/expert knowledge. The active interaction plays its role at the different stages of learning process. However, the above researches are not enough, and two problems need to be discussed further. On the one hand, in the BN structure learning environment, the problem of extraction, expression and integration of expert knowledge needs to be improved. In the specific researched background, when the expert knowledge on some relationships among the variables is relatively obvious, the extraction of expert knowledge is easy. However, the expert knowledge is ambiguous, imprecise and subjective in nature. When the specific relationship is not expressed, the extraction of expert knowledge changes to be difficult. As the strengths and knowledge degrees of every expert are different, when the group experts express their opinions on the same problem, the determination of relationship between two variables (i.e. two nodes in the BN) is more difficult. Therefore, it is necessary to search for the method to obtain more reasonable relationship between two variables by integrating the opinions of group experts. The obtained relationship will be more explicit and reliable. In such cases, it is not proper to use the crisp numbers to express the expert knowledge. The IFS theory proposed by Atanassov [3] can express and integrate the expert knowledge in a proper way. As a generalization of FS theory, it can deal with vagueness, ambiguity and hesitation. Except the membership function, the IFS are associated with the non-membership function and the hesitancy function. The hesitancy degree of IFS can express the ignorance, i.e. the lack of knowledge for the membership degree. In the process of decision making or evaluation, the hesitancy degree plays an important role when the experts cannot express their opinions in an accurate way. On the other hand, the way of integrating expert knowledge and observational data for the BN structure learning methods needs to be researched further. Taking the score-based methods using the GA search method as the example, GA is the common global search method, which is effective to search for the most suitable BN structure from the population by the score function and evolution operators. The BN structure learning method based on the GA has been developed in the paper [14]. It uses GA as the search method and analyzes the behaviour of algorithm by the different parameters. Since then, many researches have been proposed to improve the BN structure learning method based on the GA from the different angles [8, 18, 32]. A cooperative coevolutionary GA is proposed to learn BN structure in the paper [8]. The paper [32] proposes a new hybrid BN structure learning algorithm by reducing the search space using mutual dependencies and applying the GA to search over the reduced space. The paper [18] proposes a chain-model GA approach to reduce the complexity of BN structure learning. However, few researches are developed to improve the BN structure learning methods based on the GA from the angle of integrating the expert knowledge. Therefore, how to integrate the expert knowledge and GA to improve the performance of BN structure learning is an important problem to solve. In this process, two problems need to be considered: the determination of initial population based on the expert knowledge; the improvement of fitness function satisfying the constraints from the expert knowledge.

3. BN priori structure determination

3.1 Problem description

For the specific problem of complex industrial environment, for example, when monitoring and diagnosing whether an abnormal event appears or not, or predicting some key parameters related with the performance of system, the relevant variables are limited and the expert knowledge about the relationships is available. It makes the collection and integration of expert knowledge possible. The extraction of expert knowledge can be realized by inquiring the experts about the relationship between two variables. The specific process is as follows. First of all, the related variables need to be determined as the nodes of BN in the problem domain, and even the variables can be allocated at different levels of causality. The knowledge about the relationships among the nodes needs to be collected from the experts. Taking the relationship between two nodes $X$ and $Y$ as the example, it includes four possible conditions: $X\to Y$ ; $X\leftarrow Y$ ; $X\neq Y$ ; $X\cong Y$ . $X\to Y$ represents that $X$ points to $Y$ , that is to say, $X$ is the parent node (cause node) and $Y$ is the children node (result node). In the same way, $X\leftarrow Y$ represents that $Y$ points to $X$ , that is to say, $Y$ is the parent node (cause node) and $X$ is the children node (result node). $X\neq Y$ represents that the two nodes have no relationship. $X\cong Y$ represents that there is a relationship between the two nodes, but the direction of arc is uncertain. The group experts are invited to express their opinions on the four possible conditions. The membership degree of every condition is obtained. The sum of membership degrees for the four conditions is equal to one. The IFS own strengths on the aspect of representation of expert knowledge. Therefore, we use the IFS to form the initial knowledge base. In the intuitionistic fuzzy environment, except the membership degree, the non-membership degree needs to be obtained. To obtain the non-membership degree, the hesitation margin $\pi_{A}(x)$ is determined at first. The hesitancy degree $\pi_{A}(x)$ can be obtained based on the evaluation results of experts from the third institution. The evaluations are based on the understanding on the researched domain and the academic level of every expert. The guideline of determining the hesitancy degree can be described that the bigger the value of evaluation is, the smaller the hesitancy degree is. The change range of hesitancy degree $\pi_{A}(x)$ is $\left[{0,1-\mu_{A}(x)}\right]$ . Based on the above qualitative guideline, the hesitancy degree can be determined. Based on the relationship $\mu_{A}(x)+\pi_{A}(x)+\nu_{A}(x)=1$ , the non-membership degree can be determined by $\nu_{A}(x)=1-\mu_{A}(x)-\pi_{A}(x)$ . Finally, the initial knowledge base is established.

In a word, the four conditions are equivalent to four alternatives, and the group experts express their opinions on the four alternatives. The most suitable relationship between two nodes needs to be determined by integrating the opinions from the group experts. Therefore, the determination of BN priori structure based on the expert knowledge can be transformed into the GDM problem to solve based on IFS. A GDM problem can be concisely expressed in the matrix format as follow:

$\displaystyle D=(\langle\mu_{ij},\nu_{ij}\rangle)_{k\times m}=\left({{\begin{% array}[]{*{20}c}{\langle\mu_{11},\nu_{11}\rangle}&{\langle\mu_{12},\nu_{12}% \rangle}&\cdots&{\langle\mu_{1m},\nu_{1m}\rangle}\\ {\langle\mu_{21},\nu_{21}\rangle}&{\langle\mu_{22},\nu_{22}\rangle}&\cdots&{% \langle\mu_{2m},\nu_{2m}\rangle}\\ \vdots&\vdots&\ddots&\vdots\\ {\langle\mu_{k1},\nu_{k1}\rangle}&{\langle\mu_{k2},\nu_{k2}\rangle}&\cdots&{% \langle\mu_{km},\nu_{km}\rangle}\\ \end{array}}}\right)$ (1)

which $D$ represents the intuitionistic fuzzy decision matrix. The column represents the alternatives $A=\{a_{1},a_{2},\ldots,a_{m}\},(1\leqslant j\leqslant m)$ and the row represents the experts $E=\{E_{1},E_{2},\ldots,E_{k}\},(1\leqslant i\leqslant k)$ . $\langle\mu_{ij},\nu_{ij}\rangle$ represents the evaluation of expert $E_{i}$ for the alternative $a_{j}$ . Based on the above analysis, we can obtain that $m=4$ .

3.2 Proposed scheme

To select the most appropriate relationships among the BN nodes, the scheme for the BN priori structure determination is described by the following framework. Taking the relationship between two nodes as the example, in Fig. 1, for the step 1, the group expert knowledge is obtained in the form of intuitionistic fuzzy decision matrix shown in Eq. (1).

Figure 1.

The scheme for the BN priori structure determination.

For the step 2, we use the knowledge measure [17] to determine the weight of every expert. The knowledge measure of every expert can be obtained by the following equation

$\displaystyle K_{i}=1-[1/(2m)]\sum\limits_{j=1}^{m}{(1-\left|{\mu_{ij}-\nu_{ij% }}\right|)}(1+\pi_{ij})(1\leqslant i\leqslant k)$ (2)

The weights of experts are determined by following equation

$\displaystyle\omega_{K_{i}}=K_{i}/\sum\limits_{i=1}^{k}{K_{i}}(1\leqslant i% \leqslant k)$ (3)

For the step 3, the aggregation operator from the Dempster-Shafer (D-S) theory is chosen to aggregate the opinions from different experts, which is shown in Eq. (4) [34].

Let $A_{1}=\langle\mu_{1},\nu_{1}\rangle$ and $A_{2}=\langle\mu_{2},\nu_{2}\rangle$ be two intuitionistic fuzzy values (IFVs). A combination operation on $A_{1}$ and $A_{2}$ , denoted by $A_{1}*A_{2}$ , is defined by:

$\displaystyle A_{1}*A_{2}=\langle(\mu_{1}(1-\nu_{2})+\mu_{2}\pi_{1})/(1-\mu_{1% }\nu_{2}-\mu_{2}\nu_{1}),(\nu_{1}(1-\mu_{2})+\nu_{2}\pi_{1})/(1-\mu_{1}\nu_{2}% -\mu_{2}\nu_{1})\rangle$ (4)

The hesitancy degree of $A_{1}*A_{2}$ is:

$\displaystyle\pi_{A_{1}*A_{2}}=(\pi_{1}\pi_{2})/(1-\mu_{1}\nu_{2}-\mu_{2}\nu_{% 1})$ (5)

Before aggregation operation, the IFVs need be handled by the discounting operation. We use the weights of experts as the discount coefficient. The specific aggregation process can refer to the related research results [10, 34].

For the step 4, the technique for order preference by similarity to ideal solution (TOPSIS) method is used to select the most suitable relationships among the nodes. Firstly, the distances from the alternative relationship to the ideal solutions should be calculated. The ideal solutions include the positive-ideal solution and negative-ideal solution, which are represented as $\langle 1,0\rangle$ and $\langle 0,1\rangle$ . The distance is calculated by the following equation [31]

$\displaystyle d=1/2(\left|{\mu_{1}-\mu_{2}}\right|+\left|{\nu_{1}-\nu_{2}}% \right|+\left|{\mu_{2}+\nu_{2}-\mu_{1}-\nu_{1}}\right|)$ (6)

The relative closeness coefficient $\xi_{j}(1\leqslant j\leqslant m)$ of every alternative relationship can be calculated by the following equation [5]

$\displaystyle\xi_{j}=d_{j-}/(d_{j-}+d_{j+})(1\leqslant j\leqslant m)$ (7)

where $d_{j+}$ represents the distance from the jth alternative to the positive-ideal solution. $d_{j-}$ represents the distance from the jth alternative to the negative-ideal solution. The relative closeness coefficient $\xi_{j}(1\leqslant j\leqslant m)$ reflects the strength of expert knowledge. The bigger the value of relative closeness coefficient is, the more certain the expert knowledge is. Therefore, based on the obtained all relative closeness coefficients, we rank them and find the most suitable relationship between two nodes with the highest value.

Repeating the above steps, the BN priori structure can be obtained, which is represented by matrix $A$ and shown in the following form.

$\displaystyle A=\left[{{\begin{array}[]{*{20}c}0&{a_{12}}&\cdots&{a_{1n}}\\ {a_{21}}&0&\cdots&{a_{2n}}\\ \vdots&\vdots&\ddots&\vdots\\ {a_{n1}}&{a_{n2}}&\cdots&0\\ \end{array}}}\right]$

where $a_{pl}\in\{0,1,*\oplus\}(p=1,2,\ldots n;l=1,2,\ldots n)$ , $n$ is the number of variables in the BN. $a_{pl}$ represents the relationship between the variable $p$ and the variable $l$ . The possible values of $a_{pl}$ include 0, 1, * and $\oplus$ . $a_{pl}=0$ represents that there is no arc from variable $p$ to variable $l$ ; $a_{pl}=1$ represents that there is an arc from variable $p$ to variable $l$ ; $a_{pl}=*$ represents that there is no expert knowledge about the relationship between the variable $p$ and the variable $l$ ; $a_{pl}=\oplus$ represents that there is a relationship between the variable $p$ and the variable $l$ , but the direction of arc is uncertain.

For the determination of matrix $A$ , an example is shown. Based on the analysis in the Section 3.1, the relationship between two nodes in BN includes four conditions. The node $X$ represents the variable $p$ , and the node $Y$ represents the variable $l$ . $\xi_{1}$ , $\xi_{2}$ , $\xi_{3}$ and $\xi_{4}$ are the relative closeness coefficients of $X\to Y$ , $X\leftarrow Y$ , $X\neq Y$ and $X\cong Y$ respectively. The following conditions can be obtained:

(1)

If the $\xi_{1}$ owns the highest value and $\xi_{1}\geqslant 0.5$ , $a_{pl}=1$ and $a_{lp}=0$ ;

(2)

If the $\xi_{2}$ owns the highest value and $\xi_{2}\geqslant 0.5$ , $a_{pl}=0$ and $a_{lp}=1$ ;

(3)

If the $\xi_{3}$ owns the highest value and $\xi_{3}\geqslant 0.5$ , $a_{pl}=0$ and $a_{lp}=0$ ;

(4)

If the $\xi_{4}$ owns the highest value and $\xi_{4}\geqslant 0.5$ , $a_{pl}=\oplus$ and $a_{lp}=\oplus$ ;

(5)

If $\xi_{j}(1\leqslant j\leqslant 4)$ with the highest value is smaller than 0.5, or no expert knowledge is available for the relationship between the variable $p$ and the variable $l$ , $a_{pl}=*$ and $a_{lp}=*$ .

4. BN structure learning by integrating the expert knowledge

4.1 Improved score function

In this paper, we use the BIC as the score function which is proposed in the paper [28]. When the expert knowledge is available, the BIC score function should be improved to satisfy it. The traditional BIC score function consists of two parts, an accuracy term and a complexity term, which is shown as follow

$\displaystyle\textit{score}_{\textit{BIC}}=\log P(D|\Theta)-(d/2)\log m=\sum% \limits_{i=1}^{n}{\sum\limits_{j=1}^{q_{i}}{\sum\limits_{k=1}^{r_{i}}m_{% \textit{ijk}}\log(m_{\textit{ijk}}/m_{ij})}}-(1/2)\log m\sum\limits_{i=1}^{n}{% q_{i}(r_{i}-1)}$ (8)

where $D$ represents the data; $\Theta$ represents the parameter of model; $d$ represents the dimension of model and $m$ represents the sample size. $d$ is valued as $\sum_{i=1}^{n}{q_{i}(r_{i}-1)}$ . $n$ represents the number of nodes in BN model; $q_{i}$ represents the number of candidate combination states of father nodes of $i$ th node; $r_{i}$ represents the number of candidate states of $i$ th node; $m_{\textit{ijk}}$ represents the number of records which satisfy the condition that the $i$ th node is in the $k$ th state and its father node set is in the $j$ th state; $m_{ij}$ represents the number of records which satisfy the condition that the father node set of $i$ th node is in the $j$ th state. In Eq. (8), the first term is the accuracy term, which is used to measure how well a particular model fits the data. The second term is the complexity term, which is used to punish the complexity of model. The bigger the value of score function for the structure is, the better the structure is.

In the score-based method, the ways of producing new structures include adding arcs, deleting arcs and reversing arcs. The reversing arcs is equivalent to the process that adding arcs after deleting arcs. Therefore, the ways can be summarized into two kinds: adding arcs and deleting arcs. Taking adding the arc $X\to Y$ as the example, if the opinion of experts is that this arc exists, the improved BIC (IBIC) should satisfy that it is bigger than BIC. Conversely, if the opinion of experts is that this arc doesn’t exist, the IBIC should be smaller than BIC. If there is no expert knowledge about this arc or the direction of arc is uncertain, the IBIC should be equal to BIC. The similar analysis can be obtained for the deleting arcs. The above the analysis can be summarized in the following table.

Table 1

The criterion which the IBIC needs to satisfy

The ways of producing the new structure	Criterion
Adding arcs	If $a_{pl}=1$ , $\textit{score}_{\textit{IBIC}}>\textit{score}_{\textit{BIC}}$
	If $a_{pl}=0$ , $\textit{score}_{\textit{IBIC}}<\textit{score}_{\textit{BIC}}$
	If $a_{pl}=*$ or $\oplus$ , $\textit{score}_{\textit{IBIC}}=\textit{score}_{\textit{BIC}}$
Deleting arcs	If $a_{pl}=1$ , $\textit{score}_{\textit{IBIC}}<\textit{score}_{\textit{BIC}}$
	If $a_{pl}=0$ , $\textit{score}_{\textit{IBIC}}>\textit{score}_{\textit{BIC}}$
	If $a_{pl}=*$ or $\oplus$ , $\textit{score}_{\textit{IBIC}}=\textit{score}_{\textit{BIC}}$

The transform operation needs to be carried out to satisfy the criterion in Table 1 for the IBIC. For convenience of calculation, considering that the BIC score is negative, when $a_{pl}=1$ , the value of $a_{pl}$ is not changed; when $a_{pl}=0$ , the value of $a_{pl}$ is replaced by 0.0001; when $a_{pl}=*$ or $\oplus$ , the value of $a_{pl}$ is regarded as 0.5. The transform coefficient is $t_{pl}$ , which is calculated by the following equation.

$\displaystyle t_{pl}=(1/2)(1-\log_{2}a_{pl})(p=1,2,\ldots n;l=1,2,\ldots n)$ (9)

The following results can be obtained:

(1)

When $a_{pl}=0$ , $t_{pl}>1$ , that is to say, the experts negate the existence of arc from the node $p$ to the node $l$ ;

(2)

When $a_{pl}=1$ , $t_{pl}<1$ that is to say, the experts are convinced of existence of arc from the node $p$ to the node $l$ ;

(3)

When $a_{pl}=*$ or $\oplus$ , $t_{pl}=1$ that is to say, no expert knowledge exists about the arc from the node $p$ to the node $l$ , or the experts only know that there is a relationship between the two nodes, but the direction is uncertain.

The IBIC can be obtained by the way that the BIC score multiplies the corresponding transform coefficient $t_{pl}$ .

4.2 Structure search based on GA

The GA is a heuristic search method, which is inspired by the evolution theory. By simulating the natural evolution process, the selection, crossover and mutation operators are proposed. Using these operators, the global optimization problem is solved by searching the best individual from the search space until a termination criterion is satisfied. The fitness function needs be determined to measure the quality of individual. For the BN structure learning, the individuals refer to the alternative BN structures. The IBIC score function in Section 4.1 is used as the fitness function. The search process based on GA can be described by the following figure.

Figure 2.

The flow diagram of BN structure learning based on GA.

Because of the introduction of expert knowledge, some parts of flow diagram in Fig. 2 are different from the traditional BN structure learning based on GA [14]. Firstly, the BN priori structure $A$ is used to initial population. We suppose that the individual is represented by matrix $H_{g}(1\leqslant g\leqslant u)$ , where $u$ represents the number of population. $H_{g}(1\leqslant g\leqslant u)$ satisfies the following form

$\displaystyle H_{g}=\left[{{\begin{array}[]{*{20}c}0&{h_{12}}&\cdots&{h_{1n}}% \\ {h_{21}}&0&\cdots&{h_{2n}}\\ \vdots&\vdots&\ddots&\vdots\\ {h_{n1}}&{h_{n2}}&\cdots&0\\ \end{array}}}\right](1\leqslant g\leqslant u)$

where $h_{pl}\in\{0,1\}(p=1,2,\ldots n;l=1,2,\ldots n)$ . When $a_{pl}=0$ or $1$ , the value of $h_{pl}$ is the same with $a_{pl}$ ; when $a_{pl}=*$ or $\oplus$ , the rand number $r$ is produced, which satisfies $r\in[0,1]$ . If $r\geqslant 0.5$ , $h_{pl}=1$ ; If $r<0.5$ , $h_{pl}=0$ . That is to say, except the arcs which can be determined by the expert knowledge, the rest of arcs are produced in the form of rand number $r$ . The selection operator adopts the way of roulette. The crossover operator adopts the two-point crossover using the column of matrix as the element. The mutation operator refers to the operations of adding the arcs, deleting the arcs and reversing the arcs. In the above processes, if the cycles are created, they should be removed to satisfy the DAG by deleting and reversing the arcs in the cycles [9, 20]. The termination criterion is that the algorithm reaches to the maximum generation, or the quality of population doesn’t change for the successive generations.

A problem needs to be focused. When $a_{pl}=\oplus$ , the corresponding $h_{pl}$ is determined by the same way with the condition $a_{pl}=\ast$ . However, if the values of $h_{pl}$ and $h_{lp}$ are all equal to zero, this will violate the expert knowledge that there is a relationship between the two nodes. Therefore, after the operations of selection, crossover and mutation, we need to judge whether the values of $h_{pl}$ and $h_{lp}$ are all equal to zero or not. If the answer is yes, the value of $h_{pl}$ or $h_{lp}$ is forced to change using number 1.

In addition, the relative closeness coefficient $\xi_{j}(1\leqslant j\leqslant 4)$ with the highest value can reflect the uncertain degree of expert knowledge, which is used to determine the BN priori structure. It can be used as the guide to adjust the mutation probability. Based on the analysis in Section 3.2, the $\xi_{j}\geqslant 0.5$ are used to determine the BN priori structure matrix $A$ . A threshold $\delta$ is set for the relative closeness coefficient $\xi_{j}(1\leqslant j\leqslant 4)$ . The relative closeness coefficient matrix $B$ is shown in the following form, which is a symmetric matrix.

$\displaystyle B=\left[{{\begin{array}[]{*{20}c}0&{\xi_{j}^{12}}&\cdots&{\xi_{j% }^{1n}}\\ {\xi_{j}^{21}}&0&\cdots&{\xi_{j}^{2n}}\\ \vdots&\vdots&\ddots&\vdots\\ {\xi_{j}^{n1}}&{\xi_{j}^{n2}}&\cdots&0\\ \end{array}}}\right]\quad\xi_{j}^{pl}=\xi_{j}^{lp}\quad(p=1,2,\ldots n;l=1,2,% \ldots n)$

The element can be obtained by the following criterion: when $\xi_{j}<0.5$ or no expert knowledge exists, $\xi_{j}^{pl}=0.5$ ; when $0.5<\xi_{j}<\delta$ , $\xi_{j}^{pl}=\xi_{j}$ ; when $\xi_{j}\geqslant\delta$ , $\xi_{j}^{pl}=1$ .

It can obtain that the bigger the value of $\xi_{j}^{pl}$ is, the more certain the expert knowledge is. The corresponding information should not be changed. Vice versa, the smaller the value of $\xi_{j}^{pl}$ is, the more likely the corresponding information is changed. This kind of change can be reflected by the mutation probability. The mutation probability and the relative closeness coefficient are inversely proportional relationship. The fundamental mutation probability is set as $P_{m}$ . The final mutation probability can be obtained by $P_{m}$ multiplying the corresponding transform coefficient. The transform coefficient matrix $C$ of mutation probability is shown in the following form.

$\displaystyle C=\left[{{\begin{array}[]{*{20}c}0&{c_{12}}&\cdots&{c_{1n}}\\ {c_{21}}&0&\cdots&{c_{2n}}\\ \vdots&\vdots&\ddots&\vdots\\ {c_{n1}}&{c_{n2}}&\cdots&0\\ \end{array}}}\right]$

where $c_{pl}=1/\xi_{j}^{pl}(p=1,2,\ldots n;l=1,2,\ldots n)$ .

5. Experimental results

For the specific domain problem, especially when the researched process can be divided into sub-processes, the relevant variables are relatively small and the collection of expert knowledge is relatively easy [15, 23, 24, 35]. In the paper [23], for the abnormal conditions in the thickening process of gold hydrometallurgy, the established BN for the thickening sub-process only includes the relevant variables. The paper [24] proposes the diagnostic BN based on operation procedures, the variables are divided into the different layers. Above BNs are all not large networks. Therefore, to solve the similar small BN construction and verify the effectiveness of proposed approach, in this paper, the cancer and the Asia networks are chosen to verify the proposed method. They are classic BNs, which have been widely used for comparative purposes in BN structure learning [18, 20, 32]. Based on the proposed method, the BN structure is determined by the datasets generated from the true BNs. The performance is evaluated by comparing the learned BNs and the true ones. This paper improves the performance of score-based method based on GA by integrating the expert knowledge. Therefore, in the following examples, at first, the proposed method is verified and compared with the traditional GA used in the BN structure learning [14, 27], and the influence of expert knowledge is shown by the Examples 1–3. What’s more, in the Example 4, the proposed method is compared with the existing research results. In the end, the proposed method is applied to the thickening process of gold hydrometallurgy in the Example 5. The established BN is analyzed and compared with the results in the paper [23].

Figure 3.

The structure of cancer network.

Example 1: The structure of cancer network is shown in Fig. 3. The cancer network is used to generate datasets. Using sampled datasets, the proposed approach is applied to learn the BN structure and compared with the true structure to verify the proposed method. To simulate the expert knowledge, we suppose that some relationships among the nodes in the cancer network are known, which is regarded as the BN priori network and shown by matrix $A_{1}$ .

$\displaystyle A_{1}=\left[{{\begin{array}[]{*{20}c}0&1&\oplus&*&*\\ 0&0&*&*&*\\ \oplus&*&0&1&*\\ *&*&0&0&*\\ *&*&*&*&0\\ \end{array}}}\right]$

From the matrix $A_{1}$ , we can obtain that the opinions of experts are $A\to B$ , $C\to D$ and there is a relationship between the nodes $A$ and $C$ , but the direction is uncertain. By the comparison of learned BN structure with the true BN structure, we can evaluate the performance of proposed method. Compared with the true BN structure, the learned BN structure may add, delete and reverse some arcs. Some indexes are introduced to measure the performance of proposed method. The indexes are the average of spurious edges, missing edges and reversed edges. To make the comparison more reasonable, the approach runs ten times. The indexes are obtained by averaging the results of ten times. In the GA, the crossover probability is set as 0.9, the mutation probability is set as 0.01 and the max generation is set as 300. The comparison results of proposed method with the traditional GA used in the BN structure learning [14, 27] for the cancer network are shown in Table 2.

Table 2

The comparison results of proposed method with the traditional GA used in the BN structure learning [14, 27] for the cancer network

	The average of	The average of	The average of	The average of
	spurious edges	missing edges	reversed edges	generations
The traditional method	3.8	0.1	2.9	134
The proposed method	3.1	0	0.8	165.4

The termination criterion is that the algorithm reaches to the maximum generation, or the quality of population doesn’t change for the successive generations. From the table, we can obtain that the algorithm is convergent before the generation reaches to the set maximum. Comparing the network structures, we can find that the obtained network structure by the proposed method is much closer to the true cancer network than the traditional method that no expert knowledge is used. Therefore, when integrating the expert knowledge into the score-based structure learning method based on GA, the obtained BN structure owns the better performance. This proofs that the proposed method is valid to increase the performance of BN structure learning.

Example 2: To reflect the role of expert knowledge in the proposed BN structure learning method, we change the quantity of expert knowledge and observe the performance of structure learning. In this paper, the expert knowledge includes four conditions: node A points to node B; node B points to node A; there is a relationship between nodes A and B, but the direction is uncertain; there is no relationship between nodes A and B. If one of four conditions is known for random two nodes in the BN, the quantity of expert knowledge adds one. As the known relationships among the nodes increase, the results of structure learning for the cancer network are shown in Table 3.

Table 3

The results of structure learning by integrating the different quantities of expert knowledge for the cancer network

The quantity of	The average of	The average of	The average of	The average of
expert knowledge	spurious edges	missing edges	reversed edges	generations
3	3.1	0	0.8	165.4
6	1.6	0	0.4	127.1
7	0.7	0	0.2	105.2
8	0	0	0.4	116.8

From the results in Table 3, we can conclude that if the collected quantity of expert knowledge increases, the performance of structure learning will improve further. Therefore, the expert knowledge plays the important role in the improvement of performance for the BN structure learning.

Example 3: In the Examples 1 and 2, we have proved the proposed algorithm is effective, and it owns better performance than the traditional GA used in the BN structure learning [14, 27]. However, the considered cancer network is a simple BN. When the nodes of BN increase, in our opinions, the role of expert knowledge will be more obvious. In this example, the Asia network is used to generate datasets. The structure of Asia network is shown in Fig. 4. The comparison results of proposed method with the traditional GA used in the BN structure learning [14, 27] are shown for the Asia network in Table 4. As the quantity of expert knowledge increases gradually, and the results of structure learning for the Asia network are shown in Table 5.

Table 4

The comparison results of proposed method with the traditional GA used in the BN structure learning [14, 27] for the Asia network

	The average of	The average of	The average of	The average of
	spurious edges	missing edges	reversed edges	generations
The traditional method	15.35	1.4	3.3	184.4
The proposed method	10.7	0.9	0.8	200.2

Table 5

The results of structure learning by integrating the different expert knowledge for the Asia network

The quantity of	The average of	The average of	The average of	The average of
expert knowledge	spurious edges	missing edges	reversed edges	generations
4	10.7	0.9	0.8	200.2
6	8.2	1	1.3	190.9
9	6.6	1	0.8	203.4
11	5.9	0.7	0.9	195.5
13	3.8	0.9	1	144.1
17	3.3	0.7	0.5	188.6

Figure 4.

The structure of Asia network.

From the results in Tables 4 and 5, the same conclusions can be obtained with the Examples 1 and 2. We can find that when the BN nodes increase and the structure is more complex, the difference of learned structure with the true structure will become bigger for the traditional method without expert knowledge. It is very difficult to search for the suitable structure. However, when the expert knowledge for the relationships among the nodes is available, the performance of structure learning is increased gradually. The more the nodes in the BN are, the greater the role of expert knowledge is.

Example 4: In this example, we will compare the proposed method with the results in the papers [32, 33]. For the run results in the paper [32], to implement the reasonable comparison, in the proposed method, we set the same parameters with the paper [32]. The dataset includes 50 random cases; the maximum number of iterations is set as 300; the population size is set as 50; the crossover rate is set as 0.6. To make the results reasonable, the proposed approach runs ten times. The score is obtained by averaging the scores of optimal structures for ten times. The running results of different algorithms for the Asia network are shown in Table 6.

Table 6

The running results of different algorithms for the Asia network

Methods	The proposed method	HSL-GA	Chain-GA	HC	GES	K2	MWST	PC
Scores	$-$ 98.802	$-$ 130.67	$-$ 130.41	$-$ 128.94	$-$ 146.79	$-$ 129.78	$-$ 152.42	$-$ 162.08

Except the running result of proposed method, the else running results of different algorithms in Table 6 are from the paper [32]. The specific simulation process can refer to the section three in the paper [32]. By the comparison in Table 6, it can conclude that the score of proposed method is better than the methods in the paper [32].

In the paper [33], the method is carried out to learn the Asia network structure under the different data sizes 200, 500 and 800. To compare our method with the paper [33], similarly, the proposed method in this paper is used to learn the Asia network structure under the different data sizes. The obtained optimal Asia network structures under the different data sizes are shown in Fig. 5. It can obtain that the proposed method can obtain good structure learning result when the data size is 500. However, the method in the paper [33] can obtain good structure learning result when the data size is 800. The proposed method in this paper owns better performance than the method in the paper [33].

Figure 5.

The obtained optimal Asia network structures under the different data sizes (a) data size is 200; (b) data size is 500.

Example 5: In this example, the proposed method is applied to the thickening process of gold hydrometallurgy. In the paper [23], the BN has been established by the expert knowledge. In this paper, we combine the expert knowledge and data information to establish the BN. Therefore, the proposed method is used to establish the BN structure and compared with the results in the paper [23]. The specific description about the abnormal conditions in the thickening process of gold hydrometallurgy can refer to the paper [23]. In this section, we only explain the establishment process of BN structure by the proposed method in this paper. At first, the priori BN structure needs be obtained. The relationships between the abnormal conditions (E: The underflow concentration is too high; F: The buffer slot 1 under the thickener is empty) and the corresponding phenomena (G: The motor current in the thickener is too large; H: The bed pressure in the thickener is too high; I: The electricity of slurry pump 1 is not stable) are regarded as the priori expert knowledge. What’s more, the relationships among the abnormal conditions (D: The underflow rate is too low; E: The underflow concentration is too high; F: The buffer slot 1 under the thickener is empty) and the relationships between the safe control decisions (A: Change the opening degree of valve 3 and the power of slurry pump 1; B: Change the opening degree of valve 4; C: Change the opening degree of valve 1.) and the abnormal conditions D, E and F are learned by the data information. In the end, the learned BN structure is analyzed and compared with the result in the paper [23]. The learned BN structure by the proposed method and the established BN in the paper [23] for the thickening process of gold hydrometallurgy are shown in Fig. 6.

Figure 6.

(a) The learned BN structure by the proposed method. (b) The established BN in the paper [23] for the thickening process of gold hydrometallurgy.

From Fig. 6, we can obtain that the learned BN structure by the proposed method has two different arcs compared with the structure in the paper [23]. The difference is that the learned structure adds one arc $A\to E$ and reverses one arc $E\to C$ . By analyzing the thickening process of gold hydrometallurgy, we can know that the safe decision A (change the opening degree of valve 3 and the power of slurry pump 1) can influence the abnormal condition E (the underflow concentration is too high) by the abnormal condition D (the underflow rate is too low) indirectly. The existence of reversed arcderives from the Markov equivalence class. We can revise the arcs based on the specific background knowledge. Therefore, we can conclude that after the proper revising process, the learned BN structure by the proposed method can be used to solve the practical problem.

6. Conclusions

In this paper, we propose a new method of BN structure learning by integrating expert knowledge and the observational data based on the score-based method. In the BN structure learning framework, to improve the performance of expert knowledge usage, the representation and integration of expert knowledge are implemented by the IFS method. The determination of BN priori structure is transformed into the GDM based on IFS. To search for the most suitable structure under the constraints from priori structure, the score-based method based on the GA is used. The score function BIC and GA are improved to satisfy the constraints from expert knowledge. Some experiments are provided to demonstrate the validity of proposed scheme and compare the performance with the existing research results. The obtained BN structure owns better performance than the score-based method without expert knowledge. The more the quantity of expert knowledge is, the better the performance of BN structure learning would be. Finally, the proposed method is applied to the thickening process of gold hydrometallurgy to solve the practical problem. For the further research, we will consider more uncertain information when using the expert knowledge to determine the BN priori structure.

Footnotes

Acknowledgments

This work was supported by the National Nature Science Foundation of China [grant numbers 61533007; 61873049], the National Key Research and Development Program of China [grant numbers 2017YFB0304205].

References

Acid

and Campos

L.M.d.

, Searching for Bayesian network structures in the space of restricted acyclic partially directed graphs, Journal of Artificial Intelligence Research 18 (2003), 445–490.

Adabor

E.S.

Acquaah-Mensah

G.K.

and Oduro

F.T.

, SAGA: A hybrid search algorithm for Bayesian network structure learning of transcriptional regulatory networks, J Biomed Inform 53 (2015), 27–35.

Atanassov

K.T.

, Intuitionistic fuzzy sets, Fuzzy Sets and Systems 20 (1986), 87–96.

Bhandari

Abbassi

Garaniya

and Khan

, Risk analysis of deepwater drilling operations using Bayesian network, Journal of Loss Prevention in the Process Industries 38 (2015), 11–23.

Boran

F.E.

Genç

Kurt

and Akay

, A multi-criteria intuitionistic fuzzy group decision making for supplier selection with TOPSIS method, Expert Systems with Applications 36 (2009), 11363–11368.

Cano

Masegosa

A.R.

and Moral

, An Importance Sampling Approach to Integrate Expert Knowledge When Learning Bayesian Networks From Data, in: International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems, 2010, pp. 685–695.

Cano

Masegosa

A.R.

and Moral

, A method for integrating expert knowledge when learning Bayesian networks from data, IEEE Transactions On Systems, Man, and Cybernetics-Part B: Cybernetics 41 (2011), 1382–1394.

Carvalho

, A Cooperative Coevolutionary Genetic Algorithm for Learning Bayesian Network Structures, in: Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation, Dublin, Ireland, 2011, pp. 1131–1138.

Castillo

Gutiérrez

J.M.

and Hadi

A.S.

, Modeling probabilistic networks of discrete and continuous variables, Journal of Multivariate Analysis 64 (1998), 48–65.

10.

Chen

S.-M.

Cheng

S.-H.

and Chiou

C.-H.

, Fuzzy multiattribute group decision making based on intuitionistic fuzzy sets and evidential reasoning methodology, Information Fusion 27 (2016), 215–227.

11.

Chickering

D.M.

, Optimal structure identification with greedy search, Journal of Machine Learning Research 3 (2002), 507–554.

12.

de Campos

L.M.

and Castellano

J.G.

, Bayesian network learning algorithms using structural restrictions, International Journal of Approximate Reasoning 45 (2007), 233–254.

13.

Diallo

T.M.L.

Henry

and Ouzrout

, Bayesian network building for diagnosis in industrial domain based on expert knowledge and unitary traceability data, IFAC-Papers Online 48 (2015), 2411–2416.

14.

Etxeberria

Larranaga

and Picaza

J.M.

, Analysis of the behaviour of genetic algorithms when learning Bayesian network structure from data, Pattern Recognition Letters 18 (1997), 1269–1273.

15.

Fienen

M.N.

Nolan

B.T.

and Feinstein

D.T.

, Evaluating the sources of water to wells: Three techniques for metamodeling of a groundwater flow model, Environmental Modelling & Software 77 (2016), 95–107.

16.

Gheisari

and Meybodi

M.R.

, BNC-PSO: Structure learning of Bayesian networks by particle swarm optimization, Information Sciences 348 (2016), 272–289.

17.

Guo

, Knowledge measure for atanassov’s intuitionistic fuzzy sets, IEEE Transactions on Fuzzy Systems 24 (2016), 1072–1078.

18.

Kabli

Herrmann

and McCall

, A Chain-Model Genetic Algorithm for Bayesian Network Structure Learning, in: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, London, England, 2007, pp. 1264–1271.

19.

Khanteymoori

A.R.

Menhaj

M.B.

and Homayounpour

M.M.

, Structure learning in Bayesian networks using asexual reproduction optimization, ETRI Journal 33 (2011), 39–49.

20.

Kim

D.W.

and Kang

B.Y.

, Structure learning of Bayesian networks by estimation of distribution algorithms with transpose mutation, Journal of Applied Research and Technology 11 (2013), 586–596.

21.

Lam

and Bacchus

, Learning Bayesian belief networks: An approach based on the MDL principle, Computational Intelligence 10 (1994), 269–293.

22.

P.P.

Bahl

and Ungar

L.H.

, Using prior knowledge to improve genetic network reconstruction from microarray data, In Silico Biology 4 (2004), 335–353.

23.

Wang

and Li

, A safe control scheme under the abnormity for the thickening process of gold hydrometallurgy based on Bayesian network, Knowledge-Based Systems 119 (2017), 10–19.

24.

Liu

Cai

and Zheng

, An approach for developing diagnostic Bayesian network based on operation procedures, Expert Systems with Applications 42 (2015), 1917–1926.

25.

Masegosa

A.R.

and Moral

, An interactive approach for Bayesian network learning using domain/expert knowledge, International Journal of Approximate Reasoning 54 (2013), 1168–1181.

26.

Meineri

Dahlberg

C.J.

and Hylander

, Using gaussian Bayesian networks to disentangle direct and indirect associations between landscape physiography, environmental variables and species distribution, Ecological Modelling 313 (2015), 127–136.

27.

Qiu

Zhang

and Zheng

, Bayesian Network based Abnormality Detection with Genetic Algorithm Optimization, in: International Conference on Computational Problem-Solving (ICCP), Lijiang, China, 2010, pp. 222–227.

28.

Schwarz

, Estimating the dimension of a model, The Annals of Statistics 6 (1978), 461–464.

29.

Shin

Kim

and Lee

J.-M.

, Production and inventory control of auto parts based on predicted probabilistic distribution of inventory, Digital Communications and Networks 1 (2015), 292–301.

30.

Cai

Sun

and Zhang

, Integrated importance measures of multi-state systems under uncertainty, Computers & Industrial Engineering 59 (2010), 921–928.

31.

Szmidt

and Kacprzyk

, Distances between intuitionistic fuzzy sets, Fuzzy Sets and Systems 114 (2000), 505–518.

32.

Vafaee

, Learning the structure of large-scale bayesian networks using genetic algorithm, in: Proceedings of the 2014 Annual Conference on Genetic and Evolutionary Computation, Vancouver, BC, Canada, 2014, pp. 855–862.

33.

Wang

and Zhang

, Bayesian network structure learning based on unconstrained optimization and genetic algorithm, Control and Decision 28 (2013), 618–622.

34.

Wang

Zhu

Song

and Lei

, Combination of unreliable evidence sources in intuitionistic fuzzy MCDM framework, Knowledge-Based Systems 97 (2016), 24–39.

35.

Liu

Zhang

Skibniewski

M.J.

Deng

and Teng

, A dynamic Bayesian network based approach to safety decision support in tunnel construction, Reliability Engineering & System Safety 134 (2015), 157–168.

36.

Zhao

Serpedin

and Dougherty

E.R.

, Inferring gene regulatory networks from time series data using the minimum description length principle, Bioinformatics 22 (2006), 2129–2135.