The research of clinical temporal knowledge graph based on deep learning

Abstract

Temporal knowledge base exists on various fields. Take medical medicine field as example, diabetes is a typical chronic disease which evolves slowly. This paper starts from actual EMR data of hospitals by combination of experience and knowledge of clinical doctors. Link prediction on clinical knowledge base such as diabetic complication requires the analysis on temporal characteristic of temporal knowledge base, which is a great challenge for traditional link prediction models. This paper proposes temporal knowledge graph link prediction model based on deep learning. This model selects the TransR transformation model suitable for big data and makes entity projection in relation space containing different semantic meanings, so as to vector the entities and complex semantic relations in graph. Then it adopts LSTM recursive neural network and adds the top-bottom relational information of the graph for sequential learning. Finally it constantly carries out deep learning through incremental calculation and LSTM recursive network to improve the accuracy of prediction. The incremental LSTM model highlights the hidden semantic and clinical temporal information and effectively utilizes sequential learning to mining forward-backward dependent information. It compensates the deficiency of lower prediction accuracy on timely knowledge graph caused by the traditional link prediction models. Finally, it is proved that the new model has better performance over temporal knowledge graph link prediction.

Keywords

Temporal knowledge graph knowledge graph link prediction translation model TransR long short term memory (LSTM) networks incremental learning deep learning

1 Introduction

Knowledge Graph is a kind of new method to represent knowledge, which belongs to semantic network scope. Knowledge Graph is used to describe all kinds of entities and concepts existing in real world [1]. In recent years, Knowledge Graph has gradually attracted attention of medical domain. Relevant researches on Knowledge Graph of medical domain have been conducted at home and abroad. There is LynxKB Knowledge Graph for pharmacology created by Oxford University abroad [2] and genetic research in the Knowledge Graph integrated by traditional relational database by Tohoku University [3]. At the same time, Researchers of medical informatics domain have also proposed various medical Knowledge Graphs including Knowledge Graph of traditional Chinese medicine established by information institute of medicine Academy Science and visualized analysis of genome epidemiology based on Knowledge Graph [4, 5]. However, the knowledge source of these medical Knowledge Graphs is mainly from public medical literatures and parts of electric medical record (EMR) of hospitals.

As everyone knows, all kinds of clinical data generated during medical activities contain rich clinical knowledge, which is mainly reflected as medical entities such as patient entities, medicine entities, diagnosis entities and the relations between medical entities [6]. This paper uses Knowledge Graph to show clinical knowledge in EMR and constructs the temporal Knowledge Graph of clinical domain based on EMR. The Knowledge Graph describes the concepts and rich semantic relations and temporal information.

Knowledge Graph link prediction [7] is an important application of Knowledge Graph learning and reasoning, whose major task is to predict the possible relations among entities of the knowledge graph and realize the discovery and restoration of missing information of the knowledge graph [8]. For the data structure of EMR is complex, there may be some missing medical entities and entity relations or false relations among entities. Link prediction of knowledge graph of clinical domain will complement those relations or correct false relations, so as to obtain more complete and authentic knowledge graph.

Currently, mature knowledge link prediction includes tensor decomposition model [9], NTN neural network [10], transformation model [11], immune clonal selection algorithm [12], and so on. However, those prediction models can only achieve good effects on common knowledge graphs. Most of common knowledge graphs are common-sense knowledge without changes as time goes on [13]. On the contrary, in clinical domain, course of general diseases is a gradually unfolding process. It can be seen that clinical factual knowledge has timeliness containing a large amount of temporal knowledge. Take diabetes as example, during the clinical diagnosis and treatment of diabetics, blood glucose examination, glycosylation hemoglobin examination, medication, and complication diagnosis result should be marked with time. The clinical factual knowledge can be placed in order according to timing sequence, to form temporal knowledge graph of clinical domain with temporal characteristic. Current knowledge graph link prediction models are often aimed at static data [14], and do not take it into consideration that there is a large amount of temporal information in temporal knowledge graphs. So they cannot make accurate predictions about temporal knowledge graphs.

In order to solve the above problem, this paper, starting from EMR of hospitals, by combination of experience and knowledge of clinical doctors, has established the temporal Knowledge Graph of clinical domain and proposes the link prediction model based on LSTM sequential incremental learning for predicting the temporal Knowledge Graph of clinical domain. This model adopts sequential learning ability of recursive neural network of Long Short-Term Memory Unit and creates the sequential incremental learning layer to extract temporal characteristic of the clinical factual knowledge, and at the same time carries out incremental learning process of ternary-sequence of the Knowledge Graph through the manner of end-to-end, so as to realize the link prediction of temporal Knowledge Graph of clinical domain. This paper, through various comparative experiments, in respect of accuracy, and recall rate, and precision, has conducted evaluation and verification on the incremental LSTM new model. At the same time, it analyzed the time complexity of this new model and finally shown that this new model has better performance on predicting temporal Knowledge Graph.

2 Related work

Link prediction is always the hot issue of knowledge graph learning and reasoning. Many researchers have proposed different link prediction models used for learning and predicting the relations among entities. Mature methods of knowledge graph link prediction can be divided into three categories: (1) knowledge graph link prediction method based on tensor decomposition, including Neural tensor network (NTN); (2) knowledge graph link prediction method based on vector transformation model, such as TranaE [15], TranasH [16], TransR [17]; (3) knowledge graph link prediction method with deep learning [18], for example the work in Ref. [19] attempts to use deep neural network to conduct the link prediction of common knowledge graph. However, these methods are only applicable to static data of knowledge graph, and they cannot be applied to link prediction of temporal knowledge graph with the characteristic of dynamic changes.

About the application of time information in knowledge graph link prediction, the work in Ref. [20] has proposed the TransE-TAE model improved from TransE transformation model. This model makes predictions about relations through analysis of the time information of knowledge graph; the work in Ref. [21] then has further studied the prediction model of uncertain knowledge graph caused by time. However, most of the previous work has been concentrated on common knowledge graph with the characteristic of timeliness and few researches have been carried out on especially salient temporal characteristic of knowledge graph of clinical domain.

3 Study method

3.1 Clinical temporal knowledge graph link prediction

Temporal knowledge graph of the clinical domain is constructed based on EMR and its formal definition is as follows:

Definition 1. Temporal knowledge graph of clinical domain G. Temporal knowledge graph of clinical domain is a directed label graph: G_t = (t₀, t_e, E, R, σ), in which E is the vertex set of the knowledge graph used for presenting entity set; R is the frontier set of the knowledge graph used for presenting factual relation set; σ is the function of E × E → R|k to describeall of the tuplesin knowledge graph. k presents the kth relation R between two entities in triple list of knowledge graph sorted according to time during the time period [t₀, t_e]. Figure 1 shows clinical temporal knowledge graph which consists two parts. They are concept layer and instance layer.

Concept layer includes entity types such as patients, diseases, test indicators, complications and drugs. In concept layer, a conceptual relation exists among different entity types. In knowledge graph shown by Fig. 1, entity conceptual layer has three tiers: the first tier includes basic information, patients, test reports, diagnoses and medications; the second tier includes diabetes diagnosis and complication diagnosis; the rest belongs to the third tier.

In instance layer, each entity has attributes and values of attribute. For example, “Zhang San” belongs to the type of patient entity, so “Zhang San” is instantiated as the name attribute value of patient entity. The medical card number of Zhang San patient entity, “117”, as an instance, corresponds to medical card number attribute value of the patient.

Definition 2. Temporal link prediction refers to predicting the possible relation R of clinical significance between entity E₁ and E₂ in the clinical temporal knowledge graph G through analysis of the known information.

Take the temporal knowledge graph shown by Fig. 1 as example, the patient has several indicator tests during the process of treatment. The triple sequence X⁽ⁱ⁾ of temporal knowledge graph of the patient shown by Table 1 is input. The prediction of the possible relation Y between patient entity and eye disease entity is output by link prediction. The whole prediction can be presented as $P = {X, Y} = {X^{(1)}, X^{(2)}, \dots, X^{(n)}}, Y$ (1)X⁽ⁱ⁾ describes the corresponding triple sequence of various relations of the patient entity, which can be taken as the attribute of each aspect of the patient: $X^{(i)} = {X_{1}^{i}, X_{2}^{i}, \dots X_{n}^{i}}$ (2)

Table 1

Temporal knowledge base link prediction

Variables	Description	Values
X ⁽¹⁾	Sequence of Blood	(patient, blood _ sugar _ test\|1, value)
	Sugar Testing	(patient, blood _ sugar _ test\|2, value)
		⋮
X ⁽²⁾	Sequence of	(patient, insulin _ test\|1, value)
	Insulin Testing	(patient, insulin _ test\|2, value)
		⋮
X ⁽³⁾	Sequence of	(patient, leukocyte _ test\|1, value)
	Leukocyte Testing	(patient, leukocyte _ test\|2, value)
⋮	⋮	⋮
X ⁽ⁿ⁾	Sequence of	(patient, use\|1, value)
	Drug Use	(patient, use\|2, value)
y	Complication with Eye	(patient, has/nocomplication
	Disease	diagnosis, EyeDisease)

3.2 Temporal link prediction based on LSTM sequential incremental learning

This paper explains specifically elaborates on details of the model and gives the training process of the model.

3.2.1 Multiple semantic triple vectoring based on TransR

Multiple semantic triple vector level based on TransR is the first level of temporal link prediction model. This level mainly adopts TransR transformation model to embed the triple 〈E_i, R, E_j〉 of clinical temporal knowledge graphG into low-dimensional space. There are many-to-many relations of different semantic meanings in clinical temporal knowledge graph. For example, many patient entities and many different testing indicator entities are all testing relations. There is ultrasonic testing relation and puncture testing relation in these testing relations. TransR supports the processing of different entities with different semantic spaces, which complies with the characteristic of relations from different semantic spaces in clinical domain. Meanwhile, we carry out projection of each entity vector to relation space in TransR, so the similar entities originally in entity space are distinguished. Therefore, we realize the differentiation of different entities at each side of many-to-many relations in clinical factual knowledge graph and also embed the entities and relations into low-dimensional vector.

In order to take into consideration of different semantic spaces, TransR model has more accurate vector presentation of many-to-many relations. On the basis of TransE model, we carry out the projection of entity vector to relation space and then build the transformation relation from head entity to tail entity.

Figure 2 is an example of the application of TransR translation model to clinical knowledge graph in this paper. The two entities Zhang San and Shanghai are projected in_born_in relation space and then embedded into vectorial coordinate. A mapped_born_in vector transformation relation is established between them.

TransR defines a projection matrix M_r for each relation to project entity vector from entity space to the relation r subspace. l_hr and l_tr are described as: $\begin{matrix} l_{hr} = l_{h} M_{r}, \\ l_{tr} = l_{t} M_{r} \end{matrix}$ (3)l_hr and l_tr are represented as corresponding loss function of f_r (h, t). $f_{r} (h, t) = {| | l_{hr} + l_{r} + l_{tr} | |}_{L_{1} / L_{2}}$ (4)

Each entity is taken to be composed of different attributes in TransR model. As for different relations, different attributes of the entity should be paid attention. We carry out projection of relations of different semantic spaces to distinguish various semantic meanings, so as to realize the efficient vectoring of clinical knowledge graph.

3.2.2 Triple sequential incremental learning based on LSTM

Triple sequential incremental learning layer based on LSTM is the second layer of temporal link prediction model. Recursive neural network model LSTM is a kind of neural network which can learn long-term dependent information. The form of input data is denoted as f = { X_n } , n = t₁, t₂, …, t_k. The input data is a temporal data vector. The triple vector with two characteristics is outputted after clinical temporal knowledge graph is vectored in TransR. (1) After being vectored, clinical data does not lose its original semantic information to the greatest extent. (2) The output triple being vectored is arranged according to timing sequence, so it has the characteristic of time sequence. Therefore, the paper sends triple vector to LSTM. Not only it maintains the original semantic meaning, but also it obtains the temportal characteristic of LSTM input data f, so as to be applicable to LSTM. Therefore, through superposition enhancement of the semantic triple in TransR and LSTM, the memory unit can utilize the historical information in sequences to more fully and accurately dig the independent information among sequences.

Currently, the most widely used LSTM unit has three gates, input gate, output gate and forgotten gate, to store historical information. Input gate is used for controlling the effect of current data input on the status value of memory unit. Forgotten gate is used for controlling the effect of historical data on current memory unit and obtaining the memory status through calculation. Output gate is used for controlling the output of status value of memory. Targeted at the triple sequence time period 〈T₁, T₂〉 in clinical temporal knowledge graph which is taken as input, we obtain the output according to formula (5). $\begin{matrix} L_{input} = {〈 E_{i}, R, E_{j}, t 〉, t \in (T_{1}, T_{2})}, \\ X_{t} = {〈 E_{i}, R, E_{j}, t^{'} 〉, t = t}, \\ f_{t} = δ (W_{f} \times [h_{t - 1}, x_{t}] + b_{f}), \\ i_{t} = δ (W_{i} \times [h_{t - 1}, x_{t}] + b_{i}), \\ C_{t}^{'} = \tanh (W_{c} \times [h_{t - 1}, x_{t}] + b_{c}), \\ C_{t} = f_{t} \times C_{t - 1} + i_{t} \times C_{t}^{'}, \\ O_{t} = δ (W_{0} [h_{t - 1}, x_{t}] + b_{0}), \\ h_{t} = O_{t} \times \tanh (C_{t}) \end{matrix}$ (5)

X_t is controlled by the forgotten gate through δ function sigmoid. This function will generate a f_t value between 0 and 1 according to the output h_t-1 of the last moment and the current input X_t, so as to determine whether to let the information C_t-1 learned at the last moment pass or partially pass (selective amnesia). i_t is used for determining the accepting state of input information and then tanh is used for generating the new candite value $C_{t}^{'}$ which may be added to the status. Finally, the values produced by the two parts combined for update.

The triple 〈h, R, t〉 is input into transformation model for vectoring to get <l_h, l_r, l_t>. Then input the matrix formed by batch vectors into LSMT and output the more precise characteristic vector $〈 V_{h}^{'}, V_{r}^{'}, V_{t}^{'} 〉$ through using LSTM for forward and backward analysis of contexts to extract characteristics of triples.

For clinical knowledge graph with the characteristic of time series, this paper improves LSTM to incremental LSTM. This model adopts the cross entropy function as loss function and adjusts the parameters in LSTM through back propagation.

3.2.3 LSTM sequential characteristic incremental combination

LSTM sequential characteristic incremental combination is the third layer of temporal link prediction proposed by this paper. In top and bottom triple sequence, we adopt incremental form to replace the simple front and back connection process. When the next vector and the last vector have values at the same position, we use incremental addition to strengthen characteristics and take the strengthened vector as the next input. We can see the sketch map of LSTM sequential characteristic incremental combination from Fig. 3 in which each rectangle corresponds to each vector and different colors mean different values. If the color of the vector is white, the vector has no data at this position. If the color of the vector is yellow, the vector has data at this position. If the color of the vector is red, this position is the data after incremental addition. For example, if the first positions of the vector V⁽¹⁾ of in time sequential data and the current vector V⁽²⁾ have data, incremental addition should be conducted for strengthening the characteristic. Then the color of the position changes from yellow to red. Algorithm 1 shows the specific characteristic incremental combination process.

Algorithm 1 LSTM sequential characteristic incremental combination process
Input: the output vector V^(m) of LSTM
Output: the characteristic vector V after sequence incremental combination.
V: the vector after characteristic combination layer.
V^(m): the Mth vector produced by LSTM in the sequential incremental layer.
➀ V⁽⁰⁾ = V⁽¹⁾
2 fori ∈ 0 : ndo
3 V⁽ⁱ⁾ = V^(i-1) + V⁽ⁱ⁾
4 endfor
➄ V = V⁽ⁿ⁾

3.2.4 Training algorithm of temporal link prediction model

Through the processing of the above three layers, we conduct the incremental calculation of triples of each type for the above characteristic combination and extraction final characteristic vector is obtained as the input of classifier. The whole training process adopts backward propagation for adjusting parameters. The specific training algorithm of the temporal link prediction model is shown as Algorithm 2.

Algorithm 2 the training process of temporal link prediction model
Input: triple sequence organized according to time series
Output: 0 or 1, 1 represents the triple is right fact, 0 represents the contrary
N _it: the times of using batch data set training
D ^tr/ D ^te: training set/test set
M : the number of LSTM in sequential incremental layer;
E : loss value in categorization layer
^A E _m: the mth loss value of LSTM in sequential incremental layer
S : the number of batch data set
$A θ_{m}^{i} / C θ^{i}$ : the mth parameter of LSTM in sequential incrementallayer/the parameter of LSTM in categorizationlayer in the ith training
➀ ( $^{A} θ_{1}^{0}$ , …, $^{A} θ_{M}^{0}$ , ^Cθ0)←Initialize(M)
➁ for i ∈ 0 : N_itdo
➂ for j ∈ 1 : Sdo
➃ D^tr←GetMiniBatch(D)
➄ for m ∈ 1 : Mdo
V^(m)←LSTM(D^tr, A $θ_{m}^{i - 1})$
➅ V ←Incremental(V, V^(m))
endfor
➆ E←CLASSIFICATION_LSTM(V, ^cθ^i-1)
(^cθ^i-1, AE)←BACKWARD(E,cθ^i-1)
➇ for m ∈ 1 : Mdo
^A E_m = Separate(^AE,m)
$^{A} θ_{m}^{i} = BACKWARD (^{A} E_{m},^{A} θ_{m}^{i - 1})$
➈ endfor
endfor
➉ EvaluateModel(Dte, $^{A} θ_{1}^{0}$ ,…, $^{A} θ_{M}^{0}$ , ^Cθ⁰)
endfor

4 Experimental results and analysis

4.1 Experimental data

All data used in this paper is from actual EMR data of affiliated hospital of Binzhou Medical University. This knowledge graph is formed by EMR data of endocrinology department of this hospital collected in recent 10 years. The data has 61,000 entities, three different relation types and training set scale is 345,549 triples.

In order to prove the effect of the triple sequential incremental learning model prediction proposed by this paper, this paper adopts ten-fold cross validation method for evaluation on the above clinical sequential knowledge graph data. Therefore, we randomly select 34,554 triples for both validation set and testing set.

4.2 Comparative models

The paper adopts the commonly used 6 reasoning and prediction models in knowledge graph as comparative models. Tensor decomposition Rescal model [22], NTN, TransE, TransH, TransR and DNN [23].

4.3 Evaluation indicators

This paper adopts accuracy (AUC), recall rate(R) and precision (P) as the evaluation indicators of the model.

Accuracy refers to the ratio between the sample number of accurate prediction by the model and the total sample number with regards to given testing data set: $AUC = \frac{TP + TN}{TP + FP + FN + TN}$ (6)

TP is positive samples predicted by the model as positive. TN is negative samples predicted by the model as negative. FP is negative samples predicted by the model as positive. FN refers to positive samples predicted by the model as negative.

In order to reflect the proportion between the positive examples under correct judgment and total positive examples, this paper has adopted the recall rate R as an evaluation indicator which reflects the prediction ability of the model on positive samples.The recall rate R ishigher, the identification ability of the model has stronger. $R = \frac{TP}{TP + TN}$ (7)

In addition, precision P reflects the model’s separating capacity of negative samples. The higher P is, the stronger the model’s separating capacity is. P refers to the ratio between positive sample number under correct prediction and the total positive sample number: $P = \frac{TP}{TP + FP}$ (8)

Therefore, this paper also adopts Precision-Recall (PR) figure to evaluate the model’s ability of separating positive samples from negative samples. At the same time, the score of F₁ can be taken as the weighted average of accuracy and recall rate.

4.4 Result analysis and discussion

The result of comparative experiments and its analysis is explained. The incremental LSTM model of this paper has been compared with other reference models in experiments.

4.4.1 Model categorization performance comparison

This paper applies the model and other 6 reference models to clinical sequential knowledge graph. The performance comparisons of those methods are in Table 2.

Table 2
Performance comparisons on different models

Model AUC R F ₁ Time complexity

Rescal 0.55 0.58 0.57 O (n_e × k + n_r × k²)

NTN 0.6 0.58 0.59 $O (n_{e}^{2} \times k_{b} \times n_{r} \times (k_{b} + k_{a})$ +2n_rkk_a + n_ek)

TransE 0.56 0.52 0.53 O (n_e × k + n_r × k)

TransH 0.62 0.62 0.62 O (n_e × k + 2n_r × k)

TransR 0.73 0.75 0.74 O (P + n_e × k + 2n_r × k)

DNN 0.81 0.80 0.80 O (log V + n³ + k + n₁ ×n₂+ n₂ × n₃ + …+n_r-1 × n_r)

Incremental 0.85 0.87 0.86 (n_r + n + 1) × O (n³)

LSTM

Model	AUC	R	F ₁	Time complexity
Rescal	0.55	0.58	0.57	O (n_e × k + n_r × k²)
NTN	0.6	0.58	0.59	$O (n_{e}^{2} \times k_{b} \times n_{r} \times (k_{b} + k_{a})$ +2n_rkk_a + n_ek)
TransE	0.56	0.52	0.53	O (n_e × k + n_r × k)
TransH	0.62	0.62	0.62	O (n_e × k + 2n_r × k)
TransR	0.73	0.75	0.74	O (P + n_e × k + 2n_r × k)
DNN	0.81	0.80	0.80	O (log V + n³ + k + n₁ ×n₂+ n₂ × n₃ + …+n_r-1 × n_r)
Incremental	0.85	0.87	0.86	(n_r + n + 1) × O (n³)
LSTM

Note: The bold word means best choice.

We can see that the accuracy of TransR model is higher than Rescal algorithm and NTN algorithm in the clinical knowledge graph from Table 2. Compared with Rescal and NTN, TransE and TransH models have especially obvious advantages. Compared with traditional link prediction methods, deep learning model DNN has bigger advantages with control expression of hidden layer. It can be seen that targeted at the characteristics of clinical temporal knowledge graph, such as rich semantic meanings and time series, the incremental LSTM model is put forward by this paper achieves the highest accuracy and recall rate and F₁-Score categorization performance has been greatly improved.

4.4.2 ROC graph comparison of models

ROC curve graph takes true positive rate TP as vertical axis and false positive rate FP as horizontal axis. It can be seen from Fig. 4 that the area under the ROC curve of the incremental LSTM model is the largest and also the most convex and closest to the upper left point, which indicates the in these three types of models, the new model has the largest value of diagnosis, the highest accuracy and the largest utilization value.

4.4.3 PR graph comparison of models

The experiment also carries out the comparison of PR curve graphs of the three models in this paper. In PR curve, R is represents recall and P represents precision. Usually, if R is high, P will be low; conversely, if R is low, P will be high. Therefore, we can see the relation and trade-off between them through PR curve. ROC curve is being convex to the left, but PR curve is different. The more convex the curve is being to the right, the better effect this model has.

From the PR graph comparison of each model in Fig. 5, we can see that the incremental LSTM model, compared with other models, has obviously better performance in PR graph. The degree of being convex to the right is much larger than before, which indicates the effect of this model has improved. This model is the one whose degree of being convex to the right is the most obvious of the seven models. So, the incremental LSTM proposed by this paper is of the best effect.

Through analysis of accuracy AUC, recall R and comparison graphs of ROC curve and PR curve, a comprehensive conclusion has been made that the incremental model proposed by this paper, compared with the previous models has been greatly improved from each aspect.

4.4.4 Network structure comparison in model

This paper conducts the comparison of different parameters of LSTM nerve cell number in incremental LSTM model. The Table 3 is the comparison chart of the effect of different parameters on the final performance.

Table 3
Comparison on different network parameters

Numbers of Neurons in LSTM AUV R F1-Score

8 0.58 0.58 0.57

16 0.62 0.64 0.63

32 0.89 0.67 0.68

64 0.74 0.72 0.73

128 0.85 0.87 0.86

256 0.80 0.81 0.81

Numbers of Neurons in LSTM	AUV	R	F1-Score
8	0.58	0.58	0.57
16	0.62	0.64	0.63
32	0.89	0.67	0.68
64	0.74	0.72	0.73
128	0.85	0.87	0.86
256	0.80	0.81	0.81

Note: The hold word means best choice.

With the increasing number of nerve cells of LSTM, the incremental LSTM has obviously improved in respect of accuracy, recall and F1 score. When the number of nerve cells increases to 356, the performance begins to reduce. The reason may be over-fitting. Therefore, this paper finally selects 128 LSTM nerve cells after the experiment of parameter comparison of models.

4.4.5 Time complexity analysis of models

The complexity comparison between the model proposed by this paper and other models is in Table 2. n_e refers to the number of entities, n_r refers to the number of relations; k refers to vector’s dimension in vectoring; k_a, k_b, k_c respectively correspond to the size of the layer; n_r is the number of neural cells in the rth layer of neural network; P represents the projection time in TransR. In DNN model, V refers to the initial sparse vector’s dimension through One-hot method; N₀ is the sequence number of input layer in two-way LSTM; N_r is the number of neural cells in the R-th layer of neural network; n represents the input triple sequence number of the incremental LSTM.

From Table 2, we can see that Rescal’s complexity is the highest and it is the most time consuming. It also takes long time to use NTN method. If it operates in the bigger data, time constraint will be intensified. However, the time complexity based on transformation model has greatly reduced, which is suitable for big data. The time of the incremental LSTM model is closely related to input quantity. The more relation categories are and the larger the input triple sequence number is in knowledge and the higher the time complexity is.

5 Conclusions

We propose temporal knowledge graph link prediction model based on deep learning in our paper. This model creates the clinical temporal knowledge graph with the characteristic of temporal based on the EMR of hospitals. This model selects the TransR transformation model suitable for big data and makes entity projection in relation space containing different semantic meanings, so as to vectorize the entities and complex semantic relations in graph. Meanwhile, it adopts LSTM recursive neural network and adds the top-bottom relational information of the graph for sequential learning. Then it conducts incremental calculation of the sequential information to extract more accurate characteristic vector. Finally, it constantly carries out deep learning through incremental calculation and LSTM recursive network to improve the accuracy of prediction. The experiment indicates: the incremental LSTM model highlights the hidden semantic, clinical temporal information and effectively utilizes sequential learning to mine forward-backward dependent information, compensate the deficiency of lower prediction accuracy on timely knowledge graph caused by the traditional link prediction models. We will replace LSTM in the proposed frame of this paper with other LSMT variation methods for further integration of other deep learning methods, so as to optimize incremental LSTM model.

Footnotes

Acknowledgment

The research presented in this paper was supported by the scientific research fund of Binzhou Medical University (Grant: No. BY2015KYQD07). The authors acknowledge the Natural Science Foundation of Shandong province (Grant: NO. ZR2013FM024).

References

Sequeda

J.F.

, Integrating relational databases with these mantic web reflection, Procof the 13th Int. Summer School Berlin (2017), 68–120.

Sulakhe

, Balasu

, A database and knowledge extraction engine for integrative medicine, Nucleic Acids Research42 (2014), 1007–1012.

Ogishima

, Takai

, Integrated database and knowledge base for genomic prospective cohort study into medical mega bank toward personalized prevention and medicine, Proc of the 15th World Congress on Health and Biomedical Informatics (2015), 1057–1059.

Ruantong

, Haofeng

, Construction of traditional Chinese medicine knowledge graph and it’s application, Journal of Medical Informatics37 (2016), 8–13.

Qiao

, Wei

, Papers on genome epidemiology in the world: A knowledge map-based visual analysis, Chinese Journal of Medical Library and Information Science22 (2013), 2–9.

Lianyuan

, Efficient randomized Hough transform for circle detection using novel probability sampling and feature points, Optik-International Journal for Light and Electron Optics123 (2012), 1834–1840.

Sahebi

, Yu-Ru

, Brusilovsky

, Tensor factorization for student modeling and performance prediction in unstructured domain, Proc of the 9th Int. Conf. on Educational Data Mining (2016), 102–106.

Kaiwei

, Bishan

, Typed Tensor Decomposition of Knowledge Bases for Relation Extraction, Proc. of the 2014 Conf. on Empirical Methods in Natural Language Processing (2014), 1568–1579.

Morales

, Collarana

, Vidal

D.E.

, Mate Tee: A semantic similarity metric based on translation embedding for knowledge graphs, Proc of the 17th Int. Conf. on Web Engineering (2017), 246–263.

10.

Vang

K.J.

, Ethics of Google’s knowledge considerations, Journal of Information Communication and Ethics in Society11 (2013), 245–260.

11.

Bollacker

K.D.

, Evans

, Paritosh

, Freebase: A collaboratively created graph database for structuring human knowledge, Proc. of the 39th ACM SIGMOD Int. Conf. on Management of Data (2008), 1247–1250.

12.

Yun

, Zhenhua

, A study on car flow organization in the loading end of heavy haul railway based on immune clonal selection algorithm, Journal of Neural Computing and Applications29 (2018), 1–11.

13.

Feng

, Lei

, Research of Automatically Generate Mapping Mechanism Based on the Semantic Role, Journal of Database Theory and Application9 (2016), 163–176.

14.

Lijuan

, Hong

, The research of query expansion based on medical terms reweighting in medical information retrieval, Journal of EURASIP Journal on Wireless Communications and Networking (2018), 1–7.

15.

Border

, Usunier

, Uarcia-Duran

, Translating embeddings for modeling multi-relational data, Proc. of the 27th Annual Conf. on Neural Information Processing Systems (2013), 2787–279.

16.

Zhen

, Jianwen

, Jianlin

, Knowledge graph embedding by translating on hyper planes, Proc of the 28th AAAI Conf. on Artificial Intelligence (2014), 1112–1119.

17.

Yankai

, Zhiyuan

, Maosong

, Leaning entity and relation embeddings for knowledge graph completion, Proc. of the 29th AAAI Conf. on Artificial Intelligence Learning (2015), 2181–2187.

18.

Danleng

, Lei

, Li

, Deep learning in visual computing and signal Processing, Applied Computational Intelligence and Solt Computing (2017).

19.

Xiang

, Taheri

, Lifu

, Commonsense knowledge base completion, Proc. of the 54th Annual Meeting of the Association for Computational Linguistics (2016), 1445–1455.

20.

Tingsong

, Tianyu

, Towards time aware knowledge graph completion, Proc. of the 26th Int. Conf. on Computational Linguistics (2016), 1715–1724.

21.

Melisachew

W.C.

, Joerg

, Marrying uncertainty and time in knowledge graphs, Proc. of the 31th AAAI Conf. on Artificial Intelligence (2017), 88–94.

22.

Luo

, Ahmad

F.S.

, Shah

J.S.

, Tensor factorization for medicine in heart failure with preserved ejection fraction, Journal of Cardiovascular Translational Research (2017), 305–312.

23.

Robert

, Catherine

, Representing general relational knowledge in ConceptNet, Proc. of the 8th Int Conf on Language Resources and Evaluation (2012), 3679–3686.