EPSLP: Efficient and privacy-preserving single-layer perceptron learning in cloud computing

Abstract

With the synchronous development of both cloud computing and machine learning techniques, the resource constrained clients are preferring to outsource the tasks of data storage and computation to the cloud server. However, in this outsourcing paradigm, since the data owners lose the control of their data, it is of vital significance to address the privacy concern of data stored on the cloud server. Hence, many researchers have been focusing on preserving the privacy of training data in learning model. Recently, Wang et al. presented a privacy protection single-layer perceptron learning for e-healthcare (PSLP) by using Paillier cryptosystem. In this paper, we present that the cloud server can obtain the sensitive training data and weight vector in the PSLP scheme. Besides, based on a symmetric homomorphic encryption algorithm, we propose an efficient and privacy-preserving single-layer perceptron learning scheme in cloud computing, named EPSLP. Security analysis shows that the proposed EPSLP can protect the privacy of training data, intermediate results and the optimal single-layer perceptron predictive models. Finally, we implement the EPSLP scheme and PSLP scheme, and extensive experiments indicate that the EPSLP is efficient in data encryption phase and the training phase of predictive model.

Keywords

Cloud computing single-layer perceptron model symmetric homomorphic encryption privacy preservation neural network

1. Introduction

Cloud computing is the development and integration of grid computing, parallel computing, and distributed computing. It can provide not only software service, but also hardware service, which can be convenient for storing data and improving efficiency of processing data [13,20,36]. Therefore, the users with resource-constrained devices incline to outsource their data and heavy calculation workloads to the cloud server in a pay-per-use manner [11,45,46]. Nevertheless, the transaction processing of cloud computing heavily relies on virtual cloud, which may be attacked maliciously [8,12,34]. In addition, the openness and heterogeneity of the network also inevitably bring security problems to data stored on the cloud server [14,19,24]. Therefore, the outsourcing paradigm will bring potential threats to privacy information of user, which may contain some sensitive data of governments, hospitals, banks or enterprises [9,32,37]. Nowadays, in many countries, privacy protection has been written into law, such as European Union Data Protection Directive and US privacy laws (HIPAA, COPPA, GLB, FRC, etc.) [15,35].

Based on the cloud computing, one of the most promising topics is analyzing big data and building decision support models with the help of cloud server. With the development of high speed networks, social networking and electronic commerce, unprecedented amount of data is produced, which contains a great deal of valuable information, such as user behavior information, device status information and so on [40]. It is common to extract useful information from large databases by using the techniques of data mining and machine learning [16]. Neural network model with strong ability to learn the feature of big data is often used to efficiently predict outputs in many scenarios [29,39]. In the meanwhile, the development of deep learning in neural networks has made breakthroughs in image, speech, and text recognition [25,28]. In addition, single-layer perceptron model is one of the widely used neural network models. In many situations, since its simplicity, it is more efficient and suitable than other complex techniques. Moreover, single-layer perceptron model has been extensively applied in many fields such as analysing risks in finance [41], predicting a variety of diseases in e-healthcare [33,43] and so on. Therefore, neural network has greatly drawn more attention in recent years. However, considering a huge amount of data, a wide variety of data types, the high speed of data stream, constrained local storage resources and limited local computational capacity, clients incline to outsource the tasks of data storage and training model to the cloud server. In this scenario, we pay attention to the privacy of sensitive training data stored on the cloud server.

This issue has been well studied by many researchers in recent years [4,7,10,25,33,42–44]. Recently, Wang et al. proposed a privacy protection scheme for single-layer perceptron learning (PSLP) [33] by using Paillier cryptosystem. In PSLP scheme, the cloud server can obtain the sensitive medical data and the final optimal weight vector in the training process. Besides, the cloud server needs to transmit the updated weight vector to the client, and then the client needs to send back the decrypted weight vector to the cloud server. Thus, there are much more communication cost in the PSLP scheme. Furthermore, based on single-layer perceptron model, Zhang et al. proposed a privacy protection disease prediction scheme (PPDP) [43]. In PPDP scheme, plenty of matrix-matrix multiplication operations are required and weight vectors are updated on the client side in the training process, which result in the inefficiency in computational overhead and communication cost. In the meanwhile, one vector on plaintext corresponds to a matrix on ciphertext, thus, PPDP scheme needs much more storage space on the cloud server side. In addition, since all the training data need to be encrypted again when the feature dimension increases or decreases, the PPDP scheme lacks of scalability. Therefore, in order to solve the above problems, we propose a novel efficient and privacy protection scheme for single-layer perceptron learning in cloud computing.

1.1. Our contribution

In this paper, we further study the problem of privacy-preserving single-layer perceptron learning in cloud computing. Our contributions are as follows:

We point out a security weakness of the PSLP scheme mainly based on that one linear equation with one unknown number can be solved. We demonstrate that the honest-but-curious cloud server can obtain the sensitive medical data and the final optimal weight vector in the training process.

We propose a concrete privacy-preserving single-layer perceptron learning scheme with the cloud computing. Security analysis indicates that our scheme can protect the privacy of training data, intermediate results and optimal predictive model in the training process.

The proposed scheme is efficient in computational cost and communication overhead. Firstly, the symmetric homomorphic encryption scheme we employed is high-performance and most time-consuming operations are executed on the cloud server. Besides, a round of communication is reduced in each training round with updating of weight vector. In addition, the proposed scheme is scalable with the changes of feature dimension. Finally, we implement our scheme in JAVA and conduct experiments on two real datasets [5,41]. Detailed analysis and extensive simulations show that our scheme offers significant time savings on the setup and learning phases compared with the previous scheme, and the accuracy rate of our scheme is acceptable.

1.2. Related work

There are plenty of research works on the privacy-preserving neural network learning in the past decades. Generally speaking, the research directions can be divided into two categories: data perturbation [3] and cryptographic solution [23]. On the one hand, many ways of data perturbation by adding noise will reduce the availability of data and the accuracy rate of the trained model [2,18,26]. However, the differential privacy of machine learning was proposed [1,25,31] recently. Differential privacy is used to protect privacy by introducing additive noise to the data, shared values or the update function, which can ensure the availability of data while protecting the privacy. On the other hand, cryptographic method has also become a promising research issue and several related works have been proposed recently. A protocol for privacy preserving neural network learning model on horizontal partitioned data was proposed by Schlitter [27]. However, this algorithm only ensures the trained data not be revealed without protecting the intermediate results. To protect the intermediate results, Chen et al. proposed a new privacy-preserving scheme for backpropagation neural network learning [10], which could be applied to vertically partitioned data. Furthermore, a privacy preserving scheme based on neural network model was proposed by Bansal et al. [4], which supported arbitrarily partitioned data. It only allowed two parties to train model cooperatively. However, the multiparty scenario is very common in reality. Therefore, in order to solve the problem of multiparty scenario, Yuan et al. proposed a privacy-preserving scheme for back-propagation neural network learning [42]. It supported multiparty scenario with the help of the cloud server. What is more, in this scheme, each party possessed some arbitrarily partitioned data. In the above schemes, the input data is distributed, therefore the neural network model is collaboratively trained by two or more users. Different from above scenarios, the scenarios of following schemes consist of only one user on the client side, who has limited computational resources and storage resources and wants to outsource data and computational workloads to the cloud server. In order to improve the training efficiency, based on back-propagation algorithm, Bu et al. proposed a privacy preserving scheme with the cloud computing [7], which protected the privacy by using BGV (it is introduced by Z. Brakerski, C. Gentry and V. Vaikuntanathan) [6] full homomorphic encryption algorithm. Zhang et al. also presented a privacy preserving scheme in cloud computing for deep computation model, which was based on the high-order back-propagation algorithm and collaborated with the cloud server to improve the computational efficiency [44]. Wang et al. proposed a privacy protection scheme for single-layer perceptron learning (PSLP) [33], which applied the Paillier cryptosystem to protect medical information. Nevertheless, the cloud server can obtain the sensitive medical data and the final optimal weight vector in the training process of PSLP scheme. Based on single-layer perceptron model, Zhang et al. proposed a privacy protection disease prediction scheme (PPDP) for e-healthcare system [43]. However, this scheme needs much more computational cost and communication overhead in the process of training and much more storage space on the cloud side. Besides, the PPDP scheme lacks of scalability when the feature dimension increases or decreases. Our scheme is based on the single-layer perceptron learning, which not only is high-performance both in computation and communication, but also can protect the privacy of training data, intermediate results and the final optimal weight vector, i.e., the predictive model. In the meanwhile, our scheme is scalable with the changes of feature dimension.

1.3. Organization

The rest of this paper is organized as follows. In Section 2, we present some preliminaries containing some necessary definitions and the symbol classification algorithm. The PSLP scheme is briefly reviewed and its security analysis is given in Section 3. In Section 4, we propose a novel efficient and privacy-preserving scheme for single-layer perceptron learning. In Section 5, we conduct the security and efficiency analysis of our scheme. The concrete and elaborate experimental evaluations are given in Section 6. Finally, conclusions will be made in Section 7.

2. Preliminaries

In this section, we introduce some preliminaries such as single-layer perceptron model [30], symmetric homomorphic encryption scheme [22] and the symbol classification algorithm.

2.1. Single-layer perceptron model

The single-layer perceptron (SLP) can be used for classification and regression, we focus on the linear classification case of two categories in this paper, i.e., its input is the feature vector of training sample, and its output is the class of training sample denoted by $\pm 1$ . SLP is the basis of neural networks, and it is composed of two layers of neurons. The input layer nodes receive the external input signals and send them to the output layer. The output layer node is a threshold logic unit [17]. Here, we choose the sign function as the activation function of output layer, and the sign function is as follows: $\begin{array}{rcl} sign (t) = \{\begin{matrix} 1, & t ⩾ 0 \\ - 1, & t < 0 \end{matrix} \end{array}$

In the meanwhile, we regard threshold as a weight corresponding to a dummy node with a fixed input of $- 1$ . Thus, study of threshold value can be included into the learning of weights, as shown in Fig. 1.

Fig. 1.

Single-layer perceptron.

Given a SLP configuration, the input vector is denoted as ${\vec{x}}_{i} = (x_{i, 1}, \dots, x_{i, n})$ ( $1 ⩽ i ⩽ m$ ), which contains the value of dummy node. The desired outputs are denoted as ${y_{1}, \dots, y_{m}}$ ( $y_{i} \in {- 1, + 1}$ ), and the results of output layer node are ${{\tilde{y}}_{1}, \dots, {\tilde{y}}_{m}}$ ( ${\tilde{y}}_{i} \in {- 1, + 1}$ ). The weight vector is denoted as $(w_{1}, \dots, w_{n})$ . Here, m represents the number of training sample or record, and n represents the number of features. We use $e = - \sum_{i, y_{i} \neq {\tilde{y}}_{i}} (y_{i} \vec{w} \cdot {\vec{x}}_{i})$ as the loss function, and minimize the loss function by using stochastic gradient descent method. Therefore, the formula for updating weights is as follows: $\begin{array}{rcl} w_{j} = w_{j} + η y_{i} x_{i, j} (1 ⩽ j ⩽ n) \end{array}$ where η is the learning rate. Based on the above description, the SLP learning algorithm can be described as Algorithm 1.

Algorithm 1

Single-layer perceptron learning process

2.2. Symmetric homomorphic encryption scheme

In our privacy-preserving algorithm for SLP learning, an efficient symmetric homomorphic encryption scheme [22] is utilized. In this encryption scheme, if the number of consecutive homomorphic multiplication operation is larger than a certain number of times, the ciphertext of product will not be decrypted correctly. However, there is no consecutive homomorphic multiplication operation in our proposed scheme. In this section, we will review the symmetric homomorphic encryption algorithm and its properties.

Basic components.The symmetric homomorphic encryption scheme includes three parts: key generation algorithm $KeyGen (\cdot)$ , encryption algorithm $Enc (\cdot)$ and decryption algorithm $Dec (\cdot)$ .

$KeyGen (k) \to (SK, p)$ : Given a security parameter k, the key generation algorithm randomly generates a public parameter p and a secret key $SK = (s, q)$ . Both q and p are big prime numbers. The bit length of q is determined by the security parameter k, $p ≫ q$ , and s is a random number in $Z_{p}^{*}$ .

$Enc (SK, d, m) \to c$ : It takes secret key $SK$ , ciphertext degree parameter d and plaintext $m \in F_{q}$ as inputs, and outputs the ciphertext c. The formula of encryption is as follows: $\begin{array}{rcl} Enc (SK, d, m) = s^{d} (r q + m) \mod p \end{array}$ where r is a big random positive integer and is called random ingredient of ciphertext c. We note that the bit length $| r |$ needs to satisfy $| r | + | q | < | p |$ .

$Dec (SK, d, c) \to m$ : It takes secret key $SK$ , ciphertext degree parameter d and ciphertext $c \in F_{p}$ as inputs, and outputs the plaintext m. The formula of decryption is as follows: $\begin{array}{rcl} Dec (SK, d, c) = (c \times s^{- d} \mod p) \mod q \end{array}$ where $s^{- d}$ is multiplicative inverse of $s^{d}$ in $F_{p}$ .

Homomorphic properties. Let $m_{1}$ , $m_{2}$ be two plaintexts, and $c_{1} = s^{d_{1}} (r_{1} q + m_{1}) \mod p$ , $c_{2} = s^{d_{2}} (r_{2} q + m_{2}) \mod p$ are two ciphertexts correspondingly, there are following homomorphic properties.

Homomorphic multiplication: The $d_{1} + d_{2}$ degree ciphertext of $m_{1} \times m_{2}$ can be calculated by $(c_{1} \times c_{2}) \mod p$ , i.e., $\begin{matrix} Enc (SK, d_{1} + d_{2}, m_{1} \times m_{2}) = (c_{1} \times c_{2}) \mod p \end{matrix}$ Note that $| r_{1} | + | r_{2} | + 2 | q | + 1 < | p |$ should be satisfied to correctly decrypt $m_{1} \times m_{2}$ from its ciphertext.

Homomorphic addition: Here, let $d_{1} = d_{2}$ , then the $d_{1}$ degree ciphertext of $m_{1} + m_{2}$ can be calculated by $(c_{1} + c_{2}) \mod p$ , i.e., $\begin{matrix} Enc (SK, d_{1}, m_{1} + m_{2}) = (c_{1} + c_{2}) \mod p \end{matrix}$ To correctly decrypt $m_{1} + m_{2}$ from its ciphertext, $(r_{1} + r_{2}) q + m_{1} + m_{2} < p$ should be satisfied, $r_{1} + r_{2}$ is the random ingredient of the ciphertext of $m_{1} + m_{2}$ .

Homomorphic scalar multiplication: Given $c_{1}$ and a plaintext $m_{2}$ , there are $\begin{matrix} Enc (SK, d_{1}, m_{1} \times m_{2}) = (c_{1} \times m_{2}) \mod p \end{matrix}$ To correctly decrypt ciphertext, $r_{1} m_{2} q + m_{1} \times m_{2} < p$ should be satisfied, $r_{1} m_{2}$ is the random ingredient of the ciphertext of $m_{1} \times m_{2}$ .

2.3. Symbol classification algorithm

In this section, we design a symbol classification algorithm based on the above symmetric homomorphic encryption scheme. In our privacy protection scheme for SLP learning, the cloud server needs to obtain the sign of $\vec{w} \cdot {\vec{x}}_{i}$ by activation function $sign (\cdot)$ , which needs to compare the value of $\vec{w} \cdot {\vec{x}}_{i}$ with 0. However, the cloud server only holds the ciphertext of $\vec{w} \cdot {\vec{x}}_{i}$ . This comparison problem can be converted to the following form.

In our solution, the cloud server holds the ciphertext of $\vec{w} \cdot {\vec{x}}_{i}$ , i.e., $D_{i}$ , and wants to compare the sign of $\vec{w} \cdot {\vec{x}}_{i}$ with the desired output $y_{i}$ , which needs the assistance of the client. Therefore, in order to obtain the sign of $\vec{w} \cdot {\vec{x}}_{i}$ , the cloud server sends back $D_{i}$ to the client firstly, the client obtains the φ by decrypting $D_{i}$ with $SK$ , i.e., $φ = Dec (D_{i})$ . Note that φ is in $F_{q}$ , and $φ ⩾ 0$ . And then, the client can obtain the sign of $\vec{w} \cdot {\vec{x}}_{i}$ according to the φ. To solve above problem, the symbol classification solution is designed as follows.

We assume that the range of $\vec{w} \cdot {\vec{x}}_{i}$ is in $[- α, β]$ , i.e., if $\vec{w} \cdot {\vec{x}}_{i} ⩾ 0$ , $\vec{w} \cdot {\vec{x}}_{i} \in [0, β]$ ; otherwise, $\vec{w} \cdot {\vec{x}}_{i} \in [- α, 0)$ . Let $0 ⩽ β ≪ (q - 1) / 2$ , and $- (q - 1) / 2 ≪ - α < 0$ . Then, if $\vec{w} \cdot {\vec{x}}_{i} ⩾ 0$ , $φ ≪ (q - 1) / 2$ , if $\vec{w} \cdot {\vec{x}}_{i} < 0$ , $φ ≫ (q + 1) / 2$ . The correctness of above comparison algorithm can be described as follows. We consider the following two situations.

Situation of $\vec{w} \cdot {\vec{x}}_{i} ⩾ 0$ : On account of $\vec{w} \cdot {\vec{x}}_{i} \in [0, β]$ and $0 ⩽ β ≪ (q - 1) / 2$ , we have $\begin{matrix} 0 ⩽ \vec{w} \cdot {\vec{x}}_{i} ⩽ β ≪ (q - 1) / 2 \\ \Rightarrow \vec{w} \cdot {\vec{x}}_{i} = φ ≪ (q - 1) / 2 \end{matrix}$

Situation of $\vec{w} \cdot {\vec{x}}_{i} < 0$ : Since $\vec{w} \cdot {\vec{x}}_{i} \in [- α, 0)$ and $- (q - 1) / 2 ≪ - α < 0$ , we have $\begin{matrix} 0 > \vec{w} \cdot {\vec{x}}_{i} ⩾ - α ≫ - (q - 1) / 2 \\ \Rightarrow \vec{w} \cdot {\vec{x}}_{i} = q + \vec{w} \cdot {\vec{x}}_{i} = φ ≫ q - (q - 1) / 2 = (q + 1) / 2 \end{matrix}$

Therefore, $\vec{w} \cdot {\vec{x}}_{i} ⩾ 0 \Rightarrow φ ≪ (q - 1) / 2$ and $\vec{w} \cdot {\vec{x}}_{i} < 0 \Rightarrow φ ≫ (q + 1) / 2$ . Thereby, the client can obtain the sign of $\vec{w} \cdot {\vec{x}}_{i}$ by comparing φ with $(q - 1) / 2$ , and sends it to the cloud server. The sign function $sign (\cdot)$ is redefined in our scheme as follows: $\begin{array}{rcl} sign (t) = \{\begin{matrix} 0, & t = 0 \\ 1, & t ≪ (q - 1) / 2 \\ - 1, & t ≫ (q + 1) / 2 \end{matrix} \end{array}$

3. The security of PSLP scheme

In this section, we give a brief overview of the PSLP scheme and then present that the privacy of medical information cannot be well protected in PSLP scheme [38].

3.1. Overview of PSLP scheme

The PSLP scheme is composed of two stages: the system setting stage and the stage of privacy-preserving SLP learning.

System setting.The hospital runs the key generation algorithm of Paillier cryptosystem to compute the public key $PK$ , and the private key $SK$ according to the security parameter k. And then, the hospital runs the Paillier encryption algorithm $Enc (\cdot)$ to encrypt the medical cases $x = {{\vec{x}}_{1}, \dots, {\vec{x}}_{m}}$ with the public key $PK$ , and obtains the corresponding ciphertexts $e x = {e {\vec{x}}_{1}, \dots, e {\vec{x}}_{m}}$ . Afterwards, the hospital sends the ciphertexts $e x$ and the desired outputs ${y_{1}, \dots, y_{m}}$ ( $y_{i} \in {- 1, 1}$ ) to the cloud server.

Privacy-preserving SLP learning.The main steps of the PSLP scheme are described in Fig. 2. Also, the specific learning process is as follows.

Fig. 2.

The details of PSLP scheme.

Step 1: The hospital generates a random weight vector $\vec{w} = (w_{1}, \dots, w_{n})$ , and transmits it to the cloud server.

Step 2: The cloud server obtains the weight vector $\vec{w} = (w_{1}, \dots, w_{n})$ , and then selects a piece of ciphertext of medical information $e {\vec{x}}_{i} = (e x_{i, 1}, \dots, e x_{i, n})$ ( $i \in {1, \dots, m}$ ), then calculates $M_{j} = e x_{i, j}^{w_{j}}$ ( $w_{j} ⩾ 0$ ) or $M_{j} = e x_{i, j}^{N - | w_{j} |}$ ( $w_{j} < 0$ ) (for $1 ⩽ j ⩽ n$ ) and $A = \prod_{j = 1}^{n} M_{j}$ , the cloud server returns A to the hospital. Here, N represents the product of two large prime numbers in Paillier cryptosystem, it is a part of the public key $PK$ .

Step 3: The hospital obtains the ciphertext A and decrypts it with Paillier decryption algorithm $Dec (\cdot)$ , and then calculates the value of the function $sign (\cdot)$ where we denote $\tilde{y} = sign (Dec (A))$ . Then the $\tilde{y}$ is transmitted to the cloud server.

Step 4: After receiving $\tilde{y}$ , the cloud server compares $\tilde{y}$ with the desired output $y_{i}$ . If $\tilde{y} \neq y_{i}$ , the cloud server will calculate $V_{j} = e x_{i, j}^{η} (y_{i} = 1)$ or $V_{j} = e x_{i, j}^{N - η} (y_{i} = - 1)$ . In the meanwhile, the cloud server encrypts the weight vector $\vec{w}$ by using Paillier encryption algorithm $Enc (\cdot)$ , ${e w}_{j} = Enc (w_{j})$ ( $w_{j} ⩾ 0$ ) or ${e w}_{j} = Enc (N - | w_{j} |)$ ( $w_{j} < 0$ ) (for $1 ⩽ j ⩽ n$ ). And then, the cloud server calculates ${e w}_{j} = {e w}_{j} \cdot V_{j}$ (for $1 ⩽ j ⩽ n$ ) to update weight vector $\vec{w}$ . Ultimately, the cloud server sends back the ciphertext of updated weight vector $e \vec{w}$ to the hospital. If $\tilde{y} = y_{i}$ , the plaintext of weight vector $\vec{w}$ will be returned to the hospital. Note that the weight vector is encrypted only once when it is updated for the first time.

Step 5: If the hospital receives the ciphertext of weight vector $e \vec{w}$ , it will obtain $\vec{w}$ by using the Paillier decryption algorithm $w_{j} = Dec ({e w}_{j})$ (for $1 ⩽ j ⩽ n$ ), and send the decrypted plaintext $\vec{w}$ to the cloud server, then the cloud server continues to run from step 2. The training process will be terminated by the hospital if one of the following conditions is satisfied. The first condition is that the number of iteration is larger than a preset threshold, and the second condition is that there is no misclassification training case, i.e., the gradient descent algorithm has converged.

3.2. Security analysis of PSLP scheme

Because the medical information $x = {{\vec{x}}_{1}, \dots, {\vec{x}}_{m}}$ has tremendous commercial value, the honest-but-curious cloud server may want to obtain the private medical information. In this section, we will show that an honest-but-curious cloud server is able to obtain the private medical information in the PSLP scheme.

Let $A$ be an honest-but-curious cloud server, it is able to obtain the private medical information. That is, in an iteration process, if $A$ updates the weight vector $\vec{w}$ , it will obtain the plaintext of updated weight vector $\vec{w}$ returned by the hospital as described in step 5. Then, $A$ possesses the plaintext of weight vector $\vec{w}$ before updating, the plaintext of updated weight vector $\vec{w}$ , the learning rate η and the desired output $y_{i}$ . Trivially, $A$ can obtain the values of medical case ${\vec{x}}_{i} = (x_{i, 1}, \dots, x_{i, n})$ according to the linear equation with one unknown number ${\vec{w}}_{j} = {\vec{w}}_{j} + η y_{i} {\vec{x}}_{i, j}$ (for $1 ⩽ j ⩽ n$ ). Therefore, the cloud server can obtain the private medical information ${\vec{x}}_{i}$ by solving these equations. Furthermore, the more times the algorithm iterates, the more medical cases will be leaked.

Besides, if the medical cases are disclosed, it will be easier for $A$ to know the practical meaning of each feature value corresponding weight vector $\vec{w}$ . However, in the PSLP scheme, $A$ can also obtain the final optimal weight vector $\vec{w} = (w_{1}, \dots, w_{n})$ , when the terminal condition is the second one as described in step 5. Therefore, the leakage of medical cases will lead to the exposure of the specific SLP model for e-healthcare. In addition, if the hospital needs to outsource their predictive task to the cloud server, the cloud server must obtain the plaintext of weight vector. However, the model also has gigantic commercial value.

In the following, we analyze why the training data and weight vector are not well protected in the PSLP scheme. The main reason is that the cloud server must obtain the plaintext of the weight vector $\vec{w}$ in order to calculate the $\vec{w} \cdot {\vec{x}}_{i}$ ( $i \in {1, \dots, m}$ ) by using Paillier cryptosystem. Trivially, if the cloud server obtains the plaintext of weight vector $\vec{w}$ before updating and the updated weight vector $\vec{w}$ , it will be easy to obtain the medical case ${\vec{x}}_{i}$ by solving this linear equation with one unknown vector $\vec{w} = \vec{w} + η y_{i} {\vec{x}}_{i}$ . Besides, the more plaintext of medical cases the cloud server obtains, the bigger probability of the meaning of each feature will be correctly guessed by the cloud server. In addition, we described above process by a concrete example in [38].

4. The proposed construction

In order to solve the problems of previous schemes, we propose a novel efficient and privacy-preserving scheme for single-layer perceptron learning in cloud computing (EPSLP) using symmetric homomorphic encryption algorithm [22] and the symbol classification algorithm as tools. In this section, we present the high description, system model, security model and concrete EPSLP scheme.

4.1. High description

The design goal of our EPSLP scheme is to provide an efficient SLP learning scheme, while preserving the privacy of training data and predictive model. We use the efficient symmetric homomorphic encryption algorithm [22] as main building block, which supports homomorphic addition and multiplication.

In our EPSLP scheme, the client owns huge data sets, and outsources private training data to the cloud server. The cloud server trains the SLP model in a privacy-preserving way. Concretely, the EPSLP scheme is comprised of two stages: setup and learning. In the setup stage, the client encrypts all features of each sample by using symmetric homomorphic encryption algorithm, and then sends all the ciphertexts and associated desired outputs to the cloud server. In the learning stage, the client initializes a random weight vector and encrypts it. Then the encrypted weight vector is sent to the cloud server. Based on the ciphertexts of training data and weight vector, the cloud server computes the ciphertext of product $\vec{w} \cdot {\vec{x}}_{i} \cdot y_{i}$ and transmits the result to the client. The client decrypts the result and obtains the sign $t_{i}$ of product $\vec{w} \cdot {\vec{x}}_{i} \cdot y_{i}$ based on the symbol classification algorithm. And then, the client sends the sign $t_{i}$ back to the cloud server. According to the sign $t_{i}$ , the cloud server can obtain the symbol ${\tilde{y}}_{i}$ of dot product $\vec{w} \cdot {\vec{x}}_{i}$ . If the symbol ${\tilde{y}}_{i}$ is not identical to the corresponding desired output $y_{i}$ , the cloud server will update the weight vector. If the termination condition is satisfied, the cloud server will send the ciphertext of final optimal weight vector to the client, else the cloud server will perform the next round of training. After receiving the ciphertext of weight vector, the client decrypts it and obtains the SLP predictive model.

4.2. System model

The EPSLP system model is shown in Fig. 3, in which there are two participants, i.e., the client and the cloud server.

Fig. 3.

System model.

The client: It possesses limited computational and storage resources and tremendous training data. On input the secure parameter, the client generates the secret key and public parameter. The client encrypts all the training data by using secret key and transmits all the ciphertexts to the cloud server to train the SLP model.

Cloud server: It is an honest-but-curious cloud service provider possessing massive storage resources and powerful computational ability. Ciphertexts for all training data are stored on the cloud server. And the cloud server trains the SLP model in a privacy-preserving way.

4.3. Security model

Our scheme is based on the honest-but-curious model [21]. In this model, the parties implement the scheme according to the protocol. However, they may try analyzing what they can obtain. Therefore, in order to ensure the security, the cloud server cannot get any other information beyond inputs and outputs. According to the information that the cloud server can obtain, the threat model can be concluded as following.

Known plaintext attack model [22]. In this threat model, the cloud server possesses the ciphertexts of all features, the plaintexts of desired outputs, and all the encrypted weight vectors. Specially, the goal of EPSLP scheme is to prevent the cloud server from obtaining the features of each piece of training data and the optimal weight vector.

In addition, we claim security requirements of the proposed EPSLP from the following three aspects.

Correctness. If the cloud server honestly executes the scheme, the computational results of each step should be correct. Because the major computation operations are conducted on ciphertexts, the computational results under ciphertexts should be identical to the outputs under the operations of corresponding plaintexts. If there are floating-point numbers in training data, the loss of accuracy rate will be negligible.

Privacy. The training data $x = {{\vec{x}}_{1}, \dots, {\vec{x}}_{m}}$ and weight vectors $\vec{w} = (w_{1}, \dots, w_{n})$ should be well protected. On the one hand, the cloud server cannot obtain the plaintexts of training data and weight vector according to corresponding ciphertexts. On the other hand, the cloud server cannot get the plaintexts of training data and weight vector by analyzing the training process. In addition, the secret key of symmetric homomorphic encryption scheme cannot be leaked, even if some training data or weight vectors are obtained by the cloud server owing to other reasons. Note that, it is not necessary to protect the privacy of desired outputs ${y_{1}, \dots, y_{m}}$ .

Efficiency. The training data should be stored on the cloud server. The computation overhead of the client side is far less than that of the cloud server side, i.e., the time-consuming operations should be implemented on the cloud server. In addition, the scheme should be scalable with the changes of feature dimension.

4.4. The concrete EPSLP scheme

The EPSLP scheme mainly is composed of two stages: setup stage and the stage of privacy-preserving SLP learning. The specific information of two phases are described in Fig. 4.

Fig. 4.

The proposed EPSLP scheme.

Setup phase. Given the security parameter k, the client runs the key generation algorithm $KeyGen (k)$ of the symmetric homomorphic encryption scheme [22] to generate the public parameter p and the secret key $SK = (s, q)$ . $SK$ is only stored on the client side. Besides, the client calculates $s^{- 1}$ which is the multiplicative inverse of s and $s^{- 2}$ which is the multiplicative inverse of $s^{2}$ .

Then, the client runs the encryption algorithm $Enc (\cdot)$ to encrypt the training data $x = {{\vec{x}}_{1}, \dots, {\vec{x}}_{m}}$ with $SK$ , and obtains the corresponding ciphertext $e x = {e {\vec{x}}_{1}, \dots, e {\vec{x}}_{m}}$ , where ${\vec{x}}_{i} = (x_{i, 1}, \dots, x_{i, n})$ , $e {\vec{x}}_{i} = (e x_{i, 1}, \dots, e x_{i, n})$ . In our solution, let the ciphertext degree parameter $d = 1$ when using the encryption algorithm $Enc (\cdot)$ to encrypt message. The encryption formula for $x_{i, j}$ is as follows: $\begin{array}{rcl} e x_{i, j} = Enc (SK, 1, x_{i, j} \cdot y_{i}) \end{array}$ And then, the client sends the ciphertexts $e x$ and the associated desired outputs ${y_{1}, \dots, y_{m}}$ to the cloud server. We note that the leakage of ${y_{1}, \dots, y_{m}}$ is not harmful to the owners of training data, and will be no compromise on the security of this scheme.

Privacy-preserving SLP learning phase. The main steps of the EPSLP scheme are depicted as follows.

Step 1: The client initializes a random weight vector $\vec{w} = (w_{1}, \dots, w_{n})$ and runs the encryption algorithm $Enc (\cdot)$ to encrypt all weights. After obtaining the corresponding ciphertext $e \vec{w} = ({e w}_{1}, \dots, {e w}_{n})$ , the client sends $e \vec{w}$ to the cloud server (note: this step is only run once and the communication of this step can be along with the communication of setup). The encryption formula for $w_{j}$ is as follows: $\begin{array}{rcl} {e w}_{j} = Enc (SK, 1, w_{j}) \end{array}$

Step 2: After obtaining the encrypted weight vector $e \vec{w} = ({e w}_{1}, \dots, {e w}_{n})$ , the cloud server randomly selects a piece of encrypted training case $e {\vec{x}}_{i} = (e x_{i, 1}, \dots, {ex}_{i, n})$ and computes $\begin{matrix} D_{i, j} = (e x_{i, j} \times {e w}_{j}) \mod p, for 1 ⩽ j ⩽ n \\ D_{i} = \sum_{j = 1}^{n} D_{i, j} \mod p \end{matrix}$ by using homomorphic multiplication property and homomorphic addition property of the symmetric homomorphic encryption algorithm. $D_{i}$ is sent back to the client. Note that the ciphertext degree of the $D_{i, j}$ is 2, thus, the ciphertext degree of $D_{i}$ is also 2.

Step 3: The client receives the ciphertext $D_{i}$ and calculates $\begin{matrix} φ = Dec (SK, 2, D_{i}), t_{i} = sign (φ) \end{matrix}$ Then, $t_{i}$ is sent to the cloud server.

Step 4: Receiving $t_{i}$ , the cloud server compares it with 0, if $t_{i} = 0$ , i.e., $\vec{w} \cdot {\vec{x}}_{i} = 0$ , then we let ${\tilde{y}}_{i} = 1$ , i.e., $sign (\vec{w} \cdot {\vec{x}}_{i}) = 1$ , else calculates ${\tilde{y}}_{i} = y_{i} \cdot t_{i}$ .

Then, the cloud server compares ${\tilde{y}}_{i}$ with $y_{i}$ , if ${\tilde{y}}_{i} \neq y_{i}$ , the cloud server will calculate $\begin{matrix} A_{j} = η \times e x_{i, j} \mod p, {e w}_{j} = ({e w}_{j} + A_{j}) \mod p, for 1 ⩽ j ⩽ n \end{matrix}$ to update weight vector $\vec{w}$ by using homomorphic scalar multiplication property and homomorphic addition property. If ${\tilde{y}}_{i} = y_{i}$ , it is not necessary to update the weight vector $\vec{w}$ . Note that the ciphertext degree of the updated ${e w}_{j}$ is 1.

So far, if the termination condition is not satisfied, the cloud server will go to step 2 to continue iterating. If the termination condition is satisfied, the cloud server will send the ciphertext of final optimal weight vector $e \vec{w}$ to the client. The termination condition is that there is no misclassification training case, i.e., the gradient descent algorithm has converged, or the number of iteration is larger than a preset threshold.

Step 5: The client receives the ciphertext of the final optimal weight vector $e \vec{w}$ , and calculates $\begin{array}{rcl} w_{j} = Dec (SK, 1, {e w}_{j}), for 1 ⩽ j ⩽ n \end{array}$ to obtain the plaintext of the final optimal weight vector $\vec{w} = (w_{1}, \dots, w_{n})$ , i.e., the SLP predictive model. This step is only performed once when the training process is terminated.

5. Analysis of our EPSLP scheme

In this section, we conduct the security and efficiency analysis of the proposed EPSLP scheme based on the threat model and security requirements in Section 4.3. Security analysis consists of correctness and privacy analysis. Efficiency analysis is composed of theoretical analysis of computational and communication efficiency.

5.1. Security analysis

In this section, we analyze the security of our EPSLP scheme from the aspects of correctness and privacy. Firstly, we analyze the correctness of our EPSLP scheme based on the correctness of symmetric homomorphic encryption scheme [22] used in our scheme.

Claim 1.
In the proposed EPSLP, if the cloud server honestly executes the scheme, the computational results of each step should be correct.
Proof.
The correctness of EPSLP scheme is introduced as follows. Note that all plaintext messages are in $F_{q}$ .

In step 2, the cloud server computes: $\begin{matrix} D_{i, j} & = (e x_{i, j} \times {e w}_{j}) = (Enc (SK, 1, x_{i, j} \cdot y_{i}) \times Enc (SK, 1, w_{j})) \\ = Enc (SK, 2, x_{i, j} \cdot y_{i} \cdot w_{j}), for 1 ⩽ j ⩽ n \end{matrix}$ Then the cloud server calculates $D_{i}$ : $\begin{matrix} D_{i} = \sum_{j = 1}^{n} D_{i, j} = \sum_{j = 1}^{n} Enc (SK, 2, x_{i, j} \cdot y_{i} \cdot w_{j}) = Enc (SK, 2, \sum_{j = 1}^{n} x_{i, j} \cdot y_{i} \cdot w_{j}) \end{matrix}$ In step 3, the client obtains $D_{i}$ , then decrypts it: $\begin{matrix} φ & = Dec (SK, 2, D_{i}) = Dec (SK, 2, Enc (SK, 2, \sum_{j = 1}^{n} x_{i, j} \cdot y_{i} \cdot w_{j})) = \sum_{j = 1}^{n} x_{i, j} \cdot y_{i} \cdot w_{j} = y_{i} \cdot (\vec{w} \cdot {\vec{x}}_{i}) \end{matrix}$ Then the client obtains the sign of $y_{i} (\vec{w} \cdot {\vec{x}}_{i})$ , i.e., $t_{i}$ : $\begin{matrix} t_{i} & = sign (φ) = sign (y_{i} \cdot (\vec{w} \cdot {\vec{x}}_{i})) \end{matrix}$ The sign function $sign (\cdot)$ can detect whether $y_{i} (\vec{w} \cdot {\vec{x}}_{i}) > 0$ or not correctly by comparing it with $(q - 1) / 2$ as mentioned previously. In step 4, if $t_{i} = 0$ , let ${\tilde{y}}_{i} = 1$ , i.e., if $\vec{w} \cdot {\vec{x}}_{i} = 0$ , let ${\tilde{y}}_{i} = 1$ . If $t_{i} \neq 0$ , the cloud server will calculate ${\tilde{y}}_{i}$ : $\begin{matrix} {\tilde{y}}_{i} & = y_{i} \cdot t_{i} = y_{i} \cdot sign (y_{i} (\vec{w} \cdot {\vec{x}}_{i})) = y_{i} \cdot y_{i} \cdot sign (\vec{w} \cdot {\vec{x}}_{i}) = sign (\vec{w} \cdot {\vec{x}}_{i}) \end{matrix}$ Therefore, ${\tilde{y}}_{i}$ represents the sign of $\vec{w} \cdot {\vec{x}}_{i}$ , and ${\tilde{y}}_{i} \in {- 1, + 1}$ . And then, if ${\tilde{y}}_{i} \neq y_{i}$ , the weight vector will be updated as follows: $\begin{array}{l} \begin{matrix} A_{j} & = η \times e x_{i, j} = η \times Enc (SK, 1, x_{i, j} \cdot y_{i}) = Enc (SK, 1, η \cdot x_{i, j} \cdot y_{i}), for 1 ⩽ j ⩽ n \end{matrix} \\ \begin{matrix} {e w}_{j} & = ({e w}_{j} + A_{j}) = (Enc (SK, 1, w_{j}) + Enc (SK, 1, η \cdot x_{i, j} \cdot y_{i})) \\ = Enc (SK, 1, w_{j} + η \cdot x_{i, j} \cdot y_{i}), for 1 ⩽ j ⩽ n \end{matrix} \end{array}$

Based on the above analysis, the EPSLP scheme is correct. Thus, the client can obtain predictive model with the assistance of the cloud server. □

Secondly, we prove the privacy of training data and predictive model from the aspects of storage and training process. From the aspect of storage, we demonstrate the privacy of training data and predictive model based on the security of symmetric homomorphic encryption scheme [22]. From the aspect of training process, we analyze the privacy of training data and predictive model based on the hardness of solving the underdetermined nonlinear systems. Claim 2.
In the proposed EPSLP, the privacy of training data $x = {{\vec{x}}_{1}, \dots, {\vec{x}}_{m}}$ and the predictive model $\vec{w} = (w_{1}, \dots, w_{n})$ should be guaranteed under the honest-but-curious model.
Proof.
We analyze the privacy of the proposed EPSLP scheme from the aspects of storage and the process of algorithm execution.
Privacy of the training data and weight vectors stored on the cloud server. In our EPSLP scheme, the training data and weight vector stored on the cloud server are encrypted by symmetric homomorphic encryption algorithm [22] which is based on the NP-hard problem of solving the underdetermined nonlinear system. Since the symmetric homomorphic encryption algorithm is secure under known-plaintext attacks, the training data and weight vector stored on the cloud server are privacy-preserving under known-plaintext attacks. Therefore, if some of the plaintexts of training data are leaked, the security of other training data will not be weakened. In the meanwhile, the leakage of label ${y_{1}, \dots, y_{m}}$ is not harmful to the owners of training data, and will be no compromise on the security of this scheme.

Privacy of the training data and predictive model in the process of learning. In step 2, the cloud server computes $\begin{matrix} D_{i, j} = (e x_{i, j} \times {e w}_{j}) \mod p, for 1 ⩽ j ⩽ n, D_{i} = \sum_{j = 1}^{n} D_{i, j} \mod p \end{matrix}$ All these calculations are executed on ciphertexts. Besides, if the plaintext of $e x_{i, j}$ and the plaintext of ${e w}_{j}$ are leaked owing to other reasons, then $\begin{matrix} e x_{i, j} = s (r_{1} q + x_{i, j} y_{i}) \mod p, for 1 ⩽ j ⩽ n \\ {e w}_{j} = s (r_{2} q + w_{j}) \mod p, for 1 ⩽ j ⩽ n \end{matrix}$ There are $2 n + 2$ unknown numbers in these $2 n$ equations, i.e., the attacker cannot obtain the secret key $SK$ , thus, security of other information is not threatened. Based on the previous analysis, all the training data and weight vectors are well protected in this step. In step 4, the cloud server can obtain the sign ${\tilde{y}}_{i}$ of $\vec{w} \cdot {\vec{x}}_{i}$ according to the received $t_{i}$ , and then computes $\begin{matrix} A_{j} = η \times e x_{i, j} \mod p, {e w}_{j} = ({e w}_{j} + A_{j}) \mod p, for 1 ⩽ j ⩽ n \end{matrix}$ All these computations are also executed on ciphertexts of training data ${\vec{x}}_{i}$ and weight vector $\vec{w}$ , and the leakage of label ${y_{1}, \dots, y_{m}}$ , the sign ${{\tilde{y}}_{1}, \dots, {\tilde{y}}_{m}}$ , received $t_{i}$ and learning rate η do not pose any threat to the security of training data and weight vectors. In addition, if the plaintext of $e x_{i, j}$ is leaked for other problems, then $\begin{matrix} A_{j} & = η \times e x_{i, j} = s (r_{1} η q + η x_{i, j} y_{i}) \mod p, for 1 ⩽ j ⩽ n \end{matrix}$ In this equation, there are three unknown numbers: s, q, $r_{1}$ cannot be solved. Furthermore, if the weight vector before updating is also leaked, there are $\begin{matrix} {e w}_{j} & = ({e w}_{j} + A_{j}) = s ((r_{1} η + r_{2}) q + w_{j} + η x_{i, j} y_{i}) \mod p, for 1 ⩽ j ⩽ n \end{matrix}$ An unknown number $r_{2}$ is added. Thus, the secret key $SK$ cannot also be leaked. All the training data and weight vectors are also well protected in this step. Last but not least, in our EPSLP scheme, the computation of $\vec{w} \cdot {\vec{x}}_{i}$ needs the ciphertext of weight vector $\vec{w}$ and the ciphertext of training case ${\vec{x}}_{i}$ , which can avoid the linear equation with one unknown $w_{j} = w_{j} + η y_{i} x_{i, j}$ (for $1 ⩽ j ⩽ n$ ). Therefore, the training data and the predictive model are privacy-preserving in the iteration process of our scheme.
□

5.2. Efficiency analysis

In this section, we conduct the theoretical analysis of computational and communication efficiency for the proposed EPSLP algorithm. In addition, we give the concrete comparison between PSLP scheme [33] and our EPSLP scheme, and the specific comparison between PPDP scheme [43] and our EPSLP scheme.

Table 1
The comparison of computational overhead

Phase Entity PSLP PPDP EPSLP

Setup Client $m n E + m n M$ $(3 m n^{3} + 2 m n^{2}) M + 3 m n^{2} (n - 1) A$ $3 m n M + m n A$

Learning Client $(n + 1) E + (n + 1) D$ $(5 n^{2} + n) M + (4 n^{2} - 3 n) A$ $1 M$

Cloud $3 n E + (3 n - 1) M$ $(2 n^{3} + n^{2}) M + 2 n^{2} (n - 1) A$ $(2 n + 1) M + (2 n - 1) A$

Phase	Entity	PSLP	PPDP	EPSLP
Setup	Client	$m n E + m n M$	$(3 m n^{3} + 2 m n^{2}) M + 3 m n^{2} (n - 1) A$	$3 m n M + m n A$
Learning	Client	$(n + 1) E + (n + 1) D$	$(5 n^{2} + n) M + (4 n^{2} - 3 n) A$	$1 M$
Cloud	$3 n E + (3 n - 1) M$	$(2 n^{3} + n^{2}) M + 2 n^{2} (n - 1) A$	$(2 n + 1) M + (2 n - 1) A$

Computational overhead. The comparisons of computational cost are given in Table 1. For the simplicity of expression, the time cost of one addition operation, one multiplication operation, one exponentiation and one modular inverse operation are represented by A, M, E and D, respectively.

In setup, the client encrypts the training data $x = {{\vec{x}}_{1}, \dots, {\vec{x}}_{m}}$ by running the encryption algorithm $Enc (\cdot)$ . The encryption of one message needs two multiplication operations and one addition operation in the symmetric homomorphic encryption scheme. In the EPSLP scheme, we need to calculate $x_{i, j} y_{i}$ when encrypting $x_{i j}$ . Therefore, the encryption of one training data $x_{i j}$ needs three multiplication operations and one addition operation. The encryption of all the training data cases needs $3 m n$ multiplication operations and $m n$ addition operations. In Table 1, we only compare the encryption cost of two scheme in this stage. Note that this stage only runs once in the whole algorithm.

In step 1, the client initializes a random weight vector $\vec{w}$ and encrypts it, which needs $2 n$ multiplication operations and n addition operations. In step 2, the cloud server performs n multiplication operations $e x_{i, j} \times {e w}_{j}$ ( $1 ⩽ j ⩽ n$ ) for selected training data case $e {\vec{x}}_{i}$ , and $(n - 1)$ addition operations to calculate the sum of the results of all multiplications in each training round. In step 3, the client decrypts the ciphertext of dot product $D_{i}$ in each training round, which costs one multiplication operation. In step 4, the cloud server performs $(n + 1)$ multiplication operations and n addition operations to update the weight vector in each round when the weight vector needs to be updated. In step 5, the client decrypts the final optimal weight vector that costs n multiplication operations. The step 1 and step 5 perform only once in the learning process, whose time cost can be neglected relative to the overhead of entire learning phase. Thus, they are not included in learning stage in Table 1.

Compared with PSLP scheme [33], the EPSLP scheme is superior in efficiency of computational cost. Above all, in the EPSLP scheme, it is not necessary to decrypt the ciphertext of weight vector in step 5 after updating weight vector. In addition, the efficiency of symmetric homomorphic encryption scheme [22] is much more efficient than Paillier cryptosystem.

Compared with the PPDP scheme, the EPSLP scheme is efficient in the phases of setup and SLP learning. Above all, plenty of matrix multiplication operations are required in PPDP scheme. Besides, in the SLP learning phase, the computational complexity of the client side in PPDP scheme is higher than that of our scheme, which will seriously affect the efficiency of the entire scheme. And the computational cost of the client side will increase with the addition of feature dimension in PPDP scheme. In addition, the change of feature dimension is inevitable in reality, in the PPDP scheme, the client needs to generate secret key again and encrypts all the training data again whatever the increase or decrease of feature dimension. However, in our EPSLP scheme, the client only needs to encrypt the new coming features for all samples when the feature dimension increases, which is efficient in the phase of setup. Therefore, our EPSLP scheme is much more scalable than PPDP scheme when feature dimension increases or decreases.

Table 2

The comparison of communication overhead

Scheme	PSLP		PPDP		EPSLP

Message	Overhead	Iteration	Overhead	Iteration	Overhead	Iteration
$e {\vec{x}}_{i}$	$O (m n)$	Õ	$O (m n^{2})$	Õ	$O (m n)$	Õ
$\vec{w}$	$O (n)$	√	−	−	−	−
$e \vec{w}$	$O (n)$	√	$O (n^{2})$	√	$O (n)$	Õ
$D_{i}$	$O (1)$	√	−	−	$O (1)$	√
$t_{i}$	$O (1)$	√	−	−	$O (1)$	√
$η C_{i}$	−	−	$O (n^{2})$	√	−	−

Communication overhead. The comparisons of communication cost are given in Table 2. In this table, the iteration indicates the message needs to be transmitted repeatedly.

In setup stage, the client needs to send public parameter p with $| p |$ bits, m pieces ciphertexts of training data cases with $m n | p |$ bits, and the corresponding desired outputs with $m | y |$ bits to the cloud server, where y represents the desired output or the final result of step 3, $y \in {- 1, 0, + 1}$ . The communications in this stage are only performed once.

In step 1, the client sends encrypted weight vector $e \vec{w}$ with $n | p |$ bits to the cloud server. In step 2, the cloud server returns the ciphertext of dot product $D_{i}$ with $| p |$ bits to the client in each training round. In step 3, the client sends the sign $t_{i}$ with $| y |$ bits to the cloud server in each training round. In step 4, if the termination condition is satisfied, the cloud server will return the ciphertext of final optimal weight vector $e \vec{w}$ with $n | p |$ bits.

Compared with the PSLP scheme [33], our EPSLP scheme is superior in communication cost according to the Table 2. In the PSLP scheme, the cloud server needs to send the updated weight vector $e \vec{w}$ to the client, then the client decrypts it and transmits the plaintext $\vec{w}$ to the cloud server. Therefore, in our scheme, a round of communication is reduced in each training round with updating of weight vector.

Compared with the PPDP scheme [43], our EPSLP scheme is efficient in communication cost. In the Table 2, the $e {\vec{x}}_{i}$ denotes an encrypted training data sample, it is composed of a n-dimension vector in our EPSLP scheme. However, in PPDP scheme, $e {\vec{x}}_{i}$ is comprised of two matrices with dimension $n \times n$ , which lead to increasing not only in communication but also in storage cost on the cloud server side. The $e \vec{w}$ denotes the encrypted weight vector, which consists of a n-dimension vector in our EPSLP scheme. However, in the PPDP scheme, it is comprised of a matrix with dimension $n \times n$ , and needs to be transmitted repeatedly in the process of training. The $η C_{i}$ denotes the ciphertext need to be updated in PPDP, which is composed of a matrix with dimension $n \times n$ and is also transmitted repeatedly in the process of training. The number of iteration is $O (m)$ , and the round of training is also $O (m)$ . There is little difference between these two numbers. The $D_{i}$ and $t_{i}$ represent the ciphertext returned to the client and the decrypted sign sent to the cloud server in the process of training respectively. And they are just a single value rather than a matrix or a vector respectively. In addition, when the feature dimension changes, there are much more communication cost in the setup phase of PPDP scheme than that of the EPSLP scheme, owing to the limited scalability of the PPDP scheme.

6. Performance evaluation

We implement the proposed EPSLP scheme and the PSLP scheme [33] using JAVA language. The cloud server side process is implemented on a computer with Intel(R) Xeon(R) CPU E5-1620V3 processor running at 3.40 GHz, 16 GB RAM. The client side process is conducted on a laptop with Intel(R) Core(TM) i5-6200U CPU processor running at 2.30 GHz, 8 GB RAM. The datasets are depicted firstly, followed by the performance evaluation in terms of efficiency and accuracy rate.

We perform the evaluation experiments on two representative binary classification datasets, namely, Wisconsin Breast Cancer Database (January 8, 1991) [5] and Default of Credit Card Clients Data Set [41]. In the Wisconsin Breast Cancer Database, there are 683 samples with 9 features grouped into 2 categories, eliminating 16 unavailable points. In the Default of Credit Card Clients Data Set, there are 30000 samples with 23 features grouped 2 categories.

The training parameters on Wisconsin Breast Cancer Database are set as follows: preset threshold is equal to 10000, learning rate $η = 1$ , the initialized weight vector is generated randomly. The training parameters on Default of Credit Card Clients Data Set are set as follows: preset threshold is equal to 60000, learning rate $η = 1$ , the initialized weight vector is also generated randomly. Note that, the number of iteration is identical to that of training round with updating of weight vector. Thus, preset threshold is equivalent to the maximum number of training round with updating of weight vector. In addition, the training data $x_{i, j} \in {\vec{x}}_{i}$ may be floating-point number originally, while the symmetric homomorphic encryption scheme works in $F_{q}$ . For the efficient calculation, each training data $x_{i, j}$ multiplies a constant number (e.g., expands with 1000 times) and the fractional part is discarded such that the absolute values of all $x_{i, j}$ are integer values in $F_{q}$ , which are accompanied by negligible loss of accuracy. In the meanwhile, we must ensure all the $x_{i, j} ≪ q / 2$ .

To use the symmetric homomorphic encryption schemes in our solutions, the bit lengths of secret key and parameters must be carefully selected. The parameters of symmetric homomorphic encryption scheme [22] we selected are described in Table 3. The $Num$ represents the training round. In the meanwhile, the experimental parameter settings of Paillier cryptosystem are the same as that of PSLP scheme [33].

Table 3
Experimental parameter setting

Parameter Bit length Remark

k 128 security parameter

q 128 depends on the security parameter

p 640 $2 | r | + 2 | q | + | Num | + | n | + | η | + 1 < | p |$

s 640 s is in $Z_{p}^{*}$

r 128 random ingredient

Parameter	Bit length	Remark
k	128	security parameter
q	128	depends on the security parameter
p	640	$2 \| r \| + 2 \| q \| + \| Num \| + \| n \| + \| η \| + 1 < \| p \|$
s	640	s is in $Z_{p}^{*}$
r	128	random ingredient

Fig. 5.

The comparison of computation cost on each step.

Fig. 6.

Time cost varying with data set size.

Fig. 7.

Time cost varying with feature dimension.

6.1. Efficiency evaluation

Firstly, we elaborately assess the efficiency of the EPSLP scheme on each step by using Breast Cancer Database. Besides, we evaluate the effect of data set size and feature dimension on efficiency by using Default of Credit Card Clients Data Set. To evaluate the performance and give a fair comparison, we perform the PSLP scheme [33] and our EPSLP scheme on the identical client side environment, cloud side environment and training parameters.

The efficiency evaluation on each step. Figure 5 shows the time cost on each step and provides a detailed comparison between the EPSLP scheme and the PSLP scheme. As shown in Fig. 5, our EPSLP scheme is several orders of magnitude faster than PSLP scheme on the setup, step 2, step 3, step 4 and step 5. For the setup stage, PSLP scheme needs 86.8 s to encrypt all the training data, EPSLP only needs 3 ms. For PSLP, the time cost of step 1 is 0.011 ms. However, for EPSLP, it is 0.016 ms, since the weight vector needs to be encrypted on the client side in our scheme. The cost time of step 5 in PSLP and EPSLP scheme are 1121.8 s and 0.02 ms, respectively. Reason for the great disparity is that the hospital needs to decrypt the ciphertext of weight vector when the weight vector is updated in PSLP scheme. However, in our scheme, the weight vector is decrypted only once when the terminal condition is satisfied. Therefore, the cost time of step 5 in the PSLP scheme increases with the number of updating.

The influence of data set size on efficiency. We assess the impact of data set size on efficiency by making the data set range from 3000 to 30000 both in the EPSLP scheme and PSLP scheme. As demonstrated in Fig. 6(a), the time cost of setup stage increases with the number of records. For EPSLP, the time cost of setup ranges from 29 ms to 247 ms. For PSLP, the time ranges from 565 s to 5670 s. However, as demonstrated in Fig. 6(b), the time cost of learning stage is stable with the increase of records. For example, in the learning stage of EPSLP scheme, when $m = 3000$ , the time cost is 4.23 s. The time cost is 4.71 s, when $m = 30000$ . They are on the same order of magnitude. The reason is that the number of training is almost constant when the data distribution and the number of iteration are determined. In addition, the EPSLP scheme is orders of magnitude faster than the PSLP scheme both in the stage of setup and learning when the data set size ranges from 3000 to 30000.

Table 4
The comparison of accuracy rate

Dataset	SLP	PSLP	EPSLP
Breast Cancer	0.974	0.974	0.974
Default of Credit Card	0.7788	0.7788	0.7788

The influence of feature dimension on efficiency. We assess the effect of feature dimension on efficiency by making the number of feature range from 5 to 23 both in the EPSLP scheme and PSLP scheme. As shown in the Fig. 7(a) and Fig. 7(b), the time cost of setup and learning stage increase with the number of feature, respectively. For the time cost of setup stage, EPSLP scheme ranges from 51 ms to 247 ms, PSLP scheme ranges from 939 s to 5670 s. For the time cost of learning stage, EPSLP ranges from 1.62 s to 4.71 s, PSLP ranges from 1095 s to 32597 s. In our EPSLP scheme, the increase of feature dimension will result in the increase of multiplication operation. In PSLP scheme, there will be much more exponentiation operations when the number of feature increases. In addition, when the feature dimension increases, the time cost of client side varies slowly in the learning stage of our EPSLP scheme. However, it will obviously increase in the learning stage of PSLP scheme. The reason is that the client needs to decrypt the weight vector when it is updated in the PSLP scheme.

6.2. Accuracy rate evaluation

To analyze the accuracy loss of our EPSLP scheme, we classify the above two datasets by using our EPSLP scheme and the PSLP scheme. In the meanwhile, we classify the above two datasets by using non-privacy-preserving SLP learning scheme (i.e., the training program executed on plaintexts) whose parameter settings before training are the same as that of EPSLP scheme.

Table 4 shows the accuracy rates of EPSLP scheme, PSLP scheme and the non-privacy-preserving SLP learning scheme. As shown in Table 4, the accuracy rates of three schemes are completely identical for the two datasets, because that the elements of two datasets are integer. If there are floating-point numbers in dataset, the accuracy rate of our EPSLP scheme and that of PSLP scheme will reduce a little according to the number of digits reserved after the decimal point. However, the loss of accuracy rate is negligible. Besides, if the number of digits reserved in the PSLP scheme is the same as that of our EPSLP scheme, accuracy rates of the two schemes will be identical.

7. Conclusions

In this paper, we review the PSLP scheme [33], in which the honest-but-curious cloud server can obtain the sensitive training data based on that one linear equation with one unknown can be solved. Besides, we propose an efficient and privacy-preserving single-layer perceptron learning scheme in cloud computing, named EPSLP. EPSLP is characterized by using a symmetric homomorphic encryption algorithm, which enables the client with limited resources to outsource encrypted training data and training tasks to the cloud server. Security analysis indicates that the proposed EPSLP can protect sensitive information. Results of experiments constructed on two real datasets show that the training data can be encrypted expeditiously, the predictive models can be trained efficiently and the accuracy rate is acceptable.

Footnotes

Acknowledgements

This work is supported by National Natural Science Foundation of China (Nos. 61572382 and 61702401), Fundamental Research Funds for the Central Universities (No. XJS17053), Key Project of Natural Science Basic Research Plan in Shaanxi Province of China (No. 2016JZ021), Guangxi Key Laboratory of Cryptography and Information Security (No. GCIS201606), Guangxi Colleges and Universities Key Laboratory of Cloud Computing and Complex Systems (Nos. YF16101 and YF17103), and Guangxi Cooperative Innovation Center of Cloud Computing and Big Data (Nos. YD16506 and YD17X07).

References

Abadi,

Chu,

Goodfellow,

H.B.

McMahan,

Mironov,

Talwar and

Zhang, Deep learning with differential privacy, in: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, 2016, pp. 308–318.

Agrawal and

C.C.

Aggarwal, On the design and quantification of privacy preserving data mining algorithms, in: Proceedings of the Twentieth ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, 2001, pp. 247–255. doi:10.1145/375551.375602.

Agrawal and

Srikant, Privacy-preserving data mining, in: Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data, 2000, pp. 439–450. doi:10.1145/342009.335438.

Bansal,

Chen and

Zhong, Privacy preserving back-propagation neural network learning over arbitrarily partitioned data, Neural Computing and Applications 20(1) (2011), 143–150. doi:10.1007/s00521-010-0346-z.

K.P.

Bennett and

O.L.

Mangasarian, Robust linear programming discrimination of two linearly inseparable sets, Optimization Methods and Software 1(1) (1992), 23–34. doi:10.1080/10556789208805504.

Brakerski,

Gentry and

Vaikuntanathan, (Leveled) fully homomorphic encryption without bootstrapping, in: Proceedings of ACM Innovations in Theoretical Computer Science, 2012, pp. 309–325.

Bu,

Ma,

Chen and

Xu, Privacy preserving back-propagation based on BGV on cloud, in: Proceedings of 17th IEEE International Conference on High Performance Computing and Communications, 2015, pp. 1791–1795.

Castiglione,

De Santis,

Masucci,

Palmieri,

Huang and

Castiglione, Supporting dynamic updates in storage clouds with the Akl-Taylor scheme, Information Sciences 387 (2017), 56–74. doi:10.1016/j.ins.2016.08.093.

Castiglione,

Pizzolante,

A.D.

Santis,

Carpentieri,

Castiglione and

Palmieri, Cloud-based adaptive compression and secure management services for 3D healthcare data, Future Generation Computer Systems 43–44 (2015), 120–134. doi:10.1016/j.future.2014.07.001.

10.

Chen and

Zhong, Privacy-preserving backpropagation neural network learning, IEEE Transactions on Neural Networks 20(10) (2009), 1554–1564. doi:10.1109/TNN.2009.2026902.

11.

Chen,

Huang,

Li,

Ma,

Lou and

D.S.

Wong, New algorithms for secure outsourcing of large-scale systems of linear equations, IEEE Transactions on Information Forensics and Security 10(1) (2015), 69–78. doi:10.1109/TIFS.2014.2363765.

12.

Chen,

Li,

Huang,

Ma and

Lou, New publicly verifiable databases with efficient updates, IEEE Transactions on Dependable and Secure Computing 12(5) (2015), 546–556. doi:10.1109/TDSC.2014.2366471.

13.

Chen,

Li,

Ma,

Tang and

Lou, New algorithms for secure outsourcing of modular exponentiations, IEEE Transactions on Parallel and Distributed Systems 25(9) (2014), 2386–2396. doi:10.1109/TPDS.2013.180.

14.

Chen,

Li,

Weng,

Ma and

Lou, Verifiable computation over large database with incremental updates, IEEE Transactions on Computers 65(10) (2016), 3184–3195. doi:10.1109/TC.2015.2512870.

15.

Dong,

Chen and

Wen, When private set intersection meets big data: An efficient and scalable protocol, in: Proceedings of the 2013 ACM SIGSAC Conference on Computer & Communications Security, 2013, pp. 789–800.

16.

Fan and

Bifet, Mining big data: Current status, and forecast to the future, ACM SIGKDD Explorations Newsletter 14(2) (2013), 1–5. doi:10.1145/2481244.2481246.

17.

Freund and

R.E.

Schapire, Large margin classification using the perceptron algorithm, Machine learning 37(3) (1999), 277–296. doi:10.1023/A:1007662407062.

18.

Huang,

Du and

Chen, Deriving private information from randomized data, in: Proceedings of the 2005 ACM SIGMOD International Conference on Management of Data, 2005, pp. 37–48. doi:10.1145/1066157.1066163.

19.

Jiang,

Chen and

Ma, Public integrity auditing for shared dynamic cloud data with group user revocation, IEEE Transactions on Computers 65(8) (2016), 2363–2373. doi:10.1109/TC.2015.2389955.

20.

Jiang,

Chen,

Wu,

Ma,

Susilo and

Lou, Secure and efficient cloud data deduplication with randomized tag, IEEE Transactions on Information Forensics and Security 12(3) (2017), 532–543. doi:10.1109/TIFS.2016.2622013.

21.

Kissner and

Song, Privacy-preserving set operations, in: Proceedings of 25th Annual International Cryptology Conference, 2005, pp. 241–257.

22.

Li,

Lu,

K.-K.R.

Choo,

Datta and

Shao, Privacy-preserving-outsourced association rule mining on vertically partitioned databases, IEEE Transactions on Information Forensics and Security 11(8) (2016), 1847–1861. doi:10.1109/TIFS.2016.2561241.

23.

Lindell and

Pinkas, Privacy preserving data mining, in: Proceedings of 20th Annual International Cryptology Conference, 2000, pp. 36–54.

24.

Liu,

Wang,

Liu,

Peng and

Wu, Achieving reliable and secure services in cloud computing environments, Computers & Electrical Engineering 59 (2017), 153–164. doi:10.1016/j.compeleceng.2016.10.005.

25.

Mohassel and

Zhang, SecureML: A system for scalable privacy-preserving machine learning, IACR Cryptology ePrint Archive 2017 (2017), 396, http://eprint.iacr.org/2017/396 .

26.

Saranya,

Premalatha and

Rajasekar, A survey on privacy preserving data mining, in: Proceedings of 2nd International Conference on Electronics and Communication Systems, 2015, pp. 1740–1744.

27.

Schlitter, A protocol for privacy preserving neural network learning on horizontal partitioned data, PSD (2008).

28.

Schmidhuber, Deep learning in neural networks: An overview, Neural networks 61 (2015), 85–117. doi:10.1016/j.neunet.2014.09.003.

29.

Shi,

Bai and

Yao, An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition, IEEE Transactions on Pattern Analysis and Machine Intelligence 39(11) (2017), 2298–2304. doi:10.1109/TPAMI.2016.2646371.

30.

J.J.

Shynk, Performance surfaces of a single-layer perceptron, IEEE Transactions on Neural Networks 1(3) (1990), 268–274. doi:10.1109/72.80252.

31.

Song,

Chaudhuri and

A.D.

Sarwate, Stochastic gradient descent with differentially private updates, in: Proceedings of IEEE Global Conference on Signal and Information Processing, 2013, pp. 245–248.

32.

Wang,

Liu,

Li,

Yang and

Wu, Outsourcing privacy-preserving social networks to a cloud, in: Proceedings of the IEEE INFOCOM, 2013, pp. 2886–2894. doi:10.1109/INFCOM.2013.6567099.

33.

Wang,

Lu and

Huang, PSLP: Privacy-preserving single-layer perceptron learning for e-Healthcare, in: Proceedings of 10th International Conference on Information, Communications and Signal Processing, 2015, pp. 1–5.

34.

Wang,

Chen,

Huang,

You and

Xiang, Verifiable auditing for outsourced database in cloud computing, IEEE Transactions on Computers 64(11) (2015), 3293–3303. doi:10.1109/TC.2015.2401036.

35.

Wang,

Chen,

Li,

Kluczniak and

Kutylowski, TrDup: Enhancing secure data deduplication with user traceability in cloud computing, International Journal of Web and Grid Services 13(3) (2017), 270–289. doi:10.1504/IJWGS.2017.10006054.

36.

Wang,

Chen,

Li,

Zhao and

Shen, Towards achieving flexible and verifiable search for outsourced database in cloud computing, Future Generation Computer Systems 67 (2017), 266–275. doi:10.1016/j.future.2016.05.002.

37.

Wang,

Miao,

Gao and

Chen, Enabling efficient approximate nearest neighbor search for outsourced database in cloud computing, Soft Computing 20(11) (2016), 4487–4495. doi:10.1007/s00500-015-1758-6.

38.

Wang,

Zhang,

Guo and

Wang, Security analysis of “PSLP: Privacy-preserving single-layer perceptron learning for e-Healthcare”, IACR Cryptology ePrint Archive 2017 (2017), 405, http://eprint.iacr.org/2017/405 .

39.

G.A.

Werner and

Hanka, Tuning an artificial neural network to increase the efficiency of a fingerprint matching algorithm, in: Proceedings of IEEE 14th International Symposium on Applied Machine Intelligence and Informatics, 2016, pp. 105–109.

40.

Wu,

Zhu,

G.-Q.

Wu and

Ding, Data mining with big data, IEEE Transactions on Knowledge and Data Engineering 26(1) (2014), 97–107. doi:10.1109/TKDE.2013.109.

41.

I.-C.

Yeh and

C.-H.

Lien, The comparisons of data mining techniques for the predictive accuracy of probability of default of credit card clients, Expert Systems with Applications 36(2) (2009), 2473–2480. doi:10.1016/j.eswa.2007.12.020.

42.

Yuan and

Yu, Privacy preserving back-propagation neural network learning made practical with cloud computing, IEEE Transactions on Parallel and Distributed Systems 25(1) (2014), 212–221. doi:10.1109/TPDS.2013.18.

43.

Zhang,

Zhu,

Xu and

Lu, PPDP: An efficient and privacy-preserving disease prediction scheme in cloud-based e-Healthcare system, Future Generation Computer Systems 79 (2018), 16–25. doi:10.1016/j.future.2017.09.002.

44.

Zhang,

L.T.

Yang and

Chen, Privacy preserving deep computation model on cloud for big data feature learning, IEEE Transactions on Computers 65(5) (2016), 1351–1362. doi:10.1109/TC.2015.2470255.

45.

Zhang,

Jiang,

K.-C.

Li,

Castiglione and

Chen, New publicly verifiable computation for batch matrix multiplication, Information Sciences (2017). doi:10.1016/j.ins.2017.11.063.

46.

Zhang,

Jiang,

K.-C.

Li and

Chen, New publicly verifiable computation for batch matrix multiplication, in: Proceedings of 12th International Conference on Green, Pervasive, and Cloud Computing, 2017, pp. 53–65. doi:10.1007/978-3-319-57186-7_5.

EPSLP: Efficient and privacy-preserving single-layer perceptron learning in cloud computing

Abstract

Keywords

1. Introduction

1.1. Our contribution

1.2. Related work

1.3. Organization

2. Preliminaries

2.1. Single-layer perceptron model

2.3. Symbol classification algorithm

3. The security of PSLP scheme

3.1. Overview of PSLP scheme

4. The proposed construction

4.1. High description

4.2. System model

4.4. The concrete EPSLP scheme

5.1. Security analysis

Table 3 Experimental parameter setting Parameter Bit length Remark k 128 security parameter q 128 depends on the security parameter p 640 2 | r | + 2 | q | + | Num | + | n | + | η | + 1 < | p | s 640 s is in Z p ∗ r 128 random ingredient

Table 4 The comparison of accuracy rate

7. Conclusions

Footnotes

Acknowledgements

References

Table 3
Experimental parameter setting

Parameter Bit length Remark

k 128 security parameter

q 128 depends on the security parameter

p 640 $2 | r | + 2 | q | + | Num | + | n | + | η | + 1 < | p |$

s 640 s is in $Z_{p}^{*}$

r 128 random ingredient

Table 4
The comparison of accuracy rate