Source code-based defect prediction using deep learning and transfer learning

Abstract

Ensuring the quality of software products is important for them to be successful. Discovering errors and fixing defective software modules early in the project lifecycle (e.g. in the testing phase) can save resources and enhance software quality. Developers should prioritize testing procedures and continuously maintain their software projects; however, when there are few instances of a new project, it is hard to build an accurate defect prediction model. Different information about software projects is available and can be utilized through open repositories. Developers can leverage the labeled defect information to build a defect prediction model. The abundance of historical software information in similar domains can assist in transferring the knowledge gained from training this information to other domains for cross-project defect prediction models. Deep learning is a promising machine learner. Deep Belief network (DBN) is a deep learning algorithm that can discover latent relationships between input features by training them through multi-hidden layers; however, it is difficult to build a good prediction model from a dataset with few modules or instances. In this research, we utilized auxiliary datasets to initialize a DBN model and transfer the obtained knowledge to train the DBN model using a source project in a cross-project combination. The expressive features generated from the DBN model are used to build a classical classifier from the source class label and test it on other target project instances. Our evaluation of 13 open Java projects from the PROMISE repository shows that our proposed model achieves improvements based on F-measures (3.6%, 4.9%, and 5.1%) for the three settings of the DBN model measured against the best used benchmark model of TCA/ TCA+ techniques. Moreover, T_DBN and DBN_Only models achieve improvement in terms of F-measure by (11.1% and 6.2%) against the best used benchmark model of TCA/TCA+ on Relink validation dataset.

Keywords

Software testing software defect prediction cross-project deep belief network transfer learning

1. Introduction

The stage of testing and maintaining software products typically takes the highest budget and efforts of any software development life cycle. Writing test cases with limited time and resources is a challenging and difficult process [27]. Usually, testing all possible cases is impractical [10]. When there is a limited testing budget, static code metrics can be used as predictors, to indicate the possibility of fault existence, and then testing resources and budget can be focused on the predicted defective modules. The prediction of software defects is a quick lightweight sampling technique which can save effort and cost, and indicate if there is a necessity for further investigations to ensure quality assurance (QA) [39]. Most statistical machine learning algorithms can recognize faults effectively, but provide limited results for defect prediction [50]. Defect prediction can be within-the same file or two successive versions of project which is called “within-project defect prediction” (WPDP), where the training and testing datasets can be from the same project distribution, or can be a cross-project defect prediction (CPDP) model. That is, the training set is from one domain and the testing set is from other similar domains [34]. Classical machine algorithms are used to determine the probability of the existence of faults in the new modules, depending on previously collected information. In this case, applying a machine classifier directly to two different distributions between one source domain and the target domain may decrease the performance of the classifier [41]. On the other hand, some classical machine learners such as the Naive Bayes (NB) treat the predictor features as statistically independent [16, 39].

In software engineering applications, the strategy of transfer learning can be used to overcome the lack of a sufficient label training set in one domain for testing purposes. In the case of cross-project defect prediction, the performance of the model is usually poor [34, 40]. Transfer learning enhances learning and attempts to benefit from the knowledge obtained from one domain and share it when training a new domain [35]. A domain adaptation method called transfer component analysis (TCA) can be used on traditional metrics to transfer learning from the source project to another target project, in cross-project combinations, to improve the performance of the defect prediction model [17]. Nowadays, the lack of training data of one domain is not a problem as the learning model from a related domain can be utilized to transfer the common features between the two domains [14, 21]. However, building software defect prediction for a cross-project model using transfer learning only on traditional source code metrics needs more investigation for improvement [10].

In recent years, deep learning has become a hot research topic [1, 7, 9, 46]. It is a type of machine learning that encompasses a family of neural network algorithms using multi-hidden layers for supervised and unsupervised learning such as deep belief network (DBN) [22]. Researchers have used deep learning to investigate its ability to extract expressive features that improve the performance of the prediction models [50]. However, deep learning faces some challenges when the size of the training data set for a target domain is small. Deep learning cannot fine-tune the parameters of the network easily and so can be prone to over-fitting problem [22]. Transfer learning within deep learning can solve this problem. It aims to share the learned knowledge from one domain with new domains which must have some similar parameters with others that are specific for each domain. Moreover, transfer learning exploits the correlation between the problem domains. Furthermore, transfer learning can transfer better features from the bottom generalized layers than the higher specific layers in multi-layer neural networks. The transferable features improve performance and generalization even after the model is fine-tuned on a new task [19].

This research investigated a representative learner algorithm to extract more expressive common features, to assist improving the performance of a cross-project software defect prediction model. DBN with transfer learning using traditional or static metrics from the source code are applied to cross-project combinations to improve the defect prediction model.

1.1 Background

Building defective-free software is a quick sampling policy for testing less significant modules [39]. One approach is to utilize data mining algorithms on historical data to predict defects for new projects. The dependency on source code metrics to indicate faults in different granularity levels was ensured early on in software programs. Some product metrics, such as static code metrics, were proposed as a metric-based model to predict defects [16]. For a new domain with limited class labels, CPDP can be used to train the model in one domain that has plenty of labeled classes, and test the model in new similar instances. However, the CPDP model still obtains a poorer performance than within-project, because of the difference between the distributions of the two projects. Moreover, the input features that are being used to build the prediction model are treated independently and the latent relationship between these features is not investigated. One solution, to overcome the difference of the distribution of the input features between the source project and the target project when building a defect predict model, is the transfer component analysis (TCA) [37]. TCA transforms the source and target projects into the same subspace, thus it enhances cross-project software classification and defect prediction [13, 33, 34, 47]. Another enhancement of the TCA is the TCA+ for normalization options selection, since the performance of the TCA depends on the type of normalization [17].

In recent years, deep learning has appeared as a new class of machine learning methods. Deep learning is based on algorithms that use artificial neural networks to learn the latent relationship between the input features. The challenge in deep learning is the difficulty in optimizing the network parameters due to the random weight initialization. Hinton et al. [11] proposed a deep belief network (DBN) model as a fast greedy learning algorithm to pre-train the network in order to obtain good network initialization. Transfer learning is widely used in daily life when we use information we have from previous experiences to learn a new skill or task. Transfer learning enhances the learning by transferring the knowledge gained from one domain to learn a new domain with little human effort [21]. The need for transfer learning in machine learning for a new domain emerges when there is no sufficient data to train the model or when the data collection process is expensive.

1.2 Problem statement

Early fault prediction in the testing process is helpful in saving effort and cost. Many machine learning algorithms can extract a statistical relationship from the source code using static metrics. Traditional techniques extract a shallow relationship between input features using a limited number of non-linear transformation layers, while assuming the existence of the same distribution of the training and testing datasets. The DBN algorithm still needs more investigation in this field. Moreover, most defect prediction approaches work effectively for within-project combinations, In the case of the lack of a training dataset for a new project, it is desirable to train data from existing source projects, and test the model on the target project (i.e. cross-projects) [40]. However, the performance is relatively small compared with the WPDP performance; therefore, to fill this gap, this research investigates the ability of DBN with transfer learning to enhance the software defects prediction model for cross-project task by extracting the representative shared information between the input static code metrics for both the source and target projects.

Research questions

This research attempted to answer the following questions:

1.
Can a DBN Learner with the Transfer Learning technique outperform the TCA/TCA+ on traditional features for a cross-project defect prediction?
2.
How will the performance of the proposed model be different with different settings?
3.
What is the time and space cost for the proposed DBN Model with the Transfer Learning technique based on traditional metrics extraction?

2. Related work

This section reviews previous research works and experiments to describe source code-based defect prediction, deep learning, transfer learning and the methods of applying these concepts to improve defect prediction models.

2.1 Source code metric-based defect prediction

In recent years, some software researchers have started to use different techniques to predict defects early in the testing phase, based on different input feature metrics [13, 33, 34]. Prasad et al. [24] point to some static code metrics that have a significant relationship with the existence of faults. Kaur and Singh [2] tested 10 object-oriented metrics for coupling and cohesion such as CK metrics. Analysis by logistic regression shows that some metrics have a greater impact on predicting defects in a class, and outperform class size in terms of low severity faults. Ozturk and Zengin [30] ensured the need for deriving low-level metrics relations from existing ones, not to add new metrics, and to improve data preprocessing to treat skewness in the data before applying machine learners. In this research, we used product metrics for static source code from different domains found in the PROMISE repository and attempted to preprocess the skewness in the data before applying the defect prediction model.

2.2 Deep learning-based predict defects

Recently, the deep learning algorithm DBN, has been used for extracting representative features and reconstructing input features for more tasks such as recognizing and discriminating purposes.

Tzortzis and Likas [12] used the DBN model to detect spams in three email datasets, and achieved similar and better result than the support vector machine (SVM) learner. Also, Song et al. [18] used the DBN to classify Chinese text documents into different categories and got better results than the SVM classifier. Ri-Xian et al. [23] combined DBN with transfer learning for cross-project defect detection. They trained the DBN model in features from source sample of image datasets to set the initial weights for the target sample for similar domain and then transferred the parameter, including structure and offset, to the target domain, then fine-tuned the model by training the model on a sample from the target domain using BP algorithm. The DBN trained a solar cell model with four hidden layers containing 3000, 1500, 750, and 90 nodes for each layer respectively. In this research, we took the technique of using DBN with transfer learning and tried to apply it to another domain, specifically software engineering. DBN can extract latent relationships between the input features and transfer learning can share the optimized parameters between two domains to improve the source code-based defect prediction model.

Sharma et al. [28] investigated the DBN model to detect an abnormality in two datasets for a digit (MNIST) and non-digit dataset (face and handwritten characters). In the software defect prediction field, Yang et al. [50] proposed a deeper model by applying the DBN model to predict whether a change in a file is buggy or not, which is beneficial for developers as it reveals real-time defects. Wang et al. [38] proposed a DBN on abstract syntax trees (ASTs) to learn semantic features of the source code at the file level. They examined 10 open source datasets from PROMISE repository to establish within-project and cross-project defect prediction. The researchers chose 10 datasets for hidden layers with 100 nodes for each, as they reflected the best value of F-measure and 200 iterations with error rate about 0.098. The results showed improvement in values of average F-measure with 64.1, 60.0, and 59.7 for the three classifiers: alternating decision tree (ADTree), Naive Bayes, and Logistic Regression respectively. In this research, we used the same dataset as that used in [38] to compare our work with the second baseline for traditional features (PROMISE features) for cross-project defect prediction. In addition, we investigated the ability of transfer learning, when combined with DBN, to predict defects in software domain. Li et al. [15] used the convolutional neural network (CNN) to generate semantic and structural features from program (ASTs) to improve the defect prediction model for buggy or clean files. The model was applied on seven Java open source projects from PROMISE repository, they concatenated the extracted semantic features with traditional static metrics and the augmented features improved the performance of the used model better than the DBN model used in [38]. As the addition of traditional static features improves the performance of the defect prediction model, this study encouraged us to explore the role of traditional static metrics only using DBN model with transfer learning to improve defect prediction.

2.3 Transfer learning for cross-project defect prediction

Zimmermann et al. [40] showed that cross-project defect prediction between two domains faces some challenges; the predictor model requires the collection of sufficient training data with corresponding defects data, which is not always available for the new project. They utilized the available historical data from one similar domain (source domain) and tested the model on the new project (target domain). The performance was not the same as the within-project prediction because of the difference in the characteristics of data and process between the two domains. In their analysis, only 21 out of 622 open-source cross-project combinations have a value of precision, recall and accuracy of more than 0.75. Pan et al. [37] proposed the transfer component analysis (TCA) approach in order to reduce the distance between the marginal distributions of two different but related domains. TCA transfers learning between two domains by extracting common latent features using maximum mean discrepancy in Reproducing Kernel Hilbert Space (RKHS). Nam et al. [17] utilized the state-of-the-art unsupervised TCA for transfer learning for cross-project defect prediction and proposed normalization select option using TCA+ by defining dataset characteristic vector (DCV) for each project in source and target domains. To decide normalization options. TCA+ stipulates that the source and the target project must have the same number and type of metrics to learn latent features in order for the transformation process for cross-project defect prediction to succeed. The results showed an improvement in the performance of cross-project defect prediction. Some researchers performed different methods of transfer learning in deep learning models, and were able to achieve better results than by building models from scratch [20, 31, 42, 48, 49]. Different domains utilized transfer learning for a cross-project to enhance the performance of classification models such as natural language processing for sentiment classification [6, 36], image recognition and computer vision [5, 26], and software defect prediction [3, 17, 47]. In this research, we benefited from the TCA+ method and tried to compare the ability of transfer learning within DBN against the TCA+ as a baseline approach.

Based on the previous literature for defect prediction, we summarized the following points:

•
Static code for product metrics is still a significant indicator for the existence of faults in the source code, thus these metrics can still be used to build a model to predict defects.
•
In the case of the lack of class labeled instances, we can benefit from the abundance of auxiliary datasets that have enough labeled class instances to build a defect prediction model and test on similar new domain instances.
•
Most machine learning investigates either independently or the shallow relationship between the metrics and their effect on defect prediction. In addition, the effects of the combination of these metrics on defect prediction need more investigation. For example, x and y are two features which independently affect the dependable variable z, but xy and x ${}^{y}$ have different effects which need further investigation.
•
Transfer learning is about moving the knowledge obtained in one domain to easily learn another similar* domain. It achieves good results in cross-project prediction in minimizing the differences in distribution between the source and target datasets, and can then improve the performance of the classification technique.
•
The DBN model achieves good results in defect prediction and classification for different domains. However, DBN in the context of software defect prediction is still premature and needs more investigation.
•
DBN can utilize transferring knowledge from one domain to another to overcome the shortage in training dataset for the target domain and improve the learning process. Software defect prediction based on source code metrics is one of the fields that should investigate the combination of DBN with transfer learning to improve the performance of the prediction model.

3. The methodology

3.1 The overall research approach

In this section, we introduce the methodology of our thesis approach to improve defect prediction based on static metrics using DBN and transfer learning. Our methodology comprised three phases: data collection and preprocessing, building the learning model using DBN and transfer learning and the prediction model construction phase. Figure 1 shows the phases of the proposed approach.

Figure 1.

The proposed model for CPDP using DBN and transfer learning.

3.2 Data sample

As the open source dataset is publicly available for investigation, we used 13 open source datasets from 10 projects in Java language from the PROMISE data repository via the website: http://openscience.us/ repo/defect. We took two versions of them forming 13 datasets as in [38].

Table 1 summarizes the properties of the 13 datasets, which have 20 static code metrics containing object-oriented metrics. More information about these metrics can be found in [25].

3.3 Research phases

The proposed approach consists of three major phases: dataset collection and preprocessing, building the learning model, and prediction model construction.

Table 1
Properties of 13 Java dataset

Project name	Version	Number of files (instances)	Defect rate %
Ant	1.6	351	26%
Camel	1.4	872	17%
jEdit	4.1	312	25%
Log4j	1.1	109	34%
Lucene	2.2	247	58%
Xalan	2.5	803	48%
Xerces	1.3	453	15%
Ivy2	2.0	352	12%
Ivy1	1.4	241	7%
Synapse	1.1	222	27%
Synapse	1.2	256	34%
Poi2.5	2.5	385	64%
Poi3.0	3.0	442	64%

3.3.1 Phase I: Dataset collection and preprocessing

Dataset collection and preprocessing contains many steps as shown in Fig. 1.

3.3.1.1. Dataset collection

Our approach depends on the static code metrics of product metrics to predict defects at the file level. These metrics represent the input features for the proposed approach. They are quantifiable measures that are used to describe the characteristics of the source code at the design level. Table 2 shows descriptions of these metrics.

3.3.1.2. Dataset preprocessing

The datasets have to be pre-processed using different techniques: over-sampling, normalization selection and noise reduction.

1.
Over-sampling: this technique is used to overcome the imbalanced class label. As shown in Table 1, above, the collected datasets have a greater major class (non-defect) than minor class (defect) in most of the projects with the exception of Poi projects (defect rate $=$ 64%). The abnormality distribution of defect and non-defect can affect the performance of the classifier and bias to the major class. Thus, the over-sampling technique increases the minor class to balance the class label. We used the Synthetic Minority Over-sampling Technique (SMOTE) method [32] to over-sample the dataset for auxiliary dataset and source projects only in cross-project combinations.
2.
Data normalization selection: the values of the measured metrics are not on the same scale, so the comparison of the impact of these metrics on class label will be difficult. In addition, the type of normalization process affects the performance of the prediction model [17]. To overcome this problem; two normalization options were suggested for the dataset; min-max and z-score methods with variations of z-score according to TCA+ normalization selection option method in the study by Wang et al. [38].

Min-max method transforms all the metrics values between 0 and 1, for example, for the feature vector $f_{j}$ , max ( $f$ ) and min ( $f$ ) represent the maximum value of the feature $f_{j}$ and minimum value respectively, for each value of $X_{i}$ of the vector feature $f_{j}$ , and $X_{i}$ is transformed according to the following equation.

$\displaystyle X^{\prime}=\frac{X_{i-\text{min}(f_{j})}}{\text{max}(f_{j})-% \text{min}(f_{j})}$ (1)

Table 2
Static-code metrics [24, 25]

No. Metric-name Description Metrics suite No. Metric-name Description Metrics suite

1. WMC Weighted methods per class Chidamber & Kemerer 11. LOC Lines Of Code Size metric

2. DIT Depth of inheritance tree Chidamber & kemerer 12. DAM Data access metric QMOOD

3. NOC Number of children Chidamber & Kemerer 13. MOA Measure of aggregation QMOOD

4. CBO Coupling between object classes Chidamber & kemerer 14. MFA Measure of func- tional abstraction QMOOD

5. RFC Response for a class Chidamber & kemerer 15. CAM Cohesion among methods QMOOD

6. LCOM Lack of cohesion in methods Chidamber & kemerer 16. IC Inheritance coupling Tang

7. CA Afferent couplings Martin 17. CBM Coupling between methods Tang

8. CE Efferent Couplings Martin 18. AMC Average method complexity Tang

9. NPM Number of public methods QMOOD 19. MAX-CC Max-cyclomatic complexity McCabe

10. LCOM3 Lack of cohesion in methods (normalized) Henderson-sellers 20. AVG-CC AVG-cyclomatic complexity McCabe

Z-score normalization transforms the mean of the data to 0 and standard deviation to 1, for example, for vector feature $f_{j}$ , each value $X_{i}$ in the feature vector $f_{j}$ is standardized according to the Z-score formula as follows:

$\displaystyle X^{\prime}=\frac{x_{i-\text{mean}(f_{j})}}{\textit{std}(f_{j})}$ (2)

In the cross-project case, there are two variations of z-score according to the level of the difference in the distribution of the source and the target domains, so in one z-score variation (N3) we standardized the source and target according to the mean and standard deviation of the target dataset as in Eq. (3). We use this type of normalization when the data of the source is either sparse or dense and has little statistical information; thus we use the target’s statistical information to scale the two datasets. The other variation of z-score (N4) is to standardize source and target according to the mean and standard deviation of the source dataset as in Eq. (4) [17]. We also use this type of normalization when the target data has little statistical information, the data is either sparse or dense, so we use source statistical information to standardize both source and target projects.

$\displaystyle{X^{\prime}}_{\textit{src}}=\frac{X_{\textit{i\ src}}-\textit{% mean}(f_{\textit{i\ tar}})}{\textit{std}(f_{\textit{i\ tar}})}$ (3) $\displaystyle X^{\prime}_{\textit{tar}}=\frac{X_{\textit{i\ tar}}-\textit{mean% }(f_{\textit{i\ src}})}{\textit{std}(f_{\textit{i\ src}})}$ (4)

Where $f_{\textit{i\ src}}$ is the feature vector in the source domain, $X_{\textit{i \ src}}$ is the original value of the source feature vector, $X^{\prime}_{\textit{src}}$ is the new value and $f_{\textit{i\ tar}}$ is the feature vector in the target domain, $X_{\textit{i \ tar}}$ is the original value of the target feature vector and $X^{\prime}_{\textit{tar}}$ is the new value.

To choose the normalization option we have to define the similarity vectors between the source and target domain by defining DCV for each project. DCV can be calculated by measuring the Euclidean distance between each pair instances in the dataset as follows:

$\displaystyle\textit{Distance}=\left\{{\textit{Dist}}_{ij}:\forall ij,1% \leqslant i<n,1<j\leqslant,\ \ i<j\right\}$ (5)

Where $\textit{Dist}_{ij}$ defines the Euclidean distance, which is calculated between instances $i$ and $j$ in each project that have $n$ instances [17], we extracted five characteristics for each project from DCV (mean value, minimum value, maximum value, median and standard deviation), besides the number of instances of each dataset. Then we calculated similarity vector (SV) that represents the difference between source and target using DCVs for them as defined in Nam et al. [17]. Algorithm 1 shows the decision rules for normalization selection [17].

Algorithm 1: Decision Rules for Normalization Selection

Input: Similarity vector $S_{S\to T}$ , where S and T are source and target projects.

Output: Normalization option (NoN|N1|N2|N3|N4)

/ Rule #1 /

1: If $S_{S\to T}$ {dist_mean} $=$ SAME and $S_{S\to T}$ {dist_std} $=$ SAME then return NoN end If.

/ Rule #2/

3. If $S_{S\to T}$ { num_Instances} and $S_{S\to T}$ {dist_min} and $S_{S\to T}$ {dist_max} are MUCH MORE or MUCH LESS then return N1 end If.

/ Rule #3/

4. If ( $S_{S\to T}$ {dist_std} $=$ MUCH MORE and ( $S_{S\to T}$ {num_Instances} $<$ SAME) or ( $S_{S\to T}$ {dist_std} $=$ MUCH LESS and $S_{S\to T}$ {num_Instances} $>$ SAME) then return N3 end if.

/Rule # 4/

5. If ( $S_{S\to T}$ {dist_std} $=$ MUCH MORE and $S_{S\to T}$ {num_Instances} $=$ MUCH MORE) or ( $S_{S\to T}$ {dist_std} $=$ MUCH LESS and $S_{S\to T}$ {num_Instances} $=$ MUCH LESS) then return N4 end if

/ Rule # 5/ return N2.

Rule 1 is used when the mean and the standard deviation of source and target are the same (no transfer information can be used) so this is no normalization type (NON); rule 2 is used when the minimum and maximum of both source and target are different, thus we use min-max normalization (N1); rules 3 and 4 are used when the standard and number of instances are different, in other words, either the source or target has good transferable information, so we normalize both source and target using one of their statistical information (N3), (N4) respectively, otherwise we use normal z-score (N2).
3.
Noise reduction: our projects come from open source repository and may contain noise instances; which can affect the distribution of the dataset and change the overall behavior of the dataset in any process. In this work, we used auxiliary dataset comprising all dataset except the combination of cross-project (source-target) to initialize the DBN model and transfer the knowledge from this data to another similar project. Our goal was to depend on a robust dataset to some degree when transferring the obtained learning. Inter Quartile Range (IQR) rule [34] was used to define any outliers and remove them from the dataset.
4.
Domain Adaptation using Transfer Component Analysis (TCA): In cross-project defect prediction, when there are no available labeled data in the target dataset, one source domain is used to train the prediction model and test the model on another similar target domain, when the two domains are similar in some way in coding. TCA is the state-of-the-art method for domain adaptation in cross-project classification task to transfer the extracted shared latent features between two domains. TCA projects the two domains into a new latent space called a reproducing kernel Hilbert space (RKHS) to minimize the difference in the two distributions and maximize the variance but keep the data properties. Pan et al. [37] propose unsupervised TCA that learns latent features without using the labeled data from the dataset. In our proposed approach we used unsupervised TCA to transform the source and target domains into latent space. The number of features can be reduced in the final transformation, but we used the same number of features as the original dataset to keep the original information when using the DBN model. Figure 1 shows domain adaptation using TCA process in phase I.

3.3.2 Phase II: Building the learning model

No.	Metric-name	Description	Metrics suite	No.	Metric-name	Description	Metrics suite
1.	WMC	Weighted methods per class	Chidamber & Kemerer	11.	LOC	Lines Of Code	Size metric
2.	DIT	Depth of inheritance tree	Chidamber & kemerer	12.	DAM	Data access metric	QMOOD
3.	NOC	Number of children	Chidamber & Kemerer	13.	MOA	Measure of aggregation	QMOOD
4.	CBO	Coupling between object classes	Chidamber & kemerer	14.	MFA	Measure of func- tional abstraction	QMOOD
5.	RFC	Response for a class	Chidamber & kemerer	15.	CAM	Cohesion among methods	QMOOD
6.	LCOM	Lack of cohesion in methods	Chidamber & kemerer	16.	IC	Inheritance coupling	Tang
7.	CA	Afferent couplings	Martin	17.	CBM	Coupling between methods	Tang
8.	CE	Efferent Couplings	Martin	18.	AMC	Average method complexity	Tang
9.	NPM	Number of public methods	QMOOD	19.	MAX-CC	Max-cyclomatic complexity	McCabe
10.	LCOM3	Lack of cohesion in methods (normalized)	Henderson-sellers	20.	AVG-CC	AVG-cyclomatic complexity	McCabe

Algorithm 1: Decision Rules for Normalization Selection
Input: Similarity vector $S_{S\to T}$ , where S and T are source and target projects.
Output: Normalization option (NoN\|N1\|N2\|N3\|N4)
/ Rule #1 /
1: If $S_{S\to T}$ {dist_mean} $=$ SAME and $S_{S\to T}$ {dist_std} $=$ SAME then return NoN end If.
/ Rule #2/
3. If $S_{S\to T}$ { num_Instances} and $S_{S\to T}$ {dist_min} and $S_{S\to T}$ {dist_max} are MUCH MORE or MUCH LESS then return N1 end If.
/ Rule #3/
4. If ( $S_{S\to T}$ {dist_std} $=$ MUCH MORE and ( $S_{S\to T}$ {num_Instances} $<$ SAME) or ( $S_{S\to T}$ {dist_std} $=$ MUCH LESS and $S_{S\to T}$ {num_Instances} $>$ SAME) then return N3 end if.
/Rule # 4/
5. If ( $S_{S\to T}$ {dist_std} $=$ MUCH MORE and $S_{S\to T}$ {num_Instances} $=$ MUCH MORE) or ( $S_{S\to T}$ {dist_std} $=$ MUCH LESS and $S_{S\to T}$ {num_Instances} $=$ MUCH LESS) then return N4 end if
/ Rule # 5/ return N2.

This phase contains two processes: the DBN Pre_training model and Transfer DBN weights.

3.3.2.1. DBN Pre_training Model

This step involves initializing DBN, using the auxiliary dataset as shown in Fig. 1, in the building learning phase. The DBN is a probabilistic and generative model composed of several stacked RBMs. The RBM is a generative stochastic energy-based model which can be represented as a graphical model that contains two connected layers; a visible layer which represents input features and a hidden layer which represents the extracted latent features. The visible layer and hidden layer nodes are connected symmetrically, and there are no connections between nodes in the same layer [22].

From the definition of the RBM as an energy model, the goal of RBM training is to minimize the energy of the neural network. Assuming that we have n visible nodes and m hidden nodes; $w$ is the matrix weights between nodes in visible and hidden layers, $a_{i}$ , $b_{j}$ are the bias values for $i^{th}$ unit in visible layer and $j^{th}$ unit in hidden layer respectively, the energy function E of the state ( $v$ , $h$ ) of the connected visible units ( $v$ ) and hidden units ( $h$ ) is computed as in Eq. (6).

$\displaystyle E(v,h)=-\sum^{n}_{i=1}{a_{i}*v_{i}}-\sum^{m}_{j=0}b_{j}*h_{j}-% \sum^{n}_{i=1}\sum^{m}_{j=1}v_{i}*w_{ij}*h_{j}$ (6)

We can define for the state ( $v$ , $h$ ) the joint probability distribution from the energy function as in Eq. (7).

$\displaystyle P(v,h)=\frac{\text{exp}(-E(v,h))}{Z}$ (7)

Where $Z$ is the partition function, given the state of the nodes in the visible layer, the conditional probability of the activated hidden nodes can be calculated using Eq. (8).

$\displaystyle P(h_{j}=1|v)=\sigma\left(b_{j}+\sum^{n}_{i=1}v_{i}*w_{ij}\right)$ (8)

Furthermore, given the state of the hidden nodes, we can calculate the conditional probability of the visible layer nodes as in Eq. (9).

$\displaystyle Pt(v_{j}=1|h)=\sigma\left(a_{i}+\sum^{m}_{j=1}(v_{i}*w_{ij})\right)$ (9)

Where $\sigma$ is the non-linear sigmoid activation function used in Eqs (8) and (9). The $\sigma$ or sigmoid is the Logistic regression function that is defined on variable $x$ as: $\sigma(x)=\frac{1}{1+e^{-x}}$ .

Through training the RBM, the parameters $w$ , $a$ , and $b$ should be adjusted using partial derivatives. The difficulty in deep learning when using random weights to initialize the network is the probability of getting stuck in local minima problem, which can prevent the model from being generalized and thus the model can over-fit [45]. To find approximated solutions, Hinton et al. [11] propose a fast greedy RBM algorithm called contrastive divergence to pre-train the model to improve the model learning. Different algorithms can be used for parameter optimization such as gradient descent using back propagation or conjugate gradient to optimize the values of the weight matrix $w$ , and bias vectors $a$ , and $b$ . Then the weights are updated besides the bias vectors of the visible and hidden layers and the network is fine-tuned by minimizing the error between the actual output and the produced ones, and then the learn occurs [23, 50].

3.3.2.2. Transfer parameters within DBN model

Training DBN on a small limited dataset can hardly optimize the network parameters and can expose it to the over-fitting problem. Transfer learning can utilize the correlation between similar domains and the availability of an auxiliary dataset. In addition, in the case of training DBN, it can overcome the shortage in training dataset and transfer the structure of one domain project with the optimized parameters to initialize the weights and train the DBN on another domain project. In the DBN model, we can perform transfer learning using an auxiliary dataset and source dataset in un-supervised learning to generate optimized parameters for supervised learning in the next phase as follows (the main steps):

Using the auxiliary dataset (All) to initialize the DBN and train the model to get good parameters (the weight matrix $w$ , and the bias vectors $a$ , and $b$ ) as shown in the DBN model pre-training in Fig. 1 for auxiliary dataset (All).

From the above point we started to train the DBN model using TCA_ed source dataset and then transfer the optimized DBN-parameters and extract representational features from the transformed source dataset [20] as shown in DBN pre-training in Fig. 1 for new source dataset. We can perform the experiment with and without transfer learning (without transfer means initializing the DBN model with source dataset only not from auxiliary dataset) to compare and calculate the improvements. Algorithm 2 shows the steps in the DBN pre-training phase.

Algorithm 2: DBN_Model Pre-training Phase
Input: Auxiliary dataset without source-target (All),
TCAed source dataset (new source)

Output: Pre-trained DBN_Model with Optimized parameters (Weight matrix $w$ , bias vectors $a$ , $b$ ) & Representative features
For each source-target in cross-project combination
Normalize (All), (new source)// using best normalization type from TCA/TCA+
Shuffle (the normalized data)
# hidden layer $=$ 3
# Nodes $=$ n // (n: based on early setting)
DBN_.Initialize // (depend on the settings of the used library)

For epoch $=$ 1 to m // (m: based on early setting)
DBN.Fit (All) // pre-train DBN using (All) dataset to get good parameters
DBN.Fit (new source) // starting from the previous step for pre-trained DBN model
// pre-train DBN model using (source) dataset
Return pre-trained DBN model with Optimized Parameters (w, a, b)

3.3.3 Phase III: Prediction model construction

The pre-trained DBN model must be fine-tuned using forward neural network and back-propagation algorithm to optimize the error rate, the TCA_ed source and TCA_ed target projects with class labels are needed to build the prediction model and evaluate the performance of the model respectively. The SMOTE technique is used to balance the label class on the transformed source dataset so the prediction model will not be biased to any class. In this phase, an output layer is added on the top of the DBN model for the classification task and is trained on the balanced and TCA_ed source dataset. To evaluate the prediction model, we need to test it on the TCA_ed target dataset. In this phase we used the transferred DBN model as the discriminator model for the classification task in supervised learning. According to Zhang [20] the representational transfer in the DBN model improves the accuracy of the classification task compared to the representational transfer in the back-propagation neural network model. Moreover, to utilize the techniques of transfer learning within DBN model, the learned parameters from the bottom layers are more generic than the higher ones [43]. Thus, they were transferred to the next phase, therefore the following steps are included.

1.
To keep the learning generated from the above unsupervised steps, we removed the higher layer and froze the first hidden layer while keeping the rest, since the generality of the extracted features decreases from bottom to the top, but the transferable features still have good impact on network learning than random features [29]. The frozen layers will prevent the good parameters from changing.
2.
An output layer was added on the top of the extracted features with logistic regression function (Softmax) for classification task, and then the whole transferred DBN model was fine-tuned by updating the weights of the network parameters. Then the probabilities for the predicted value for the label class using transformed source dataset only with its label class were obtained.
3.
The predicted model was tested on the transformed target dataset for cross-project defect prediction. The model was tested using DBN with and without transfer learning. Figure 2 shows Prediction model.
4.
The performance of the prediction model was calculated using evaluation metrics. Algorithm 3 shows the steps in the DBN fine-tune phase and prediction model construction.

Algorithm 3: DBN Model Fine-tune Phase

Input: Pre-trained DBN Model with parameters (Weight matrix, bias vectors a & b)

& Representative Features, new source (TCA_ed source with class label), new target (TCA_ed Target with class label)

Output: Prediction Model (DBN2), Best f-measure

Best f-measure $=$ 0, DBN2 $=$ DBN

Remove higher layer from (DBN2), Freeze the bottom ones in (DBN2)

Add output layer on (DBN2) with Logistic Regression function

For epochs $=$ 1 to m

DBN2.Fit (new Source) // fine-tune the transferred Pre-trained model (DBN2) using source dataset and

//minimize the error between actual inputs & the predicted ones update the (w, a, b)

Return the predicted model (DBN2)

DBN2.Evaluate (new target) // test the predicted model (DBN2) using

Calculate f-measure // (normalized new target dataset with class label)

If f-measure $>$ Best f-measure then

Best f-measure $=$ f-measure

End if

End

Return Predicted Model (DBN2), Best f-measure

Figure 2.
Tuning # of neurons.

3.4 Data analysis and interpretation

Algorithm 3: DBN Model Fine-tune Phase
Input: Pre-trained DBN Model with parameters (Weight matrix, bias vectors a & b)
& Representative Features, new source (TCA_ed source with class label), new target (TCA_ed Target with class label)
Output: Prediction Model (DBN2), Best f-measure
Best f-measure $=$ 0, DBN2 $=$ DBN
Remove higher layer from (DBN2), Freeze the bottom ones in (DBN2)
Add output layer on (DBN2) with Logistic Regression function
For epochs $=$ 1 to m
DBN2.Fit (new Source) // fine-tune the transferred Pre-trained model (DBN2) using source dataset and
//minimize the error between actual inputs & the predicted ones update the (w, a, b)
Return the predicted model (DBN2)
DBN2.Evaluate (new target) // test the predicted model (DBN2) using
Calculate f-measure // (normalized new target dataset with class label)
If f-measure $>$ Best f-measure then
Best f-measure $=$ f-measure
End if
End
Return Predicted Model (DBN2), Best f-measure

To evaluate the proposed approach results, different evaluation metrics were used to assess the performance of the machine classifiers. We used Recall, Precision, and f-measures that are popular metrics for defect prediction models. Moreover, we compared the performance of the different settings of the model in the experiment. Precision and recall can be calculated using confusion matrix that describes the result of the prediction model compared to the actual results in the dataset. Table 3 shows the confusion matrix.

The precision measure represents the ratio of the correctly instances predicted as a defect to all number predicted as defective instances as shown in Eq. (10).

$\displaystyle\textit{Precision}=\frac{\textit{TP}}{\textit{TP}+\textit{FP}}$ (10)

Recall measure represents the ratio of the instances correctly predicted as defects to the actual number of defective instances as shown in Eq. (11).

$\displaystyle\textit{Recall}=\frac{\textit{TP}}{\textit{TP}+\textit{FN}}$ (11)

F-measure is a combination of the two precision and recall measures and represents the weighted harmonic mean value from these two metrics; f-measure can be calculated as in Eq. (12).

$\displaystyle\textit{F-measure}=\frac{\textit{2*Precision*Recall}}{\textit{% Precision}+\textit{Recall}}$ (12)

The high precision value indicates the ability of the prediction model to discover the defective instances in the dataset, while, the high recall value indicates the ability of the prediction model to discover the defective instances in the project. F-measure takes the summary of the two measures [38, 50].

Table 3

Confusion matrix

	Predict defect	Predict non-defect
Actual defect	True positive… (TP)	False negative… (FN)
Actual non-defect	False positive… (FP)	True negative… (TN)

3.5 Research tools

The proposed approach was achieved using a PC with the following properties; Intel corei5 CPU 2.5 GHz, 8 GB RAM, Windows 10. Matlab R2017a to find Euclidean distance for DCV and run TCA. Microsoft Excel 2010 was used to implement our version of normalization selection using decision rules as in TCA+. For DBN and transfer learning we used open source toolkit for Java code, Deeplearning4j. Deeplearning4j is widely used to build and train neural networks, especially deep learning models. More information about deeplearning4j can be found on the website: https://deeplearning4j.org/overview. We installed the prerequisites for deeplearning4j to run the library including: a Java Development Kit (JDK) such as Java 1.7; an automated build and dependency manager (Apache Maven); and an Integrated Development Environment (IDE) such as Netbeans IDE1

¹
https://netbeans.org/community/releases/80/install.html.

8.0.2). For data preprocessing, we used the open source Weka tools2

https://www.cs.waikato.ac.nz/ml/weka/downloading.html.

3.8.1, IQR for noise reduction and SMOTE for oversampling. We also used Weka 3.8.1 implementation for Logistic regression for the three baselines.

4. Results and analysis

4.1 Experimental setup

4.1.1 Dataset

To evaluate the DBN model and facilitate reproducibility of the experiment, we used 13 open source projects from the PROMISE datasets repository.

4.1.2 Baselines of traditional features

To evaluate the performance of the DBN model based on traditional static code metrics for defect prediction, we compared the extractive features generated from the DBN model on traditional features using three baselines, the first baseline uses 20 traditional features from the source dataset to train logistic regression classifier and test on target dataset features. The second baseline (TCA) uses the domain adaptation approach TCA with different normalization options on source and target traditional features. The transformed source dataset was used to train logistic regression classifier and we tested it on the transformed target dataset. The third baseline (TCA+) is similar to baseline two but with normalization option selection depending on decision rule in Fig. 4. Based on the degree of similarities between source and target DCVs’, the type of normalization is selected before applying TCA following the method of Nam et al. [17]. Logistic regression is a probabilistic linear model that is widely adopted in data mining as a machine classifier [4, 38]. We used the default settings implemented in Weka3.8.1 tool for the three baselines.

4.1.3 Cross-project defect prediction

When there are few instances in a domain it is difficult to build an accurate machine classifier for a new dataset, therefore we can benefit from the availability of other similar domains to build a defect prediction model and test it on the new domain. This technique is called cross-project defect prediction. This means building a machine classifier on the source dataset and examining it on the target dataset. Thus, we prepared a dataset to perform cross-project defect prediction by conducting 22 combinations for cross-project settings for the source-target project as in [38]. When training the DBN model we used the auxiliary dataset to initialize the model. We used 11 out of 13 projects, excluding the source-target combination from the auxiliary dataset. We called them All_without_source-target (All) for each combination, so we had 11 auxiliary datasets. For example, when training ant-camel in cross-project settings we excluded these two projects from the all auxiliary and added the remaining projects in one file. The auxiliary dataset is treated independently from the cross-project combinations. After that, we applied noise reduction (IQR) method using Weka 3.8.1 implementation. IQR detects outlier and extreme instances, which we then removed from the auxiliary dataset using Remove-With-Values filter implemented in Weka 3.8.1.

To have equal participation of the instances we also applied oversampling on the auxiliary dataset using SMOTE technique implemented in Weka 3.8.1. We repeated the process of balancing percentages until the two classes were equal. These two steps on the auxiliary dataset were essential as the data features should be clean and balanced when the learning is transferred to the source dataset when building the DBN model. SMOTE was applied to the source dataset only for the source-target combination after the domain adaptation (TCA) was applied, and we did not apply it to the target dataset, only to the target dataset domain adaptation (TCA) to mimic the reality.

4.1.4 Parameter settings for the DBN model

The most important part of the deep learning model is to find the appropriate architecture which is a heuristic and endeavor issue. Deep learning models have many parameters that need to be adjusted before training the proposed model. Some of these hyper parameters are related to the deep model and some are related to the optimizer function. There are different methods of adjusting these parameters using automatic procedures such as grid search, random search or hyper parameter optimization. All of these approaches need a lot of resources and time. Nevertheless, we can carry out a manual search for the main hyper-parameters [44].

The main effective parameters that need fine-tuning are the number of hidden layers, the number of hidden neurons in each layer and the number of epochs [38]. Bengio [44] recommends picking standard model choices from other papers and conducting some search for others. For the number of hidden layers, as our dataset has a small number of instances to train, choosing too few hidden layers may result in poor performance and the model may not store all the expressive features of the training set in order to improve cross-project defect prediction. However, choosing too many layers may take a lot of time and may cause the model to over-fit. Thus, we took three hidden layers, as many papers reported that this number of hidden layers obtains good result for the DBN model [8, 50]. To tune the other two parameters, we began by preparing two settings for the experiment. For the first setting, we trained cross-project using the DBN model with transfer learning (using auxiliary dataset), while for the second setting, we trained the DBN model without transfer learning (without using auxiliary dataset). To measure the performance of the proposed model, we suggested naming the two model settings. When training the auxiliary dataset All_without_source-target in pre-training phase, fine-tuned on transformed source dataset, and then tested on transformed target dataset to be named T_DBN model. While we suggested naming the model training on transformed source dataset only for pre-training and fine-tuning phases and then tested on transformed target dataset to be named DBN_Only model for both with and without transfer learning respectively.

To measure the effects of the two parameters we tested the model on five cross-project combinations from the dataset. Wang et al. [38] selected five projects of two consecutive versions to adjust the DBN parameters for within-project defect prediction and validated the model on cross-project combinations. In this thesis, we have no within-project setting, so we chose random five combinations, and added a validation dataset experiment for our model on another dataset for cross-project combinations. We tested each combination individually and took the average f-measure for the five cross-project combinations in the defect prediction model. The five cross-project combinations included: ant-camel, log4j-jEdit, xerces-xalan, ivy2-synapse12, and ant-poi3.

Moreover, for the purpose of comparison between models with different settings, in order to answer research question 2, we took two different combinations of datasets to measure the effect of the two parameters on these models and get independent results. Thus, we were able to differentiate the performance of the two models. In this setting, we removed noise reduction using IQR step from All-without_source-target but kept SMOTE step. The five cross-project combinations for comparison purposes included: ant_poi3, lucene-log4j, xerces-ivy2, camel-jEdit, and xerces-xalan.

4.1.4.1. Setting the number of neurons in each layer

For the number of neurons in each hidden layer we tested a fixed number of neurons in each hidden layer as, according to Bengio [44], having the same number of neurons in each hidden layer works better than pyramid like or upside-down pyramid where the number of neurons decreases or increases respectively through the hidden layers. We chose seven discrete values to test for number of neurons including: 3, 5, 15, 18, 30, 50, 60 and tested with three hidden layers with fixed number of epochs $=$ 15. We used the five cross-project combinations mentioned earlier for T_DBN model. We took the average of the f-measure for the five results. Figure 2 shows the result of the average F-measure for tuning the number of neurons in each layer, in the Fig. 2 peaks at 50 neurons that have the highest f-measure. Consequently, we chose 50 neurons in each layer as the result from Fig. 2.

4.1.4.2. Setting the number of epochs

In training DBN model the number of epochs is an effective parameter, in each epoch the model tries to update the weights of the model and to decrease the difference between input data and the generated ones. In other words, as the number of epochs increases, the error rate decreases. In each epoch, the model updates the weights in one pass using all datasets; having many epochs decreases the differences but takes too much time [38]. To tune the number of epochs we used squared error (squared loss in deeplearning4j library) and f-measure to evaluate the number of epochs. We used 50 epochs as the maximum number and we scored f-measure for the five cross-combinations every 5 epochs, along with the squared loss. We took the average for both f-measure and squared error for the five combinations. Figure 3 shows the result of tuning the number of epochs. In Fig. 3, the squared error decreases as the number of epochs increases. The best average f-measure was scored with a value of 49.5 when the number of epochs was 35 and the average squared was 0.035; as a result, we took the value of 35 for the set number of epochs in T_DBN model.

Figure 3.

Tuning # of epochs.

4.2 Results and results analysis

This section presents the result of the proposed model through different measures for the model and answers the research questions (RQs). It is followed by an illustration of the results and an analysis of these outputs for the different settings of the deep belief network models, with and without transfer learning, in addition to the justification of the differences in the results. Moreover, we present the general effects of some approaches on the different models.

4.2.1 DBN model with transfer learning results

RQ1: Can a DBN Learner with the Transfer Learning technique outperform the TCA/TCA+ on traditional features for cross-project defect prediction?

To answer this question, we used the first setting of the DBN model with transfer learning. We compared our proposed T_DBN based cross-project for defect prediction with the three baselines mentioned early. In baseline one, we built the logistic regression classifier on the source project and tested it on the target project. In baseline two, we used three normalization options (N2, N3, N4) with TCA technique for domain adaptation and built logistic regression on the source project, and then tested it on the transformed target project. While in baseline three, we used TCA+ technique that returns normalization option depending on decision rules shown in Fig. 2 before applying TCA. After that, we built the logistic regression on the transformed source project and tested it on the transformed target project. We were encouraged by the result of the type of normalization in baseline two and took the best average one for all 22 combinations from (N2, N3, N4), which was N3 (normalize source and target using mean and standard deviation of the source project). Thus, we normalized all 22 cross-project combinations using N3 for T_DBN model before applying TCA. In addition, deep belief network needs its own normalization type, so we adopted N3 for all combinations before applying T_DBN model.

The All dataset was used to pre-train the DBN model and transfer the parameter (weight matrix $w$ between connected nodes and bias vectors $a$ , and $b$ for visible and hidden layers respectively) to the transformed source dataset which was used to start training on the model and fine-tune the transferred parameters. The last layer of the model used source class label to build a classifier such as logistic regression. To evaluate the model, the transformed target dataset was used to predict the new unknown class and compare it with actual ones. Table 4 shows the result of the three measures of the Precision, Recall and F-measure of the T_DBN model. The values of the three measures range between 0 and 1. Table 5 shows comparisons of f-measure in percentages for T_DBN model and the three baseline models mentioned earlier: baseline1; TCA with normalization options (N2, N3, and N4) (baseline2); and TCA+ (baseline3) respectively. In addition, the baseline1 (TCA+) on PROMISE traditional features in [38] is included. The difference between TCA+ in [38] and our work is that they used ADTree classifier instead of logistic regression. The best f-measure between the five models is highlighted in bold.

4.2.1.1. DBN model with transfer learning results analysis

The experimental results in Table 4 for T_DBN model show the average f-measure to be around 49 with high values of recall measure for most of the cross-project combinations. Although the results show a low value for the precision values for most combinations, the average recall achieved 86%. In the case of defect prediction models, we were interested to achieve a small false negative (FN) value in Eq. (10) for the recall measure. In other words, it is better to predict a small number of a real defective file as non-defective than to predict a large number as clean when they are, in fact, defective. However, the low precision value can be overcome by carrying out more inspections in the testing phase. The low precision value comes from the high false positive (FP) value which means that the model predicts the instances as defective when they are in fact clean.

Table 4
Precision, recall, and f-measure of the T_DBN model

Source	Target	Precision	Recall	F-measure
ant1.6	camel1.4	0.18	0.82	0.30
ant1.6	poi3.0	0.76	0.88	0.81
jEdit	camel	0.17	1.00	0.29
camel1.4	ant1.6	0.27	1.00	0.42
poi3.0	ant1.6	0.28	0.95	0.43
camel1.4	jEdit	0.29	0.72	0.42
log4j	jEdit	0.30	0.78	0.43
jEdit	log4j	0.34	1.00	0.51
lucene2.2	log4j	0.35	0.97	0.51
lucene2.2	xalan2.5	0.48	1.00	0.65
xerces1.3	xalan2.5	0.48	1.00	0.65
xalan2.5	lucene2.2	0.59	1.00	0.74
log4j	lucene2.2	0.58	1.00	0.74
xalan2.5	xerces1.3	0.25	0.55	0.35
ivy2.0	xreces1.3	0.17	0.77	0.28
xerces1.3	ivy2.0	0.22	0.40	0.29
synapse1.2	ivy2.0	0.19	0.35	0.25
ivy1.4	synapse1.1	0.30	0.87	0.45
poi2.5	synapse1.1	0.29	0.95	0.44
ivy2	synapse1.2	0.34	1.00	0.50
poi3.0	synapse1.2	0.38	0.81	0.52
synapase1.2	poi3.0	0.64	1.00	0.78
Average		0.36	0.86	0.49

The results in Table 5 show that the T_DBN model returns better average f-measure than the three baselines with values (48.6) against 39.8 for baseline1, 37.9 (N2), 40.1 (N3), and 37.9 (N4) for TCA (baseline2) and 36.6 for TCA+ (baseline3). Moreover, T_DBN achieves eight values (with four values above 70%) of f-measure out of 22 combinations higher than the four models including baseline1 (TCA+) for Wang et al. [38] with ADTree classifier. For example, T_DBN scores best f-measure for ant1.6-poi3.0 (78.2), xalan2.5-lucene2.2 (73.7), log4j-lucene2.2 (73.7) and synapse1.2-poi3.0 (77.9). While TCA+ for Wang et al. [38] scored best value (65.1) for synapse 1.2-poi3.0 and TCA [N3] had two best values for jEdit-log4j (65.5) and lucene2.2-log4j (65.3). Furthermore, TCA+ did not get better results than TCA since the authors of TCA+ point out that the type of normalization cannot be generalized. Moreover, the average f-measure for TCA with normalization type N3 had the highest values between the three baselines of our own implementation. This result is compatible with that of Nam et al. [17] for the type of normalization with TCA for cross-project combinations. However, T _DBN produced bad results in some cross-project combinations, for example came1.4-ant1.6 (42.0) while baseline1 was 59.9 f-measure, and three worse results were produced from the three baselines in lucene2.2-log4j (52.5), xerces1.3-ivy2.0 (20.4) and poi3.0-synapse1.2 (51.7). This can be attributed to the small defective rate in the target project or the type of normalization in both processes, domain adaptation and deep learning, since we took the N3 normalization type as the best average result for all cross-project combinations.

Table 5

Comparisons of F-measure (%) with the five models, with the best F-measure highlighted in bold

Source	Target	F-measure %
		Baseline1	TCA (Baseline2)			TCA+	(TCA+) ADTree	T_DBN
			N2	N3	N4	(Baseline3)	classifier [38]
ant1.6	camel1.4	32.0	28.7	26.0	16.0	28.7	29.2	30.6
ant1.6	poi3.0	59.9	45.3	39.8	28.8	45.3	34.3	78.2
jEdit	Camel	29.3	28.1	20.8	24.8	20.8	33.0	28.5
camel1.4	ant1.6	59.9	13.1	45.9	45.9	13.1	61.6	42
poi3.0	ant1.6	41.1	55.2	51.3	55.2	55.2	59.8	41.8
camel1.4	jEdit	55.1	5.0	34.5	43.6	34.5	53.7	42.1
log4j	jEdit	44.6	56.2	50.0	52.7	52.7	41.9	45
jEdit	log4j	49.2	65.5	65.5	31.1	31.1	57.4	54
lucene2.2	log4j	53.2	59.8	65.3	62.9	59.8	57.1	52.5
lucene2.2	xalan2.5	53.5	59.2	64.3	63.8	63.8	53.0	65.0
xerces1.3	xalan2.5	10.8	14.0	43.9	44.9	14.0	58.1	65.0
xalan2.5	lucene2.2	54.2	63.7	45.2	41.4	63.7	56.1	73.7
log4j	lucene2.2	51.8	56.4	51.5	52.5	52.5	52.4	73.7
xalan2.5	xerces1.3	25.2	24.6	37.8	37.4	24.6	39.4	27.2
ivy2.0	xreces1.3	23.4	22.5	11.2	24.7	22.5	39.8	27.7
xerces1.3	ivy2.0	21.4	29.0	27	24.6	29.0	40.9	20.4
synapse1.2	ivy2.0	42.7	45.1	31.2	41.2	45.1	38.3	32.2
ivy1.4	synapse1.1	11.6	7.0	22.9	21.1	7.0	34.8	45.5
poi2.5	synapse1.1	47.0	40.2	49.7	44.9	40.2	37.6	43.1
ivy2	synapse1.2	27.7	27.5	22	7	27.5	57.0	50.3
poi3.0	synapse1.2	55.6	57.9	55.6	52.5	57.9	54.2	51.7
synapase1.2	poi3.0	26.9	30.1	21.4	17.1	17.1	65.1	77.9
Average		39.8	37.9	40.1	37.9	36.6	47.9	48.6

From the results in Section 4.2.1 we can summarize the following points:

•

DBN with transfer learning improves recall measure but does not improve precision measure; low values for precision measure come from high values in FP which can be resolved by carrying out more inspections.

•

DBN with transfer learning (T_DBN) improves cross-project defect prediction and outperforms TCA/TCA+ on average f-measure for the same classifier (logistic regression) and achieves a little improvement comparable with other classifiers (ADTree for TCA+) in [38].

•

However, T_DBN does not improve the result for some cross-project combinations either from the type of normalization or the small defective rate in the target project.

To prove that T_DBN is the best model we used one-way analysis of variance (ANOVA) to test if the means of F-measure for the best five models presented in Table 5 (Baseline1, TCA N3, TCA+, TCA+ ADTree classifier [38], T_DBN) are equal, and to determine if there is one mean that is significantly different from the others. The hypotheses are:

$\displaystyle H_{0}:μ_{1}=μ_{2}=\ldots=\mu_{5}$ $\displaystyle H_{1}:\text{at least one mean is different from the others.}$

Table 6 provides the ANOVA results for the best five models presented in Table 5. As shown in the table, the P value for all the models is less than 5%. Therefore, $H_{0}$ will be rejected at 5% significance level and $H_{1}$ will be accepted for all the models, and since the average f-measure of the T_DBN model has the highest value we conclude that the T_DBN is the best model.

Table 6

ANOVA test

Results details*
Source	SS	Df	Ms
Between groups	2491.8269	4	622.9567	F $=$ 2.51228
Within groups	26036.3036	105	247.9648
Total	28528.1305	109

*The F-ratio value is 2.51228. The p-value is 0.046041. The result is significant at $p<$ 0.05.

4.2.2 Different settings of DBN model result

RQ2: How will the performance of the proposed model be different with different settings?

4.2.2.1. DBN with and without transfer learning result

To answer this question, we prepared for the second setting DBN without transfer learning and compared it with the first model. The second setting is DBN_Only that trains the DBN model without utilizing an auxiliary dataset, meaning that the two phases of the DBN model are trained on a TCA_ed source dataset and tested on a TCA_ed target dataset without transfer learning. We used a different set of source-target combinations to obtain independent result from the first setting. The set of combinations contains: ant-poi3, lucene-log4j, xerces-ivy2, camel-jEdit, and xerces-xalan.

Figure 4 shows the average f-measure for the five combinations for 50 epochs for transfer learning with DBN model T_DBN (A) left and for model without transfer learning DBN_Only (B) right respectively.

Figure 4.

Average F-measure for T_DBN model (A) left and DBN_Only model (B) right. (A) Average F-measure for T_DBN; (B) Average F-measure for DBN_only.

Figure 5 shows T_DBN in the small squared line that uses transfer learning and the large squared line for DBN_Only model.

Table 7 shows the performance in terms of the best f-measure of the two models for 50 neurons and 50 epochs. The best values are highlighted in bold.

DBN with and without transfer learning result analysis

The result of the first two settings for the proposed T_DBN model with transfer learning and DBN_Only without transfer learning is shown in Fig. 4A and B respectively. Figure 4A shows the stability of the average of the best f-measure values for T_DBN more than the result for DBN_Only in Fig. 4B. Figure 5 shows the two values of the average best-f-measure for the two models. They have comparable values with higher average values for T_DBN. Table 7 shows that in 14 out of 22 combinations T_DBN outperforms DBN_Only, and they have equal values for 4 combinations while DBN_Only has four combinations better than T_DBN. The result of the T_DBN with average f-measure (48.6) outperforms deep belief network without transfer learning DBN_Only with average f-measure (47.3), this means that both models, with and without transfer learning, improve defect prediction for cross-project combinations.

Table 7

The best F-measure for T_DBN and DBN_Only models

Source	Target	T_DBN	DBN_Only
ant1.6	camel1.4	30.6	28.5
ant1.6	poi3.0	78.2	78.0
jEdit	camel	28.5	29.5
camel1.4	ant1.6	42.0	41.5
poi3.0	ant1.6	41.8	41.5
camel1.4	jEdit	42.1	40.4
log4j	jEdit	45.0	36.2
jEdit	log4j	54.0	50.7
lucene2.2	log4j	52.5	50.7
lucene2.2	xalan2.5	65.0	65.0
xerces1.3	xalan2.5	65.0	65.0
xalan2.5	lucene2.2	73.7	74.0
log4j	lucene2.2	73.7	73.7
xalan2.5	xerces1.3	27.2	26.4
ivy2.0	xreces1.3	27.7	27.8
xerces1.3	ivy2.0	20.4	27.2
synapse1.2	ivy2.0	32.2	20.4
ivy1.4	synapse1.1	45.5	42.6
poi2.5	synapse1.1	43.1	42.6
ivy2	synapse1.2	50.3	50.3
poi3.0	synapse1.2	51.7	50.3
synapase1.2	poi3.0	77.9	77.7
Average		48.6	47.3

Figure 5.

Average F-measure for T_DBN and DBN_Only models.

4.2.2.2. DBN models with SOMTE technique results

Furthermore, as we wanted to show the effect of SMOTE technique on the used models, we first applied SMOTE technique on the best model of the three baselines, which is TCA with N3 normalization option, which we called TCA [N3] Smote, and we compared this with our three models. The first model was T_DBN model with SMOTE technique plus noise reduction (NR) using IQR on auxiliary dataset (All_without_source-target) which we renamed T_DBN [NR $+$ SMOTE]. The second model, T_DBN with SMOTE technique only on the auxiliary dataset (All_without_source-target) is called T_DBN [SMOTE] and the third model is DBN_Only without using auxiliary dataset to transfer learning. Table 7 shows the f-measure for the three settings of our work, and the extended baseline two of TCA (TCA [N3] Smote). The best f-measure is highlighted in bold.

DBN models with SOMTE technique results analysis

For the second settings of the proposed model and the effect of SMOTE technique only on the prediction model, Table 8 shows that the T_DBN [SMOTE] outperforms the TCA (N3) [Smote] with average F-measures of 48.8 and 43.7 respectively. It outperforms the other two settings of the deep belief network with average F-measure (48.6) for T_DBN [NR $+$ SMOTE], and (47.3) for DBN_Only. T_DBN [SMOTE] and T_DBN [NR $+$ SMOTE] have comparable values.

Table 8

F-measure for the three settings of the DBN models and TCA method

Source	Target	TCA[N3] Smote	T_DBN [NR $+$ SMOTE]	DBN_Only	T_DBN [SMOTE]
ant1.6	camel1.4	33.8	30.6	28.5	29.8
ant1.6	poi3.0	40.0	78.2	78.0	81.3
jEdit	camel	28.7	28.5	29.5	28.5
camel1.4	ant1.6	55.6	42.0	41.5	42.0
poi3.0	ant1.6	48.2	41.8	41.5	43.1
camel1.4	jEdit	50.5	42.1	40.4	41.6
log4j	jEdit	50.5	45.0	36.2	43.1
jEdit	log4j	42.3	54.0	50.7	51.0
lucene2.2	log4j	61.9	52.5	50.7	51.1
lucene2.2	xalan2.5	62.2	65.0	65.0	65.0
xerces1.3	xalan2.5	59.5	65.0	65.0	65.1
xalan2.5	lucene2.2	46.9	73.7	74.0	74.0
log4j	lucene2.2	55.7	73.7	73.7	73.7
xalan2.5	xerces1.3	37.3	27.2	26.4	34.7
ivy2.0	xreces1.3	23.7	27.7	27.8	28.0
xerces1.3	ivy2.0	21.2	20.4	27.2	28.6
synapse1.2	ivy2.0	31.4	32.2	20.4	25.0
ivy1.4	synapse1.1	30.1	45.5	42.6	44.8
poi2.5	synapse1.1	43.1	43.1	42.6	43.9
ivy2	synapse1.2	52.4	50.3	50.3	50.3
poi3.0	synapse1.2	53.6	51.7	50.3	51.7
synapase1.2	poi3.0	31.7	77.9	77.7	77.7
Average		43.7	48.6	47.3	48.8

The highest average f-measure for the T_DBN [SMOTE] model illustrates that the deep belief network can deal with some degree of noise instances and gives them low weights in the higher hidden layer. Thus, they will be discarded in the classification phase. While in T_DBN [NR $+$ SMOTE], the noise reduction process may remove some instances that may have some participation in the classification phase and thus IQR may remove good data. The TCA (N3) [SMOTE] model has an average f-measure of 43.7 that is better than TCA (N3) alone which is shown in Table 5 with average f-measure of 40.1. This means that oversampling using SMOTE technique improves the defect prediction model for cross-project combinations.

4.2.2.3. DBN models on validation dataset result

We also extended our work and used ReLink validation dataset that is used in [17]. We applied T_DBN and DBN_Only on ReLink dataset. Table 9 shows information about Relink dataset.

Table 9

ReLink dataset information for validation test

Project	Type	# of instances (files)	Percent buggy files %	# of metrics
Apache	Web server	194	50.52	26
Safe	Security	56	39.29	26
Zxing	Bar-code reader library	399	29.57	26

The number of files is small and the dataset contain 26 metrics of complexity metrics such as average cyclomatic, average line, maximum cyclomatic, and more details for metrics and meaning are found in Understand website tools3

http://www.scitool.com/products/understand/.

[17]. The last field in the dataset shows whether the file is buggy or clean. In order to prepare the dataset for cross-project settings we followed the same steps as for the PROMISE dataset. We took 50 neurons and 60 epochs for setting the T_DBN and DBN_Only. Table 10 shows the result of the validation test on the Relink dataset. It also shows the TCA+ result for Nam et al. [17].

Table 10

The best f-measure for T_DBN and DBN_Only for Relink dataset

Source	Target	TCA			TCA+	(TCA+)	T_DBN	DBN only
		N2	N3	N4		Nam et al. [17]
Apache	Safe	65.2	60.6	47.1	47.0	72.0	68.3	50.0
Apache	Zxing	38.5	20.9	24.1	21.0	49.0	50.0	48.7
Safe	Apache	45.5	62.8	67.9	68.0	64.0	68.1	67.4
Safe	Zxing	39.1	37.1	48.1	39.0	43.0	50.8	47.4
Zxing	Apache	39.7	50.3	65.6	50.0	72.0	73.0	70.7
Zxing	Safe	47.4	41.2	57.1	41.0	64.0	66.7	63.2
Average		45.9	45.5	51.6	44.3	61.0	62.8	57.9

DBN models on validation dataset result analysis

Table 10 shows that in the validation test for the Relink dataset both T_DBN and DBN_Only outperform both TCA with (N2), (N3), and (N4) and TCA+ approaches for our version of implementation of TCA/TCA+ with logistic regression. Also, our results are comparable and slightly better than Nam et al. [17] for the same cross-project combinations with average f-measure of 62.8 for our models T_DBN against 61.0 for TCA+ for Nam et al. [17], with an improvement of 1.8%. Also, T_DBN achieved higher f-measure values in five out of six combinations. Nam et al. [17] used LIBLINEAR for logistic regression classifier implementation.

From the analysis of Section 4.2.2 we can summarize the following points:

•

DBN with transfer learning that utilizes the availability of the auxiliary dataset improves defect prediction for cross-project tasks. For example, some of the combination values (8 out of 22) for the T_DBN model are the highest among the baselines on PROMISE projects, and all the T_DBN values are higher than those for the DBN_Only model and the baseline models on Relink dataset.

•

DBN training on source project only can extract expressive features and improve cross-project defect prediction models (such as DBN_Only model).

•

The DBN model can deal with some degree of noise instances and extract expressive features to improve cross-project defect models (such as T_DBN [SMOTE] against T _DBN [NR $+$ SMOTE]).

•

Balancing class label using SMOTE technique improves the prediction defect model for cross-project combinations (such as T_DBN [SMOTE] model).

•

Our proposed model T_DBN outperforms the second baseline for Wang et al. [38] on traditional features on PROMISE dataset. Furthermore, T_DBN outperforms [17] on validation ReLink dataset.

4.2.3 Time and space cost for T_DBN model and DBN_Only result

RQ3: What is the time and space cost for the T_DBN based features extraction?

To answer this question, we kept track of cost of time and memory space for T_DBN to generate expressive features in two phases of the DBN model when training with All_without_source-target and fine-tuned with the source dataset, and carried out prediction on the target dataset. We also recorded time and memory cost for the setting without transfer learning on DBN_Only. For pre-processing and domain adaptation we excluded them from calculation of time and space cost because they are normal procedures. Table 11 shows the costs of time and space memory for T_DBN and DBN_Only models.

Time and space cost for T_DBN model and DBN_Only result analysis

The result in Table 10 shows the time in minutes (m) and memory space in megabytes (MB) for T_DBN and DBN_Only models. The values of time for generating expressive features and building the prediction model, then testing on target project ranges between 6 and 12:30 minutes for T_DBN model with an average of about 9 minutes. While it ranges between nearly 0:30 and 2 minutes with an average of about 1 minutes for the DBN_Only model. For example, log4j-jEdit combination took about 8 minutes (7:53.2) in the T_DBN model while the same model took nearly half a minute (00:27.9) for the DBN_Only model. The excessive time in the first model was due to the pre-training phase for auxiliary dataset compared to the log4j as source dataset which is nearly small dataset with (109) instances.

Table 11
Time & Memory Space cost for T_DBN &DBN_Only (m: minute)

Source	Target	T_DBN		DBN _Only
		Time/m	Memory Space/MB	Time/m	Memory Space/MB
ant1.6	camel1.4	06:05.5	11	01:02.7	11
jEdit	camel	07:13.3	11	01:08.0	11
ant1.6	poi3.0	08:15.5	10	01:03.4	11
synapase1.2	poi3.0	11:28.2	11	00:45.6	11
camel1.4	ant1.6	08:12.3	11	01:46.2	11
poi3.0	ant1.6	08:27.6	10	01:06.8	11
camel1.4	jEdit	09:13.0	11	01:51.0	11
log4j	jEdit	07:53.2	11	00:27.9	10
jEdit	log4j	09:10.4	11	00:56.8	11
lucene2.2	log4j	08:28.0	12	00:37.4	11
lucene2.2	xalan2.5	07:55.0	10	00:38.0	11
xerces1.3	xalan2.5	10:44.1	11	01:35.7	10
xalan2.5	lucene2.2	10:45.0	12	01:38.2	11
log4j	lucene2.2	11:38.7	11	00:38.4	11
xalan2.5	xerces1.3	12:29.1	11	01:38.9	11
ivy2.0	xreces1.3	08:02.4	11	01:19.4	11
xerces1.3	ivy2.0	08:14.7	11	01:21.6	11
synapse1.2	ivy2.0	08:03.8	11	00:49.0	10
ivy1.4	synapse1.1	08:01.9	11	00:55.9	11
poi2.5	synapse1.1	08:25.2	11	01:00.7	11
ivy2	synapse1.2	08:32.1	11	01:13.7	10
poi3.0	synapse1.2	09:06.2	11	01:06.9	11
Average		08:55.7	11.0	01:07.4	10.8

In general, the time taken is fairly applicable in practice. The memory space taken also ranges between 10 MB–12 MB for T_DBN and 10 MB–11 MB for DBN_Only. For example, xalan2.5-lucene2.2 needs 12 MB for T_DBN model and 11 MB for DBN_Only model.

From the analysis of the results for Section 4.2.3 we summarize the following:

•

DBN with its two different settings, with and without transfer learning, is acceptable and applicable for extracting expressive features, in practice, in terms of time and memory space.

5. Conclusion and future work

5.1 Conclusion

In this study we proposed a representative learning based DBN to learn latent features from source code metrics to enhance the defect prediction process on different target projects. We leveraged the concept of transfer learning within the DBN model to improve learning between two similar projects in cross-project combinations. We benefited from the availability of the auxiliary dataset to initialize the DBN model and transfer the learned parameters to fine-tune the model using the source project generated using representative features. Then, we built a prediction model for the target project. We designed three settings for our proposed model: T_DBN, DBN_Only and T_DBN [SMOTE]. We evaluated them on open source projects from the PROMISE repository for different cross-project combinations and they outperformed baseline 1, for Wang et al. [38] by 0.7%. In this paper, the three models outperformed the three baselines and improved the defect prediction model for cross-projects on an average F-measure with 3.6%, 4.9%, and 5.1% for DBN_Only, T_DBN [NR $+$ SMOTE] and T_DBN [SMOTE] respectively against the best model of the three baseline models TCA (N3) [SMOTE]. Moreover, we evaluated T_DBN and DBN_Only on ReLink validation dataset used in [17] and achieved improvement for average f-measure with 6.2 and 11.1% for DBN_Only and T_DBN respectively against the best baseline TCA (N4) of our version implementation. and outperformed TCA+ for Nam et al. [17] by 1.8 %. Furthermore, our traditional features extraction based model achieved significant results comparable to the semantic features extraction learner used by Wang et al. [38]. However, our traditional extraction model needs less effort and costs less, and utilizes the available information in different repositories. Moreover, in our work, we observed that the DBN_based model can deal with some degree of noise instances. Balancing class label using SMOTE technique has an improving effect on cross-project defect prediction model.

5.2 Future work

In the future, we would like to extend our representative features DBN_model to be evaluated on different language projects like C/C++, and C#. In addition, we could evaluate our model using different granularity levels such as class or package level. Moreover, we could test different techniques of transfer learning for cross-project combination for different domains and within the DBN model. Additionally, we could examine the performance of transfer learning with the DBN model with different classifiers. We could also conduct more analysis on source metrics to select the most important features to improve defect prediction model. Finally, we could search for a new technique to improve precision measure value in defect prediction models.

5.3 Threats to validity

5.3.1 Traditional feature selection

In our experiment we used some static code metrics that contained different features of source code but not all product metrics, and we cannot generalize these metrics to different cross-project combinations in different code languages. Adding more or different static code metrics may or may not improve the results. Moreover, the performance of the proposed model is still unknown for metrics of different source code languages and/or closed software projects.

5.3.2 Implementation of TCA+

Our second and third baselines depended on TCA/TCA+ approaches. We implemented our own version of normalization type option selection for TCA+ with the help of the instructions in [17] and the assistant of the first author of the same paper. We followed all the procedures defined in the paper.

Footnotes

Acknowledgments

This paper is part of the Master’s thesis submitted by Nawzat Alsmadi to the computer information systems, Faculty of IT, Yarmouk University, in April 2018.

References

Fischer

and Igel

, Training restricted Boltzmann machines: An introduction, Pattern Recognition 47(1) (2014), 25–39.

Kaur

and Singh

, Impact of Coupling and Cohesion metrics on fault tolerance prediction, International Journal of Computer Science and Communication Engineering 4(3) (2015).

Panichella

Oliveto

and De Lucia

, February. Cross-project defect prediction models: L’union fait la force, In Software Maintenance, Reengineering and Reverse Engineering (CSMR-WCRE), 2014 Software Evolution Week-IEEE Conference, 2014, pp. 164–173.

Sobhani

, P300 classification using deep belief nets, Doctoral dissertation, Colorado State University, 2014.

Kulis

Saenko

and Darrell

, What you saw is not what you get: Domain adaptation using asymmetric kernel transforms, In Computer Vision and Pattern Recognition (CVPR), IEEE Conference, 2011, pp. 1785–1792.

Wang

and Mahadevan

, Heterogeneous domain adaptation using manifold alignment, In IJCAI Proceedings-International Joint Conference on Artificial Intelligence 22(1) (2011), 1541.

, Human Action Recognition Using Deep Probabilistic Graphical Models, Doctoral dissertation, University of Sheffield, 2014.

Wang

and Shang

, Modeling physiological data with deep belief networks, International Journal of Information and Education Technology (IJIET) 3(5) (2013), 505.

F.Y.

et al., Fault diagnosis based on deep learning, Proceedings of the American Control Conference (ACC2016), Boston, MA, USA: IEEE, 2016, pp. 6851–6856.

10.

Zhang

, Towards Generalizing Defect Prediction Models, Doctoral dissertation, Queen’s University Kingston, Ontario, Canada, 2016.

11.

Hinton

G.E.

Osindero

and Teh

Y.W.

, A fast learning algorithm for deep belief nets, Neural Computation 18(7) (2006), 1527–1554.

12.

Tzortzis

and Likas

, Deep belief networks for spam filtering, In Tools with Artificial Intelligence, ICTAI 2007. 19th IEEE International Conference on 2, 2007, pp. 306–309.

13.

Qing

et al., Cross-Project Software Defect Prediction Using Feature-Based Transfer Learning, In Proceedings of the 7th Asia-Pacific Symposium on Internetware, ACM. 2015, pp. 74–82.

14.

Guyon

et al., Unsupervised and transfer learning challenges in machine learning, Brookline, Massachusetts 7 (2013).

15.

et al., Software defect prediction via convolutional neural network, In Software Quality, Reliability and Security (QRS), 2017 IEEE International Conference, 2017, pp. 318–328.

16.

Nam

, Survey on software defect prediction, Department of Computer Science and Engineering, The Hong Kong University of Science and Technology, Tech. Rep, 2014.

17.

Nam

Pan

and Kim

, Transfer defect learning, Proceedings of 2013International Conference on Software Engineering, IEEE Press, (2013), pp. 382–391.

18.

Song

Qin

and Zhang

, June. Chinese text categorization based on deep belief networks, In Computer and Information Science (ICIS), IEEE/ACIS 15th International Conference, 2016, pp. 1–5.

19.

Yosinski

et al., How transferable are features in deep neural networks? In Advances in Neural Information Processing Systems, 2014, pp. 3320–3328.

20.

Zhang

, Deep transfer learning via restricted Boltzmann machine for document classification, In Machine Learning and Applications and Workshops (ICMLA), 2011 10th International Conference on, 1, 2011, pp. 323–326.

21.

Weiss

Khoshgoftaar

T.M.

and Wang

, A survey of transfer learning, Journal of Big Data 3(1) (2016), 1–40.

22.

Deng

and Yu

, Deep learning: methods and applications, Published as Foundations and Trends® in Signal Processing 7(3–4) (2014), 197–387.

23.

Ri-Xian

Ming-Hai

and Xian-Bao

, Defects detection based on deep learning and transfer learning, Metallurgical & Mining Industry (7), (2015), 312–321.

24.

Prasad

M.C.

Florence

and Arya

, A Study on Software Metrics based Software Defect Prediction using Data Mining and Machine Learning Techniques, International Journal of Database Theory and Application 8(3) (2015), 179–190.

25.

Jureczko

, Significance of different software metrics in defect prediction, Software Engineering: An International Journal 1(1) (2011), 86–95.

26.

Kan

Shan

and Chen

, Domain adaptation for face recognition: Targetize source domain bridged by common subspace, International Journal of Computer Vision 109(1–2) (2014), 94–109.

27.

Kim

et al., REMI: defect prediction for efficient API testing, In Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering, ACM, 2015 August, pp. 990–993.

28.

Sharma

M.K.

Sheet

and Biswas

P.K.

, Abnormality Detecting Deep Belief Network, In Proceedings of the International Conference on Advances in Information Communication Technology & Computing, ACM, 2016 August, p. 11.

29.

Long

et.al., Learning transferable features with deep adaptation networks, In International Conference on Machine Learning, 2015, pp. 97–105.

30.

Ozturk

M.M.

and Zengin

, Towards a Better Understanding of Static Code Attributes for Defect Prediction, In Proceedings of the tenth International Conference on Software Engineering Advances ICSEA, 2015, pp. 40–44.

31.

Oquab

et. al. Learning and transferring mid-level image representations using convolutional neural networks, In Proceedings of the IEEE Conferenceon Computer Vision and Pattern Recognition, 2014, pp. 1717–1724.

32.

Chawla

N.V.

Bowyer

K.W.

Hall

L.O.

and Kegelmeyer

W.P.

, SMOTE: synthetic minority over-sampling technique, Journal of Artificial Intelligence Research 16 (2002), 321–357.

33.

and Ma

, Towards Cross-Project Defect Prediction with Imbalanced Feature Sets, arXiv: 1411.4228v1 [cs.SE] 16 Nov 2014.

34.

Cao

Sun

Cao

and Tan

, Software defect prediction via transfer learning based neural network, In Reliability Systems Engineering (ICRSE), 2015 First International Conference, IEEE, 2015, pp. 1–10.

35.

Pan

S.J.

, Transfer Learning, Nanyang Technological University, Singapore, 2014, pp. 553–570.

36.

Pan

S.J.

et al., Cross-domain sentiment classification via spectral feature alignment, In Proceedings of the 19th international conference on World Wide Web. ACM, 2010, pp. 751–760.

37.

Pan

S.J.

et al., Domain adaptation via transfer component analysis, IEEE Transactions on Neural Networks 22(2) (2011), 199–210.

38.

Wang

Liu

and Tan

, Automatically learning semantic features for defect prediction, In Proceedings of the 38th International Conference on Software Engineering, ACM, 2016, pp. 297–308.

39.

Menzies

et al. Defect prediction from static code features current results, limitations, new approaches, Automated Software Engineering 7(4) (2010), 375–407.

40.

Zimmermann

et al., Cross-project defect prediction: a large-scale experiment on data vs. domain vs. process. In Proceedings of the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT Symposium on the foundations of software engineering, ACM, 2009, pp. 91–100.

41.

et al., Deep model based domain adaptation for fault diagnosis, IEEE Transactions on Industrial Electronics 64(3) (2017), 2296–2305.

42.

Zhao

, Research on the deep learning of the small sample data based on transfer learning, In Proceedings of AIP Conference 1864(1) (2017), 020018.

43.

Bengio

, Deep learning of representations for unsupervised and transfer learning, ICML Unsupervised and Transfer Learning 27 (2012), 17–36.

44.

Bengio

, Practical recommendations for gradient-based training of deep architectures, In Neural networks: Tricks of the trade. Springer. Berlin, Heidelberg. (2012), pp. 437–478.

45.

Bengio

, Learning deep architectures for AI. Foundations and trends®, Machine Learning 2(1) (2009), 1–127.

46.

LeCun

Bengio

and Hinton

, Deep Learning, Nature 521(7553) (2015), 436–44.

47.

et al., Transfer learning for cross-company software defect prediction, Information and Software Technology 54(3) (2012), 248–256.

48.

et al., Transfer learning for video recognition with scarce training data, CoRR, 2014, pp. 1–12.

49.

and Ji

, Constrained Deep Transfer Feature Learning and its Applications, In Computer Vision and Pattern Recognition (CVPR), IEEE. 2016, pp. 5101–5109.

50.

Yang

et al., Deep Learning for Just-In-Time Defect Prediction, Proceedings of Software Quality, Reliability and Security (QRS), 2015 IEEE, 2015, pp. 17–26.

Source code-based defect prediction using deep learning and transfer learning

Abstract

Keywords

1. Introduction

1.1 Background

1.2 Problem statement

2.1 Source code metric-based defect prediction

2.2 Deep learning-based predict defects

2.3 Transfer learning for cross-project defect prediction

3.1 The overall research approach

3.3 Research phases

Table 1 Properties of 13 Java dataset

3.3.1.1. Dataset collection

3.3.1.2. Dataset preprocessing

3.3.2.1. DBN Pre_training Model

3.3.2.2. Transfer parameters within DBN model

1 https://netbeans.org/community/releases/80/install.html.

4.1 Experimental setup

4.1.1 Dataset

4.1.2 Baselines of traditional features

4.1.3 Cross-project defect prediction

4.1.4 Parameter settings for the DBN model

4.1.4.1. Setting the number of neurons in each layer

4.1.4.2. Setting the number of epochs

4.2.1 DBN model with transfer learning results

4.2.1.1. DBN model with transfer learning results analysis

Table 4 Precision, recall, and f-measure of the T_DBN model

4.2.2.1. DBN with and without transfer learning result

4.2.2.2. DBN models with SOMTE technique results

4.2.2.3. DBN models on validation dataset result

Table 11 Time & Memory Space cost for T_DBN &DBN_Only (m: minute)

5.1 Conclusion

5.2 Future work

5.3 Threats to validity

5.3.1 Traditional feature selection

5.3.2 Implementation of TCA+

Footnotes

Acknowledgments

References

Table 1
Properties of 13 Java dataset

¹
https://netbeans.org/community/releases/80/install.html.

Table 4
Precision, recall, and f-measure of the T_DBN model

Table 11
Time & Memory Space cost for T_DBN &DBN_Only (m: minute)