A novel approach to fuzzy c-Means clustering using kernel function

Abstract

Clustering is an unsupervised procedure that divides a set of objects into homogeneous groups. Two types of clustering are possible, Hard clustering and Soft clustering/Fuzzy clustering. Hard clustering is not feasible for complex datasets that contain uncertainty and overlapping clusters, whereas fuzzy clustering efficiently handles it. FCM is sensitive to the initial values and challenging to cluster nonlinear data. A new approach is implemented here with the Fuzzy c-Means (FCM) clustering algorithm to improve the performance. The Kernel function ensures the linear separability of complex clusters by projecting the feature space into a higher dimension and not subject to the initial values. The Kernel-based FCM (KFCM) optimized the clustering. The relevant features are considered for clustering, and it improves the validity of clusters. The irrelevant features blur the clusters and reduce the quality. Silhouette index (SI) and Davies-Bouldin index (DBI) have been used as the evaluation function. The experiments are conducted on two benchmark datasets and one artificial dataset. The result justifies Kernel-based FCM, and the superiority of features reduced Kernel-based FCM clustering over other traditional fuzzy clustering techniques.

Keywords

Fuzzy c-Means kernel function feature selection silhouette index Davies-Bouldin index

1. Introduction

The rapid growth of the digital world contributes to the exponential growth of data in the contemporary scenario. In the early days of digitization, the data was produced by computers, which later moved on to different platforms and to different formats such as image, audio, video. Retrieving relevant information from this vast data is very difficult. Data Clustering is one of the standard data mining techniques and has been widely used in Big Data research. Clustering is used to break down the data into associated components to make patterns and order visible. Typically, a cluster is interpreted as a group of related data points around a core known as a centroid. Vast volumes of data are extensively analyzed to derive valuable information in the form of new interactions, trends, or clusters to appropriate a user’s decision.

Clustering is an unsupervised classification, which does not require an assumption about category labels that identify objects with a prior identifier. It separates data points into homogeneous groups so that data points in the same group are as identical as possible and data points are as dissimilar as possible in separate groupings. The classification demands human interaction to choose the criteria and unsupervised clustering that handpick the clustering criteria itself.

The clusters with well-defined boundaries are considered hard clusters, and those without a well-defined boundary are soft or fuzzy clusters. In hard clustering algorithms, each data point of the data set is assigned to one and only one cluster, and in fuzzy clustering, the data points are assigned to more than one cluster with different membership values [1]. The performance of the hard clustering algorithm, such as the k-means algorithm, depends on the initial values of centroids and gives no guarantee for an optimum solution. Clustering plays a vital role in identifying the underlying pattern structure since big data involves too much uncertainty, and fuzzy clustering is one of the easiest ways to catch uncertainty [2].

Most conventional clustering algorithms use centres and basic distance measurements in the original space and only cluster data into spherical or elliptical shapes [3] without recognizing the irregular shape. Hard clustering investigates only the distance between data patterns and is not adequate for finding clusters with complicated internal structures. Clusters found using such techniques cannot be easily described. The Fuzzy clustering algorithms should be capable of identifying clusters with non-spherical or random shapes and some points in one cluster, which essentially be closer to other points in other clusters than the points in their clusters [4].

For complex datasets where there are overlapping clusters, hard allocation of points to clusters is not feasible. A fuzzy clustering algorithm can be used for such overlapping clusters [5]. Fuzzy clustering is a sophisticated data processing strategy that is unlabeled, involves outliers and has irregular patterns. Fuzzy clustering algorithms are better to deal with discrimination of similar structures, data set in n-dimensional spaces than hard clustering [6]. It is more valuable for unlabeled data and the outliers of the dataset [7]. Hard clustering such as the k-means and Ward’s clustering lack the precision for image processing and machine learning methods than FCM clustering [6]. FCM algorithm is sensitive to the noise and isolated data and initial clustering centre. It is a local hill-climbing algorithm, which is easy to converge to a local maximum [8]. The kernel function is used to figure out the data points, which cannot be linearly separated from original space to high dimensional space. This results in the new space giving better performance [9].

2. Literature review

Zadeh introduced the fuzzy sets in 1965. Zadeh introduced the grade of membership. In a crisp set, the data points have the membership either 0 or 1. In fuzzy sets, membership for each object varies from 0 to 1 [5]. Fuzzy c-Means (FCM) is an improvised version of the traditional c-means/k-means algorithm for data classification. The FCM is more flexible than k_means in partitioning the objects. FCM is the most commonly used fuzzy clustering method, allowing one data to belong to more than one cluster. This method is developed by J. Dunn in 1973 [10] and improved by James C Bezdek in 1981. In 1996, Pal proposed the fuzzy possibilistic c-means (FPCM) model and algorithm that generated both membership and possibilities when clustering unlabeled data. FPCM resolved the defect of FCM, i.e. noise sensitivity defect, and conquers the coincident cluster problem of PCM [11]. Fuzzy clustering technique is used to determine the policies and plans in the enlarged cases of coronavirus [12].

Zhang and Chen initiated a novel clustering algorithm called the kernel fuzzy c-means algorithm. It is resilient to outliers and noise and also tolerates clusters of varying sizes [13]. Timothy C Havens extended the FCM to massive data by implementing sampling techniques followed by non-iteration extension and kernel methods [14]. Telmo M. Silva Filho combined a metaheuristic optimization, like particle swarm optimization, with a traditional clustering, FCM, to improve the accuracy. FCM applied in different realms like image segmentation [15, 16], detection and correction of insufficient data in the analysis of power system load data [17], mode-seeking framework [18]. Zhang Siqing proposed a clustering protocol based on fuzzy logic for Multi-hop WSN to extend the life of wireless sensor networks (WSNs) and lessen energy consumption [19]. The possibilistic fuzzy c-means algorithm used in the sphere detects the earthquake and releases a warning alarm [20]. The Isolation forest method is used to find the outliers in the databases. P. Karczmare and others enriched the Isolation Forest on the base of Fuzzy c-means [21].

Ding and Fu launched a kernel-oriented fuzzy c-means clustering algorithm, the GAKFCM clustering algorithm, which used the adaptive genetic algorithm to optimize the initial clustering centre and improve the performance of the FCM by using the KFCM algorithm. It is oriented to optimise the genetic algorithm juxtaposed with the enhanced genetic algorithm and the kernel technique [8]. C Jiang and others proposed an FCM based on a Simulated annealing algorithm and kernel technology that outperformed conventional FCM and resolved the three defects: discoveries of outliers, an unknown number of clusters, and an unbalanced process of clustering [22]. FCM is responsive to the initial cluster centres and easy to converge to local optimization. Wang, Zhang and Xiao introduced a Fruit Fly Optimization KFCM clustering algorithm based on the fruit fly algorithm. This algorithm optimises the initial clustering centre and then categorises data using the kernel-based fuzzy c-means clustering algorithm [23].

The adaptive kernel-based fuzzy C-means clustering with spatial constraints model, presented by Guang Hu and Zhenbin Du has increased the efficiency of image segmentation [24]. Wenke Zang and others established a kernel-based intuitionistic fuzzy C-means clustering using the improved multi-objective artificial immune algorithm (KIFCM-IMOIA). The KIFCM-IMOIA uses the kernel trick and the intuitionistic fuzzy entropy (IFE) which improves the robustness to noises [25]. M. Giri and S. Majumder developed a new eigenvalue-based technique for spectrum sensing. That paper used kernel based Fuzzy C means clustering to cluster the test vectors [26]. Webiao Qiao and Zhe Yang introduced kernel-based fuzzy C-means into dolphin swarm algorithm [27].

2.1 Fuzzy clustering

Fuzzy clustering is more flexible than hard clustering in that each object may have membership in more than one cluster [14]. In hard clustering, the data points have the membership of either 0 or 1. The membership of each object in different clusters in the fuzzy clustering is varying from 0 to 1 [5]. In fuzzy clustering, the degree of membership of an object 1 $\leqslant j\leqslant N$ is $u_{ij}\in$ [0, 1], and the total membership of each object across all clusters must equal to 1, i.e. $\sum u_{ij}=$ 1 where 1 $\leqslant j\leqslant c$ and 1 $\leqslant i\leqslant N$ .

$\displaystyle 0<\sum\nolimits_{j=1}^{N}u_{ij}<N\quad\text{[5,28]}$

where $N$ is the total number of data objects.

2.2 Fuzzy c-Means clustering

Fuzzy c-Means is the most commonly used fuzzy-based clustering algorithm, a modification of the c-means clustering technique. FCM divides the data samples into C fuzzy clusters. The cost function of FCM is minimized as a result of calculating the cluster centre. The cost function of the FCM is as follows:

$\displaystyle J(U,V)=\sum_{j=1}^{n}\sum_{i=1}^{c}u_{ij}^{m}\|x_{j}-v_{i}\|^{2}$ (1)

Where $U=\{u_{ij}\}_{c\times n}$ is the membership matrix, $V=\{v_{1},v_{2},v_{3},\ldots,v_{c}\}$ center of $c$ clusters and $X=$ $\{x_{1},x_{2},x_{3},\ldots,x_{n}\}$ is the data set with $n$ data points. $c$ is the number of clusters and $n$ the number of data points, $u_{ij}$ the membership of $x_{j}$ in class $i$ , $m$ the degree of fuzzification ie $m>$ 1, $\|x_{j}-v_{i}\|^{2}$ Euclidean distance between the data point $x_{j}$ and the cluster centre $v_{i}$ . FCM tries to get the optimal value of U and V to minimize the objective function. Each data point is assigned a high membership in a class whose centre is close to the data point; otherwise, low membership. Algorithm 1 gives the Fuzzy c-Means algorithm [29].

The FCM partition of dataset $X\!=\!\{x_{j},j=1,2,\ldots,\linebreak n\}$ with $V_{i}(i=1,2,\ldots,c)$ the centers of each cluster and $U_{ij}(i=1,2,3,\ldots,c,j=1,2,\ldots,n)$ as the fuzzy membership of data point $j$ belonging to class $i$ . $d_{ij}=U_{ij}\|x_{j}-v_{i}\|$ is called the fuzzy deviation of $x_{j}$ from class $i$ . $d_{ij}$ is the Euclidean distance between $x_{j}$ and $v_{i}$ . FCM has partitioned the dataset into $c$ clusters by using the initial features of the dataset.

[t] Fuzzy c-Means algorithm Input: $X$ , $c$ , $m$ , $\varepsilon$ Output: $U$ , $V^{\prime}$ [1] Initialize the cluster center $V=\{v_{1},v_{2},v_{3},\ldots,v_{n}\}$ Compute the membership matrix

$\displaystyle u_{ij}=\sum_{k=1}^{c}\left[\frac{\|x_{j}-v_{i}\|}{\|x_{j}-v_{k}% \|}\right]^{\frac{-2}{m-1}}$ (2)

Update the cluster center

$\displaystyle v_{i}^{\prime}=\frac{\sum_{j=1}^{n}u_{ij}^{m}x_{j}}{\sum_{j=1}^{% n}u_{ij}^{m}}\quad i\in(1,2,3,\ldots,c)$ (3)

$\|V^{\prime}-V\|<\varepsilon$ break Goto step 2

The cluster centre is initialized randomly. The corresponding membership values are evaluated using Eq. (2) and updated the cluster centres using Eq. (3). FCM algorithm stopped when the change from the previous cluster centre to the new cluster centre has reached the target value. The convergence speed of the FCM algorithm is greatly influenced by the initial value, mainly in the case of a large number of clusters and easy to fall into local optimization [30]. The steps (2) to (4) are iterated till the algorithm terminates. The estimation of the modified membership function is a consequence of the objective function being minimized. The centroid is calculated with FCM as the mean of all points, weighted by their degree of cluster membership. FCM transfers the cluster centres within a data set to the correct position through iteratively reviewing the cluster centres and the membership grades for each data point. In the end, a hard classification is reached by assigning each data point exclusively to the class with the highest membership value. The performance of the FCM depends on the initial cluster centres.

2.3 Kernel clustering

Kernel Methods enables researchers to examine nonlinear relations with the same competence as linear algorithms. A kernel-based FCM is proposed to simplify FCM clustering, to resolve some of the problems in the FCM clustering algorithm. The weakness of fuzzy c-means is its sensitivity to the prototypes [6] and ease to fall into local optimum [8]. In many clusters, the convergence speed is subjective to the initial value [30]. Kernel clustering algorithm does not rely on the initialization of the clusters. It discovers nonlinearly separable clusters and finds optimal solutions due to its search procedure and incremental nature [31]. The kernel approach project data into higher dimensional space is an efficient way to solve the problem of FCM. Kernelization is an effective mathematical technique that helps one to carry out tacit calculations on the data set, thereby allowing a clustering process to removing clusters of nonlinear separating hypersurfaces [20, 21, 32].

3. Experiments and analysis

This section illustrates the experiments conducted on FCM and KFCM to validate the analysis. In the first experiment, FCM and FCM using kernel trick is evaluated. In the second experiment, are used only the relevant features of a dataset for clustering. A complete program was created using the Python programming language to find the optimal result value for the FCM and KFCM algorithms.

3.1 FCM with kernel function and feature selection

The FCM effectiveness is limited mainly to spherical clusters. The kernel function attempts to address this problem by mapping data with a nonlinear relationship to appropriate feature space. The kernel tricks are used for the data analysis problems such as classification, clustering, categorization. The kernel function transforms the features to a higher dimension. It repeats until the changes of the cluster converge to the fixed value. It changes any linear model into a non-linear model by replacing the initial features with the higher features. Thus the projection of the dataset turns from the initial representation space to a higher-dimensional space. Since it is an implicit feature expansion it needs the inner products between all pairs of data. The result shows that while applying the kernel function, the optimal clusters are defined. The kernel FCM method outperforms the traditional FCM method.

[h] FCM with Kernel Function (KFCM) Input: $X=(a_{1},a_{2},a_{3},\ldots,a_{n}),c,\gamma,l_{c}$ [1] $X_{\textit{new}}=\phi$ $C,V_{m}=\textit{FCM}(X,c)$ $i=$ 1 to $l_{c}$ $X_{p}=$ APPLYKERNEL(X) $C,V_{m}^{\prime}=\textit{FCM}(X_{p},c)$ $e=|V_{m}-V_{m}^{\prime}|$ $e<\gamma$ break $X=X_{\textit{new}}$ return C

Algorithm 2 illustrates FCM Clustering with Kernel Function which is denoted as KFCM. The KFCM algorithm accepts dataset $X=\{x_{j},j=1,2,\ldots,n\}$ with the attribute $a_{1}$ , $a_{2}$ , $a_{3}$ , …, $a_{n}$ no of clusters $c$ , target value $\gamma$ and iteration count $l_{c}$ as input parameters. $X_{\textit{new}}$ initializes with $\emptyset$ . The second step performs the FCM algorithm on dataset $X$ and partitions the dataset into $c$ clusters using the dataset’s whole features and calculating the validity of clusters $V_{m}$ and the clusters $C$ . Then function APPLYKERNEL is applied. In this experiment, the Linear kernel function is applied on x ${}_{i}$ , the data element of dataset $X$ . The kernel value $f$ is extracted by finding the dot product of the data and merged into each data point in the dataset. The updated dataset with the kernel values is clustered again using the FCM algorithm. It finds out the clusters $C$ and the new validity of clusters $V_{m}^{\prime}$ . The difference between $V_{m}$ and $V_{m}^{\prime}$ is evaluated as $e$ . The algorithm terminates when there is a minor change in the validity measures, i.e. $e<\gamma$ and returns the clusters $C$ . $\gamma$ is a predetermined constant and is defined as 10 ${}^{-3}$ in this experiment. Otherwise, repeat the algorithm until the iteration count $l_{c}$ is reached.

[t] function: APPLYKERNEL(X) [1] $X_{\textit{new}}=X$ $x_{i}\in X_{\textit{new}}$ $f=\sqrt{x_{i1}^{2}+x_{i2}^{2}+x_{i3}^{2}+\ldots+x_{in}^{2}}$ $X_{\textit{new}}=X_{\textit{new}}\cup\{x\ll f\}$ return $X_{\textit{new}}$ [t] FCM with Feature Selection and Kernel Function Input: $X=(a_{1},a_{2},a_{3},\ldots,a_{n}),c,\alpha,\gamma,l_{c}$ [1] $X_{\textit{new}}=\phi$ $a_{i}\in X$ $a_{i}\in X/a_{j}$ $\textit{Cor}_{ij}=\textit{Correlation}(a_{i},a_{j})$ $\textit{Cor}_{ij}<\alpha$ and $a_{i}\not\in X_{\textit{new}}$ $X_{\textit{new}}=X_{\textit{new}}\cup\{a_{i}\}$ $\textit{Cor}_{ij}<\alpha$ and $a_{j}\not\in X_{\textit{new}}$ $X_{\textit{new}}=X_{\textit{new}}\cup\{a_{j}\}$ $X=X_{\textit{new}}$ $C,V_{m}=\textit{FCM}(X,c)$ $i=$ 1 to $l_{c}$ $X_{p}=$ APPLYKERNEL(X) $C,V_{m}^{\prime}=\textit{FCM}(X_{p},c)$ $e=|V_{m}-V_{m}^{\prime}|$ $e<\gamma$ break $X=X_{\textit{new}}$ return C Algorithm 3 is the extension of KFCM. It outlines the KFCM with feature selection. The predefined value $\alpha$ , target value of correlation coefficient is also added to the parameter list of the algorithm KFCM. The correlation coefficient, $\textit{Cor}_{ij}$ between the attributes $a_{i}$ and $a_{j}$ is calculated. The attributes are selected and added into $X_{\textit{new}}$ , the temporary dataset if the correlation coefficient $\textit{Cor}_{ij}$ is less than $\alpha$ . The temporary dataset $X_{\textit{new}}$ is updated with selected attributes. While applying the Linear kernel function, APPLYKERNEL, the kernel value $f$ was originated and updated the dataset by adding $f$ with each data point. Again the data was clustered using FCM and evaluate $V_{m}^{\prime}$ . The value $e$ is calculated as the difference between $V_{m}$ and $V_{m}^{\prime}$ . The algorithm terminated when $\gamma$ is less than $e$ or at the value of count $l_{c}$ . The algorithm returns the clusters $C$ .

3.2 Validity measures

Figure 1.

SI value of FCM and Linear Kernel FCM on IRIS dataset without feature selection.

Figure 2.

SI value of FCM and Linear Kernel FCM on Fruits dataset without feature selection.

Figure 3.

SI value of FCM and Linear Kernel FCM on Wine dataset without feature selection.

Figure 4.

DBI value of FCM and Linear Kernel FCM on IRIS dataset without feature selection.

Figure 5.

DBI value of FCM and Linear Kernel FCM on Fruits dataset without feature selection.

Figure 6.

DBI value of FCM and Linear Kernel FCM on Wine dataset without feature selection.

Unsupervised clustering is found without any cluster labels. Therefore, the inter-cluster distance and the interspace between the data points in a cluster is used to evaluate the clusters. In this paper, two clustering metrics are used to measure the consistency of clustering: the Silhouette Index (SI) and Davies-Bouldin index (DBI). Silhouette index displays a measure of intra-cluster distance and inter-cluster difference [33]. The range of SI is 1 to $-$ 1. The best value is one that indicates the clusters are well defined. The Davies-Bouldin index (DBI) [34] is evaluated the average similarity between clusters, where similarity is the ratio of within-cluster distances to between-cluster distances. Thus, clusters that are farther apart and less dispersed will result in a better score. Values closer to zero indicate a better partition. These validity measures serve the dual function of minimizing cluster distribution, calculating cluster separation.

3.3 Datasets

The performance of FCM and Kernel FCM is evaluated effectively using three test data sets. The benchmark datasets IRIS and Wine were taken from the UCI machine learning repository, and the other dataset, Fruits, is an artificial dataset. The IRIS is a two-dimensional dataset, which consists of 150 samples with four features. The four features of IRIS are the length and width of sepals and petals of three species of Iris. The dataset WINE with 13 features and consists of 179 samples. The dataset is multivariate and attributes characteristics are integer and real. It provides information on the chemical analysis of different wines. The amounts of 13 ingredients are identified in each of the three varieties of wines. The dataset Fruits contain 70 images of different coloured fruits. The image file cannot be used directly for clustering. Preprocessing of data is conducted on the dataset and converts each sample to the corresponding combination of Red, Green, and Blue. Then the dataset becomes a two-dimensional dataset with Red, Green, and Blue values corresponding to the image as the attributes of the dataset Fruits.

3.4 Results

Figure 7.

SI value of FCM and Linear Kernel FCM on IRIS dataset with feature selection.

Figure 8.

SI value of FCM and Linear Kernel FCM on Fruits dataset with feature selection.

Figure 9.

SI value of FCM and Linear Kernel FCM on Wine dataset with feature selection.

Figure 10.

DBI value of FCM and Linear Kernel FCM on IRIS dataset with feature selection.

Figure 11.

DBI value of FCM and Linear Kernel FCM on Fruits dataset with feature selection.

Figure 12.

DBI value of FCM and Linear Kernel FCM on Wine dataset with feature selection.

This section summarises the findings of the experiments performed. In this section, the results demonstrations the efficacy of KFCM over FCM and the importance of feature selection in clustering.

Table 1 shows the Silhouette index (SI) and Davies-Bouldin index (DBI) values of the FCM cluster. The SI is varying with the number of clusters. The highest value of the SI is considered an optimal cluster. The smallest values of the DBI are measured as better clusters. As per the values of SI and DBI, cluster two is optimal in the IRIS dataset. From Table 1, it is understood that cluster two has the highest SI value and cluster three has the smallest DBI value for the Fruits dataset. There is an inconsistency about the optimal partition. Among the clusters of dataset WINE, cluster two is optimal because of the highest SI and small DBI value.

Table 1

SI and DBI value of FCM clustering with whole features

FCM with whole features
Dataset and	Validity	Clusters
(Features)	Method	2	3	4	5
IRIS-150	SI	0.68081	0.54928	0.49249	0.48720
Features-4	DBI	0.40483	0.66960	0.78173	0.81667
Fruits-70	SI	0.56849	0.53828	0.50803	0.43178
Features-3	DBI	0.64171	0.62532	0.78071	0.85809
Wine-178	SI	0.70397	0.63861	0.49603	0.57503
Features-13	DBI	0.41787	0.54340	0.63422	0.50102

Table 2

SI and DBI value of Linear Kernel FCM with whole features

Linear Kernel FCM with whole features
Dataset and	Validity	Clusters
(Features)	Method	2	3	4	5
IRIS-150	SI	0.64628	0.57533	0.57522	0.59335
Features-4	DBI	0.45593	0.56161	0.51912	0.46448
Fruits-70	SI	0.58208	0.63708	0.63528	0.61550
Features-3	DBI	0.59593	0.43861	0.44106	0.45109
Wine-178	SI	0.70397	0.63861	0.49603	0.57503
Features-13	DBI	0.41787	0.54340	0.63422	0.50102

Table 3

SI and DBI value of FCM clustering with the selected number of features

FCM with selected number of features
Dataset and	Validity	Clusters
(Features)	Method	2	3	4	5
IRIS-150	SI	0.684239	0.550545	0.508085	0.488689
Features-3	DBI	0.402098	0.656347	0.711802	0.81279
Fruits-70	SI	0.592839	0.472240	0.483442	0.382204
Feature-2	DBI	0.597660	0.722145	0.724848	0.880602
Wine-178	SI	0.659067	0.575184	0.566827	0.514101
Features-8	DBI	0.477561	0.526340	0.53271	0.565719

Table 4

SI and DBI value of Linear Kernel FCM with the selected number of features

Linear Kernel FCM with selected number of features
Dataset and	Validity	Clusters
(Features)	Method	2	3	4	5
IRIS-150	SI	0.763307	0.546997	0.506185	0.477789
Features-3	DBI	0.311321	0.658129	0.726425	0.842092
Fruits-70	SI	0.631776	0.459067	0.483642	0.375137
Feature-2	DBI	0.619930	0.791961	0.730815	0.884368
Wine-178	SI	0.63367	0.584509	0.567533	0.554965
Features-8	DBI	0.492629	0.508725	0.533364	0.51586

The kernel trick is applied to the datasets, and the results illustrate in Table 2. In the Linear kernel dot product of the data points is evaluated. The values of SI are increased for the datasets IRIS and Fruits. In the case of dataset Fruits, while applying FCM, according to SI, cluster two is better, and by DBI, cluster three is better. This inconsistency is changed while using Kernel Trick. The result shows that cluster three is better than two. It indicates that the clustering implemented by Kernel Trick is more optimal than the traditional FCM. The result of the Wine dataset is not changed. It means the algorithms FCM and Linear Kernel FCM (LKFCM) have the same effect on this dataset.

Figures 1, 2 and 3 demonstrates the comparison study of the SI value of the FCM algorithm and Linear Kernel FCM (LKFCM) Clustering. Figures 4, 5 and 6 shows the DBI value of the FCM algorithm and Linear Kernel FCM (LKFCM) Clustering algorithm. The exclusive features of the datasets are used in this section.

In the second section, only the relevant features are considered for clustering. All the features are not equally significant for clustering. Some features are essential, while some others are less important. The feature selection approach aims to select a small subset of features that maximize relevance to the target and minimize redundancy. Most clustering algorithms do not apply feature selection or the relation between the features. The same dataset is implemented here with fewer but relevant features. The similarity is distance-dependent, so there are at least a few characteristics on which the points are identical to each other for a pair of data points in a cluster. Various methods of clustering presume that all characteristics are equally relevant for clustering. This is one of the causes that most clustering algorithms may not perform well in high dimensional data. The time required to cluster high dimensional data is more than the time required for low dimensional data.

A significant feature helps to create better clusters, while an insignificant one does not help for the same and blurring the clusters can negatively affect the clustering algorithms. Unimportant features are considered noisy or obsolete and may be omitted to reduce the data size for more effective clustering. The reduction of features reduces the time for clustering. Pearson’s correlation coefficient between the variables is considered for the feature selection. Pearson’s correlation coefficient gives the measures of linear correlation between two variables. According to the correlation coefficient, features are selected for the clustering.

Tables 3 and 4 illustrate the experimental result of feature reduced, FCM and Linear Kernel-based FCM. The selected features are three from four of IRIS, two from three of Fruits, eight from thirteen of Wine. The experiment results are better than the results of clustering without feature selection in IRIS and Fruits datasets. The resultant SI value of the feature reduced, IRIS and Fruits is greater than the traditional FCM and simple Linear Kernel-based FCM. Furthermore, the DBI value is smaller than the result of the simple FCM and simple Kernel-based FCM. The Wine dataset has less value of SI and a greater value of DBI than the simple FCM and Kernel-based FCM without feature selection. The correlation coefficient between the variables is significantly less in the Wine dataset. So as the number of relevant features reduces the quality will also be decreased.

Figures 7, 8, 9 Illustrates the SI value of FCM and Linear Kernel FCM (LKFCM) Clustering applied on the datasets IRIS, Fruits, Wine, respectively. Figures 10, 11, 12 shows the DBI value of the same.

4. Conclusion

The paper proposes the FCM clustering and a Kernel based FCM algorithm for clustering the data. The contributions of this paper are as follows. Primarily, a comparison study of FCM and Kernel-based FCM is performed. Secondly, the selected features are used for clustering. The experiments have been conducted on two benchmarking datasets and one artificial dataset. The results show that the Kernel-based FCM has better optimization than simple FCM, and the use of selected features for clustering improves the result of clustering. The evaluation is performed in terms of two validity metrics, the SI and DBI. Another possible extension of the topic would be to solve the problem that FCM is sensitive to the initial cluster centroids. Further study on this topic will also include the multiple kernel functions for clustering in real-world problems.

References

Aggarwal

Reddy

. DATA Custering Algorithms and Applications. 2013. p. 652.

Maheshwari

Sharma

. A Survey on Big Data Challenges in Fuzzy Algorithms. Int J Comput Appl. 2016; 145(5): 15-7.

Lee

Ouyang

. A neuro-fuzzy system modeling with self-constructing rule generation and hybrid SVD-based learning. IEEE Trans Fuzzy Syst. 2003; 11(3): 341-53.

Wang

Lee

. A Kernel-based Fuzzy Clustering Algorithm. First International Conference on Innovative Computing, Information and Control – Volume I. 2006. pp. 4-7.

Zadeh

. Fuzzy Sets. Information and Control. 1965; 8: 338-58.

Lewis

. Fuzzy Clustering Algorithms – Review of the Applications. 2016 IEEE Int Conf Smart Cloud. 2016. pp. 282-8. Available from: http://ieeexplore.ieee.org/document/7796188/.

Feher

Schmidt

. Fuzzy clustering as a means of selecting representative conformers and molecular alignments. J Chem Inf Comput Sci. 2003; 43(3): 810-8.

Ding

. Kernel-based fuzzy c-means clustering algorithm based on genetic algorithm. Neurocomputing. 2016; 188: 233-8. doi: 10.1016/j.neucom.2015.01.106.

Breneman

. Kernel Methods for Pattern Analysis. Technometrics. 2005; 47(2): 237-237.

10.

Dunn

. A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters. Journal of Cybernetics. 1973; 3(3): 35-57.

11.

Pal

NRR

Pal

Keller

JMM

Bezdek

JCC

. A possibilistic fuzzy c-means clustering algorithm. IEEE Trans Fuzzy Syst. 2005; 13(4): 517-30. Available from: http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=1492404.

12.

Mahmoudi

Baleanu

Mansor

Tuan

Pho

. Fuzzy clustering method to compare the spread rate of COVID-19 in the high risks countries. Chaos, Solitons and Fractals. 2020; 140: 1-9.

13.

Zhang

Chen

. Clustering incomplete data using kernel-based fuzzy C-means algorithm. Neural Process Lett. 2003; 18(3): 155-62.

14.

Havens

Member

Bezdek

Fellow

Leckie

Hall

, et al. Fuzzy c-Means Algorithms for Very Large Data. IEEE Transactions on Fuzzy Systems. 2012; 20(6): 1130-46.

15.

Xiang

Member

Tang

, et al. A Kernel Clustering Algorithm With Fuzzy Factor: Application to SAR Image Segmentation. IEEE Geoscience and Remote Sensing Letters. 2014; 11(7): 1290-4.

16.

Lei

Jia

Zhang

Meng

Nandi

. Significantly Fast and Robust Fuzzy C-Means Clustering Algorithm Based on Morphological Reconstruction and Membership Filtering. IEEE Trans Fuzzy Syst. 2018; 26: 1-15.

17.

Dong

Liu

Kong

Zeng

, et al. Power system bad load data detection based on an improved fuzzy C-means clustering algorithm. 2017 IEEE Power Energy Soc Gen Meet. 2017. pp. 1-5. Available from: http://ieeexplore.ieee.org/document/8274732/.

18.

Bonis

Oudot

. A fuzzy clustering algorithm for the mode-seeking framework. Pattern Recognit Lett. 2018; 102: 37-43. doi: 10.1016/j.patrec.2017.11.019.

19.

Zhang

Tao

Yang

. Fuzzy Logic-Based Clustering Algorithm for Multi-hop Wireless Sensor Networks. Procedia Comput Sci. 2018; 131: 1095-103. doi: 10.1016/j.procs.2018.04.270.

20.

Saad

Shalaby

Samy

Sayed

. Automatic arrival time detection for earthquakes based on Modified Laplacian of Gaussian filter. Comput Geosci. 2018; 113: 43-53.

21.

Karczmarek

Kiersztyn

Pedrycz

Czerwiński

. Fuzzy C-Means-based Isolation Forest. Appl Soft Comput. 2021; 106: 107354. doi: 10.1016/j.asoc.2021.

22.

Jiang

Cui

Zhang

. Kernel FCM based on simulated annealing algorithm. 2017 7th Int Work Comput Sci Eng WCSE 2017. 2017. pp. 110-5.

23.

Wang

Zhang

Xiao

. Kernel-based Fuzzy C-means Clustering Based on Fruit Fly Optimization Algorithm. 2017 International Conference on Grey Systems and Intelligent Services. 2017. pp. 251–6.

24.

. Adaptive Kernel-Based Fuzzy C-Means Clustering with Spatial Constraints for Image Segmentation. Int J Pattern Recognit Artif Intell. 2019; 33(1): 1-15.

25.

Zang

Wang

Jiang

Liu

. A Kernel-Based Intuitionistic Fuzzy C-Means Clustering Using Improved Multi-Objective Immune Algorithm. IEEE Access. 2019; 7: 84565-79.

26.

Giri

Majumder

. Eigenvalue-based cooperative spectrum sensing using kernel fuzzy c-means clustering. Digit Signal Process A Rev J. 2021; 111: 102996. doi: 10.1016/j.dsp.2021.

27.

Qiao

Yang

. An Improved Dolphin Swarm Algorithm Based on Kernel Fuzzy C-Means in the Application of Solving the Optimal Problems of Large-Scale Function. IEEE Access. 2020; 8: 2073-89.

28.

Bezdek

. Pattern Recognition with Fuzzy Objective Function Algorithms. Plenum Press. 1981.

29.

Bezdek

Ehrclich

William

. FCM: The fuzzy c-means clustering algorithm. Computers & Geosciences. 1984; 10(2): 191-203.

30.

Wang

Han

Huang

. Application of Fuzzy C-Means Clustering Algorithm Based on Particle Swarm Optimization in Computer Forensics. Phys Procedia. 2012; 24: 1186-91. doi: 10.1016/j.phpro.2012.02.177.

31.

Tzortzis

Likas

. The Global Kernel. 2008 Int Jt Conf Neural Networks (IJCNN 2008). 2008. pp. 1977-84.

32.

Cleuziou

Moreno

. Kernel methods for point symmetry-based clustering. Pattern Recognit. 2015; 48(9): 2812-30. doi: 10.1016/j.patcog.2015.03.013.

33.

Rousseeuw

. Silhouettes: A graphical aid to the interpretation and validation of cluster analysis. J Comput Appl Math. 1987; 20(C): 53-65.

34.

Davies

Bouldin

. A Cluster Separation Measure. IEEE Trans Pattern Anal Mach Intell. 1979; PAMI-1(2): 224-7.