Abstract
In order to solve the problem of noise sensitivity and low precision in image segmentation, a new image segmentation method based on super-pixel with neighborhood constrained and multi-level FCM clustering is proposed. Firstly, an improved SLIC algorithm of FCM is designed based on the local pixel region of the image to get the refined super-pixel. Then, the improved FCM clustering algorithm is combined with the spatial relationship of neighborhood super-pixel to mark the super-pixel categories. In order to verify the effectiveness of the algorithm, the artificial synthetic image with noise and natural image without noise are used for comparison experiments. The experimental result show the algorithm proposed in this paper can not only suppress the noise, but also improve the processing accuracy.
Introduction
Fuzzy C-Means clustering is a nonlinear programming process with constraints. The algorithm has been successfully applied in image processing [1, 2, 3, 4] pattern recognition [5, 6], data mining [7], control decision [8] and other fields. However, in the field of image processing, due to the inherent characteristics of image data, there are still many shortcomings when using FCM algorithm for image segmentation. That are as follows: Firstly, the pixels that make up the image are many and complex, and the pixel distribution is uneven, especially the color image of “a little red in all green trees”. Secondly, traditional FCM algorithm is applied to image segmentation, it ignores the influence of pixel position, so the segmentation effect is not good and the accuracy is not high. Thirdly, FCM algorithm is an iterative algorithm based on “hill climbing”, which seek all pixels in image segmentation, so the execution time of the algorithm will increase sharply with the increase of image size and resolution, especially in processing of massive color image, its performance of real-time is even worse.
In view of the fact that the FCM algorithm ignores the influence of pixel position and noise sensitivity, scholars have proposed a series of improved algorithms [3, 4, 9, 10, 11, 12, 13, 14, 15, 16, 17]. Aiming at the spatial relationship of pixels, FCM_S1 proposed by reference 9̧ and FCM_S2 proposed by reference [10] are the early improved FCM algorithms that introduce neighborhood spatial information into image segmentation, FCM_S1 is an algorithm about robust image segmentation using FCM with spatial constraints based on new kernel-induced distance measure, and FCM_S2 is an algorithm of fast and robust fuzzy c-means clustering algorithms incorporating local information for image segmentation. After that, reference [11] defines a robust C-means clustering algorithm based on local information, its control parameters of local information influence factors proposed in this algorithm are obtained entirely from the calculation of pixel information, and no one sets other parameters for it.
Aiming at the processing of image noise and outliers, Guo et al. proposes an algorithm of Local Noise Detecting Adaptive Fuzzy C-mean Clustering (NDFCM) [2] for local noise detecting, which is effective for noise processing, but the execution time is long and the efficiency of image segmentation is not high. Tao et al. [3] proposed an improved Super-pixel Fast FCM clustering algorithm (SFFCM), based on morphological gradient reconstruction, which improves the segmentation accuracy and efficiency of color images, but the segmentation accuracy of images with a lot of noise is lower than that of NDFCM. Gu et al. [14] proposed fuzzy double C-means clustering based on sparse self-representation model, its main feature is that can process two different dimensional data sets at the same time, and has good data classification and recognition ability, high robustness on noise and data adaptability, however, data feature extraction is complex and time-consuming for it.
Zhang et al. [4] proposed a Deviation Sparse Fuzzy C-Means clustering algorithm with Neighborhood information constraints (DSFCM_N), which suppresses noise by introducing deviation tolerance distance vector
In view of this, based on the comprehensive analysis of the latest research progress at home and abroad, and based on the theory of SLIC (Simple Linear Iterative Clustering) [21], a neighborhood multi-level FCM clustering method for super-pixel image segmentation is proposed in this paper. The first-level FCM clustering algorithm is used to improve the SLIC super-pixel segmentation technique to obtain fine super-pixels, and the second level improves FCM clustering based on the neighborhood super-pixel constraint relationship on the basis of the first level, thus the super-pixel category marking is realized. Through two-level FCM clustering, we can further improve the segmentation accuracy and suppress noise in image segmentation.
Super-pixel segmentation based on FCM improved SLIC algorithm
Super-pixel is the use of image pixel color and position and other information to segment the image into a polygonal region composed of multiple adjacent pixels. SLIC is a classical super-pixel segmentation algorithm based on K-Means clustering. It uses the color and position information of the image format of CIELab to measure the similarity of pixels, CIELAB is an image colour standard formulated by CIE in 1931 the L represents image grey level, a and b are two colour channels. See relevant literature for details. And then generates super-pixel blocks with compact pixel distribution and uniform size. In each iteration, the algorithm updates the clustering center with the average of all pixels in the neighborhood block. However, due to the existence of noise pixels in the neighbourhood, not all the pixels in the neighborhood block have high similarity with the clustering center, so the clustering center is unstable.
In addition, when the distance between the pixel and the clustering center is used to measure the similarity, it does not been taken into account the relative importance of color and position. But in fact, the similarity of color (grayscale) is more important in the image with rich details, for the image with uniform colour (grayscale) distribution, the similarity of pixel position is more important. So when the classical SLIC algorithm is used for super-pixel segmentation of images with different structures, it may produce wrong segmentation, including under-segmentation and over-segmentation. Based on this, this paper designs a SLIC super-pixel segmentation algorithm based on FCM, which achieves good results in suppressing noise, and the accuracy and efficiency of real time in image segmentation is greatly improved.
Algorithm description
Let a CIELab image have
Then the initial clustering center
Where
The color distance between pixel p and the cluster center
Where
It can be seen from the Eq. (4) that the closer colour of the pixel
Similarly, the distance between pixel p and the cluster center
The position similarity after normalization by Gaussian kernel function is:
Considering the importance of colour and position in the similarity calculation, this paper introduces the parameter
With the above definition, when using the FCM algorithm for super-pixel clustering segmentation, we only need to search the pixels in the 2
S1: set the number of
In Eq. (8),
The
The above SLIC iterative method based on FCM realizes the initial division of super-pixels. FCM is a fuzzy clustering algorithm, which assigns pixels to a certain class according to the principle of maximum membership degree. However, in the process of pixel classification, some pixels, such as boundary pixels, have little difference in membership degree to several classes, which will lead to miss-division when iterating many times, especially when the initial clustering center is not selected correctly. In addition, if the neighborhood block contains noise pixels, FCM clustering will also classify them into a certain category, therefore, using the above super-pixel segmentation method will result in under-segmentation (a super-pixel block contains a number of different categories of pixels) and over-segmentation (pixels of the same nature may be divided into different super-pixel blocks), which need to be refined, and the processing flow is shown in Fig. 1.
Super-pixel refinement processing method.
The analysis is as follows:
(1) Firstly, the super-pixel is divided by the improved SLIC method in this paper, and the initial super-pixel set of the image is obtained. Then, the standard deviation
In Eq. (10),
(2) After obtaining the standard deviation of the super-pixel, compared with the pre-given threshold
(3) After the super-pixel segmentation is completed, the standard deviation
The formula for calculating the standard deviation
In Eq. (11), NP
By analyzing the above steps, we can see that in the process of solving the problem of under- segmentation, there is a large difference between the noise pixel in the image and the clustering center of any super-pixel, and the noise will be “separated” from the initial super-pixel after many iterations. Similarly, in the process of solving the over-segmented merged super-pixel, the noise pixel is not merged, so this method not only eliminates the under-segmentation and over-segmentation in the primary super-pixel segmentation. And effectively separate the noise pixels in the image.
In the previous section, we proposed the technique of SLIC super-pixel partition that combines FCM iterative clustering, and obtained fine super-pixels in the image. Because the super-pixels only segment the image in the neighborhood, they cannot classify the objects in the image from the angle of the whole image. In addition, it is difficult to suppress the noise far away from the local area. FCM algorithm can also achieve super-pixel classification from a global perspective. Therefore, based on the combination of super-pixel and traditional FCM clustering, this paper proposes an improved super-pixel FCM clustering algorithm based on neighborhood constraints, which aims to achieve super-pixel category marked in the whole image and suppress the influence of noise.
Location constraint of neighborhood spatial
From the super-pixel division method in Section 2, we can see that the super-pixel in the image not only has its own inherent color or gray scale characteristics, but also related to its location. For example, the super-pixels around another super-pixel is likely to constitute the same object in the image, that is, they are the same kind of super-pixel. While the super-pixel block far away from another the super-pixel is less likely to belong to the same class. Therefore, in the super-pixel classification and marking of the image, we should consider not only the information of the super-pixel itself, but also the position relationship among the super-pixels.
Let
While
Compared with the FCM_S1 and FCM_S2 proposed in reference [10] and the FGFCM method proposed in reference [19], Eq. (12) fuses the information of the super-pixel itself and its spatial location information. Through the use of
In the improved FCM clustering (Section 3.2 will describe the FCM clustering after the introduction of factor
Using the super-pixel impact factor
Where
The iterative execution process of the algorithm is as follows:
S1: set clustering number
Algorithm analysis:
It can be seen from the above algorithm description, the robustness of the algorithm to noise is mainly reflected in the neighborhood constraint impact factor
It specifically includes two cases. Firstly, suppose that in a neighborhood, the central super-pixel
After repeated iterative calculation of Eq. (14), the membership degree
In order to verify the effectiveness of the algorithm proposed in this paper, we use the Super-pixel Fast FCM clustering algorithm (SFFCM) based on morphological gradient reconstruction proposed in reference [3], the improved FCM algorithm based on local spatial information FCM_S1 and FCM_S2 proposed in reference [9] and literature [10], and the Deviation Sparse Fuzzy C-means clustering algorithm under Neighborhood(DSFCM_N) information constraints proposed in reference [4]. Among them, FCM_S1 and FCM_S2 first introduced neighborhood spatial information, which is a classical clustering and segmentation algorithm based on neighborhood information. SFFCM algorithm is a new algorithm based on super-pixel processing published on IEEE Transaction on Fuzzy System, the algorithm has sufficient theory and good application effect. DSFCM_N algorithm proposes to identify noise and outliers by applying sparsity to the deviation between measured values and theoretical values under the constraint of neighborhood information and the clustering accuracy is improved, the design of the algorithm is novel, and it has a great correlation with the algorithm proposed. So it is feasible to use these algorithms for experimental comparison. For the convenience of description, the super-pixel image segmentation algorithm based on neighborhood multi-level FCM clustering proposed in this paper is referred to as SN_FCM.
Experimental environment: CPU Intel(R) Core™ i7-6700@3.4GHZ, RAM 16G, Operating system: Windows 10, Programming environment: Matlab 2012a.
The parameters setting in the experiment are: FCM clustering fuzzy weighted index
Experimental data: we use two kinds of image data for experiments, the first is the artificial synthetic image with 40% salt and pepper noise, the second is the natural colour image set without noise. Because the first data set is a synthetic image set with simple colour and a large amount of noise, and the second natural image data set is not only rich in colour but also free of noise, these two data sets reflect the composition of image data from different sides. Experiments are carried out on these two data sets to verify the processing of noisy images by the proposed method It can better illustrate the effectiveness of the method proposed in this paper. Experimental evaluation indicators: there are three evaluation indicators in this experiment, segmentation accuracy (SA), similarity score (CS) and execution timeSA represents the percentage between the correct pixel obtained after segmentation and the real correct pixel; CS represents the percentage of pixels obtained after segmentation and total classified pixels (including correct and wrong). SA and CS are defined as follows:
In the formula,
Super-pixel blocks of the synthetic color image.
The synthetic color image is shown in Fig. 2a. The size of the image is 135
Evaluation index of 5 algorithms for synthetic image
Evaluation index of 5 algorithms for synthetic image
Synthetic color image.
From Fig. 3, we can see that among the five algorithms, the segmentation effect of FCM_S1 is the worst, DSFCM_N and SN_FCM are the best, there are little difference between them, they are close to the original Fig. 2a, and SN_FCM is more prominent to the details of image reconstruction, and the boundary of each component object is smoother. At the same time, by analyzing the evaluation indexes in Table 1, it can be concluded that SN_FCM has the largest segmentation SA and CS, followed by DSFCM_N, and has little difference from the SN_FCM, and FCM_S1 is the worst. This is because although FCM_S1 has introduced the spatial influence adjustment parameter “a”, but the parameter value is given subjectively, and FCM_S2 has improved on the basis of FCM_S1, so the segmentation effect of FCM_S2 is better than that of FCM_S1. SFFCM combines super-pixel information and morphological techniques, so it has high segmentation accuracy in image segmentation.
However, in the morphology-based MMGR-WT algorithm, the minimum value of structural elements
In addition, in terms of execution time, DSFCM_N not only calculates the membership degree and clustering center, but also repeatedly calculates the deviation of the data
In order to further verify the effectiveness of the algorithm, this experiment selects the Pascal VOC 2012 natural image dataset, which is commonly used to test the classification in the field of artificial intelligence. It contains 4 categories and 20 kinds of natural and scene images without artificial noise, such as airplane, bicycle, bird, bus, bed, cat, chair, cow, dog, sheep, etc. At present, the capacity of the Pascal VOC 2012 dataset has exceeded 12000 images. And it is being updated and expanded every year. Pascal VOC 2012 dataset has become a standard dataset for measuring the performance of computer vision algorithm [21]. The scene images of some Pascal VOC 2012 dataset are shown in Fig. 4.
Pascal VOC 2012 dataset partial scene images.
Super pixel blocks of the image corresponding to Fig. 4.
In the experiment, we select seven images (cat, horse, plane, car, motorcycle, TV, living room) from the Pascal VOC 2012 dataset. In this experiment, considering that the color information of pixels is a little more important than the position information, so colour weight parameter
The evaluation index values of 5 algorithms in natural image
Image segmentation comparison of the SN_FCM algorithm proposed in this paper.
The analysis of the data in Table 2 shows that for the natural color image processing without noise, except for the algorithm FCM_S1, the other four algorithms have high accuracy and similarity. Although the SN_FCM has achieved better accuracy and similarity, but comparing with other algorithms, such as DSFCM_N, its value of SA and CS is not significantly improved, and even slightly lower than the DSFCM_N. It shows that this algorithm has no obvious advantage in dealing with images without noise, but compared with experiment 1, this algorithm has great advantage in image processing with noise, so as to further verify the robustness of the algorithm to noise. Similarly, in the comparison of the execution time of the five algorithms, the execution time of the DSFCM_N algorithm is much longer than that of the other four algorithms. The execution time of the SN_FCM is relatively long, but within the tolerable range, and much shorter than that of the DSFCM_N algorithm.
In this paper, an image super-pixel partition algorithm is given based on SLIC and FCM iterative clustering, and then a neighborhood constrained super-pixel FCM clustering impact factor is designed by combining the information of super-pixel itself and the spatial information of neighborhood super-pixel. Over this, an improved super-pixel FCM clustering algorithm based on neighborhood constraint is proposed. Theoretical analysis and experiments show that the algorithm proposed in this paper achieves good result in image clustering segmentation. Especially for the processing of images with a lot of noise, it has high accuracy and similarity.
The main contributions of this paper are as follows: one is to design a SLIC super-pixel fine partition method based on FCM, and the other is to propose a super-pixel FCM clustering algorithm based on neighborhood spatial impact factor
Footnotes
Acknowledgments
Yu Yang (1986.3-), Female, Master, Deputy senior engineer, Maoming Guangdong, Major in Big data technology and Application; Corresponding author: Zhicheng Wen(1972-), male, Dong’an county, Hunan province, Han, Ph.D., professor, China, E-mail: zcwen@mail.shu.edu.cn.
This paper supported by the Industrial Support Project of Xinyu Science and Technology Bureau, Jiangxi Province: Research on Reliability Control Technology for Train Communication Network Transmission System (2019).
