An Ultrasound Image Enhancement Method Using Neutrosophic Similarity Score

Abstract

Ultrasound images, having low contrast and noise, adversely impact in the detection of abnormalities. In view of this, an enhancement method is proposed in this work to reduce noise and improve contrast of ultrasound images. The proposed method is based on scaling with neutrosophic similarity score (NSS), where an image is represented in the neutrosophic domain through three membership subsets T, I, and F denoting the degree of truth, indeterminacy, and falseness, respectively. The NSS measures the belonging degree of pixel to the texture using multi-criteria that is based on intensity, local mean intensity and edge detection. Then, NSS is utilized to extract the enhanced coefficient and this enhanced coefficient is applied to scale the input image. This scaling reflects contrast improvement and denoising effect on ultrasound images. The performance of proposed enhancement method is evaluated on clinical ultrasound images, using both subjective and objective image quality measures. In subjective evaluation, with proposed method, overall best score of 4.3 was obtained and that was 44% higher than the score of original images. These results were also supported by objective measures. The results demonstrated that the proposed method outperformed the other methods in terms of mean brightness preservation, edge preservation, structural similarity, and human perception-based image quality assessment. Thus, the proposed method can be used in computer-aided diagnosis systems and to visually assist radiologists in their interactive-decision-making task.

Keywords

neutrosophic sets image enhancement neutrosophic similarity score ultrasound images liver

Introduction

Ultrasound imaging is one of the most prevalent diagnostic techniques due to its advantage of being non-invasive, portable and cost-effective. However, ultrasound images are affected by blurring due the presence of speckle, therefore it is hard for an observer to interpret the images and obtain quantitative information from them. Noise in ultrasound can be modeled as the combined effect of two components: one is additive, such as electronic and thermal noise, and the other is multiplicative, called “speckle.” Speckle is the result of the constructive and destructive coherent summation of ultrasound echoes when ultrasound pulses randomly interfere with objects of comparable size to the sound wavelength and then the superposition of acoustical echoes produces an intricate interference pattern.¹ Noise is considered as undesirable consequence of the image formation process in coherent imaging, directly impact the visualization of the ultrasound image by the physician, deteriorate the quality, and the perceivable resolution of diagnostically important features and thus lead to inaccuracy in clinical diagnosis. Therefore, it is essential to remove speckle in the ultrasound images without compromising important image details. The commonly used adaptive statistic filters based on multiplicative nature of speckle include Lee filter, Kaun filter, Wiener filter, frost filter, and adaptive median filters.^2
-6 Speckle noise is reduced by these adaptive filters by adjusting the size of the filtering window. However, image resolution gets decreased inside the window, which in turn blurs the edges. These low-pass filtering methods are simple and reduces speckle noise in homogeneous area; however, these methods display limitations on preserving sharp textures of images or reducing speckle noise in area close to the edges.²

The anisotropic diffusion (AD) method developed by Perona and Malik was based on heat equation and method performs better in homogeneous regions with preserving the edge information in images which are corrupted by additive noise but does not perform well for the multiplicative speckle noise.⁷ Yu et al. applied coefficient of variation to AD method and proposed speckle reduction anisotropic diffusion (SRAD) method.³ In this method, the instantaneous coefficient of variation manages the detection of edges in speckled images. The function exhibits high values at edges and produces low values in homogeneous regions. The method excels in terms of mean-preserving, edge-preserving and edge-enhancing over adaptive filter algorithms in speckled images. Although it makes fine details more visible, it still has limitation in retaining subtle features because of its smoothing and eradicating nature. The other anisotropic diffusion filter methods are oriented speckle reducing AD, smallest univalue segment assimilating nucleus-controlled AD and discrete topological derivative.^8
-10 Most of these methods perform well in the suppression of speckle noise but result in over-smoothening that is, important diagnostic details get lost. Mittal et al. proposed a modified SRAD (termed as MSRAD in this paper) by limiting speckle reduction and combining regularization method to preserve the fine-details of ultrasound liver images.¹¹ However, the selection of the parameters is subjective and image enhancement is highly dependent on the value of parameters. Although nonlocal means filters such as probabilistic patch-based filter, the optimized bayesian nonlocal means filter and Guo et al. have better speckle noise removing effect, the complexity of these filters is usually very high.^12
-14 Consequently, they cannot meet the real-time need of computer-aided system. The methods based on wavelets, shearlet, curvelets, bandelets, etc. are also used for de-speckling.^15
-18 Bedi et al. combined the MSRAD with the non-subsampled shearlet transform for despeckling the ultrasound images (termed as Shearlet-MSRAD in this paper).¹⁸ The method involves the decomposition of images using the shearlet transform. Then, shearlet transform thresholding function was applied on the detailed, that is, the high frequency coefficients and MSRAD was applied on the approximate, that is, the low frequency coefficients. This multidimensional and multidirectional method enhances the visual characteristics of the ultrasound images and remove speckle noise.¹⁸ These methods perform better on edge preservation; however, artifacts often occur. Chandra and Bajpai worked on a linear fractional mesh-free partial differential equation for enhancement of medical images.¹⁹ The proposed enhancement method was used to increase the accuracy in tumor segmentation.

The total variation regularization-based image restoration method was first presented by Rudin et al.²⁰ This method performs well in preserving edges and image sharpness while removing noise. Aubert and Aujol gave a non-convex model used for reducing speckle noise by utilizing a maximum a posteriori estimator.²¹ Koundal et al. proposed variation method based on Nakagami distribution that reduces the speckle noise in medical ultrasound images.²² Their method preserved the edge information well however the fine-details of texture were lost in processing. Additionally, these variational methods usually display staircase effect due to the loss of fine-details of texture in ultrasound images. These methods fail to preserve fine-details of texture because of the fuzziness caused by echo patterns in ultrasound images.

Singh et al. worked on ultrasound video segmentation and classification.²³ They reduced speckle noise using an algorithm based on a convolutional neural network. Then, enhanced texture features, contrast, resolvable details, and image structures in ultrasound video frames by using contrast-limited adaptive histogram equalization method. The contrast-limited adaptive histogram equalization method considered an automatic system for evaluating the grid size using entropy, and three target distribution functions (uniform, Rayleigh, and exponential), and interpolation techniques (B-spline, cubic, and Lanczos-3). Although the model showed preservation of texture, the complexity and time taken by model increased. Mustafa et al. carried out a comparative study on nine speckle reducing filters and five edge detection operators.²⁴ Authors found best results with linear neighborhood averaging and sobel edge detection.²⁴ It was observed that indeterminacy information in ultrasound images often gets overlooked in the traditional de-speckling methods. This problem is mostly solved by using, fuzzy sets.^25
-27 However, the fuzzy set can only handle the membership degree and fails to deal with non-membership and indeterminacy degree of pixels. Zhuang et al. enhanced ultrasound images by using fuzzy-based technique and then used these enhanced images for image segmentation.²⁸ Here, region of interest was highlighted so that it can help in segmentation whereas, fine details of image were compromised.

The Neutrosophic sets (NS) are generalization of fuzzy set and can deal with indeterminant information in images. Neutrosophic sets have been successfully applied into image de-noising applications.^29,30 The method proposed by Guo et al. was based on NS approach wherein entropy in NS domain was measured to estimate the indetermination.³¹ In order to remove the image noise and to reduce the sets indetermination, γ-median, a filtering operation was applied. Faraji and Xiaojun used NS-based technique to remove noise and enhance facial features.³² Qi et al. used pixel-wise adaptive neutrosophic filter based on neutrosophic indeterminacy to remove salt-pepper noise.³³ Guo and Şengür applied NS and directional α-mean operation for detection of edges.³⁴ The method removed the noise effect and detected the edges. Shahin et al. applied neutrosophic similarity score (NSS) scaling for enhancement of pathological images.³⁵ The NSS was computed under multi-criteria and then utilized to scale the input image. The similarity measurement employed to measure the similarity between two elements in NS under multi-criteria was given by Guo et al.³⁶ It was observed that the image quality and enhancement using the NSS were improved compared to traditional contrast enhancement methods.

Based on the reviewed literature, it is evident that image enhancement in ultrasound images is still an open area for further research. In this paper, a liver ultrasound image enhancement method based on NSS is proposed. A liver ultrasound image is represented in the NS domain, and NSS is defined. The NSS measure is employed to measure the belonging degree to the true texture and is utilized to extract the enhanced coefficient. The resultant coefficient is then finally applied to scale the input image to get an enhanced image. Experiments have been conducted on a four classes of clinical liver ultrasound images. The objective is to denoise and enhance the ultrasound image and preserve the texture details which have diagnostic importance. Both subjective and objective evaluators are used to evaluate and compare the proposed method’s performance.

The paper is organized as follows. Sections “Material and Method” and “Evaluation of the Enhancement Method” describes the material, proposed method, and evaluation measures. Section “Results and Discussions” provides the experimental results and comparisons, and in section “Conclusion” conclusions are drawn.

Material and Method

In this research, a database of liver ultrasound images was collected from the patients who underwent a medical examination at the Department of Radiodiagnosis, Manipal Hospital of Bangalore, India during the period March 2013 to August 2014. The study was approved by the medical research ethics committee of the hospital. Database of 189 B-mode ultrasound images were acquired from 94 different patients: 67 males (mean age: 50 years) and 27 females (mean age: 42 years) with an age range from 21 to 70 years over this time period. All images were obtained using the same ultrasound equipment (GE LOGIQ E9). The protocol, followed during image acquisition, includes (1) fasting for 4 to 5 hours prior to examination, (2) image capturing via longitudinal scan/subcostal transverse scan/intercostal scan depend on varied patient conditions, (3) imaging with 3 to 5 MHz broadband curvilinear probe with extended width beam, (4) finer adjustments of machine settings such as focus and time-gain-compensation was done according to body habitus, and (5) image acquisition of right lobe is through segments 7 and 8 or segments 5 and 6. The acquired image database comprises of 48 normal liver (NOR), 50 chronic liver (CHL), 50 cirrhosis (CIR), and 41 HCC evolved over cirrhosis.

Neutrosophic Image

Neutrosophy, a branch of philosophy, introduced by Smarandache³⁷ as a generalization of dialectics, which studies the neutralities’ origin, nature, and scope. It represents every entity ‹Y›, the opposite ‹Anti-Y›, and the neutralities ‹Neut-Y› that is neither ‹Y› nor ‹Anti-Y›.

An image is defined in the NS as: let U be a universe of discourse, B_p be a bright pixel set in U, and an image $f (i, j)$ defined by using NS is called neutrosophic image $f_{N S} (i, j)$ . The neutrosophic image $f_{N S} (i, j)$ is characterized by three membership sets T, I, F that is, true, indeterminacy and false set. A pixel P in the image is described as P (T, I, F) and belongs to Bp in the following way: it is t% true in the bright pixels set, i% indeterminate, and f% false, where t varies in T, i varies in I, and f varies in F. The membership values can be defined under different criteria. At the intensity image criterion c_in they are defined as shown in equations (1)-(3).

T_{C i n} (i, j) = \frac{f (i, j) - f_{m i n}}{f_{m a x} - f_{m i n}}

(1)

I_{C i n} (i, j) = 1 - \frac{G d_{i} (i, j) - G d_{i m i n}}{G d_{i m a x} - G d_{i m i n}}

(2)

F_{C i n} (i, j) = 1 - T_{C i n} (i, j)

(3)

where $f (i, j)$ and $G d_{i} (i, j)$ are the intensity value and gradient value at the position of $(i, j)$ on the image.

Neutrosophic Similarity Score

Neutrosophic similarity score is defined by Ye to measure the similarity degree between different elements, and has been applied widely in image processing due to its ability to describe the indeterminate information such as noises and vague edges in images.³⁸

A NS can be defined under different alternatives and criteria as: let A = {A₁; A₂; . . ; A_m} be a set of alternatives in neutrosophic set, and C = {C₁; C₂; . . .; C_n} be a set of criteria. The alternative A_m at C_n criterion is denoted as {T_Cn(A_m); I_Cn(A_m); F_Cn(A_m)}/A_m, where T_Cn(A_m), I_Cn(A_m), and F_Cn(A_m) are the membership values to the true, indeterminacy and false set at the C_n criterion.

A similarity measurement employed to evaluate the similarity degree between two elements in NS under multi-criterion is given in equation (4):

S_{C j} (A_{m}, A_{n}) = \frac{\begin{array}{l} T_{C j} (A_{m}) T_{C j} (A_{n}) + I_{C j} (A_{m}) I_{C j} (A_{n}) + \\ F_{C j} (A_{m}) F_{C j} (A_{n}) \end{array}}{\begin{array}{l} \sqrt{T_{C j}^{2} (A_{m}) + I_{C j}^{2} (A_{m}) + F_{C j}^{2} (A_{m})} \\ \sqrt{T_{C j}^{2} (A_{n}) + I_{C j}^{2} (A_{n}) + F_{C j}^{2} (A_{n})} \end{array}}

(4)

In multi-criteria environment, the concept of ideal element can be used to identify the best alternative. The ideal alternative $A^{*}$ is denoted as: { $T_{C j}^{*} (A_{i}), I_{C j}^{*} (A_{i}), F_{C j}^{*} (A_{i})} / A_{i}^{*}$ . The similarity to the ideal alternative is computed as given in equation (5):

S_{C j} (A_{i}, A^{*}) = \frac{\begin{array}{l} T_{C j} (A_{i}) T_{C j} (A^{*}) + I_{C j} (A_{i}) I_{C j} (A^{*}) \\ + F_{C j} (A_{i}) F_{C j} (A^{*}) \end{array}}{\begin{array}{l} \sqrt{T_{C j}^{2} (A_{i}) + I_{C j}^{2} (A_{i}) + F_{C j}^{2} (A_{i})} \\ \sqrt{T_{C j}^{2} (A^{*}) + I_{C j}^{2} (A^{*}) + F_{C j}^{2} (A^{*})} \end{array}}

(5)

The similarity score for a pixel P(i,j) is calculated to identity the degree to the ideal texture for a gray scale intensity image is given in equation (6)

S_{C j} (P (i, j), A^{*}) = \frac{\begin{array}{l} T_{C j} (i, j) T_{C j} (A^{*}) + I_{C j} (i, j) I_{C j} (A^{*}) \\ + F_{C j} (i, j) F_{C j} (A^{*}) \end{array}}{\begin{array}{l} \sqrt{T_{C j}^{2} (i, j) + I_{C j}^{2} (i, j) + F_{C j}^{2} (i, j)} \\ \sqrt{T_{C j}^{2} (A^{*}) + I_{C j}^{2} (A^{*}) + F_{C j}^{2} (A^{*})} \end{array}}

(6)

The similarity value is sensitive to noise on image. As it is used for image enhancement, the noisy regions on images can get labeled into a wrong group. In order to make the enhancing results robust to noise and enhance edges multi-criteria that are intensity criteria c_in, local mean intensity criterion c_mi, and edge detection criterion c_e, are considered to map the pixel to NS domain, and NSS is calculated under them.

The mean filter is the simple and intuitive method that has been used in digital image processing for reducing noise and smoothing of images. In this filtering method intensity value of each pixel is replaced with the mean intensity value of its neighboring pixels and itself. The enhancement is affected by the size of filter; thus, experiments are conducted with varying the filter size, that is, 3 × 3 and 5 × 5 size. Thereafter, the local mean intensity criterion c_mi can be defined by using pixel values of the resulting mean filtered image and transforming them into NS domain using the definitions given in equations (7) -(10).

T_{Cmi} (i, j) = \frac{f_{m} (i, j) - f_{m m i n}}{f_{m m a x} - f_{m m i n}}

(7)

f_{m} (i, j) = \frac{1}{w \times w} \sum_{l = i - w / 2}^{i + w / 2} \sum_{k = j - w / 2}^{j + w / 2} f (l, k)

(8)

I_{C m i} (i, j) = 1 - \frac{G d_{m} (i, j) - G d_{m m i n}}{G d_{m m a x} - G d_{m m i n}}

(9)

F_{C m i} (i, j) = 1 - T_{C m i} (i, j)

(10)

where $f_{m} (i, j)$ and $G d_{m} (i, j)$ are the intensity value and gradient magnitude value at the position of (i,j) on the image after mean filter processing. $f_{m m i n}$ and $f_{m m a x}$ are minimum and maximum intensity values, respectively. $G d_{m m i n}$ and $G d_{m m a x}$ are minimum and maximum of the gradient value in the image.

The edge detection is aimed at identifying points in an image at which the image brightness changes sharply. The edge information is extracted to evaluate the liver diseases and is utilized to define the enhancement criterion. Thus, to obtain the edge features, an edge operator is applied to the image and the edge image criterion is defined using the edge image values and transforming into neutrosophic set domain as described in equations (11) -(13).

T_{Ce} (i, j) = \frac{E (i, j) - E_{m i n}}{E_{m a x} - E_{m i n}}

(11)

I_{C e} (i, j) = 1 - \frac{G d_{e} (i, j) - G d_{e m i n}}{G d_{e m a x} - G d_{e m i n}}

(12)

F_{C e} (i, j) = 1 - T_{C e} (i, j)

(13)

where $E (i, j)$ is the edge value at (i,j) computed by using Sobel operator²⁴ because of its simplicity and high speed. $G d_{e} (i, j)$ is the gradient value on $E (i, j)$ . $E_{m i n}$ and $E_{m a x}$ are minimum and maximum edge values, respectively. $G d_{e m i n}$ and $G d_{e m a x}$ are minimum and maximum of the gradient value.

Proposed Method

In this section, a method based on neutrosophic is proposed to enhance ultrasound images by computing scale coefficient NSS. The algorithm improves contrast, remove noise and enhances edges of ultrasound liver images. The proposed algorithm consists of three main stages: transformation to neutrosophic sets domain, NSS calculation, and scaling.

Transformation to neutrosophic sets domain

The NS {T_Cn(A_i); I_Cn (A_i); F_Cn (A_i)} can be interpreted for specific pixel $P g (i, j) .$ has its own neutosophic sets at a specific criterion n. In the proposed method, NSS calculations are performed using alternatives and multi-criteria. The multi-criteria are the intensity criterion, local mean intensity criterion and edge detection criterion as described in sections “Neutrosophic Image” and “Neutrosophic Similarity Score.”

Neutrosophic similarity score calculations

A similarity value is calculated to identify the degree to the ideal object under multi-criteria:

S_{C i n} (P (i, j), A^{*}) = \frac{\begin{array}{l} T_{C i n} (i, j) T_{C i n} (A^{*}) \\ + I_{C i n} (i, j) I_{C i n} (A^{*}) \\ + F_{C i n} (i, j) F_{C i n} (A^{*}) \end{array}}{\begin{array}{l} \sqrt{T_{C i n}^{2} (i, j) + I_{C i n}^{2} (i, j) + F_{C i n}^{2} (i, j)} \\ \sqrt{T_{C i n}^{2} (A^{*}) + I_{C i n}^{2} (A^{*}) + F_{C i n}^{2} (A^{*})} \end{array}}

(14)

S_{C m i} (P (i, j), A^{*}) = \frac{\begin{array}{l} T_{C m i} (i, j) T_{C m i} (A^{*}) \\ + I_{C m i} (i, j) I_{C m i} (A^{*}) \\ + F_{C m i} (i, j) F_{C m i} (A^{*}) \end{array}}{\begin{array}{l} \sqrt{T_{C m i}^{2} (i, j) + I_{C m i}^{2} (i, j) + F_{C m i}^{2} (i, j)} \\ \sqrt{T_{C m i}^{2} (A^{*}) + I_{C m i}^{2} (A^{*}) + F_{C m i}^{2} (A^{*})} \end{array}}

(15)

S_{C e} (P (i, j), A^{*}) = \frac{\begin{array}{l} T_{C e} (i, j) T_{C e} (A^{*}) \\ + I_{C e} (i, j) I_{C e} (A^{*}) \\ + F_{C e} (i, j) F_{C e} (A^{*}) \end{array}}{\begin{array}{l} \sqrt{T_{C e}^{2} (i, j) + I_{C e}^{2} (i, j) + F_{C e}^{2} (i, j)} \\ \sqrt{T_{C e}^{2} (A^{*}) + I_{C e}^{2} (A^{*}) + F_{C e}^{2} (A^{*})} \end{array}}

(16)

The value of ideal alternative A* under specific criteria are: False set at intensity criteria, $S_{c i n}^{f} (i, j)$ that is, A*= {0,0,1}; True set at intensity criteria, $S_{c i n}^{t} (i, j)$ that is, A* = {1,0,0}; True set at local mean criteria, $S_{c m i}^{t} (i, j)$ that is, A* = {1,0,0}; True set at edge detection criteria, $S_{c e}^{t} (i, j)$ that is, A* = {1,0,0}. The false set overcomes the change in image brightness intensity. The true sets of all the criteria favors contrast and edges enhancement. In the proposed method, the average value of S_cin, S_cmi, and S_ce as the final NSS is defined as in equation (17):

\begin{array}{l} N S S (i, j) = [S_{c i n}^{f} (i, j) + S_{c i n}^{t} (i, j) \\ + S_{c m i}^{t} (i, j) + S_{c e}^{t} (i, j)] / 4 \end{array}

(17)

NSS scaling

Finally, the resultant NSS coefficient is utilized to scale the input image pixel $f (i, j)$ . The final enhanced image ( $f_{e n}$ ) is reconstructed as:

f_{e n} (i, j) = N S S (i, j) \times f (i, j)

(18)

This method denoises and enhances the contrast and edges of ultrasound image without producing any distortion. The advantage of this scaling is that it adjusts the brightness of image such that the texture features of ultrasound image are more noticeable.

Evaluation of the Enhancement Method

The experiments are taken on a platform with Intel core i5-8250U@1.60 GHz with 8 GB RAM and the method is implemented using the software of Matlab 2018a. In the present study, mean filter is implemented with filters of size 3 × 3 and 5 × 5. Hereafter proposed method with filter 3 × 3 and 5 × 5 will be termed as template-3 and template-5 respectively.

Evaluation of proposed method was done by two ways, that is, subjective and objective. The subjective metric for image quality assessment belong to the study of relationship between physical detectable change and human perception. In the present work, senior radiologist having more than 20 years of experience rated the image based on visual image quality. The other performance assessment criteria are objective metrics. The measure quantitatively assesses the speckle and additive noise reduction is peak signal-to-noise ratio (PSNR). Further the measures such as edge preservation index (EPI), universal image quality index (UIQI), and absolute mean brightness error (AMBE) give assessment about the edge preservation, mean brightness and similarity of the processed image in comparison to original one. Finally, the multi-scale structural similarity (MS-SSIM) measure consider the assessment of perceptual quality based on human visual system.

Subjective Evaluation

A dataset of 189 ultrasound liver images comprising of 48 NOR, 50 CHL, 50 CIR, and 41 HCC were processed with proposed method using template-3 and -5, and state-of-the-art method.³⁵ One hundred and eight-nine sets of images was provided to radiologist for subjective evaluation where each set contained four images, that is, one original image and three processed images. The three processed images are (1) enhanced image with state-of-art method proposed by Shahin et al.,³⁵ (2) enhanced image with proposed method using template-3 and, (3) enhanced image with proposed method using template-5. The criterion used by the radiologist for image quality assessment is to assign a visual grading score to an ultrasound image (either original or processed one) by estimating the availability of diagnostically important features as per the rubric shown in Table 1. These diagnostically important features are (1) blurriness, (2) textural contrast, (3) preservation of structural information, (4) visualization of resolvable details, and (5) usefulness for the better diagnosis. Absolute visual grading analysis is used to score the interpretation on above-mentioned features to assess image quality by the radiologist and it facilitates the quantification of subjective opinions within the images that are graded against each other. The expert radiologist was asked to score them in the range of 1 to 5. Here “one” was assigned to image having poor quality and “five” with excellent quality. The score of “three” was average score. The score of “two” and “four” was below and above average, respectively. The score marked for images was finally collected and score assigned to each image was tabulated.

Table 1.

Rubric for Subject Quality Assessment of Ultrasound Image.

Features		Excellent image quality: no limitations for clinical use (score 5)	Good image quality: minimal limitations for clinical use (score 4)	Sufficient image quality: moderate limitations for clinical use but no substantial loss of information (score 3)	Restricted image quality: relevant limitations for clinical use, clear loss of information (score 2)	Poor image quality: image not usable, loss of information, image must be repeated (score 1)
1.	Blurriness	All the five features are visible	All except one feature are exhibited.	At least 50% features are exhibited.	25% of the features are accommodated appropriately.	Only exhibits one or does not exhibit any of the features.
2.	Textural contrast
3.	Preservation of structural information
4.	Visualization of resolvable details
5.	Usefulness for the better diagnosis

Objective Evaluation

Numerical values were computed to quantify the quality of a processed image in comparison to its original (unprocessed) counterpart. The proposed method is evaluated in terms of (1) additive noise and speckle reduction, (2) mean brightness preservation, (3) edge preservation, (4) similarity between original and processed images, and (5) human perception-based image quality assessment. The detail of objective performance measures is mentioned as follows:

Peak signal-to-noise ratio

The peak signal to noise ratio measures how closely the processed image resembles to the original image.²³ It is the ratio of the maximum possible intensity value in image to the distortion between the original and processed image. It is defined by the following equation:

P S N R = - 10 l o g_{10} [\frac{\sum_{i = 0}^{i = M - 1} ({(f (i, j) - f_{e n} (i, j))}^{2}}{M \times N \times 255^{2}}]

(19)

Where $f (i, j)$ and $f e n (i, j)$ represent intensity of pixels $(i, j)$ in original and enhanced image.

Absolute mean brightness error

Absolute mean brightness error is mostly used to evaluate brightness preservation in processed image after image enhancement.³⁵ It is defined as:

ABME; = | \bar{f} - \bar{f_{e m}} |

(20)

where $\bar{f}$ is mean values of the original image and $\bar{f_{e m}}$ is mean values of the processed image. The value of error should be minimum for high quality images.

Edge preservation index

Edge preservation index in the processed image is a pointer of maintaining the hidden texture information during noise removal process.¹¹ This performance measure should be close to unity for an optimal effect of edge preservation. It is defined by the following equation:

EPI = \frac{E P (Δ f - \bar{Δ f}, Δ f_{e n} - \bar{Δ f_{e n}})}{\sqrt{\begin{array}{l} E P (Δ f - \bar{Δ f}, Δ f - \bar{Δ f}) . \\ E P (Δ f_{e n} - \bar{Δ f_{e n}}, Δ f_{e n} - \bar{Δ f_{e n}}) \end{array}}}

(21)

In the above equation (18), $Δ f$ and $Δ f_{e n}$ represent the high-pass filtered version of $f$ and $f_{e n}$ respectively, calculated with a 3 × 3 pixel standard approximation of Laplacian operator. $\bar{Δ f}$ and $\bar{Δ f_{e n}}$ are mean intensity value of $Δ f$ and $Δ f_{e n}$ respectively. EP is defined as EP(s₁,s₂) = $\sum_{i = 1}^{M} \sum_{j = 1}^{N} s 1 (i, j) . s 2 (i, j)$ .

Universal image quality index

Universal image quality index models any distortion as a combination of three different factors: loss of correlation, luminance distortion, and contrast distortion.³⁰ It is expressed as:

UIQI = \frac{σ_{f f_{e n}}}{σ_{f} σ_{f_{e n}}} . \frac{2 \bar{f} \bar{f_{e n}}}{{(\bar{f})}^{2} + {(\bar{f_{e n}})}^{2}} . \frac{2 σ_{f} σ_{f_{e n}}}{σ_{f}^{2} + σ_{f_{e n}}^{2}}

(22)

where $\bar{f}$ and $\bar{f_{e n}}$ , and $σ_{f}$ and $σ_{f_{e n}}$ represent the mean and standard deviation of the original and processed image. $σ_{f f_{e n}}$ represents the covariance. The first component measures the linear correlation (similarity) between two images and its range is [−1,1]. The second component measures how close the mean luminance is between images and its range is [0,1]. The third component measures how similar the global contrast is of two images and its range is also [0,1]. If the final value of UIQI is high then there is less distortion or high structural similarity in between the original and processed image.

Multi-scale structural similarity

The structural similarity index (SSIM) is a method for measuring the similarity between two images. The SSIM index is a full reference metric, in other words, measurement of image quality based on the original image as reference. The SSIM method is a single-scale approach. Whereas, the detail perception depends on other factors such as the resolution of image and the observer-to-image distance. Wang et al. incorporated observer viewing distances and developed MS-SSIM index. MS-SSIM simulates different spatial resolutions by iterative down-sampling and weighting the different values of each component of SSIM (luminance, contrast, and structure) at different scales. This index has been proved to be more accurate than SSIM for certain conditions.³⁹

In MS-SSIM the original image as Scale S₁ and the highest scale as Scale S_d, which is obtained after d-1 iterations. At the S_p^th scale, the contrast and structure comparison are calculated and denoted as c_p(i,j) and s_p(i,j), respectively. The comparison of luminance is computed only at Scale S_d and is denoted as l_d(i,j). The overall SSIM evaluation is obtained by combining the measurement at different scales using

\begin{array}{l} SSIM (i, j) = [l_{d} (f, f_{e n})]^{α_{d}} . \\ \prod_{p = 1}^{d} [c_{p} (f, f_{e n})]^{β_{p}} [s_{p} (f, f_{e n})]^{γ_{p}} \end{array}

(23)

l (f, f_{e n}) = \frac{2 μ_{f} μ_{f_{e n}} + C o_{1}}{μ_{f}^{2} + μ_{f_{e n}}^{2} + C o_{1}}

(24)

c (f, f_{e n}) = \frac{2 σ_{f} σ_{f_{e n}} + C o_{2}}{σ_{f}^{2} + σ_{f_{e n}}^{2} + C o_{2}}

(25)

s (f, f_{e n}) = \frac{σ_{f f_{e n}} + C o_{3}}{σ_{f} σ_{f_{e n}} + C o_{3}}

(26)

where C_o₁, C_o₂, and C_o₃ are small constants given by C_o₁ = (K₁ L)²; C_o₂ = (K₂ L)², and Co₃ = C_o₂/2, respectively. L is dynamic range of pixel values (L = 255), K₁ = 0.01, and K₂ = 0.03. The $μ_{f}$ , $σ_{f}^{2}$ , and $σ_{f f_{e n}}$ are the mean of $f$ , the variance of $f$ and the covariance of $f$ and $f_{e n}$ , respectively. The exponents in eq 21, $α_{m},$ $β_{p}$ , and $γ_{p}$ are used to adjust the relative importance of different components. Here $α_{m} = β_{p} = γ_{p}$ for all p’s and d = 5.

Results and Discussions

The proposed method with template-3 and -5 is verified on clinically acquired liver ultrasound image database. The performance of methods is evaluated by using both subjective and objective measures. The mean scores of subjective evaluations by the expert radiologist are given in Table 2. The total score is summation of the scores given by the radiologist on each ultrasound image. The scores of original images are lowest for all liver classes. The template-3 performs well in all liver classes except in enhancement of HCC images in which template-5 is better. The visual score for NOR, CHL and CIR with template-5 and Shahin et al.³⁵ methods are almost same that is 3.8 and 3.9, respectively. Overall score of subjective evaluation with template-3 and template-5 were obtained as 4.3 and 3.8, respectively and that were 44% and 27% higher than the score of original images. Whereas, the overall best score of 4.3 was obtained with template-3. The expert radiologist has recommended enhancement with template-3 as the diagnostic features were more noticeable as compared to original images.

Table 2.

Scores of Subjective Evaluation.

Image category (total number of images)	The average score per ultrasound image (total score)
Image category (total number of images)	Original image	Proposed template-3	Proposed template-5	Shahin et al.³⁵
NOR (48)	2.87 (138)	4.8 (216)	4.1 (198)	4.1 (200)
CHL (50)	2.92 (147)	4.5 (225)	3.7 (186)	4.0 (203)
CIR (50)	3.16 (156)	4.3 (218)	3.8 (194)	3.8 (192)
HCC (41)	3.01 (126)	3.9 (160)	4.0 (164)	3.6 (148)
Total (189)	3.0 (567)	4.3 (819)	3.8 (742)	3.9 (743)

NOR = normal; CHL = chronic, CIR = cirrhosis; HCC = hepatocellular carcinoma.

Figure 1 shows original and processed images of CHL and HCC. The images of CHL and HCC are marked as 1 and 2. The ROI is marked in a rectangular region and termed as A1and A2 of original image. Similarly, B1 and B2, C1 and C2, and D1 and D2 are marking for images processed by template-3, template-5 and Shahin et al.³⁵ method. It is observed that the texture information in original image are not quite visible as compared to processed images by template-3 and template-5. In case of HCC and background texture, the contrast is more noticeable in template-5 as compared to template-3. This can be the reason that HCC got more score in subjective evaluation. However, template-3 provided appropriate overall visual appearance in comparison to original images with enhanced texture contrast and no blurring effect whereas with template-5 there is more averaging of intensity which results in smoothening of features inside the HCC lesion. All these visual representations of original and processed images can be observed in the marked region of Figure 1. The clarity of improvements in Figure 1 may be less but the improvements were clearly seen when they were visualized on computer screen with black background.

Figure 1.

Original and processed images by proposed methods and Shahin et al. method³⁵. 1: Chronic liver; 2: HCC. (A) Original image. (B) Processed by template-3. (C) Processed by template-5. (D) Processed by Shahin et al.³⁵

The subjective results are supported by objective measures. Table 3 summarizes the results of objective evaluation by the proposed method using template-3 and template-5 and the method of Shahin et al.³⁵ in terms of mean and standard deviation of measured values to reflect the results on 189 ultrasound images. The mean value of PSNR with the proposed method is higher than that of Shahin et al.³⁵ It clearly indicates that the speckle reduction is higher with the proposed method either using template-3 or template-5. The value of AMBE obtained by the proposed method is comparative smaller than that of Shahin et al.³⁵ and it is 0.0304 with template-3 and 0.0326 with template-5. Thus, the proposed method with template-3 shows the best preservation of mean brightness in the processed images. The mean value of EPI of proposed method with template-3 is 0.9835 which is the highest and shows efficient preservation of edges in processed image. Universal image quality index measures the structural distortion in between original (unprocessed) and enhanced (processed) images and gives a fair idea about the perceived variations in structural information due to processing. The value of UIQI is close to 1 that is 0.9777 with template-3 which is much higher than that of Shahin et al.³⁵ and depicts higher structural similarity of processed image with the original image. Further, MS-SSIM, that generalizes UIQI, is highly adaptable for extracting structural information by modeling human visual perception that includes the ideas like changes in image due to processing tend to be less visible in (1) bright regions and (2) areas of significant activity, that is, texture, objects in the image. In brief, it defines similarity in structural information independent of average luminance and contrast. The mean value of MS-SSIM with both template-3 and -5 is 0.9996 that is higher than that of Shahin et al.³⁵ The variance around mean is lesser with template-3 than that of template-5 and it reflects that the proposed method with template-3 have comparatively better perceptual visualization of details than that of template-5. It can be observed from Table 3 that the values of objective measures have best performances with template-3. These results support the finding of subjective evaluation and reflects that the template-3 have better candidacy with proposed method than that of template-5. The proposed method with template-3 is further compared with SRAD³ and MSRAD¹¹ and recent enhancement techniques, that is, Shearlet-MSRAD¹⁸ and Shahin et al.³⁵ Figure 2 shows clinically acquired ultrasound image and processed images for visual illustration with compared enhancement methods. It can be observed that processed image by SRAD method have fading textural information. MSRAD as well as Shearlet-MSRAD methods preserve textural details by limited smoothing for noise filtering and improvement of the contrast of the image. The method of Shahin et al.³⁵ enhances the contrast much more effectively than the other methods. It can be seen clearly that the proposed method further improves the visualization of image textural quality in terms of reduced blurring and better contrast. Furthermore, Figure 3 presents the graphical illustration of all objective performance metrics with proposed method, SRAD,³ MSRAD,¹¹ Shearlet-MSRAD,¹⁸ and Shahin et al.³⁵ The PSNR values of proposed method in comparison with other methods show that a limited reduction in speckle. That is supported by the fact that speckle removal has been done up to an extent to reduce blurring not the textural details. Textural details are important for diagnosis. Further, the mean value of AMBE with proposed NSS method is minimum that indicate the best preservation of overall brightness of image after enhancement. The highest mean value of EPI with less variation around mean reflects that the best preservation of textural information with limited speckle removal. Also, the measures like UIQI and MS-SSIM reflect higher values with the proposed method in comparison to other methods and are close to unity and therefore it implies that the enhanced images with proposed method and their original counterparts are structurally similar and maintain perceptual image quality. In conclusion, all the results significantly highlight that the proposed method outperforms the other methods in terms of reduction of blurring due to speckle, mean brightness preservation, edge preservation, structural similarity, and perceived image quality.

Table 3.

Results of Objective Performance on 189 Ultrasound Images by Three Enhancement Methods.

Objective measures	Template-3 (mean ± SD)	Template-5 (mean ± SD)	Shahin et al.³⁵ (mean ± SD)
PSNR	33.18 ± 1.80 × 10^-1	34.18 ± 1.80 × 10^-1	27.93 ± 3.5 × 10^-1
AMBE	0.0304 ± 8.71 × 10^-3	0.0326 ± 9.87 × 10^-3	0.0401 ± 1.53 × 10^-2
EPI	0.9835 ± 4 × 10^-3	0.9687 ± 2.1 × 10^-3	0.9722 ± 1.4 × 10^-3
UIQI	0.9779 ± 1.08 × 10^-2	0.9757 ± 1.17 × 10^-2	0.9053 ± 3.18 × 10^-2
MS-SSIM	0.9996 ± 1.42 × 10^-4	0.9996 ± 1.63 × 10^-4	0.9994 ± 2.64 × 10^-4

Figure 2.

Processed and zoomed image of liver. 1: Original image; 2: Processed by SRAD³; 3: Processed by MSRAD¹¹; 4: Processed by Shearlet-MSRAD¹⁸; 5: Processed by Shahin et al.³⁵; 6: Processed by proposed method.

Figure 3.

Boxplots of (A) PSNR, (B) AMBE, (C) EPI, (D) UIQI, and (E) MS-SSIM making comparison with state-of-the-art image enhancement methods.

Finally, Figure 4 provides a straight comparison in between original and enhanced images by proposed method with template-3 for four different classes of liver image. These images represent the class of normal liver, chronic liver, cirrhosis liver, and liver with HCC as A1, B1, C1, and D1, respectively in original (unprocessed) form and A2, B2, C2, and D2 in enhanced form by the proposed template-3 method. The diagnostically important regions are marked in the rectangular box. It can be seen clearly on comparing these images that the proposed method provides clear visualization of resolvable details and improved textural contrast with the preservation of structural information. Thus, the proposed enhancement method shows its usefulness for the better diagnosis.

Figure 4.

Original and enhanced images (by the proposed method using template-3) for four different liver image classes.

Conclusion

The proposed enhancement method based on NSS scaling include (1) intensity, (2) local mean intensity and (3) edge detection criteria. The intensity criterion is used for sharpness, local mean intensity is for noise removal or speckle reduction and edge detection is for enhancement of edges. Then NSS with multicriteria is utilized to extract the enhanced coefficient and this enhanced coefficient is applied to scale the input image. In the proposed method (1) true set is taken to consider the belongingness of pixel to image sharpness, texture and edges and (2) false set to overcome the changes in image brightness. Two templates of filter window, that is template-3 and template-5 are assessed in the present work. The subjective and objective analysis, together, clearly demonstrate that the proposed enhancement method with template-3 shows impressive results by effectively reducing burring due to speckle and enhancing the diagnostically important features of ultrasound images.

In summary, the proposed enhancement method is designed in such a way that it reduces noise and blurring introduced by the speckle up to the extent that texture of ultrasound image should not be impacted and furthermore the fine textural details and other objects are made prominent by introducing contrast enhancement. Thus, enhancing the image addresses here the blurring issue due to speckle, and contrast improvement for easy visualization of textural details which in turn make texture features more noticeable in ultrasound image and help radiologists in precise diagnosis.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Puja Bharti

Deepti Mittal

References

Thijssen

. Ultrasonic speckle formation, analysis and processing applied to tissue characterization. Pattern Recognit Lett. 2003;24(4-5):659-75.

Shrimali

Anand

Kumar

. Comparing the performance of ultrasonic liver image enhancement techniques: a preference study. IETE J Res. 2010;56(1):4-10.

Acton

. Speckle reducing anisotropic diffusion. IEEE Trans Image Process. 2002;11(11):1260-70.

Gupta

Kaur

Chauhan

Saxena

. A versatile technique for visual enhancement of medical ultrasound images. Digit Signal Process. 2007;17(3):542-60.

Poddar

Tewary

Sharma

Karar

Ghosh

Pal

. Non-parametric modified histogram equalisation for contrast enhancement. IET Image Process. 2013;7(7):641-52.

Bharti

Mittal

Ananthasivan

. Computer-aided characterization and diagnosis of diffuse liver diseases based on ultrasound imaging: a review. Ultrason Imaging. 2016;39(1):33-61.

Perona

Malik

. Scale-space and edge detection using anisotropic diffusion. IEEE Trans Pattern Anal Mach Intell. 1990;12(7):629-39.

Tan

Wang

. Ultrasound speckle reduction by a SUSAN-controlled anisotropic diffusion method. Pattern Recognit. 2010;43(9):3083-92.

Damodaran

Ramamurthy

Velusamy

Kanakaraj Manickam

. Speckle noise reduction in ultrasound biomedical B-scan images using discrete topological derivative. Ultrasound Med Biol. 2012;38(2):276-86.

10.

Krissian

Westin

C-F

Kikinis

Vosburgh

. Oriented speckle reducing anisotropic diffusion. IEEE Trans Image Process. 2007;16(5):1412-24.

11.

Mittal

Kumar

Saxena

Khandelwal

Kalra

. Enhancement of the ultrasound images by modified anisotropic diffusion method. Med Biol Eng Comput. 2010;48(12):1281-91.

12.

Guo

Wang

Hou

. Speckle filtering of ultrasonic images using a modified non local-based algorithm. Biomed Signal Process Control. 2011;6(2):129-38.

13.

Coupe

Hellier

Kervrann

Barillot

. Nonlocal means-based speckle filtering for ultrasound images. IEEE Trans Image Process. 2009;18(10):2221-9.

14.

Deledalle

Denis

Tupin

. Iterative weighted maximum likelihood denoising with probabilistic patch-based weights. IEEE Trans Image Process. 2009;18(12):2661-72.

15.

Gleich

Kseneman

Datcu

. Despeckling of TerraSAR-X data using second-generation wavelets. IEEE Geosci Remote Sens Lett. 2010;7(1):68-72.

16.

Hou

Zhang

Feng

. SAR image despeckling based on nonsubsampled shearlet transform. IEEE J Sel Top Appl Earth Obs Remote Sens. 2012;5(3):809-23.

17.

Bhutada

Anand

Saxena

. Edge preserved image enhancement using adaptive fusion of images denoised by wavelet and curvelet transform. Digit Signal Process A Rev J. 2011;21(1):118-30.

18.

Bedi

Sunkaria

Mittal

. Enhancement of ultrasound images using modified anisotropic diffusion model in non-subsampled shearlet domain. In: Proceeding - IEEE International Conference on Computing, Communication and Automation, ICCCA 2017, pp. 1119-24 (Institute of Electrical and Electronics Engineers Inc., 2017).

19.

Chandra

Bajpai

. Fractional mesh-free linear diffusion method for image enhancement and segmentation for automatic tumor classification. Biomed Signal Process Control. 2020;58:101841.

20.

Rudin

Lions

P-L

Osher

. Multiplicative denoising and deblurring: theory and algorithms. In: Geometric Level Set Methods in Imaging, Vision, and Graphics. New York: Springer-Verlag; 2003, pp. 103-19.

21.

Aubert

Aujol

J-F

. A variational approach to removing multiplicative noise. SIAM J Appl Math. 2008;68(4):925-46.

22.

Koundal

Gupta

Singh

. Nakagami-based total variation method for speckle reduction in thyroid ultrasound images. Proc Inst Mech Eng H. 2016;230(2):97-110.

23.

Singh

Mukundan

De Ryke

. Feature enhancement in medical ultrasound videos using contrast-limited adaptive histogram equalization. J Digit Imaging. 2020;33(1):273-85.

24.

Mustafa

Abrahim

Omara

Mohammed

Hassan

Mustafa

. Reduction of speckle noise and image enhancement in ultrasound image using filtering technique and edge detection. J Clin Eng. 2020;45(1):51-65.

25.

Keerthivasan

Babu

Sudha

. Speckle noise reduction in ultrasound images using fuzzy logic based on histogram and directional differences. In: 2013 International Conference on Communication and Signal Processing, pp. 499-503 (IEEE, 2013).

26.

Bayani

Langarizadeh

Radmard

Nejad

. Quality improvement of liver ultrasound images using fuzzy techniques. Acta Inform Med. 2016;24(6):380-4.

27.

Guo

Cheng

Huang

Tian

Zhao

Sun

, et al. Breast ultrasound image enhancement using fuzzy logic. Ultrasound Med Biol. 2006;32(2):237-47.

28.

Zhuang

Lei

Joseph Raj

Qiu

. Application of fractal theory and fuzzy enhancement in ultrasound image segmentation. Med Biol Eng Comput. 2019;57(3):623-632.

29.

Koundal

Gupta

Singh

. Neutrosophic based nakagami total variation method for speckle suppression in thyroid ultrasound images. IRBM. 2017;39(1):43-3.

30.

Singh

Koundal

Gupta

. Speckle reduction method for thyroid ultrasound images in neutrosophic domain. IET Image Process. 2016;10(2):167-75.

31.

Guo

Cheng

Zhang

. A new neutrosophic approach to image denoising. New Math Nat Comput. 2009;5(3):653-62.

32.

Faraji

. An effective neutrosophic set-based preprocessing method for face recognition. In: 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), pp. 1–4 (IEEE, 2013).

33.

Liu

. A neutrosophic filter for high-density Salt and Pepper noise based on pixel-wise adaptive smoothing parameter. J Vis Commun Image Represent. 2016;36:1-10.

34.

Guo

Şengür

. A novel image edge detection algorithm based on neutrosophic set. Comput Electr Eng. 2014;40(8):3-25.

35.

Shahin

Amin

Sharawi

Guo

. A novel enhancement technique for pathological microscopic image using neutrosophic similarity score scaling. Optik (Stuttg). 2018;161:84-97.

36.

Guo

Şengür

. A novel image thresholding algorithm based on neutrosophic similarity score. Measurement. 2014;58:175-86.

37.

Smarandache

. Neutrosophic logic - a generalization of the intuitionistic fuzzy logic. SSRN Electron J. 2016. doi:10.2139/ssrn.2721587.

38.

. Multicriteria decision-making method using the correlation coefficient under single-valued neutrosophic environment. Int J Gen Syst. 2013;42(4):386-94.

39.

Singh

Shree

. A new homomorphic and method noise thresholding based despeckling of SAR image using anisotropic diffusion. J King Saud Univ - Comput Inf Sci. 2020;32(1):137-48. doi:10.1016/j.jksuci.2017.06.006.