BPENet: Boundary perception enhancement network for retinal vessel and coronary angiogram segmentation

Abstract

Vessel segmentation is crucial for assisting in the diagnosis and treatment of a range of diseases, such as retinal diseases and coronary artery diseases. However, current vessel segmentation methods often face the problem of poor vessel boundary segmentation, making the vessel segmentation task still challenging. This study aims to alleviate this problem, thereby improving the vessel segmentation results. By introducing a boundary detection auxiliary task to the vessel segmentation main task, we propose a boundary perception enhancement network (BPENet) for retinal vessel and coronary angiogram segmentation. Among them, to enhance vessel feature extraction capability, BPENet introduces the feature extraction enhancement (FE2) module. To enhance vessel boundary perception capability, BPENet introduces the boundary enhancement (BE) module. In addition, to fully leverage feature information from different layers, BPENet introduces the deep feature aggregation (DFA) module. Experimental results on retinal vessel datasets (DRIVE, CHASE-DB1, and STARE) and coronary angiogram datasets (DCA1 and XCAD) show that BPENet outperforms the existing mainstream segmentation methods. BPENet is expected to provide a reliable vessel segmentation result for doctors during the diagnosis and treatment of related diseases.

Keywords

retinal vessel segmentation coronary angiogram segmentation boundary detection feature extraction enhancement boundary enhancement deep feature aggregation

Introduction

The vessel system plays a crucial role in nourishing other organs and tissues of the human body.¹ Among them, retinal vessels are the only vessels of the human body that can be directly observed by non-invasive methods. Its morphological characteristics (e.g., branching angle, branching pattern, length, tortuosity, and width) are utilized to diagnose, screen, treat, and evaluate a wide range of ophthalmologic diseases (e.g., glaucoma and diabetic retinopathy, etc.).² Coronary arteries are vessels that supply blood to the heart. Its morphological characteristics (e.g., branching angle, diameter, length, and tortuosity) can reveal initial coronary artery disease symptoms.^3,4 In traditional diagnosis and treatment processes, vessel segmentation is mainly performed manually by experienced professional doctors.^5,6 Due to the complexity of vessel structures, the professional level of the doctors also affects the segmentation results. Therefore, manual segmentation of vessels is a very challenging task. Automated computer segmentation technology can significantly optimize medical processes, not only saving valuable human resources but also helping doctors quickly and accurately complete disease screening, improving overall medical efficiency and quality. Therefore, researching and designing an accurate automated vessel segmentation approach is crucial for assisting doctors in diagnosing and treating related diseases, including retinal and coronary artery diseases.

In recent years, deep learning (DL) has been widely applied to various medical image processing tasks, including breast abnormality recognition,⁷ skin lesion recognition,⁸ liver tumor segmentation,⁹ Alzheimer's disease diagnosis,¹⁰ and COVID-19 lesion segmentation.¹¹ Convolutional neural networks (CNN) have become one of the mainstream architectures in DL due to their outstanding capabilities in image feature extraction. Among them, FCN¹² has notable contributions to semantic segmentation. It pioneered a CNN-based pixel-level prediction method, providing a basic framework for subsequent semantic segmentation methods. Then emerged U-Net,¹³ which uses an encoder-decoder architecture with skip connections and employs traditional square convolutions for feature extraction, achieving significant success in medical image segmentation. Inspired by U-Net, many researchers have improved upon it, thus proposing various models for vessel segmentation. In order to decrease parametric quantities in the original U-Net, M2U-Net¹⁴ mainly employed depthwise separable convolutions for feature extraction. In order to expand the receptive field, AMF-Net¹⁵ mainly employed atrous convolutions for feature extraction. As one of the core parts of a vessel segmentation model, the feature extraction method is crucial for a model. In this paper, the feature extraction enhancement (FE2) module, which mainly comprises four-angle strip convolution (FSC) and adaptive channel selection (ACS), is employed to enhance the model's vessel feature extraction capability, thereby improving the model segmentation performance.

Multi-task learning can enhance model performance through mutual promotion, so this approach is also employed in vessel segmentation studies. Ma et al.¹⁶ designed a network capable of concurrently performing artery/vein classification and vessel segmentation tasks. This network introduced a multi-task output block incorporating the spatial activation mechanism, which leverages the outcome of the relatively easy vessel segmentation task to boost the outcome of the artery/vein classification task. DMAN¹⁷ was a network that integrated quantification and segmentation tasks for main coronary vessels. Among them, quantification can serve as a global restraint to enhance segmentation performance, while segmentation can indicate stenosis lesion locations to gain better quantification accuracy. Furthermore, in the field of building extraction from remote sensing images, some studies have treated boundary detection as one of the tasks in multi-task learning to improve boundary extraction effects. For example, Xu et al.¹⁸ adopted multi-task learning strategies to capture more boundary information by introducing the additional boundary detection decoder. BFL-Net¹⁹ also introduced the additional boundary detection decoder, differing from Xu et al.¹⁸ in that BFL-Net fused the outputs of different decoder branches. Specifically, it concatenated the output feature maps from the segmentation and boundary detection decoder branches and then applied convolution operations to obtain the building extraction results. However, few researchers focus on the problem of poor boundary segmentation in existing vessel segmentation methods. Given the advantages of multi-task learning and the use of boundary detection as an auxiliary task in other fields, this paper introduces a boundary detection auxiliary task to the vessel segmentation main task. In this way, the vessel boundary detection auxiliary task's powerful extraction ability for vessel boundary information can be utilized to enhance the vessel segmentation main task's extraction ability for vessel boundary information, thus alleviating the problem of poor vessel boundary segmentation. In medical image segmentation, attention mechanisms are extensively applied because of their ability to focus on important information. In order to increase sensitivity to information features, DEU-Net²⁰ introduced the attention skip module with channel attention in U-Net's skip connections. In order to strengthen retinal vessel features in spatial features and highlight salient features transmitted through skip connections, AAU-net²¹ introduced gate attention in U-Net's skip connections. Through existing studies, it is seen that the introduction of attention mechanisms in skip connections can improve the model's performance. Given this, this paper innovatively introduces the boundary enhancement (BE) module with boundary attention into the skip connections, which can also alleviate the problem of poor vessel boundary segmentation. Furthermore, unlike BFL-Net,¹⁹ the BE module doesn’t directly fuse the feature maps from the boundary branch with those from the segmentation branch, but instead fuses the boundary detection results into the segmentation task as a boundary attention map. This design allows boundary information to participate in segmentation decisions through attention guidance rather than feature supplementation, so that the segmentation main task has stronger boundary perception capabilities.

Besides attention mechanisms that enhance model performance, the fusion of features from different layers of the U-Net decoder can better utilize multi-scale information, thus improving the model's capability of understanding and segmenting complex images. In order to prevent feature information loss, AMF-Net¹⁵ presented a multi-scale fusion (MSF) module, by upsampling feature maps from each decoder stage to the model input image size using bilinear interpolation and then obtaining the output of this module after concatenation and 1 × 1 convolution (1 × 1 Conv) operations. Although the MSF module can improve model performance, its feature fusion process is so simple that it does not fully leverage feature information from different layers. Given this, this paper designs the deep feature aggregation (DFA) module to further improve model performance.

U-Net is an interesting innovation.²² Through the aforementioned research work, it can also be noted that the U-Net architecture is very suitable for the vessel segmentation task, but there is still a lot of room for improvement. Given this, we improve on U-Net, thus proposing a boundary perception enhancement network (BPENet) for retinal vessel and coronary angiogram segmentation. Firstly, BPENet introduces the FE2 module to strengthen the model's vessel feature extraction capability. This module mainly comprises FSC and ACS. Among them, FSC primarily employs strip convolutions²³ at four angles ( $0 \circ$ , $45 \circ$ , $90 \circ$ , and $135 \circ$ ) for vessel feature extraction. The ACS is used to better attend to the significant channel features. Secondly, since existing vessel segmentation methods suffer from poor vessel boundary segmentation, BPENet introduces the boundary detection auxiliary task and the BE module to alleviate this problem. It is notable that retinal vessel datasets DRIVE,²⁴ CHASE-DB1,²⁵ and STARE,²⁶ as well as coronary angiogram datasets DCA1²⁷ and XCAD,²⁸ do not provide vessel boundary labeled maps. We apply image processing approaches to vessel segmentation labeled maps, thereby obtaining corresponding vessel boundary labeled maps. For details, see Subsection 3.2 (Data preprocessing). Overall, the vessel boundary is obtained through the boundary detection auxiliary branch (BDAB). The boundary refinement is achieved via boundary detection supervision, which progressively optimizes the boundary prediction results output by the BDAB. The boundary information is fused into the vessel segmentation main network through the following steps: first, applying a sigmoid to the boundary predictions to generate the boundary attention maps, which are then element-wise multiplied with the encoder feature maps in the skip connections, followed by concatenation and convolution operations to complete the final fusion, as implemented in the BE module. Additionally, to fully leverage feature information from different layers, BPENet introduces the DFA module, which deeply fuses features from multiple decoder layers and combines ACS. Figure 1 shows the comparison of vessel boundary segmentation effects between BPENet and the baseline model (U-Net). From the figure, it can be seen that BPENet achieves better segmentation results in the vessel boundary regions than the baseline model, indicating that our boundary perception enhancement mechanism is effective.

Figure 1.

Comparison of vessel boundary segmentation effects between BPENet and the baseline model. (a1) and (a2) Original image, (b1) and (b2) Ground truth (GT), (c1) and (c2) Baseline model, (d1) and (d2) BPENet (ours).

The main contributions of this paper are as follows:

We propose a boundary perception enhancement network (BPENet) that introduces a boundary detection auxiliary task to the vessel segmentation main task. Experimental results on five datasets show that BPENet is highly competitive in the vessel segmentation task.

We introduce the feature extraction enhancement (FE2) module to enhance the model's vessel feature extraction capability, which mainly comprises four-angle strip convolution (FSC) and adaptive channel selection (ACS).

We introduce the boundary enhancement (BE) module with boundary attention to enhance the model’s vessel boundary perception capability.

We introduce the deep feature aggregation (DFA) module to fully leverage feature information from different layers, which mainly generates richer and more comprehensive feature representations by deeply fusing features from different layers.

Methodology

Overall overview

This paper introduces the FE2, BDAB, BE, and DFA modules to the U-Net, thereby proposing BPENet. As shown in Figure 2, BPENet mainly consists of the encoder, decoder, skip connections, and BDAB. Among them, to enhance vessel feature extraction capability, we employ the FE2 module mainly comprising strip convolutions at four angles ( $0 \circ$ , $45 \circ$ , $90 \circ$ , and $135 \circ$ ) and ACS to replace convolution blocks in original U-Net encoder and decoder. Secondly, to alleviate the problem of poor vessel boundary segmentation, we embed boundary perception mechanisms into the network. Specifically, we add the BDAB to the deep layer of the encoder and construct its prediction results as boundary attention. In the BE module, this attention is element-wise multiplication with the encoder features to achieve feature modulation, followed by concatenation and convolution operations to complete the final fusion. The fused features are fed to the decoder. Finally, to fully leverage feature information from different layers, we introduce the DFA module in the decoder. This module deeply fuses features from multiple decoder layers and combines ACS. In addition, in Figure 2, Output_bd represents the output of the boundary detection auxiliary task, and Output_vs represents the output of the vessel segmentation main task.

Figure 2.

Network architecture of BPENet. This model uses FE2 to replace U-Net's convolution blocks, thus strengthening the model's vessel feature extraction capability. Next, BDAB is added to the encoder for vessel boundary detection. Then, BE is introduced in the skip connections to strengthen the model's vessel boundary perception capability. Finally, DFA is added to the decoder to fully leverage feature information from different layers.

Feature extraction enhancement (FE2) module

It is crucial for a model to have excellent feature extraction capability. BPENet introduces the FE2 module to enhance the model's vessel feature extraction capability. As shown in Figure 3, the FE2 module mainly comprises FSC and ACS, where four angles refer to the $0 \circ$ , $45 \circ$ , $90 \circ$ , and $135 \circ$ that can better fit the directions of vessel paths, while ACS is used to better attend to the significant channel features. Firstly, feature map F passes through FSC, as described in detail:

F^{'} = [S C_{0 \circ} (F); S C_{45 \circ} (F); S C_{90 \circ} (F); S C_{135 \circ} (F)]

(1)

where SC_x denotes a strip convolution operation in the x direction. [X1; X2; X3; X4] denotes concatenating the feature maps X1, X2, X3, and X4.

Figure 3.

FE2 module. This module mainly consists of four-angle strip convolution (FSC) and adaptive channel selection (ACS), where FSC mainly comprises strip convolutions at four angles: $0 \circ$ , $45 \circ$ , $90 \circ$ , and $135 \circ$ , while ACS is mainly dual-branch channel attention.

ACS is mainly a dual-branch channel attention. Next, F′ passes through ACS, as described in detail:

M_{m a x} = σ (C 1 D_{k} (G M P (F^{'})))

(2)

F_{m a x}^{'} = F^{'} \otimes M_{m a x}

(3)

M_{a v g} = σ (C 1 D_{k} (G A P (F^{'})))

(4)

F_{a v g}^{'} = F^{'} \otimes M_{a v g}

(5)

F^{″} = f ([{F^{'}}_{m a x}; {F^{'}}_{a v g}])

(6)

where GMP denotes global max pooling,

σ

denotes sigmoid function, and

C 1 D_{k}

denotes one-dimensional convolution with kernel size k. M_max denotes channel attention map of one branch, and

\otimes

denotes element-wise multiplication. GAP denotes global average pooling, and M_avg denotes channel attention map of another branch. f denotes combined operations of 1 × 1 Conv, BN (batch normalization), and ReLU. In addition, the kernel size k of the one-dimensional convolution is determined by an adaptive selection function of kernel size,²⁹ as described by the following formula:

k = ψ (C) = {| \frac{\log_{2} (C)}{γ} + \frac{b}{γ} |}_{o d d}

(7)

where

| t |_{o d d}

denotes the nearest odd number of t. C denotes channel quantities of F′. This paper sets parameters

γ

and b to 2 and 1, respectively.

Finally, F″ and F perform residual connection and get F_out by 1 × 1 Conv, BN, ReLU, and Dropout operations. The specific process is described as follows:

F_{o u t} = f^{'} (F \oplus F^{″})

(8)

where

\oplus

denotes element-wise summation, and f′ denotes combined operations of 1 × 1 Conv, BN, ReLU, and Dropout. In addition, in all FE2 modules, the kernel size of strip convolutions in the

0 \circ

direction is set to 1 × 9 so that it has equivalent parameters as the traditional 3 × 3 convolution kernel, and the remaining three angle directions are expressed similarly.

Boundary detection auxiliary branch (BDAB)

Because multi-task learning can improve the model performance through mutual promotion, so combining vessel segmentation and vessel boundary detection for multi-task learning, the vessel boundary detection auxiliary task's powerful extraction ability for vessel boundary information can be utilized to enhance the vessel segmentation main task's extraction ability for vessel boundary information. Based on this, we add the auxiliary branch in the encoder for vessel boundary detection, as shown in Figure 2(a). Firstly, an encoder deep feature map sequentially passes through the FE2 module and bilinear interpolation (upsampling by 4 times), followed by concatenating the feature map obtained from upsampling with the encoder shallow layer feature map along the channel dimension. Finally, the concatenated feature map sequentially goes through the FE2 module, bilinear interpolation (upsampling by 2 times), 1 × 1 Conv, and sigmoid function to get the vessel boundary prediction result.

Boundary enhancement (BE) module

Attention mechanisms can highlight important features while suppressing unimportant features. To enhance the model's vessel boundary perception capability, we introduce the BE module. As shown in Figure 4, the BE module mainly takes the boundary prediction results obtained from BDAB as boundary attention and introduces them to the skip connections, which enables the segmentation main network to focus more on the vessel boundary regions and capture the boundary information more effectively, thereby improving the boundary segmentation effects. Firstly, encoder feature map E is element-wise multiplied with boundary attention map M_b to obtain feature map E′. Then, E′ and E perform concatenation. Finally, the concatenated feature map sequentially passes through 1 × 1 Conv, BN, and ReLU to get E_out. The overall details of the BE module are described as follows:

E^{'} = E \otimes M_{b}

(9)

E_{o u t} = f ([E^{'}; E])

(10)

Figure 4.

BE module. This module mainly utilizes boundary attention to enhance the model's vessel boundary perception capability.

Deep feature aggregation (DFA) module

Feature maps from different layers carry different feature information. By fusing these information, the model can understand the image content more comprehensively, thereby enhancing model performance. To fully leverage feature information from different layers, we introduce the DFA module in the decoder. As shown in Figure 5, the DFA module deeply fuses features from multiple decoder layers and combines ACS that can better attend to the significant channel features.

Figure 5.

DFA module. This module deeply fuses features from multiple decoder layers and combines ACS that can focus on important channel features.

Firstly, the decoder feature maps D₁, D₂, and D₃ by using bilinear interpolation to upsample them by 8, 4, and 2 times, respectively, to obtain feature maps $D_{1}^{'}$ , $D_{2}^{'}$ , and $D_{3}^{'}$ . The specific process is described as follows:

D_{1}^{'} = U P_{8} (D_{1})

(11)

D_{2}^{'} = U P_{4} (D_{2})

(12)

D_{3}^{'} = U P_{2} (D_{3})

(13)

where UP_x denotes bilinear interpolation upsampling by x times.

Next, some convolution and concatenation operations are performed in $D_{1}^{'}$ , $D_{2}^{'}$ , $D_{3}^{'}$ , and $D_{4}$ to get $D_{1}^{^{″}}$ , $D_{2}^{^{″}}$ , $D_{3}^{^{″}}$ , and $D_{4}^{'}$ , as described in detail:

D_{1}^{^{″}} = f (D_{1}^{'})

(14)

D_{2}^{^{″}} = f ([D_{2}^{^{'}}; D_{1}^{^{″}}])

(15)

D_{3}^{^{″}} = f ([D_{3}^{^{'}}; D_{2}^{^{″}}])

(16)

D_{4}^{'} = f ([D_{4}; D_{3}^{^{″}}])

(17)

Next, $D_{1}^{^{″}}$ , $D_{2}^{^{″}}$ , $D_{3}^{^{″}}$ , $D_{4}^{'}$ , and $D_{4}$ perform concatenation. Immediately following, performing ACS and convolution operations obtains $D^{'}$ . The specific process is described as follows:

D = A C S ([D_{1}^{^{″}}; D_{2}^{^{″}}; D_{3}^{^{″}}; D_{4}^{'}; D_{4}])

(18)

D^{'} = f (D)

(19)

Finally, $D^{'}$ and $D_{4}$ are element-wise summed to get $D_{o u t}$ , as described in detail:

D_{o u t} = D^{'} \oplus D_{4}

(20)

Experimental setup

Datasets

BPENet was evaluated on three retinal vessel datasets, DRIVE,²⁴ CHASE-DB1,²⁵ and STARE,²⁶ as well as two coronary angiogram datasets, DCA1²⁷ and XCAD.²⁸ If the dataset image has two expert's annotation maps, we employ the first expert's annotation map as ground truth (GT). The images in the DRIVE, CHASE-DB1, and STARE datasets are all color images, with 40, 28, and 20 annotated images, respectively. The images in the DCA1 dataset are all grayscale images, with 134 annotated images. The images in the XCAD dataset are all 1 channel, with 126 annotated images. The DRIVE dataset has been officially divided into training and test sets. We use the first 20, 16, 100, and 100 images of CHASE-DB1, STARE, DCA1, and XCAD datasets as the training set and the last 8, 4, 34, and 26 as the test set, respectively. Detailed information about the five datasets is illustrated in Table 1.

Table 1.

Dataset information.

Dataset	Subject	Training / test	Resolution
DRIVE	40	20/20	584 × 565
CHASE-DB1	28	20/8	999 × 960
STARE	20	16/4	700 × 605
DCA1	134	100/34	300 × 300
XCAD	126	100/26	512 × 512

Data preprocessing

The image preprocessing process consists of converting images to grayscale images (if dataset images are color images), normalization, contrast limited adaptive histogram equalization (CLAHE) to strengthen contrast, and gamma correction to deal with the unevenness of image brightness. In addition, for boundary (or contour) labeled maps acquisition, we draw on the approach adopted by Xu et al.¹⁸ and Liu et al..¹⁹ Specifically, we go through the findContours function of OpenCV to get corresponding vessel boundary labeled maps from vessel segmentation labeled maps of training samples.

For training samples of DRIVE, CHASE-DB1, STARE, and DCA1, we crop the overlapping blocks from the preprocessed training set images to expand the training sample numbers using the sliding cropping window with a stride of 24 and a size of 128 × 128. For training samples of XCAD, we crop the overlapping blocks from the preprocessed training set images to expand the training sample numbers using the sliding cropping window with a stride of 36 and a size of 128 × 128. For test samples of DRIVE, CHASE-DB1, STARE, and DCA1, we perform the 0-value padding on the right and bottom sides of the images to fill the resolution sizes to 608 × 608, 1024 × 1024, 704 × 704, and 320 × 320, respectively. For test samples of XCAD, we do not perform the 0-value padding process.

We process the overlapping blocks by random vertical flipping, horizontal flipping, and rotations of $90 \circ$ , $180 \circ$ , or $270 \circ$ in the training phase to improve sample diversity. In addition, in the test phase, if the test sample has been previously processed by 0-value padding, we first crop the prediction result map obtained from the test to obtain the prediction result map of the original sample area (namely, the area before 0-value padding) and then perform the evaluation metrics calculations and save the prediction result map of the original sample area.

Evaluation metrics

In order to comprehensively evaluate BPENet, we employ five evaluation metrics: accuracy (ACC), F1-score (F1), area under curve (AUC), sensitivity (SE), and specificity (SP). Among them, AUC denotes the area under the receiver operating characteristic (ROC) curve, while ACC, F1, SE, and SP can be expressed as formulas:

A C C = \frac{T P + T N}{T P + F N + T N + F P}

(21)

F 1 = \frac{2 T P}{2 T P + F P + F N}

(22)

S E = \frac{T P}{T P + F N}

(23)

S P = \frac{T N}{T N + F P}

(24)

where TP, FP, TN, and FN represent true positive, false positive, true negative, and false negative, respectively.

Implementation details

BPENet was implemented by PyTorch 1.7.1 framework, and the related experiments were performed on a single NVIDIA RTX A6000 with 48GB of GPU memory. During training, we set the batch size to 100 and employed Adam as the optimizer. For experiments on DRIVE, CHASE-DB1, DCA1, and XCAD, the initial learning rate and weight decay were both set to 1e-3. For experiments on STARE, the initial learning rate and weight decay were both set to 1e-4. We employed the cosine annealing algorithm to gradually reduce the learning rate over 30 epochs. For testing, we used obtained model parameters in the 30th epoch.

Both the vessel segmentation main task and the boundary detection auxiliary task exhibit class imbalance, while the Dice loss optimizes based on the overlap between predicted results and ground truth labels, exhibiting strong robustness against class imbalance. Furthermore, both tasks are essentially pixel-level classification problems, while the binary cross-entropy (BCE) loss provides fine-grained pixel-level supervision, which helps improve the model's ability to capture local details, particularly in complex regions such as vessel boundaries. Given this, we jointly use both loss functions, specifically as follows:

L_{v s} = λ_{v s} \times L_{D i c e} ({\hat{y}}_{v s}, y_{v s}) + (1 - λ_{v s}) \times L_{B C E} ({\hat{y}}_{v s}, y_{v s})

(25)

L_{b d} = λ_{b d} \times L_{D i c e} ({\hat{y}}_{b d}, y_{b d}) + (1 - λ_{b d}) \times L_{B C E} ({\hat{y}}_{b d}, y_{b d})

(26)

L_{t o t a l} = L_{v s} + L_{b d}

(27)

where

L_{v s}

L_{b d}

, and

L_{t o t a l}

represent the vessel segmentation main task loss, boundary detection auxiliary task loss, and total loss, respectively.

λ_{v s}

and

λ_{b d}

are hyperparameters, and we set them both to 0.5.

L_{D i c e}

and

L_{B C E}

represent Dice and BCE loss, respectively. The inputs

{\hat{y}}_{v s}

and

y_{v s}

L_{D i c e}

and

L_{B C E}

L_{v s}

represent predicted values and true values in the vessel segmentation main task, respectively. The inputs

{\hat{y}}_{b d}

and

y_{b d}

L_{D i c e}

and

L_{B C E}

L_{b d}

represent predicted values and true values in the boundary detection auxiliary task, respectively.

Experimental results

Retinal vessel segmentation

Table 2 demonstrates performance comparisons of BPENet with advanced methods on DRIVE and CHASE-DB1. Table 3 demonstrates performance comparisons of BPENet with advanced methods on STARE. As shown in Table 2, the values of ACC, F1, AUC, SE, and SP metrics obtained by BPENet on DRIVE are the best, which are 0.9715, 0.8378, 0.9886, 0.8522, and 0.9878, respectively. The values of ACC, SE, and SP metrics obtained by BPENet on CHASE-DB1 are the best, which are 0.9764, 0.8818, and 0.9883, respectively. Although the value of the F1 metric of BPENet on CHASE-DB1 is 0.97% lower compared to DS2TUNet, the values of ACC, SE, and SP metrics of BPENet are higher than DS2TUNet. Among them, the value of the SE metric of DS2TUNet is 7.17% lower compared to BPENet, so considering the values of several metrics together, BPENet performance on CHASE-DB1 is better than DS2TUNet. Although the value of the AUC metric of BPENet on CHASE-DB1 is 0.21% lower compared to AMF-Net, the values of ACC, SE, and SP metrics of BPENet are higher than AMF-Net. Among them, the value of the SE metric of AMF-Net is 4.74% lower compared to BPENet, so considering the values of several metrics together, BPENet performance on CHASE-DB1 is better than AMF-Net. Additionally, although the value of the AUC metric of BPENet on CHASE-DB1 is 0.03% lower compared to ACC-UNet, the values of ACC, F1, SE, and SP of BPENet are higher than ACC-UNet, and the values of these four metrics are higher by more than 0.03%, so considering the values of several metrics together, BPENet performance on CHASE-DB1 is better than ACC-UNet.

Table 2.

Performance comparisons of BPENet with advanced methods on DRIVE and CHASE-DB1.

Method	DRIVE					CHASE-DB1
Method	ACC	F1	AUC	SE	SP	ACC	F1	AUC	SE	SP
UNet++³⁰	0.9698	0.8242	0.9842	0.8192	0.9842	0.9745	0.8057	0.9885	0.8392	0.9838
ResUnet³¹	0.9696	0.8258	0.9864	0.8290	0.9833	0.9738	0.8000	0.9859	0.8307	0.9836
M2U-Net¹⁴	0.9630	0.8091	0.9714	-	-	0.9703	0.8006	0.9666	-	-
DEU-Net²⁰	0.9567	0.8270	0.9772	0.7940	0.9816	0.9661	0.8037	0.9812	0.8074	0.9821
RVSeg-Net³²	0.9681	-	0.9817	0.8107	0.9845	0.9726	-	0.9833	0.8069	0.9836
DoubleU-Net³³	0.9698	0.8261	0.9859	0.8232	0.9841	0.9736	0.8059	0.9893	0.8604	0.9806
Pyramid U-Net³⁴	0.9615	-	0.9815	0.8213	0.9807	0.9639	-	0.9832	0.8035	0.9787
MSU-Net³⁵	0.9695	0.8235	0.9846	0.8182	0.9842	0.9741	0.8036	0.9876	0.8413	0.9832
AMF-Net¹⁵	0.9581	-	0.9824	0.8139	0.9826	0.9729	-	0.9919	0.8344	0.9881
Super U-Net³⁶	0.9660	0.8080	-	0.8180	-	0.9660	0.7520	-	0.7690	-
SBDF-Net³⁷	0.9692	0.8208	0.9843	0.8115	0.9845	0.9733	0.7972	0.9858	0.8355	0.9827
AAU-net²¹	0.9526	0.8135	-	0.8156	0.9728	-	-	-	-	-
CANet³⁸	0.9687	0.8208	0.9848	0.8242	0.9827	0.9739	0.8026	0.9880	0.8430	0.9828
ACC-UNet³⁹	0.9694	0.8237	0.9860	0.8211	0.9838	0.9745	0.8102	0.9901	0.8642	0.9820
DS2TUNet⁴⁰	0.9664	0.8195	-	0.8071	0.9823	0.9741	0.8362	-	0.8101	0.9869
FSE-Net⁴¹	0.9627	-	0.9839	0.8150	0.9834	0.9743	-	0.9871	0.7789	0.9874
ESDMR-Net⁴²	0.9699	0.8287	0.9853	0.8320	0.9832	0.9749	0.8211	0.9881	0.8380	0.9850
U-Net v2⁴³	0.9631	0.7970	0.9778	0.7931	0.9806	0.9722	0.7998	0.9853	0.8603	0.9769
EHMCANet⁴⁴	0.9696	0.8231	0.9850	0.8234	0.9839	0.9736	0.8012	0.9876	0.8451	0.9824
BPENet(ours)	0.9715	0.8378	0.9886	0.8522	0.9878	0.9764	0.8265	0.9898	0.8818	0.9883

Bold font indicates the best result of each evaluation metric, and “-” indicates missing values of the evaluation metric.

Table 3.

Performance comparisons of BPENet with advanced methods on STARE.

Method	STARE
Method	ACC	F1	AUC	SE	SP
UNet++³⁰	0.9719	0.7664	0.9588	0.7632	0.9862
ResUnet³¹	0.9715	0.7485	0.9698	0.7140	0.9866
DoubleU-Net³³	0.9684	0.7679	0.9552	0.8009	0.9782
MAU-Net⁴⁵	0.9581	0.7826	0.9721	0.7536	0.9808
MSU-Net³⁵	0.9718	0.7640	0.9573	0.7564	0.9863
SBDF-Net³⁷	0.9712	0.7508	0.9392	0.7253	0.9864
CANet³⁸	0.9699	0.7599	0.9600	0.8006	0.9812
TUnet-LBF⁴⁶	0.9681	0.8198	-	0.8004	0.9852
ACC-UNet³⁹	0.9711	0.7644	0.9632	0.7752	0.9844
FSE-Net⁴¹	0.9710	-	0.9876	0.6810	0.9818
U-Net v2⁴³	0.9696	0.7850	0.9806	0.8008	0.9800
EHMCANet⁴⁴	0.9711	0.7995	0.9786	0.7849	0.9863
BPENet(ours)	0.9742	0.8091	0.9876	0.8161	0.9884

Bold font indicates the best result of each evaluation metric, and “-” indicates missing values of the evaluation metric.

As shown in Table 3, the values of ACC, AUC, SE, and SP metrics obtained by BPENet on STARE are the best, which are 0.9742, 0.9876, 0.8161, and 0.9884, respectively. It is worth noting that the value of the AUC metric of FSE-Net on STARE is as good as our BPENet. Although the value of the F1 metric of BPENet on STARE is 1.07% lower compared to TUnet-LBF, the values of ACC, SE, and SP metrics of BPENet are higher than TUnet-LBF. Among them, the value of the SE metric of TUnet-LBF is 1.57% lower compared to BPENet, so considering the values of several metrics together, BPENet performance on STARE is better than TUnet-LBF.

In summary, it can be noted that BPENet shows excellent performance on all three retinal vessel datasets. In addition, to display the segmentation effects of BPENet more intuitively, we visualize the retinal vessel segmentation results of BPENet and nine models, namely UNet++, ResUnet, DoubleU-Net, MSU-Net, SBDF-Net, CANet, ACC-UNet, U-Net v2, and EHMCANet. Among them, except for SBDF-Net and ACC-UNet, in which the batch size is 60 during training, the rest of the implementation details used for these nine models are the same as our BPENet setup, as described in Subsection 3.4 (Implementation details). The performance of these nine models on the DRIVE, CHASE-DB1, and STARE datasets is shown in Tables 2 and 3. The visualization of retinal vessel segmentation results of BPENet and these nine models is shown in Figure 6. By observation, the segmentation effects of BPENet are better than other methods in many details, including the segmentation effects of vessel boundaries and so on. This suggests that BPENet is able to accomplish superior segmentation effects in the complicated retinal vessel trees compared to other methods.

Figure 6.

Visualization of segmentation results on retinal vessel datasets. (a) Original Image, (b) Partial Image, (c) GT, (d) UNet++, (e) ResUnet, (f) DoubleU-Net, (g) MSU-Net, (h) SBDF-Net, (i) CANet, (j) ACC-UNet, (k) U-Net v2, (l) EHMCANet, (m) BPENet(ours).

Coronary vessel segmentation

Table 4 demonstrates performance comparisons of BPENet with advanced methods on DCA1 and XCAD. As shown in Table 4, the values of ACC, F1, AUC, SE, and SP metrics obtained by BPENet on DCA1 are the best, which are 0.9788, 0.8119, 0.9930, 0.8564, and 0.9885, respectively. The values of ACC, AUC, and SE metrics obtained by BPENet on XCAD are the best, which are 0.9758, 0.9869, and 0.8518, respectively. Although the value of the F1 metric of BPENet on XCAD is 0.05% lower compared to MSU-Net, the values of ACC, AUC, SE, and SP metrics of BPENet are higher than MSU-Net. Among them, the value of the SE metric of MSU-Net is 2.19% lower compared to BPENet, so considering the values of several metrics together, BPENet performance on XCAD is better than MSU-Net. Additionally, although the value of the SP metric of BPENet on XCAD is 0.05% lower compared to SBDF-Net, the values of ACC, F1, AUC, and SE of BPENet are higher than SBDF-Net, and the values of these four metrics are higher by more than 0.05%, so considering the values of several metrics together, BPENet performance on XCAD is better than SBDF-Net.

Table 4.

Performance comparisons of BPENet with advanced methods on DCA1 and XCAD.

Method	DCA1					XCAD
Method	ACC	F1	AUC	SE	SP	ACC	F1	AUC	SE	SP
UNet++³⁰	0.9771	0.7866	0.9876	0.8142	0.9863	0.9723	0.7637	0.9829	0.8230	0.9812
ResUnet³¹	0.9775	0.7917	0.9914	0.8257	0.9860	0.9716	0.7557	0.9842	0.8043	0.9818
DoubleU-Net³³	0.9786	0.8018	0.9894	0.8297	0.9870	0.9698	0.7565	0.9859	0.8391	0.9774
MSU-Net³⁵	0.9778	0.7945	0.9880	0.8239	0.9866	0.9742	0.7792	0.9863	0.8299	0.9829
SBDF-Net³⁷	0.9769	0.7874	0.9885	0.8199	0.9858	0.9719	0.7521	0.9812	0.7778	0.9835
CANet³⁸	0.9771	0.7905	0.9897	0.8312	0.9854	0.9730	0.7685	0.9825	0.8236	0.9820
ACC-UNet³⁹	0.9774	0.7921	0.9909	0.8284	0.9859	0.9729	0.7724	0.9865	0.8390	0.9807
U-Net v2⁴³	0.9753	0.7812	0.9902	0.8401	0.9826	0.9630	0.7462	0.9816	0.8408	0.9709
EHMCANet⁴⁴	0.9772	0.7919	0.9889	0.8315	0.9855	0.9697	0.7510	0.9850	0.8249	0.9787
BPENet(ours)	0.9788	0.8119	0.9930	0.8564	0.9885	0.9758	0.7787	0.9869	0.8518	0.9830

Bold font indicates the best result of each evaluation metric.

In summary, it can be noted that BPENet shows excellent performance on both coronary angiogram datasets. In addition, to display the segmentation effects of BPENet more intuitively, we visualize the coronary vessel segmentation results of BPENet and nine models, namely, UNet++, ResUnet, DoubleU-Net, MSU-Net, SBDF-Net, CANet, ACC-UNet, U-Net v2, and EHMCANet. Among them, the implementation details used for these nine models are the same as those introduced in Subsection 4.1 (Retinal vessel segmentation). The performance of these nine models on the DCA1 and XCAD datasets is shown in Table 4. The visualization of coronary vessel segmentation results of BPENet and these nine models is shown in Figure 7. Through observation, the segmentation effects of BPENet are superior to other methods, indicating that it can better accomplish pixel-level coronary vessel segmentation.

Figure 7.

Visualization of segmentation results on coronary angiogram datasets. (a) Original Image, (b) GT, (c) UNet++, (d) ResUnet, (e) DoubleU-Net, (f) MSU-Net, (g) SBDF-Net, (h) CANet, (i) ACC-UNet, (j) U-Net v2, (k) EHMCANet, (l) BPENet(ours).

Complexity analyses

In Figure 8, we demonstrate the floating point operations (FLOPs) and parametric quantities (Params) of BPENet (ours) and nine models, namely, UNet++, ResUnet, DoubleU-Net, MSU-Net, SBDF-Net, CANet, ACC-UNet, U-Net v2, and EHMCANet. The input image size used is 128 × 128. G represents the hundred million, and M represents the million. Among them, BPENet's FLOPs and Params are 28.53G and 38.26 M, respectively. As shown in Figure 8, compared with other models, BPENet's FLOPs and Params are within the acceptable range.

Figure 8.

Complexity comparison of different models.

Ablation studies

BPENet mainly introduces the FE2, BDAB, BE, and DFA modules to the U-Net. To verify the effectiveness of these modules, we perform ablation experiments on DRIVE and DCA1, as shown in Table 5.

Table 5.

Ablation experiments.

Model	DRIVE					DCA1
Model	ACC	F1	AUC	SE	SP	ACC	F1	AUC	SE	SP
Baseline (U-Net)	0.9689	0.8110	0.9727	0.8001	0.9828	0.9770	0.7823	0.9860	0.8291	0.9854
Baseline + FE2	0.9696	0.8260	0.9828	0.8283	0.9834	0.9772	0.7865	0.9905	0.8331	0.9862
Baseline + BDAB	0.9693	0.8212	0.9817	0.8189	0.9829	0.9773	0.7883	0.9892	0.8323	0.9867
Baseline + BDAB + BE	0.9701	0.8265	0.9839	0.8307	0.9840	0.9777	0.8008	0.9906	0.8352	0.9877
Baseline + DFA	0.9695	0.8234	0.9826	0.8216	0.9831	0.9772	0.7949	0.9894	0.8317	0.9870
Baseline + FE2 + BDAB	0.9698	0.8259	0.9833	0.8302	0.9841	0.9775	0.7988	0.9913	0.8346	0.9867
Baseline + FE2 + BDAB + BE	0.9704	0.8381	0.9869	0.8356	0.9847	0.9780	0.8032	0.9916	0.8357	0.9889
Baseline + FE2 + BDAB + BE + DFA	0.9715	0.8378	0.9886	0.8522	0.9878	0.9788	0.8119	0.9930	0.8564	0.9885

Bold font indicates the best result of each evaluation metric.

On DRIVE, we first perform ablation experiments of four independent modules, namely FE2, BDAB, BE, and DFA, referred to as Baseline + FE2, Baseline + BDAB, Baseline + BDAB + BE, and Baseline + DFA, respectively. It is worth noting that the BE module inputs must utilize BDAB outputs, so we treat Baseline + BDAB + BE as the independent module ablation experiment of the BE module, rather than Baseline + BE. Through experimental results, it can be observed that on five metrics, including ACC, F1, AUC, SE, and SP, the ablation experiments of four independent modules (Baseline + FE2, Baseline + BDAB, Baseline + BDAB + BE, and Baseline + DFA) all outperform the Baseline, and Baseline + BDAB + BE outperform Baseline + BDAB. This indicates that the FE2, BDAB, BE, and DFA modules we designed can all play their corresponding functions. Among them, the FE2 module can indeed enhance the model's vessel feature extraction capability, the vessel boundary detection auxiliary task introduced by BDAB can indeed enhance the extraction ability of the vessel segmentation main task for vessel boundary information, the BE module can indeed enhance the model's vessel boundary perception capability, and the DFA module can indeed fully leverage feature information from different layers of the decoder. Next, to obtain the final BPENet model (namely, Baseline + FE2 + BDAB + BE + DFA), we employ the module stacking strategy to sequentially complete the remaining experiments on the Baseline + FE2 foundation, including Baseline + FE2 + BDAB, Baseline + FE2 + BDAB + BE, and Baseline + FE2 + BDAB + BE + DFA (ours). Through experimental results, it can be observed that the values of ACC, AUC, SE, and SP metrics obtained by Baseline + FE2 + BDAB + BE + DFA (ours) are the best. Additionally, although the value of the F1 metric of Baseline + FE2 + BDAB + BE + DFA (ours) is 0.03% lower compared to Baseline + FE2 + BDAB + BE, the values of ACC, AUC, SE, and SP of Baseline + FE2 + BDAB + BE + DFA (ours) are higher than Baseline + FE2 + BDAB + BE, and the values of these four metrics are higher by more than 0.03%, so considering the values of several metrics together, Baseline + FE2 + BDAB + BE + DFA (ours) performance is better than Baseline + FE2 + BDAB + BE. In summary, Baseline + FE2 + BDAB + BE + DFA (ours) demonstrates the best overall performance in the ablation experiments on DRIVE. On DCA1, we apply a similar analysis approach, which yields conclusions consistent with those from the ablation experiments on DRIVE.

Through ablation experiments, it is evident that incorporating FE2, BDAB, BE, and DFA modules into U-Net is effective for retinal and coronary artery vessel segmentation, and the overall architectural design of BPENet is rational. In addition, to show the segmentation effects of BPENet more intuitively, we visualize the vessel segmentation results obtained by ablation experiments, as illustrated in Figure 9. Through observation, it can be seen that the segmentation effects of BPENet (namely, Baseline + FE2 + BDAB + BE + DFA) are the closest to the expert's labeled maps, indicating its superiority.

Figure 9.

Visualization of ablation experiments. (a) Original Image, (b) GT, (c) Baseline, (d) Baseline + FE2, (e) Baseline + BDAB, (f) Baseline + BDAB + BE, (g) Baseline + DFA, (h) Baseline + FE2 + BDAB, (i) Baseline + FE2 + BDAB + BE, (j) Baseline + FE2 + BDAB + BE + DFA(ours).

Cross-dataset validations

To further evaluate the BPENet ability, we perform cross-dataset validations. Among them, for cross-dataset validations in the retinal vessel datasets, DRIVE is employed as the training set, while CHASED-B1 and STARE are employed separately as test sets. For cross-dataset validations in the coronary angiogram datasets, DCA1 is employed as the training set, while XCAD is employed as the test set. In this experiment, the implementation details of the nine models (UNet++, ResUnet, DoubleU-Net, MSU-Net, SBDF-Net, CANet, ACC-UNet, U-Net v2, and EHMCANet) are the same as those described in Subsection 4.1 (Retinal vessel segmentation). The results of cross-dataset validations are illustrated in Table 6. We use ACC, AUC, and SP metrics for performance evaluation. Through Table 6, it can be seen that BPENet has the best values of ACC and SP metrics in the “DRIVE (Training), CHASE-DB1 (Test)” experiment, while the value of AUC metric is second-best. Although the value of AUC metric of BPENet in the “DRIVE (Training), CHASE-DB1 (Test)” experiment is 0.13% lower compared to U-Net v2, the values of ACC and SP metrics of U-Net v2 are separately 0.28% and 1.64% lower compared to BPENet. Therefore, the effect of BPENet is still better than the U-Net v2 in the “DRIVE (Training), CHASE-DB1 (Test)” experiment. BPENet has the best values of ACC and AUC metrics in the “DRIVE (Training), STARE (Test)” experiment, while the value of SP metric is second-best. Although the value of SP metric of BPENet in the “DRIVE (Training), STARE (Test)” experiment is 0.02% lower compared to MSU-Net, the values of ACC and AUC metrics of MSU-Net are separately 0.13% and 2.20% lower compared to BPENet. Therefore, the effect of BPENet is still better than the MSU-Net in the “DRIVE (Training), STARE (Test)” experiment. The values of all three metrics of BPENet are the best in the “DCA1 (Training), XCAD (Test)” experiments. To sum up, the results of cross-dataset validations in Table 6 demonstrate that BPENet has better generalization ability compared to other models.

Table 6.

Cross-dataset validations.

Model	DRIVE (Training), CHASE-DB1 (Test)			DRIVE (Training), STARE (Test)			DCA1 (Training), XCAD (Test)
Model	ACC	AUC	SP	ACC	AUC	SP	ACC	AUC	SP
UNet++	0.9451	0.8254	0.9811	0.9655	0.9003	0.9859	0.9618	0.9564	0.9801
ResUnet	0.9291	0.8670	0.9653	0.9637	0.8997	0.9860	0.9632	0.9601	0.9811
DoubleU-Net	0.9427	0.8075	0.9804	0.9634	0.9000	0.9754	0.9632	0.9374	0.9770
MSU-Net	0.9441	0.8024	0.9814	0.9656	0.9263	0.9863	0.9624	0.9443	0.9808
SBDF-Net	0.9462	0.8494	0.9806	0.9625	0.9343	0.9830	0.9622	0.9515	0.9806
CANet	0.9446	0.8143	0.9808	0.9652	0.9206	0.9800	0.9619	0.9566	0.9790
ACC-UNet	0.9470	0.8998	0.9819	0.9624	0.9380	0.9831	0.9634	0.9640	0.9786
U-Net v2	0.9469	0.9224	0.9715	0.9579	0.9179	0.9604	0.9535	0.9490	0.9691
EHMCANet	0.9445	0.8682	0.9809	0.9649	0.9280	0.9855	0.9605	0.9520	0.9738
BPENet(ours)	0.9497	0.9211	0.9879	0.9669	0.9483	0.9861	0.9650	0.9705	0.9822

Bold font indicates the best result of each evaluation metric.

Conclusion

In response to the existing problem of poor vessel boundary segmentation in current vessel segmentation methods, we propose a BPENet to alleviate the problem. BPENet not only introduces a boundary detection auxiliary task to the vessel segmentation main task, but also skillfully introduces our proposed three modules: FE2, BE, and DFA module. Among them, the FE2 module is introduced to enhance the model's vessel feature extraction capability, the BE module is introduced to enhance the model's vessel boundary perception capability, and the DFA module is introduced to fully leverage feature information from different layers. Experimental results on retinal vessel and coronary angiogram datasets show that BPENet is highly competitive in the vessel segmentation task, indicating its superiority. Of course, the vessel boundary perception enhancement brought by BPENet has practical clinical value. More precise vessel boundaries help improve the measurement accuracy of geometric parameters, such as vessel diameter, curvature, and stenosis degree. These parameters are important bases for diagnosing related diseases, such as diabetic retinopathy and coronary artery disease. In addition, accurate vessel boundary information can provide reliable structural references for clinical operations such as stent implantation and laser surgery navigation.

Although our approach demonstrates good performance on retinal vessel and coronary angiogram datasets, it still has limitations: (1) In the aspect of retinal vessel segmentation, this paper focuses primarily on segmenting the retinal vessel structure without further distinguishing between arteries and veins. However, in retinal analysis, artery/vein classification is equally important for diagnosing related diseases. For example, the arteriovenous ratio (AVR) is a key clinical indicator in diseases such as diabetic retinopathy and hypertensive retinopathy; (2) The network architecture in this paper is primarily based on the CNN implementation. Although some innovative modules have been introduced to improve vessel segmentation performance, convolution operations are inherently limited by their local receptive fields, resulting in a limited ability to extract global contextual information. However, this ability is crucial for global topological relationship modeling of complex vessel structures. In future work, we will focus on addressing the limitations mentioned above. Specifically, through multi-task learning, we will combine the retinal artery/vein classification task with the approach presented in this paper, while introducing the Transformer, which has global modeling capabilities. The new model constructed in this way will not only retain the advantages of vessel boundary perception but also have the ability to capture global contextual information, and will be able to simultaneously obtain both vessel segmentation and artery/vein classification outputs.

Footnotes

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Natural Science Foundation of China (grant number 62266011).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

ORCID iDs

Linfeng Kong

Yun Wu

References

Camasão

Mantovani

. The mechanical characterization of blood vessels and their substitutes in the continuous quest for physiological-relevant performances. a critical review. Materials Today Bio 2021; 10: 100106.

Fraz

Remagnino

Hoppe

, et al. Blood vessel segmentation methodologies in retinal images – a survey. Comput Methods Programs Biomed 2012; 108: 407–433.

Kerkeni

Benabdallah

Manzanera

, et al. A coronary artery segmentation method based on multiscale analysis and region growing. Comput Med Imaging Graph 2016; 48: 49–61.

Wan

Feng

Tong

, et al. Automated identification and grading of coronary artery stenoses with x-ray angiography. Comput Methods Programs Biomed 2018; 167: 13–22.

Guo

Chen

, et al. Retinal vessel segmentation combined with generative adversarial networks and dense unet. IEEE Access 2020; 8: 194551–194560.

Jun

Kweon

Kim

, et al. T-net: nested encoder–decoder architecture for the main vessel segmentation in coronary angiography. Neural Netw 2020; 128: 216–233.

Bakkouri

Afdel

. Multi-scale CNN based on region proposals for efficient breast abnormality recognition. Multimed Tools Appl 2019; 78: 12939–12960.

Bakkouri

Afdel

. Computer-aided diagnosis (cad) system based on multi-layer feature fusion network for skin lesion recognition in dermoscopy images. Multimed Tools Appl 2020; 79: 20483–20518.

Sahli

Ben Slama

Labidi

. U-net: a valuable encoder-decoder architecture for liver tumors segmentation in ct images. J Xray Sci Technol 2022; 30: 45–56.

10.

Bakkouri

Afdel

Benois-Pineau

, et al. Bg-3dm2f: bidirectional gated 3d multi-scale feature fusion for Alzheimer’s disease diagnosis. Multimed Tools Appl 2022; 81: 10743–10776.

11.

Bakkouri

Afdel

. Mlca2f: multi-level context attentional feature fusion for COVID-19 lesion segmentation from CT scans. Signal Image Video Process 2023; 17: 1181–1188.

12.

Long

Shelhamer

Darrell

. Fully convolutional networks for semantic segmentation. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp.3431–3440.

13.

Ronneberger

Fischer

Brox

. U-net: convolutional networks for biomedical image segmentation. In: Navab N, Hornegger J, Wells WM, et al. (eds) Medical image computing and computer-assisted intervention – MICCAI 2015. Cham: Springer International Publishing, 2015, pp.234–241. ISBN 978-3-319-24574-4.

14.

Laibacher

Weyde

Jalali

. M2u-net: effective and efficient retinal vessel segmentation for real-world applications. In: 2019 IEEE/CVF conference on computer vision and pattern recognition workshops (CVPRW), pp.115–124.

15.

Yang

Cui

, et al. Amf-net: attention-aware multi-scale fusion network for retinal vessel segmentation. In: 2021 43rd Annual international conference of the IEEE engineering in medicine & biology society (EMBC), pp.3277–3280.

16.

, et al. Multi-task neural networks with spatial activation for retinal vessel segmentation and artery/vein classification. In: Shen D, Liu T and Peters TM, et al. (eds) Medical image computing and computer assisted intervention – MICCA 2019. Cham: Springer International Publishing, 2019, pp.769–778. ISBN 978-3-030-32239-7.

17.

Zhang

Gao

, et al. Joint segmentation and quantification of main coronary vessels using dual-branch multi-scale attention network. In: de Bruijne M, Cattin PC and Cotin S, et al. (eds) Medical image computing and computer assisted intervention – MICCAI 2021. Cham: Springer International Publishing, 2021, pp.369–378. ISBN 978-3-030-87193-2.

18.

Zhu

Luo

, et al. Extracting buildings from remote sensing images using a multitask encoder-decoder network with boundary refinement. Remote Sens (Basel) 2022; 14: 564.

19.

Liu

Xia

Feng

, et al. A novel building extraction network via multi-scale foreground modeling and gated boundary refinement. Remote Sens (Basel) 2023; 15: 5638.

20.

Wang

Qiu

. Dual encoding u-net for retinal vessel segmentation. In: Shen D, Liu T, Peters TM, et al. (eds) Medical image computing and computer assisted intervention – MICCAI 2019. Cham: Springer International Publishing, 2019, pp.84–92. ISBN 978-3-030-32239-7.

21.

. Retinal vascular segmentation algorithm based on atrous attention u-net. In: 2022 International conference on image processing, computer vision and machine learning (ICICML), pp.517–521.

22.

Wang

Zou

Liu

. Hybrid dilation and attention residual u-net for medical image segmentation. Comput Biol Med 2021; 134: 104449.

23.

Hua

, et al. Deep supervision feature refinement attention network for medical image segmentation. Eng Appl Artif Intell 2023; 125: 106666.

24.

Staal

Abramoff

Niemeijer

, et al. Ridge-based vessel segmentation in color images of the retina. IEEE Trans Med Imaging 2004; 23: 501–509.

25.

Owen

Rudnicka

Mullen

, et al.

Measuring retinal vessel tortuosity in 10-year-old children: validation of the computer-assisted image analysis of the retina (CAIAR) Program.

Investigative Ophthalmology & Visual Science 2009; 50: 2004–2010.

26.

Hoover

Kouznetsova

Goldbaum

. Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Trans Med Imaging 2000; 19: 203–210.

27.

Cervantes-Sanchez

Cruz-Aceves

Hernandez-Aguirre

, et al. Automatic segmentation of coronary arteries in x-ray angiograms using multiscale analysis and artificial neural networks. Appl Sci 2019; 9: 5507.

28.

Hua

Deng

, et al. Self-supervised vessel segmentation via adversarial learning. In: 2021 IEEE/CVF international conference on computer vision (ICCV), pp.7516–7525.

29.

Wang

Zhu

, et al. Eca-net: efficient channel attention for deep convolutional neural networks. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp.11531–11539.

30.

Zhou

Rahman Siddiquee

Tajbakhsh

, et al. Unet++: a nested u-net architecture for medical image segmentation. In: Stoyanov D, Taylor Z, Carneiro G, et al. (eds) Deep learning in medical image analysis and multimodal learning for clinical decision support. Cham: Springer International Publishing, 2018, pp.3–11. ISBN 978-3-030-00889-5.

31.

Zhang

Liu

Wang

. Road extraction by deep residual u-net. IEEE Geosci Remote Sens Lett 2018; 15: 749–753.

32.

Wang

Zhong

, et al. Rvseg-net: an efficient feature pyramid cascade network for retinal vessel segmentation. In: Martel AL, Abolmaesumi P, Stoyanov D, et al. (eds) Medical image computing and computer assisted intervention – MICCAI 2020. Cham: Springer International Publishing, 2020, pp.796–805. ISBN 978-3-030-59722-1.

33.

Jha

Riegler

Johansen

, et al. Doubleu-net: a deep convolutional neural network for medical image segmentation. In: 2020 IEEE 33rd international symposium on computer-based medical systems (CBMS), pp.558–564.

34.

Zhang

. Pyramid u-net for retinal vessel segmentation. In: ICASSP 2021 - 2021 IEEE international conference on acoustics, speech and signal processing (ICASSP), pp.1125–1129.

35.

Zhang

Liu

, et al. Msu-net: Multi-scale u-net for 2d medical image segmentation. Front Genet 2021; 12: 639930.

36.

Beeche

Singh

Leader

, et al. Super u-net: a modularized generalizable architecture. Pattern Recognit 2022; 128: 108669.

37.

Wang

Tian

, et al. Sbdf-net: a versatile dual-branch fusion network for medical image segmentation. Biomed Signal Process Control 2022; 78: 103928.

38.

Xie

Zhang

Pan

, et al. Canet: context aware network with dual-stream pyramid for medical image segmentation. Biomed Signal Process Control 2023; 81: 104437.

39.

Ibtehaz

Kihara

. Acc-unet: a completely convolutional unet model for the 2020s. In: Greenspan H, Madabhushi A, Mousavi P, et al. (eds) Medical image computing and computer assisted intervention – MICCAI 2023. Cham: Springer Nature Switzerland, 2023, pp.692–702. ISBN 978-3-031-43898-1.

40.

Liu

Zhou

Shen

, et al. A fundus vessel segmentation method based on double skip connections combined with deep supervision. Front Cell Dev Biol 2024; 12: 1477819.

41.

Pan

, et al. Fse-net: rethinking the up-sampling operation in encoder-decoder structure for retinal vessel segmentation. Biomed Signal Process Control 2024; 90: 105861.

42.

Khan

Naqvi

Meijering

. Esdmr-net: a lightweight network with expand-squeeze and dual multiscale residual connections for medical image segmentation. Eng Appl Artif Intell 2024; 133: 107995.

43.

Peng

Chen

Sonka

. U-net v2: rethinking the skip connections of u-net for medical image segmentation. In: 2025 IEEE 22nd International symposium on biomedical imaging (ISBI), pp.1–5.

44.

Wang

Wei

, et al. Efficient hierarchical multiscale convolutional attention for accurate medical image segmentation. Vis Comput 2025; 41: 10805–10826.

45.

Wang

Wan

, et al. Mau-net: a retinal vessels segmentation method. In: 2020 42nd annual international conference of the IEEE engineering in medicine & biology society (EMBC), pp.1958–1961.

46.

Zhang

Luo

, et al. Tunet-lbf: retinal fundus image fine segmentation model based on transformer unet network and lbf. Comput Biol Med 2023; 159: 106937.