Segmenting multi-modal abdominal multi-organ using the CADSTransN-Net model

Abstract

Background:

Deep learning advances medical imaging segmentation, but the insufficient diversity of datasets limits its performance. The AMOS22 dataset addresses this by providing large-scale, varied clinical data to enhance algorithm robustness.

Purpose

This study develops and validates CADSTransN-Net (Convolutional Attention and Deep Supervision TransN-Net) to optimize abdominal organ segmentation for the AMOS22 challenge.

Methods

CADSTransN-Net integrates three core innovations: a novel N-shaped feature flow path (departing from symmetric architectures for efficient encoder-decoder fusion), a convolutional attention mechanism (prioritizing anatomically relevant regions), and layer-wise deep supervision (ensuring meticulous gradient propagation and faster convergence).

Results

Evaluated on the full AMOS22 dataset, CADSTransN-Net achieved outstanding comprehensive performance: average Dice Similarity Coefficient (DSC) of 0.907, Normalized Surface Dice (NSD) of 0.850, 95th Percentile Hausdorff Distance (HD(95%)) of 3.98 mm, Average Surface Distance (ASD) of 0.75 mm, Absolute Volumetric Difference (AVD) of 39,755.88 mm³, and Relative Volumetric Difference (RVD) of 1.53%. These metrics confirm its high accuracy in region overlap, boundary consistency, and volume estimation for multi-modal abdominal multi-organ segmentation.

Conclusions

CADSTransN-Net effectively meets AMOS22's challenges, delivering robust performance across region, boundary, and volume metrics. It provides a reliable solution for multi-modal abdominal multi-organ segmentation, with significant clinical potential for tasks such as surgical navigation.

Keywords

abdominal image segmentation CADSTransN-Net model CT MRI AMOS22

Introduction

Segmenting abdominal organs from computed tomography (CT) and magnetic resonance imaging (MRI) scans is a crucial task in medical image analysis, as its accuracy significantly impacts diagnostic precision, treatment planning, and surgical guidance.¹ Recent advances in deep learning have significantly improved the accuracy of automatic multi-organ segmentation. However, the lack of diversity in existing datasets restricts the performance of deep learning models.² These datasets often lack sufficient variation in disease states, imaging modalities, and anatomical differences. Such variations are crucial for developing models that can accurately reflect real-world clinical scenarios. Moreover, the high costs associated with collecting and annotating 3D medical data further limit the scope of these datasets, often resulting in a focus on a limited number of organs and insufficient sample sizes. Such constraints hinder the development of models that can robustly handle the complexities of real-world medical imaging tasks. In response to these limitations, the AMOS dataset provides comprehensive data specifically for the abdominal organ segmentation task.³ It includes 500 CT and 100 MRI scans collected from a diverse range of patients across multiple centers, vendors, modalities, stages, and disease states. Each scan is carefully annotated with voxel-level accuracy for 15 different abdominal organs, offering unprecedented detail and variability. The AMOS dataset offers a robust platform for developing more advanced segmentation algorithms, thereby facilitating comprehensive evaluations of deep-learning models for abdominal image segmentation across various clinical conditions.

Recent studies have demonstrated the potential of deep learning to overcome the challenges associated with medical image segmentation.^4,5 However, traditional convolutional neural networks (CNNs) frequently face challenges in managing the complex spatial relationships and variability inherent in medical images. Some studies have begun integrating CNNs with innovative mechanisms, such as Transformers and attention models.^6,7 These approaches utilize transformers to capture long-range dependencies within image data and develop a contextual understanding of organ structure, offering new methods to tackle issues like complex spatial relationships and variability. The urgency of developing more accurate and robust segmentation models is driven not only by the clinical demand for precise diagnostics and treatment but also by the need for efficient and reliable analysis tools to manage the increasing volume of medical imaging data. As medical datasets become larger and more complex, the ability of models to scale and maintain high accuracy is critical. Moreover, incorporating advanced models into clinical practice can significantly enhance the efficiency of medical procedures and improve patient outcomes.

This study develops a new deep-learning model that enhances abdominal image segmentation performance by integrating TransN-Net with a CA mechanism and DS module. The Transformer layer in CADSTransN-Net employs a multi-head self-attention mechanism, which differs from traditional CNNs and enables the capture of long-distance dependencies in image data. This mechanism allows the model to integrate distant contextual information, improving segmentation accuracy and detail. In medical image processing, introducing the Transformer layer optimizes the model's ability to process complex tissue structures and non-homogeneous image features through the self-attention mechanism, which is often difficult to achieve with traditional CNN structures.

The CA layer is designed to focus the model's learning on more anatomically relevant regions by dynamically adjusting the response of the feature map. This fine spatial attention capability is not present in traditional attention mechanisms. In experiments, the CA layer demonstrated enhanced sensitivity to detailed features, which is particularly important for identifying and segmenting small organs or structures with complex morphology in medical images, thereby significantly improving segmentation accuracy.

The DS strategy introduces loss functions at multiple depth levels of the CADSTransN-Net model, enabling it to more effectively utilize information from intermediate layers during training, accelerate convergence, and improve adaptability to various segmentation tasks. Compared with traditional single-output supervised models, deep supervision allows the model to correct errors early in training, which is particularly important for training deep networks. It helps avoid the gradient vanishing problem and improves the model's training stability and performance.

The CA mechanism enables the model to focus on key features, thereby enhancing the detection of details crucial for accurate segmentation. At the same time, the DS module can effectively utilize the information of the intermediate layers during the training process, promoting enhanced convergence and generalization. The primary contributions of this research are as follows:

The CADSTransN-Net model integrates the strengths of both Transformer and N-Net architectures. It introduces a cross-layer branch connection and path fusion mechanism, and designs an N-shaped feature flow path. In this structure, the encoded features from specific intermediate layers are directly transmitted to higher-resolution decoding layers for fusion, enhancing feature extraction capabilities. This integrated approach more effectively captures the spatial hierarchy and contextual information required for precise organ segmentation.

The CADSTransN-Net model uses a CA mechanism to dynamically refine feature maps and focus on relevant anatomical regions, enhancing the model's ability to depict complex abdominal structures.

The CADSTransN-Net model employs a DS module to ensure detailed gradient propagation and rapid convergence. This multi-level supervision fosters a more comprehensive optimization of the network during training. It enhances the utilization of information from intermediate layers and enables self-correction at various levels, thereby boosting overall learning efficiency and improving model performance.

The CADSTransN-Net model achieves excellent abdominal organ segmentation capabilities, meets the technical requirements of medical image segmentation, and enhances the diagnosis process and treatment planning.

Related research

Due to the development of deep learning technology, abdominal organ segmentation has achieved remarkable progress, with performance significantly better than traditional image processing methods. CNN has become the mainstay of image segmentation tasks due to its ability to learn hierarchical features directly from data.⁸ U-Net was developed for biomedical image segmentation and has quickly become a benchmark model for medical images, as its structure is well-suited for capturing the spatial hierarchy of various segmentation tasks.⁹ Despite the excellent performance of CNN in medical image segmentation, challenges such as the high variability of patient anatomy, the presence of noise and artifacts in imaging data, and the need for substantial computing resources remain.

Researchers often consider automatic segmentation of abdominal organs a resolved issue, given that state-of-the-art methods achieve results comparable to inter-rater variability on many benchmark datasets.¹⁰ However, most of these datasets predominantly feature cases from a single center, stage, vendor, or disease, raising questions about the generalizability of these high performances to other datasets. Ma et al.¹¹ conducted a comprehensive study on the segmentation of the liver, kidney, spleen, and pancreas using the large and diverse AbdomenCT-1K dataset. Their findings revealed that current state-of-the-art methods face challenges in generalization across different medical centers, stages, and diseases that the models have not previously encountered.

Gibson et al.¹² studied the automatic segmentation of abdominal anatomical structures on CT images. They proposed a deep learning-based registration-free segmentation algorithm for eight organs related to endoscopic pancreatic and biliary surgical navigation, including the pancreas, gastrointestinal tract (esophagus, stomach, and duodenum), and peripheral organs (liver, spleen, left kidney, and gallbladder). When they cross-validated the proposed method on a multicenter dataset of 90 subjects, it produced significantly higher DSC on all organs and lower mean absolute distances for most organs, demonstrating that deep learning-based segmentation represents a registration-free approach for multi-organ abdominal CT segmentation that may support image-guided navigation in gastrointestinal endoscopy procedures.

Shen et al.¹³ developed a U-Net-based segmentation model tailored for hepatobiliary and pancreatic surgery-related organs, including the pancreas, duodenum, gallbladder, liver, and stomach. The model addresses the complexity of CT backgrounds and the variability in size and shape among different organs. To enhance feature extraction, they introduced a spatial attention block designed to learn a spatial attention map through explicit external supervision, effectively highlighting regions of interest. Additionally, they incorporated a deformable convolution block, which adapts the receptive fields for each organ by using trainable offsets to accommodate shape and size variations. The model proposed by Shen et al. significantly enhances overall segmentation performance, achieving an average DSC of 80.46%, thereby establishing it as a competitive method for multi-organ segmentation.

Irshad et al.¹⁴ observed that the complex representation and indistinct boundaries of abdominal organs impede the accuracy of deep learning-based segmentation methods. They proposed enhancing abdominal image segmentation by incorporating organ boundary prediction as a supplementary task. Their approach involved training a 3D encoder-decoder network through multi-task learning, which enabled the simultaneous segmentation of abdominal organs and their boundaries. The study explored two network topologies within a unified multi-task framework, differing in the extent of shared weights between the two tasks. This multi-task training approach proved advantageous, compelling the network to concentrate on the obscure boundaries of organs and, consequently, enhancing the segmentation accuracy of abdominal CT data.

In the abdominal MRI segmentation, Amjad et al.¹⁰ identified that deep learning efforts have predominantly focused on developing automatic segmentation models for a single MRI sequence. They introduced a multi-sequence deep learning-based automatic segmentation model (mS-DLAS) for multi-sequence abdominal MRI. The mS-DLAS model, utilizing the 3DResUnet network, was trained and tested on four T1- and T2-weighted MRIs obtained during routine RT simulation from 71 patients with abdominal tumors. The method also used strategies such as data preprocessing, Z-normalization, and data augmentation to improve model performance. Furthermore, two sequence-specific models, T1-weighted (T1-M) and T2-weighted (T2-M), were also developed to assess the performance of sequence-specific DLAS. The final DLAS model demonstrated its efficacy by generating accurate contours of 12 upper abdominal organs for each test case within 21 s.

Labeling multiple organs on a single MR sequence is a time-consuming and labor-intensive task, and manual labeling across multiple MR sequences introduces even greater complexity. One approach to alleviating the burden of manual annotation involves training a model on one sequence and generalizing it to others; however, the presence of domain gaps often results in poor generalization performance. To address the significant challenges in MR image segmentation, Xue et al.¹⁵ introduced a unified framework named OMUDA for one-to-many unsupervised domain adaptive segmentation. This framework utilizes the disentanglement of content and style to effectively transform images from the source domain into those of multiple target domains. OMUDA also employs generator reconstruction and style constraints to enhance cross-modal structural consistency and minimize domain aliasing. The effectiveness of OMUDA is demonstrated by its average Dice Similarity Coefficients (DSCs) across multiple sequences and organs, with values of 85.51% on the internal test set, 82.66% on the AMOS22 dataset, and 91.38% on the CHAOS dataset. These quantitative results confirm the segmentation performance and training efficiency of OMUDA, proving its practical applicability in the early stages of product development.

Although recent years have seen significant progress in the automatic segmentation of abdominal multi-organ structures from CT/MRI scans, the comprehensive evaluation of model capabilities continues to be hindered by the lack of large-scale benchmarks from varied clinical scenarios. The high costs associated with collecting and labeling 3D medical data limit most deep learning models to datasets with a small number of organs of interest or samples. This limitation restricts the capabilities of modern deep learning models and complicates the provision of a comprehensive and fair assessment of different methods. To address these challenges, Yuanfeng et al.¹⁶ introduced AMOS, a large-scale and diverse clinical dataset specifically for abdominal organ segmentation. AMOS comprises 500 CT and 100 MRI, collected from multi-center, multi-vendor, multi-modal, multi-stage, and multi-disease settings, each annotated at the voxel level for 15 abdominal organs. This dataset provides a challenging environment and a testbed for studying robust segmentation algorithms across various targets and scenarios. Furthermore, Yuanfeng et al. benchmarked several state-of-the-art medical segmentation models to evaluate the performance of existing methods on the AMOS dataset.

AMOS 2022³ encompasses two distinct tasks: CT-only abdominal organ segmentation and combined CT/MRI abdominal organ segmentation. The CT-only task is a fundamental routine that evaluates the performance of various segmentation methods across a large and diverse collection of 500 CT scans. The CT/MRI abdominal organ segmentation task incorporates an additional 100 MRI scans, all annotated consistently with the CT scans, to expand the scope of imaging modalities. In this cross-modality context, developing a unified algorithm that can accurately segment abdominal organs from both CT and MRI images is crucial.

CT scans provide excellent resolution and speed, making them ideal for visualizing bone structures and detecting tumors or abnormalities that exhibit high contrast with surrounding tissues. However, they often fall short of providing a clear contrast for soft tissues. On the other hand, while MRI excels in soft tissue contrast and delivers diverse physiological information, it is generally more costly, requires longer scanning times, and is more susceptible to motion artifacts. Relying solely on CT or MRI can lead to incomplete or biased medical interpretations and diagnoses, potentially missing critical information accessible only through alternative modalities. Given the unique characteristics of these imaging modalities, this study aims to develop a versatile image segmentation framework that is highly effective for both CT and MRI scans. This framework seeks to enhance the comprehensiveness and accuracy of medical analyses by leveraging the unique advantages of each technology without requiring extensive modality-specific adjustments. The ultimate goal is to advance the performance of abdominal imaging technology segmentation, thereby contributing to the field by developing a model that performs effectively across different imaging modalities. These improvements will enhance accuracy and utility in clinical settings, ultimately leading to improved patient outcomes.

Methods

This paper proposes a CADSTransN-Net model to tackle the CT/MRI abdominal organ segmentation task in the AMOS 2022 challenge. The CADSTransN-Net synergistically combines the Transformer and N-Net models, leveraging their advantages in processing large receptive fields and precise positioning to enhance the feature extraction capabilities of medical image segmentation. This integration enhances the model's capability for feature extraction in medical image segmentation. Additionally, the CA mechanism and DS technology are incorporated into the CADSTransN-Net structure to improve segmentation accuracy and efficiency. Figure 1 shows the CADSTransN-Net model structure.

Figure 1.

The CADSTransN-Net model structure.

The following provides a detailed description of the various components and functionalities within the CADSTransN-Net model architecture:

Input Layer: The model receives a CT or MRI scan of the abdomen as its input, which includes the anatomical structures that need to be segmented.

Encoder: The encoder features multiple convolutional layers that progressively down-sample the input image. This reduction in spatial dimensions while increasing the depth of the feature map allows the model to capture higher-level features and contextual information from the image.

Transformer layer: The Transformer layer includes linear projection, multi-head self-attention, and a feedforward network. Linear projection converts high-dimensional feature maps into a format suitable for processing in the Transformer layer. The multi-head self-attention mechanism enables the model to simultaneously focus on different parts of the input image, thereby capturing long-range dependencies and interactions between various regions. The feedforward neural network further processes these features after the self-attention mechanism.

CA Module: The CA module dynamically refines the feature map, zeroing in on the most relevant anatomical regions. This precise targeting substantially enhances the model's ability to accurately segment complex structures within the abdomen.

Decoder: The decoder block up-samples the refined feature maps back to their original input dimensions using deconvolution layers or transposed convolutions to enhance the spatial resolution.

Skip Connections: These skip connections bridge encoder and decoder layers, enabling the model to effectively combine low-level details with high-level semantic information, helping to preserve spatial information.

DS Module: The model employs DS by incorporating loss layers at multiple depths within the network, which ensures detailed gradient propagation. This approach fosters the learning of more robust features, consequently accelerating convergence and enhancing overall performance.

Output Layer: The final output of the model is a segmented image, with each voxel or pixel classified into one of 15 distinct abdominal organ categories.

By combining these components, the CADSTransN-Net model effectively leverages the advantages of the Transformer architecture in capturing long-range dependencies and the N-Net architecture in precise localization, as well as CA mechanisms and DS to improve overall segmentation performance. This comprehensive approach enables the model to achieve high-precision segmentation of abdominal organs from CT and MRI scans.

TransN-Net

TransN-Net is a 3D medical image segmentation model built upon an improved version of the classic U-Net architecture. Its core innovation lies in redesigning the traditional symmetrical U-shaped structure into an "N-shaped feature flow path" and integrating a Transformer module at the bottom of the network to capture global contextual information. This architecture fully leverages the strength of CNNs in extracting local features and the capability of Transformers in modeling long-range dependencies. As a result, it enhances the model's ability to perceive complex anatomical structures while maintaining high-resolution detail reconstruction, thereby improving the accuracy and robustness of medical image segmentation.

As illustrated in Figure 2, the network architecture of TransN-Net comprises three main components: an encoder, a Transformer module, and a decoder. The entire network exhibits a topological layout resembling an "N-shaped" path. Compared to the U-shaped structure of the classic U-Net, this design is more flexible, breaks the constraint of structural symmetry, and constructs an asymmetric yet efficient feature interaction mechanism that flows from shallow to deep and then back to shallow layers.

Figure 2.

Structure of the TransN-Net model.

Feature flow and the formation mechanism of the N-type structure

TransN-Net takes a three-dimensional medical image $X_{0} \in R^{H \times W \times D}$ as input and gradually constructs feature maps with different semantic levels through successive encoding operations. In the encoding path, each layer typically includes two main operations: three-dimensional convolution (Conv3D) and max pooling (MaxPool), as shown in Equation (1):

x_{i + 1} = M a x p o o l (C o n v 3 D (x_{i})), i = 0, 1, 2, \dots, l - 1

(1)

Where l denotes the number of encoder layers, this process progressively reduces the spatial resolution of the input volume to

1 / 2^{l}

of its original size, while increasing the number of feature channels, thereby constructing a typical multi-scale feature pyramid. This hierarchical representation allows the model to capture a wide range of information, from fine edge details to high-level semantic contours.

Unlike U-Net, which performs symmetric up-sampling directly from the bottom layer, TransN-Net introduces cross-layer branch connections and path fusion mechanisms. The proposed architecture allows features from specific intermediate encoder layers to be transmitted directly to decoder layers with higher resolution, breaking away from the constraint of symmetric skip connections.

The feature flow trajectory in the encoding path forms an N-shaped topology. Features are not only propagated downward for semantic abstraction, but also flow across layers toward different decoding levels. The proposed framework establishes an asymmetric information pathway following the pattern of shallow → deep → shallow. The decoder layers are no longer limited to receiving features from their symmetric encoder counterparts; instead, they aggregate information from multiple encoding layers.

Through the construction of additional connection branches, features from intermediate layers (e.g., the second or third encoder layers) are directly fused into multiple asymmetric positions in the decoder. This multi-path feature fusion enhances semantic information flow and improves the model's perception of multi-scale structures. The N-type structure also avoids the bottleneck of information being funneled exclusively through the network's bottom layer, as seen in the U-Net. By enabling feature jumps directly at intermediate layers, it reduces the risk of gradient vanishing and promotes training stability.

Transformer block

The Transformer model, first proposed by Vaswani et al.¹⁷ in 2017, utilizes a self-attention mechanism that enables the model to process data while considering the internal relationships within the sequence. This structure is exceptionally suited for handling sequential data, such as text or time series. However, its versatility extends to various other data types, including images. The key advantage of multi-head self-attention in Transformer layers over traditional convolutional layers is its ability to capture long-range dependencies in data. Traditional convolutional layers process data using filters that operate on local receptive fields. This localized operation limits their ability to incorporate information from distant parts of the input, unless very deep or extensive networks are used. In contrast, the multi-head self-attention mechanism directly calculates the interaction between any two elements in the input, regardless of their distance in the sequence or spatial arrangement.

The multi-head self-attention mechanism enables a model to attend to different positions within the input sequence, capturing various features from distinct subspaces at different locations. Each ‘head’ in the multi-head self-attention can potentially focus on different aspects of the input data, providing a richer representation of the input by aggregating diverse perspectives across multiple positions and feature subspaces. Referring to Figure 3, let us explore how the Transformer layer functions within this architecture:

Figure 3.

The transformer block.

Hidden Feature Extraction: After the encoder and decoder blocks perform preliminary processing, they linearly project the input image's features into a higher-dimensional space. This transformation prepares the features for the Transformer layer to process effectively.

Multi-Head Self-Attention: The Transformer's attention mechanism is then mapped through N attention mechanisms to obtain N sets of feature vectors, including Q (query vector), K (key vector), and V (value vector). Equations (2)–(4) describe the multi-head attention mechanism.

Q_{h}, K_{h}, V_{h} = p r o j_{Q_{h}} (z), p r o j_{K_{h}} (z), p r o j_{V_{h}} (z)

(2)

attention (Q_{h}, K_{h}, V_{h}) = s o f t m a x (\frac{Q_{h} K_{h}}{\sqrt{d}}) V_{h}

(3)

Z_{M H S A} = c o n c a t (a t t e n t i o n (Q_{1 \sim H}, K_{1 \sim H}, V_{1 \sim H}))

(4)

Where

p r o j_{Q_{h}}

p r o j_{K_{h}}

, and

p r o j_{V_{h}}

represent the linear projection of the Hth head of

Q_{h}

, respectively. d is the dimension of the input z, which is used to compress the correlation score that increases with the feature dimension. Softmax represents the exponential normalization function. attention represents the inner product function of the attention mechanism. Concat represents the concatenation of H discrete attention outputs.

Feed-Forward Network: The feed-forward neural network, composed of linear layers, subsequently processes the output from the attention and normalization stages. This network further transforms the data and integrates non-linearities, which enhance the model's learning capabilities.

Output Mapping: The feed-forward neural network, composed of linear layers, subsequently processes the output from the attention and normalization stages. This network further transforms the data and integrates non-linearities, which enhance the model's learning capabilities.

By employing multi-head self-attention, Transformer models not only learn isolated local patterns but also integrate global information, which traditional convolutional layers cannot efficiently achieve. This Transformer layer structure enables the model to effectively learn complex patterns and dependencies in medical images, which are crucial for accurate segmentation and analysis.

Multi-path connections enhance information propagation

To further strengthen cross-level feature fusion, TransN-Net introduces a multi-path branch connection mechanism that enables features from multiple encoder layers to be directly and concurrently transmitted to any decoder layer, rather than being restricted to symmetric skip connections. This design significantly expands the space for feature aggregation. Each decoder node not only integrates its own up-sampled features but also simultaneously fuses feature representations from several encoder layers. Theoretically, this structure improves the redundancy of information representation and enhances model robustness.

Y_{j} = U p c o n v (Y_{j + 1}) + \sum_{i \in S_{j}} f_{i, j} (X_{i})

(5)

Where:

Y_{j}

is the feature at the

j

-th decoder layer,

Y_{j + 1}

is the output of the next deeper decoder layer,

X_{i}

is the output of the

i

-th encoder layer,

f_{i, j} (\cdot)

denotes the cross-layer fusion projection function (typically a 1 × 1 × 1 convolution or a channel-alignment transformation),

S_{j}

is the set of encoder layers connected to the

j

-th decoder layer.

This formulation can be interpreted as an ordered aggregation operation over a set of features. By applying the cross-layer transformation function $f_{i, j}$ , features from different semantic levels are mapped into a unified representation space before fusion. The multi-path connection mechanism creates shorter gradient feedback routes, effectively reducing the risk of gradient vanishing in deep networks. As a result, each decoder layer can simultaneously access features from shallow layers (rich in spatial details) and deeper layers (containing abstract semantics), enabling joint enhancement of local detail preservation and semantic reconstruction.

This design is particularly advantageous for medical image segmentation tasks, where organs often exhibit blurred boundaries and intricate structural variations. The multi-path fusion strategy helps the network maintain high sensitivity to subtle anatomical details while preserving overall structural integrity.

CA module

The CA module specifically engineers dynamic refinement of feature maps at each stage of the CADSTransN-Net. It achieves this by closely focusing on the region of interest in the abdominal area and learning spatial dependencies to adjust feature responses in each channel. Importantly, this enhancement does not significantly increase computational complexity. These adjustments collectively enhance the model's ability to perform high-accuracy segmentation. Figure 4 shows the CA module used in this paper.

Figure 4.

Ca module structure.

The CA module dynamically adjusts the significance of the network's feature maps by assigning variable attention weights based on the differing information content at each position. The following is the detailed calculation process:

The input feature map X passes through a convolution layer to reduce the channel dimension of the feature map. This convolution layer utilizes a convolution kernel of a specific size to modify the number of channels in the feature map without altering its spatial dimensions. The calculation method is Equation (6):

X^{'} = C o n v 1 (X) = W_{1} \times X

(6)

Among them, $W_{1}$ is the weight of the convolution kernel, its dimension is $(C / R) \times C \times 1 \times 1 \times 1$ . This step reduces the number of channels from C to $C / R$ to decrease computational complexity and model parameters, R is the dimensionality reduction ratio, and the commonly used value is 16.

$X^{'}$ passes through the ReLU activation function. The role of the ReLU activation function is to introduce nonlinearity and help the model learn more complex function mappings. It sets all negative values to 0 and retains all positive values. Equation (7) is the calculation result:

X^{″} = R e l u (X^{'})

(7)

This step ensures that the feature map after the first convolution layer is nonlinear before being passed to the next layer so that the model can capture more complex features.

The number of channels of the feature map is expanded from $C / R$ back to C using another $1 \times 1 \times 1$ convolutional layer. This process generates the attention weights for each position, resulting in Equation (8):

A = C o n v 2 (X^{″}) = W_{2} \times X^{″}

(8)

The dimension of $W_{2}$ is $C \times (C / R) \times 1 \times 1 \times 1$ .

The Sigmoid function converts the convolution result A into a weight value within the range [0, 1]. These values adjust the importance of each element in the original input feature map. Equation (9) illustrates the calculation method:

A^{'} = S i g m o i d (A)

(9)

This step ensures that the generated attention weights are positive and constrained between 0 and 1, suitable for use as weights.

The final step applies the attention map A’ to the original feature map X to adjust the strength of each feature position through element-wise multiplication, also known as the Hadamard product. The calculation method is Equation (10):

Y = X ⊙ A^{^{'}}

(10)

By adjusting the weights of the original feature map, the model focuses on areas considered more important while suppressing the influence of less important ones. This adjustment allows CADSTransN-Net to process input features dynamically, thereby enhancing the model's performance and flexibility.

DS module

The DS module is a technique used in deep learning models, especially in tasks such as medical image segmentation. Its primary purpose is to enhance the learning ability and performance of the model by introducing additional supervised signals at different levels of the model.¹⁸ The CADSTransN-Net model integrates DS operations across three distinct layers of the decoding stage, allowing the model to calculate loss not only in the last layer but also in multiple intermediate layers. Consequently, each intermediate layer possesses its own loss function, enabling direct optimization. Figure 5 shows the DS module. The architecture comprises three main parts: feature extraction, a multi-level supervision unit, and final prediction.

Figure 5.

The DS module. (a) feature extraction, (b) multi-level supervision unit, (c) final prediction.

In the feature extraction stage (Figure 5(a)), the DS module employs average pooling to reduce the dimension of the feature map by calculating the average value of each pixel. This reduction helps decrease computational demands in subsequent layers while retaining essential feature information. Subsequently, the convolutional layer processes the input data using a filter and activates it through the ReLU function, enhancing the network's nonlinear capabilities and improving the training process.

In the multi-level supervision unit (Figure 5(b)), each unit comprises Dropout, ReLU activation, and inner product layers. The Dropout is a regularization technique that helps prevent network overfitting by randomly deactivating some neurons. This random deactivation allows the network to develop more robust feature representations. Following each Dropout, a ReLU activation layer preserves the network's nonlinear characteristics. The inner product layer's output in each DS unit directly compares with the ground truth labels to generate a loss, thereby facilitating optimization at specific network depths. This method ensures that the model fully leverages the features of each level, allowing each layer to contribute to the overall task.

The final prediction unit features an inner product layer (Figure 5(c)). This layer produces the terminal prediction result, aiming at the overall target of the network. The system compares this output with the final ground truth labels of the layer to calculate key performance indicators, such as accuracy and loss. These indicators directly determine the effectiveness of the network training.

Considering that the model utilizes the DS module, the output of each layer employs the same cross-entropy loss function, and the calculation of the total loss must follow Equations (11) and (12).

L = 0.8 L_{m a i n} + 0.2 \sum_{i = 1}^{n} λ_{i} L_{i}

(11)

\sum_{i = 1}^{n} λ_{i} = 1

(12)

$L_{m a i n}$ is the loss of the final output of the network. We set the weight to 0.8 so that the model prioritizes optimizing important feature information during the optimization process, thereby emphasizing the importance of the final output. $L_{i}$ is the loss of the output of the i-th intermediate layer. We set the total weight of the intermediate layer losses to 0.2 to prevent these auxiliary losses from overly affecting the main task's learning process while still utilizing them to promote good feature extraction and gradient flow. $λ_{i}$ is the weight of the i-th layer loss. The $λ_{i}$ before each intermediate layer loss $L_{i}$ should ensure that $\sum_{i = 1}^{n} λ_{i} = 1$ so that the proportion of the total $0.2 \sum_{i = 1}^{n} λ_{i}$ in the total loss remains 0.2.

The DS module enables the CADSTransN-Net model to receive feedback not only at the final output stage but also at the intermediate stages. This multi-level supervision allows the model to utilize information from intermediate layers more effectively during training, thereby optimizing learning efficiency and enhancing model performance. The multi-level supervision module facilitates self-correction at various levels, proving particularly beneficial for solving complex pattern recognition problems. Additionally, the DS module mitigates the issue of vanishing gradients, thus making the training of deep networks more feasible.

Experimental results

Dataset

The AMOS22 dataset is the core part of the Multimodal Abdominal Multi-Organ Segmentation Challenge, which aims to advance and evaluate medical image segmentation technology. Table 1 shows the composition of the AMOS22 data. The dataset comprises 500 CT scans and 100 MRI scans, with the training set, validation set, and test set containing 240, 120, and 240 scans, respectively. Each scan provides voxel-level annotations of 15 different abdominal organs. Including spleen, right kidney, left kidney, gallbladder, esophagus, liver, stomach, aorta, vena cava, pancreas, right adrenal gland, left adrenal gland, duodenum, bladder, prostate/uterus.

Table 1.

AMOS22 data composition.

Dataset	Modality	Number of data	Training sets	Validation sets	Test sets	Categories
AMOS22	CT, MRI	600 (500 CT + 100 MRI)	240	120	240	15

Figure 6 illustrates an example of dataset visualization using ITK-Snap medical image visualization software. Figure 6(a) displays the axial plane image of amos_0001, Figure 6(b) shows the sagittal plane image, Figure 6(c) presents the coronal plane image, and Figure 6(d) features the 3D view. For additional details about AMOS22, please visit the official challenge page at AMOS22 Challenge (https://amos22.grand-challenge.org).

Figure 6.

Example of dataset visualization. (a) axial plane, (b) sagittal plane, (c) coronal plane, (d) 3D view.

We use the loss function to quantify the error between the model's predictions and the actual values, guiding the optimization of the model's parameters. Specifically, we employ cross-entropy loss (CE loss) as the training loss, applying it to the outputs of multiple stages for the DS module.¹⁹ This approach enables the model to learn effective feature representations at each layer, thus enhancing the training process and overall model performance. Equation (13) illustrates the calculation of CE loss, which involves the ground truth label values and the predicted probabilities for the segmentation targets. CEloss is not only simple to implement but also offers high computational efficiency.

C E l o s s = - \sum_{c = 1}^{C} p_{c} l o g q_{c}

(13)

Where C represents the total number of categories,

p_{c}

denotes the ground truth distribution for observation category C, typically represented by one-hot encoding, and the model assigns

q_{c}

as the predicted probability that the current output belongs to category C.

Experimental results

The Amos22 Challenge evaluates the performance of models using two metrics: the DSC and NSD. In addition to DSC and NSD, this study further adopts four clinically relevant metrics—HD (95%), ASD, AVD, and RVD to comprehensively evaluate segmentation performance from boundary accuracy and volume consistency perspectives, which are critical for clinical applicability in abdominal multi-organ segmentation.

The DSC is a statistical measure that quantifies the similarity between two sample sets and is widely used in medical image segmentation to assess the accuracy of segmentation results. Equation (14) shows the calculation method for DSC, where X and Y represent the two sample sets, and |X| and |Y| denote the number of elements in each set.

DSC = \frac{2 | X \cap Y |}{| X | + | Y |}

(14)

NSD refers to the degree of boundary coincidence within a given error range and is used to evaluate the results of boundary-based segmentation. The NSD metric reports the good surface portion compared to the total surface (the sum of the predicted surface area and the ground truth surface area). A higher value represents high performance. Equation (15) presents the calculation method for NSD.

N S D (X, Y) = \frac{| B_{\partial X} (τ) \cap \partial Y | + | B_{\partial Y} (τ) \cap \partial X |}{| \partial X | + | \partial Y |}

(15)

$B_{\partial X} (τ)$ and $B_{\partial Y} (τ)$ are the boundary areas around the ground truth X and the segmentation surface Y, respectively, and τ is the tolerance defined according to clinical requirements or consistency between radiologists. In this paper, we set τ = 1 mm, and the ground truth and segmentation surface boundaries are each extended by 1 mm when calculating NSD. We will consider two boundaries consistent if they overlap within 1 mm. Such a setting allows for a certain amount of error during evaluation, which enables a more fair and practical assessment of the segmentation algorithm's performance, as perfect boundary alignment is often difficult to achieve in practice. $\partial X$ and $\partial Y$ represent the ground truth and segmentation boundaries extracted from the image segmentation results. In our experiments, we adhere to the standard nnU-Net framework and employ a sliding window method with a step size of 0.5. This method ensures comprehensive coverage of the input volume, allowing the model to utilize all available information and enhance evaluation robustness.

HD (95%) quantifies the maximum boundary deviation between the predicted segmentation and ground truth, while excluding the top 5% of extreme outlier distances. This metric avoids overestimation caused by individual abnormal boundary points, making it more robust and clinically meaningful than the full Hausdorff Distance—especially for small organs (e.g., gallbladder, adrenal glands) where local segmentation errors may otherwise distort evaluation results. Equation (16) presents the calculation method for HD (95%).

\begin{aligned} H D (95) (X, Y) = p e r c e n t i l e (m a x {s u p_{x \in ϑ X} i n f_{y \in ϑ Y} | | x - y | |_{2}, \\ s u p_{y \in ϑ Y} i n f_{x \in ϑ X} | | x - y | |_{2}}, 95) \end{aligned}

(16)

$ϑ X$ : Organ boundaries extracted from the real labeled area; $ϑ Y$ : The organ boundary extracted from the predicted segmentation region (consistent with the boundary definition in NSD); $| | x - y | |_{2}$ : The Euclidean distance between the boundary point x (belonging to $ϑ X$ ) and the boundary point y (belonging to $ϑ Y$ ); $s u p_{x \in ϑ X} i n f_{y \in ϑ Y} | | x - y | |_{2}$ : Forward Hausdorff distance, that is, the maximum value among the minimum distances from all points on the true boundary to the predicted boundary; $s u p_{y \in ϑ Y} i n f_{x \in ϑ X} | | x - y | |_{2}$ : Backward Hausdorff distance, that is, the maximum value among the minimum distances from all points on the predicted boundary to the true boundary; $p e r c e n t i l e (\cdot, 95)$ : Take the 95th percentile of the set of “maximum values of forward and backward Hausdorff distances”, eliminate the 5% extreme outliers, and ensure the robustness of the evaluation; Unit: Millimeters (mm). The smaller the value, the higher the boundary alignment accuracy.

ASD complements HD (95%) by measuring the average of minimum Euclidean distances between all corresponding boundary points of the ground truth and predicted segmentation. It reflects the overall smoothness and consistency of the segmentation boundary, rather than focusing on extreme values, making it suitable for evaluating the overall quality of large organs (e.g., liver, spleen) with extensive boundary surfaces. Equation (17) presents the calculation method for ASD.

\begin{aligned} A S D (X, Y) = & \frac{1}{| ϑ X | + | ϑ Y |} (\sum_{x \in ϑ X} i n f_{y \in ϑ Y} | | x - y | |_{2} \\ + \sum_{y \in ϑ Y} i n f_{x \in ϑ X} | | x - y | |_{2}) \end{aligned}

(17)

$| ϑ X |$ : Discrete number of points on the true boundary $ϑ X$ ; $| ϑ Y |$ : Predict the discrete points on the boundary $ϑ Y$ (extracted using the sliding window method consistent with NSD, with a step size of 0.5 to ensure uniform point density); $\sum_{x \in ϑ X} i n f_{y \in ϑ Y} | | x - y | |_{2}$ : The sum of the minimum distances from all points on the true boundary to the predicted boundary; $\sum_{y \in ϑ Y} i n f_{x \in ϑ X} | | x - y | |_{2}$ : The sum of the minimum distances from all points on the predicted boundary to the true boundary; Unit: Millimeters, the smaller the value, the higher the overall matching degree of the boundary.

AVD quantifies the actual numerical difference between the volume of the predicted segmentation region and the ground truth organ volume. It directly reflects the accuracy of the model in reconstructing the spatial size of abdominal organs, which is essential for clinical tasks such as evaluating organ atrophy or enlargement. Equation (18) presents the calculation method for AVD.

A V D (X, Y) = | V_{p r e d} (Y) - V_{g t} (X) |

(18)

$V_{g t} (X)$ denotes the ground truth organ volume, calculated as $V_{g t} (X) = | X | \times v$ , where |X| is the number of voxels in the ground truth region X, and v is the volume of a single voxel (derived from the AMOS22 dataset's DICOM metadata: $v = d_{x} \times d_{y} \times d_{z}$ , with $d_{x}, d_{y}, d_{z}$ representing the voxel spacing in the x, y, z directions, respectively; $V_{p r e d} (Y)$ denotes the predicted organ volume, calculated identically to $V_{g t} (X)$ using the number of voxels in the predicted region Y (|Y|); The absolute value |·| ensures AVD is a non-negative metric, avoiding cancellation of positive and negative volume deviations. AVD is expressed in cubic millimeters (mm³), with smaller values indicating closer alignment between the predicted and actual organ volumes.

RVD normalizes AVD by the ground truth organ volume, converting the absolute volume difference into a percentage of the ground truth organ volume. This metric eliminates the impact of organ size differences. AVD may be acceptable for the liver (a large organ), but excessive for the gallbladder (a small organ)—making it suitable for comparing volume accuracy across different abdominal organs. Equation (19) presents the calculation method for RVD.

R V D (X, Y) = \frac{| V_{p r e d} (Y) - V_{g t} (X) |}{V_{g t} (X)} \times 100 %

(19)

$V_{p r e d} (Y)$ and $V_{g t} (X)$ retain the same volume definitions as in AVD; Division by $V_{g t} (X)$ normalizes the absolute difference to a proportion relative to the true organ size; Multiplication by 100% converts the proportion to a percentage for intuitive interpretation. Notably, if $V_{g t} (X) = 0$ (i.e., the organ is absent in the ground truth), RVD is marked as “not applicable (NA)” to avoid division by zero. In all other cases, smaller RVD values (typically < 5% for clinical acceptability) indicate higher relative volume accuracy.

We tested the performance using the same training, validation, and test sets, and the data-splitting strategy is identical to that used in the AMOS22 competition. We trained all models on a computer equipped with an NVIDIA GeForce RTX 4090 D 24GB GPU. For training parameters, we set the initial learning rate to 0.001 (applied uniformly across all models) and adopted a ReduceLROnPlateau schedule. Suppose the validation Dice score does not improve for 30 consecutive epochs. We reduce the learning rate by a factor of 0.2 (with a threshold of 1e-3 to prevent trivial adjustments) until it drops below 1e-6 or we have completed the total number of epochs (set to 10,000). We used the Adam optimizer for all models, configuring it with default momentum parameters (β₁ = 0.9, β₂ = 0.999, and ε = 1e-8) and a weight decay of 3e-5 for L2 regularization. We applied gradient clipping with a maximum norm of 12 to prevent gradient explosion. We set the batch size to 2 and implemented complementary regularization strategies: data augmentation (3D rotation within [−30°, 30°], scaling in [0.7, 1.4], symmetric mirroring with 0.5 probability, and disabled elastic deformation to avoid anatomical noise) and dropout with a probability of 0 (to retain fine-grained anatomical features), paired with L2 regularization to suppress overfitting.

Table 2 focuses on validating the necessity of the N-shaped feature flow path. We replaced the original N-shaped flow (derived from N-Net) with three mainstream feature flow architectures: U-Net (symmetric encoder-decoder with skip connections), ResU-Net (U-Net integrated with residual blocks), and DenseU-Net (U-Net with dense connection blocks). The experiments were conducted under the same experimental settings (using the full AMOS22 dataset, consistent training hyperparameters, and evaluation metrics of DSC and NSD).

Table 2.

Segmentation results of U-Net, ResU-Net, DenseU-Net, and N-Net models.

Model	U-Net		ResU-Net		DenseU-Net		N-Net
Metric	DSC	NSD	DSC	NSD	DSC	NSD	DSC	NSD
Spleen	96.20	87.61	96.49	88.01	96.02	87.66	96.81	89.88
R.Kid	95.85	88.36	95.78	88.03	95.81	87.88	96.66	89.27
L.Kid	95.63	87.1	95.04	86.72	95.62	86.93	96.63	89.41
Gall	84.18	79	77.17	72.82	89.61	84.41	83.40	77.08
Eso	82.86	79.34	82.46	78.83	83.63	81.36	86.43	80.49
Liver	97.19	86.86	97.31	87.53	97.65	87.82	97.25	86.53
Stom	91.75	82.26	93.71	84.88	92.47	82.91	92.47	81.93
Aorta	94.33	84.22	94.51	83.51	94.88	84.64	95.60	87.20
Pos	86.27	71.6	87.37	72.85	88.26	72.46	91.13	80.74
Panc	85.20	80.18	85.99	81.28	86.36	80.54	88.51	80.52
RAG	74.74	78.58	76.33	79.13	77.32	90.53	77.91	79.88
LAG	76.53	79.12	78.46	79.86	79.12	80.69	80.52	80.46
Duo	78.29	72.14	78.21	72.07	79.50	73.26	83.62	74.53
Blad	85.53	82.12	87.29	82.95	93.29	87.03	91.88	88.20
Pros	70.64	64.32	72.98	64.91	56.23	51.08	85.85	75.24
average	86.35	80.19	86.61	80.23	87.05	81.28	89.64	82.76

Exegesis: Spleen - spleen, R.Kid - right kidney, L.Kid - left kidney, Gall - gallbladder, Eso - esophagus, Liver - liver, Stom - stomach, Aorta - aorta, Pos - postcava, Panc - pancreas, RAG - right adrenal gland, LAG - left adrenal gland, Duo - duodenum, Blad - bladder, Pros - prostate/uterus.

The N-Net model outperforms U-Net, ResU-Net, and DenseU-Net across overall performance metrics, achieving the highest average Dice Similarity Coefficient (DSC) of 89.64 and average Normalized Surface Dice (NSD) of 82.76, which are 2.59 and 1.48 percentage points higher than DenseU-Net (the second-best model), respectively, and significantly superior to U-Net (+3.29 DSC, +2.57 NSD) and ResU-Net (+3.03 DSC, +2.53 NSD), validating the effectiveness of the N-shaped feature flow path design in multi-organ segmentation tasks. N-Net shows remarkable advantages in segmenting several challenging organs, particularly the prostate (Pros), pancreas (Panc), and esophagus (Eso); the most significant improvement is observed in prostate segmentation, with DSC increasing by 12.87 percentage points and NSD by 10.33 percentage points compared to ResU-Net (the best-performing comparative model), demonstrating that N-Net's feature fusion strategy substantially enhances segmentation accuracy for small organs and those with blurred boundaries. A special case is the gallbladder (Gall) segmentation, where DenseU-Net achieves the best performance (DSC 89.61, NSD 84.41). At the same time, N-Net ranks third (DSC 83.40), possibly due to the gallbladder's small volume and susceptibility to interference from surrounding tissues in images, suggesting varying adaptability of different feature flow paths to specific organs. For larger organs with relatively clear boundaries, such as the liver (Liver) and spleen (Spleen), all four models achieve high DSC (>95%). However, N-Net still maintains a slight advantage, especially in spleen segmentation, where its NSD is 1.87 percentage points higher than the second-ranked ResU-Net, indicating its superiority in predicting organ surface contours. DenseU-Net performs second-best in most organs and ranks first in bladder (Blad) segmentation with a DSC of 93.29, highlighting the advantage of dense connection structures in predicting regional consistency for specific organs; ResU-Net excels in stomach (Stom) segmentation (DSC 93.71), reflecting the effectiveness of residual connections in capturing boundaries of complex structures; U-Net, as the baseline model, shows stable but unremarkable overall performance. In contrast, N-Net outperforms other models in segmenting most organs (11/15) and particularly excels in improving the accuracy of hard-to-segment organs, confirming the universality and effectiveness of its feature flow design (Table 3).

Table 3.

Segmentation results of N-Net, TransN-Net, TransN-Net + CA, TransN-Net + DS, and CADSTransN-Net models.

Model	N-Net		TransN-Net		TransN-Net + CA		TransN-Net + DS		CADSTransN-Net
Metric	DSC	NSD	DSC	NSD	DSC	NSD	DSC	NSD	DSC	NSD
Spleen	96.81	89.88	97.00	89.99	97.26	90.62	97.34	90.74	97 . 52	91 . 47
R.Kid	96.66	89.27	96.51	89.32	96.73	89.66	96.88	89.92	97 . 00	90 . 27
L.Kid	96.63	89.41	96.62	89.55	96.66	89.77	96.77	89.84	96 . 96	90 . 31
Gall	83.40	77.08	83.40	77.39	84.90	78.88	85.24	79.22	85 . 64	80 . 23
Eso	86.43	80.49	86.25	80.61	86.94	81.58	87.01	81.96	87 . 68	82 . 68
Liver	97.25	86.53	97.43	86.90	97.71	88.11	97.78	88.16	97 . 96	89 . 13
Stom	92.47	81.93	92.24	82.22	92.60	83.20	92.65	83.28	93 . 51	84 . 62
Aorta	95.60	87.20	95.72	87.57	95.87	88.01	95.94	88.31	96 . 07	88 . 76
Pos	91.13	80.74	92.15	81.10	92.44	81.61	92.61	81.98	92 . 89	82 . 68
Panc	88.51	80.52	88.57	81.17	89.31	82.15	89.33	82.15	89 . 61	82 . 77
RAG	77.91	79.88	78.90	80.49	79.61	81.27	79.55	81.08	80 . 42	81 . 96
LAG	80.52	80.46	80.67	81.02	81.40	81.71	81.02	81.31	82 . 17	82 . 54
Duo	83.62	74.53	83.52	75.18	84.47	76.34	84.61	76.58	84 . 27	76 . 62
Blad	91.88	88.20	91.53	88.73	92.26	89.44	91.49	88.73	92 . 73	90 . 19
Pros	85.85	75.24	87.62	81.16	86.15	80.69	86.41	78.96	86 . 93	80 . 83
average	89.64	82.76	89.74	83.09	90.28	84.19	90.31	84.12	90 . 76	85 . 00

N-Net has demonstrated strong capabilities in abdominal CT/MRI segmentation, achieving a DSC score of 89.64 and an NSD score of 82.76, thereby affirming its prominence in medical image segmentation. The TransN-Net model combines the strengths of the Transformer and N-Net architectures, providing enhanced feature extraction capabilities. This combination effectively manages the contextual information essential for spatial hierarchies and detailed organ segmentation. As a result, it surpasses N-Net's performance in abdominal CT/MRI segmentation, obtaining a DSC score of 89.74 and an NSD score of 83.09. The CA module dynamically optimizes the feature map by prioritizing relevant anatomical regions, focusing on critical areas while reducing the impact of less pertinent ones. Integrating this module into TransN-Net enhances performance and flexibility, achieving a DSC score of 90.28 and an NSD score of 84.19. The DS module setting ensures that the model receives feedback at the final output stage and throughout various intermediate stages. This approach optimizes the use of information from intermediate layers, enhancing overall learning efficiency and model performance. The TransN-Net + DS module achieves a DSC score of 90.31 and an NSD score of 84.12. CADSTransN-Net utilizes both the CA and the DS modules, achieving a DSC score of 90.76 and an NSD score of 85.00 in abdominal CT/MRI segmentation. Additionally, it excels in organ segmentation, establishing CADSTransN-Net as a promising solution for abdominal organ segmentation.

Table 4 presents a comprehensive comparison of core performance metrics for eight segmentation models on the AMOS22 dataset, with transparent performance gradients observed as model complexity and improved modules increase. Among the basic CNN models, U-Net (the foundational model) shows the lowest performance across all metrics (DSC = 86.35%, HD(95%) = 5.59 mm, RVD = 2.19%), revealing limitations in accurate boundary capture and volume estimation for multi-organ segmentation; ResU-Net, which incorporates residual connections, outperforms U-Net with a 0.26-percentage-point increase in DSC, a 0.39 mm reduction in HD(95%), and a 2512.24 mm³ decrease in AVD, demonstrating that residual connections effectively alleviate gradient vanishing and enhance feature reuse. DenseU-Net, leveraging dense connections, further improves performance to 87.05% DSC, a 4.02 mm HD (95%) (the best among basic models), and 2.09% RVD, validating its advantage in segmenting the acceptable boundaries of small organs, such as the gallbladder and adrenal glands. Moving to advanced models, N-Net achieves a significant performance leap compared to DenseU-Net, with a 2.59-percentage-point increase in DSC (from 87.05% to 89.64%), a 1.48-percentage-point increase in NSD (from 81.28% to 82.76%), and a 3611.78 mm³ reduction in AVD, likely due to its optimized feature fusion modules (e.g., cross-scale feature concatenation) that address the challenge of significant multi-organ scale differences. TransN-Net, which adds Transformer attention to N-Net, only sees a 0.1-percentage-point DSC increase (from 89.64% to 89.74%) but a rise in HD(95%) to 5.56 mm and RVD to 2.22%, possibly because Transformers struggle with capturing local features of small organs (attention weights tend to focus on large organs, leading to boundary deviations in small ones). Among the fused improved models, TransN-Net + CA (with channel attention) outperforms TransN-Net with a 0.54-percentage-point DSC increase (from 89.74% to 90.28%), a 1.27 mm HD(95%) reduction (from 5.56 mm to 4.29 mm), and a 1.81% RVD, showing that channel attention can precisely focus on organ-specific feature channels (e.g., grayscale difference channels between the liver and spleen) to correct boundary deviations; TransN-Net + DS (with dense dilated convolution) reaches 90.31% DSC, a minimal 0.78 mm ASD, and 49,101.66 mm³ AVD, confirming the advantage of dilated convolution in expanding the receptive field while preserving resolution, making it suitable for complete segmentation of large organs like the liver and stomach. CADSTransN-Net, a multi-module fused model, achieves the best performance across all metrics (90.76% DSC, 85.00% NSD, 3.98 mm HD(95%), 1.53% RVD), proving that its "Transformer + channel attention + dense dilated convolution" architecture meets the multi-organ segmentation requirements of complete large-organ segmentation, accurate small-organ boundary capture, and precise volume estimation.

Table 4.

Comparison of core performance metrics of different segmentation models in AMOS22.

Model	DSC	NSD	HD(95%)	ASD	AVD	RVD
U-Net	86.35	80.19	5.59mm	1.00 mm	62,510.00 mm³	2.19%
ResU-Net	86.61	80.23	5.20 mm	0.97 mm	59,997.76 mm³	2.11%
DenseU-Net	87.05	81.28	4.02 mm	0.77 mm	58,995.70 mm³	2.09%
N-Net	89.64	82.76	4.97 mm	0.93 mm	55,383.92 mm³	2.03%
TransN-Net	89.74	83.09	5.56 mm	0.99 mm	60,075.10 mm³	2.22%
TransN-Net + CA	90.28	84.19	4.29 mm	0.82 mm	47,802.88 mm³	1.81%
TransN-Net + DS	90.31	84.12	4.05 mm	0.78 mm	49,101.66 mm³	1.88%
CADSTransN-Net	90.76	85.00	3.98 mm	0.75 mm	39,755.88 mm³	1.53%

Figure 7 provides a detailed visualization of the segmentation results for amos_0008 using ITK-Snap medical image visualization software. The first row displays the ground truth, while each subsequent row showcases the segmentation outcomes from different models.

Figure 7.

Ground truth and prediction results of amos_0008. (a) axial plane, (b) sagittal plane, (c) coronal plane, (d) 3D view.

The first image in each row of Figure 7 depicts the 79th axial plane of amos_0008, allowing viewers to examine the segmentation across different models: N-Net, TransN-Net, TransN-Net + CA, TransN-Net + DS, and the fully integrated CADSTransN-Net. These models are shown from the second row onwards, illustrating incremental improvements in segmentation accuracy.

Figure 7(b) extends the comparison by displaying the 386th sagittal plane. This visualization highlights how each model manages complex anatomical structures when viewed from this specific orientation.

Similarly, Figure 7(c) presents the 401st coronal plane. It further showcases the models’ abilities to handle intricate anatomical details from a different perspective.

Finally, Figure 7(d) presents a 3D view of the segmented organs, offering a holistic and spatial understanding of the model's performance. This 3D representation effectively highlights the depth and precision of organ delineation for each model, with the CADSTransN-Net demonstrating significantly improved accuracy and detail, particularly when directly compared to the conventional N-Net model.

This layered visual analysis not only showcases the specific capabilities of each segmentation approach but also underscores the progressive enhancements achieved by incorporating CA and DS techniques in the TransN-Net architecture, culminating in the advanced performance of the CADSTransN-Net model.

Tables 2 –4 demonstrate the excellent performance of CADSTransN-Net in abdominal CT/MRI segmentation, achieving a DSC of 90.76 and an NSD score of 85.00. These metrics highlight the high accuracy and precision of CADSTransN-Net in delineating individual organs in abdominal images. Furthermore, Figure 7 vividly illustrates the ability of CADSTransN-Net to accurately segment each organ class, providing clear visual evidence of its effectiveness. These results confirm the usefulness of CADSTransN-Net in complex medical imaging tasks and demonstrate its potential to significantly enhance the diagnostic process by providing precise anatomical details.

Discussion

Table 5 shows a detailed comparison of CADSTransN-Net and state-of-the-art methods in 15 categories of abdominal CT/MRI segmentation results, with two decimal places for each result. Amos22 ranked all participants, with yushi13 winning first place in the CT/MRI segmentation challenge, achieving a DSC score of 89.72 and an NSD score of 81.50.³ Details of the segmentation results are available on the Amos22 official leaderboard (https://amos22.grand-challenge.org/evaluation/amos-ctmri-regular-evaluation/leaderboard). The CADSTransN-Net, as proposed in this paper, surpassed yushi13 in most organ segmentation categories, achieving higher DSC and NSD scores. These results demonstrate the superior segmentation capability of CADSTransN-Net.

Table 5.

Comparison with state-of-the-art methods.

Model	CADSTransN-Net		1st³		GMIM²⁰		A-Eval²¹		3D UX-N ET²²
metric	DSC	NSD	DSC	NSD	DSC	NSD	DSC	NSD	DSC	NSD
Spleen	97.52	91.47	96.88	91.28	73.34	80.63	92.28	94.37	97.00	/
R.Kid	97.00	90.27	96.3	90.23	73.6	83.91	90	94.07	96.70	/
L.Kid	96.96	90.31	96.99	90.98	73.6	83.91	89.9	92.8	96.10	/
Gall.	85.64	80.23	85.79	76.59	51.42	58.71	79.97	81.56	92.30	/
Eso.	87.68	82.68	85.83	79.61	38.72	49.73	81.72	93.03	83.20	/
Liver	97.96	89.13	97.96	86.50	78.9	71.53	95.59	96.9	98.40	/
Stom.	93.51	84.62	94	79.79	52.1	64.59	90.57	94.42	92.00	/
Aorta	96.07	88.76	95.39	91.33	74.86	80.49	/	/	95.10	/
Pos	92.89	82.68	91.23	81.01	64.88	80.41	/	/	91.40	/
Panc.	89.61	82.77	89.52	74.94	52.36	74.78	82.04	93.06	85.60	/
RAG	80.42	81.96	77.06	81.77	37.16	74.87	/	/	82.50	/
LAG	82.17	82.54	80.24	82.68	37.16	74.87	/	/	73.90	/
Duo.	84.27	76.62	84.41	71.93	45.24	68.7	/	/	85.30	/
Blad.	92.73	90.19	91.58	79.16	55.34	77.12	/	/	90.60	/
Pros.	86.93	80.83	83.38	63.05	54.22	62	/	/	87.60	/
average	90.76	85.00	89.72	81.50	57.86	71.32	87.66	92.53	90.00	/

Qi et al.²⁰ developed a Siamese framework for GMIM, featuring an online branch and a target branch, to guide the network in learning correlations between organs and tissues by reconstructing the original image from partial observations. The online branch utilizes an adaptive hierarchical masking strategy to identify boundaries or areas with minor contextual changes within the image, and it learns high-level semantic representations from deeper layers of the multi-scale encoder. Meanwhile, the target branch supports contrastive learning by providing representations that help reduce redundancy. GMIM standardizes identification by collectively treating the left and right kidneys as a single kidney and both LAG and RAG as AG. This method achieved a DSC score of 57.86 and an NSD score of 71.32. Unlike GMIM, which primarily uses a hierarchical masking strategy for identification and redundancy reduction, CADSTransN-Net employs a CA mechanism and DS that directly enhance the model's focus on relevant anatomical features and facilitate gradient propagation at multiple network depths. These features significantly contribute to the model's superior DSC and NSD scores, where CADSTransN-Net achieves a DSC of 90.76 and an NSD of 85.00, surpassing GMIM's performance.

Huang et al.²¹ developed the A-Eval benchmark, a cross-dataset evaluation tool for abdominal multi-organ segmentation across five different datasets. This benchmark evaluates the generalizability of various models across diverse data usage scenarios, including independent training on a single dataset, leveraging unlabeled data via pseudo-labeling, combining different modalities, and joint training on all available datasets. Additionally, the study examined the impact of model size on cross-dataset generalization, highlighting the importance of effective data utilization in enhancing model generalizability. This research provides valuable insights into assembling large-scale datasets and refining training strategies. Ultimately, their efforts yielded segmentation results for 9 out of the 15 abdominal organs, achieving a DSC of 87.66 and an NSD score of 92.53. While their approach underscores the importance of model scalability and data diversity, CADSTransN-Net extends these principles by integrating multiple advanced architectural elements that not only handle data diversity efficiently but also ensure high precision in segmentations across various imaging conditions. This results in CADSTransN-Net's robust performance across different organs.

Lee et al.²² discovered that the 3D Medical VIT (SwinUNETR) achieved state-of-the-art performance across several 3D volumetric data benchmarks, including 3D medical image segmentation. The approach combines Layered Transformers, such as the Swin Transformer, with elements from convolutional neural networks, significantly improving the practical feasibility of volumetric segmentation in 3D medical datasets. The success of this hybrid model is mainly due to the extensive receptive field provided by non-local self-attention and the high number of model parameters. They developed a lightweight volumetric convolutional neural network, 3D UX-Net, which integrates convolutional layers into a hierarchical Transformer framework to enhance the robustness of volume segmentation. Specifically, they revisited volumetric depthwise convolutions with large kernel sizes (starting from 7 × 7 × 7), inspired by the Swin Transformer, to expand the global receptive field. By replacing the Multi-Layer Perceptron (MLP) in the Swin Transformer block with point-wise depthwise convolutions and reducing the number of normalization and activation layers, 3D UX-Net not only surpassed the top score previously held by yushi13 in the CT/MRI segmentation challenge but also achieved a DSC score of 90.00, with no NSD score reported. Their model, while effective in providing an extensive receptive field through non-local self-attention, contrasts with the CADSTransN-Net, which leverages a specialized blend of Transformer layers for global contextual awareness and N-Net architectures for precise localization. Additionally, CADSTransN-Net's inclusion of a CA mechanism uniquely refines feature maps to focus more intensively on critical structures, leading to an improvement in segmentation accuracy, evidenced by surpassing the previously high scores in both DSC and NSD.

CADSTransN-Net integrates the robust feature extraction capabilities of the N-Net architecture with the global contextual awareness of Transformer layers, and CA mechanisms and DS strategies further enhance this integration. Unlike traditional models that rely heavily on local contextual information, the Transformer layers in CADSTransN-Net process image data with an emphasis on long-range dependencies—this enables the model to integrate information across the entire image. This capability is particularly beneficial for medical imaging, where understanding the relationship between adjacent anatomical structures is crucial.

The CA mechanism introduces a novel dimension to the model, dynamically focusing on anatomically significant areas and refining feature maps more effectively than traditional attention mechanisms. This targeted approach enables CADSTransN-Net to improve segmentation precision, particularly in regions with complex morphology or closely packed organs. It stands in sharp contrast to models that apply uniform attention across all features—such models can dilute focus on the critical regions required for accurate medical analysis.

Furthermore, the DS modules introduced into the model architecture enhance learning from the network's earliest layers. By implementing loss functions at multiple depths, CADSTransN-Net corrects errors not only at the output layer but throughout the entire network, resulting in a more robust and error-resilient model. This method addresses a key limitation of traditional approaches, which often struggle with gradient vanishing or exploding—especially in deep neural networks.

Experimental evaluations underscore the superior performance of CADSTransN-Net, which achieves high scores in both DSC and NSD. These results reflect the model's effectiveness in handling the variability inherent in medical images, such as differences in organ size, shape, and location across patients. CADSTransN-Net's ability to outperform traditional segmentation methods highlights its potential to enhance diagnostic accuracy and efficiency in clinical settings.

In conclusion, the CADSTransN-Net model not only advances the technological capabilities of segmentation algorithms but also aligns closely with clinical needs by providing a versatile, robust, and efficient tool for medical image analysis. Its development marks a considerable advancement in applying deep learning to healthcare, offering significant improvements in patient care and operational efficiency in medical diagnostics.

Conclusion

This paper proposes CADSTransN-Net, an innovative model that fuses Transformer and N-Net architectures, designed explicitly for abdominal organ segmentation from CT and MRI scans. To address key challenges in medical image segmentation, the model integrates enhanced Coordinate Attention mechanisms and incorporates a Deep Supervision module—two core design optimizations that collectively elevate the precision of abdominal organ delineation. Empirical results demonstrate that CADSTransN-Net outperforms conventional methods in capturing complex anatomical structures, validating the effectiveness of its architecture fusion and module enhancements. Beyond technical advancements, the model exhibits significant potential for practical clinical integration, as its reliable segmentation performance can support downstream clinical workflows, such as preoperative planning and postoperative evaluation, by providing accurate anatomical references. Looking forward, CADSTransN-Net offers a promising direction for future research in medical imaging technology—for instance, it can serve as a baseline framework to explore multi-modal data fusion, or be extended to segment more complex anatomical regions, further bridging the gap between AI-driven segmentation and clinical practice.

Footnotes

Ethical considerations

Not applicable.

Informed consent statement

Not applicable.

Author contributions

Data curation, GW and YT; Formal analysis, LM; Investigation, TM; Methodology, PS; Validation, ZC.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported in part by the Guangxi Natural Science Foundation (2026GXNSFBA00640003), Guangxi Key Laboratory of Automatic Detecting Technology and Instruments (YQ26102), Middle-aged and Young Teachers’ Basic Ability Promotion Project of Guangxi (2025KY0258), Guilin University of Electronic Technology Scientific Research Fund Project (UF24014Y), National Natural Science Foundation of China (62263006), 2021 Director's Fund of the Guangxi Key Laboratory for Automatic Detection Technology and Instruments (YQ21107).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

References

Barat

Pellat

Hoeffel

, et al. CT And MRI of abdominal cancers: current trends and perspectives in the era of radiomics and artificial intelligence[J]. Jpn J Radiol 2024; 42: 246–260.

Ching

Himmelstein

Beaulieu-Jones

, et al. Opportunities and obstacles for deep learning in biology and medicine[J]. J R Soc Interface 2018; 15: 20170387.

Multi-Modality abdominal multi-organ segmentation challenge 2022: https://amos22.grand-challenge.org.

Zhang

Shen

Jiao

. Segment anything model for medical image segmentation: current applications and future directions[J]. Comput Biol Med 2024;171: 108238.

Lambert

Forbes

Doyle

, et al. Trustworthy clinical AI solutions: a unified review of uncertainty quantification in deep learning models for medical image analysis[J]. Artif Intell Med 2024;150: 102830.

Yuan

Zhang

Fang

. An effective CNN and transformer complementary network for medical image segmentation[J]. Pattern Recognit 2023; 136: 109228.

Xiao

Liu

, et al. Transformers in medical image segmentation: a review[J]. Biomed Signal Process Control 2023; 84: 104791.

Jiang

Huang

, et al. FDB-Net: fusion double branch network combining CNN and transformer for medical image segmentation[J]. J Xray Sci Technol 2024; 32: 931–951.

Ronneberger

Fischer

Brox

. U-net: convolutional networks for biomedical image segmentation[C]. In: Medical Image Computing and Computer-assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18, 2015, pp. 234–241. Springer International Publishing.

10.

Amjad

Thill

, et al. Deep learning auto-segmentation on multi-sequence magnetic resonance images for upper abdominal organs[J]. Front Oncol 2023; 13: 1209558.

11.

Zhang

, et al. Abdomenct-1k: is abdominal organ segmentation a solved problem?[J]. IEEE Trans Pattern Anal Mach Intell 2021; 44: 6695–6714.

12.

Gibson

Giganti

, et al. Automatic multi-organ segmentation on abdominal CT with dense V-networks[J]. IEEE Trans Med Imaging 2018; 37: 1822–1834.

13.

Shen

Wang

, et al. Multi-organ segmentation network for abdominal CT images based on spatial attention and deformable convolution[J]. Expert Syst Appl 2023; 211: 118625.

14.

Irshad

Gomes

DPS

Kim

. Improved abdominal multi-organ segmentation via 3D boundary-constrained deep neural networks[J]. IEEE Access 2023; 11: 35097–35110.

15.

Chen

, et al. A novel one-to-multiple unsupervised domain adaptation framework for abdominal organ segmentation[J]. Med Image Anal 2023; 88: 102873.

16.

Bai

, et al. Amos: a large-scale abdominal multi-organ benchmark for versatile medical image segmentation[J]. Adv Neural Inf Process Syst 2022; 35: 36722–36732.

17.

Vaswani

Shazeer

Parmar

, et al. Attention is all you need[J]. arXiv e-prints 2017; arXiv: 1706.03762.

18.

Wang

Lee

, et al. Training deeper convolutional networks with deep supervision[J]. arXiv preprint arXiv:1505.02496, 2015.

19.

Mao

Mohri

Zhong

. Cross-entropy loss functions: theoretical analysis and applications[C]. In: International Conference on Machine Learning. PMLR, 2023, pp. 23803–23828.

20.

Jiang

Shi

, et al. GMIM: self-supervised pre-training for 3D medical image segmentation with adaptive and hierarchical masked image modeling[J]. Comput Biol Med 2024; 176: 108547.

21.

Huang

Deng

, et al. A-Eval: A Benchmark for Cross-Dataset Evaluation of Abdominal Multi-Organ Segmentation[J]. arXiv preprint arXiv:2309.03906, 2023.

22.

Lee

H H

Bao

Huo

, et al. 3d ux-net: A large kernel volumetric convnet modernizing hierarchical transformer for medical image segmentation[J]. arXiv preprint arXiv:2209.15076, 2022.