TGMS-UNet: A dual-branch segmentation network for ultrasound endometrium based on sequence guidance and multi-scale feature correction

Abstract

Objective

Ultrasound imaging of the endometrium is often affected by speckle noise, low contrast, and physiological cycle variations, which lead to blurred boundaries and ambiguous structural representation. To address these challenges, this study proposes a segmentation method specifically designed for ambiguous ultrasound scenarios, which provides stronger attention guidance to indistinct boundaries, thereby improving the accuracy of structural contour segmentation.

Methods

The proposed TGMS-UNet is a dual-branch segmentation network that converts geometric contours into sequential representations by calculating the distance and angle variations from boundary points to the centroid. The sequence-image feature alignment module employs global attention and scanning mechanisms to achieve cross-modal fusion and integrate contour prior knowledge, while the feature correction and adaptive fusion module uses a gating mechanism to rectify multi-scale feature misalignment and suppress erroneous information. The network is trained end-to-end, enabling the fusion of multi-scale image features and sequence-guided boundary information within a unified framework.

Results

Experimental results show that TGMS-UNet achieves a Dice of 0.9226 and an IoU of 0.8822 on a private endometrial ultrasound dataset of 1,063 images, and a Dice of 0.8355 and an IoU of 0.8012 on the public BUSI breast ultrasound dataset, outperforming existing mainstream segmentation models on key metrics.

Conclusion

The proposed TGMS-UNet demonstrates strong ability to handle ambiguous, low-contrast ultrasound images, particularly in accurately delineating blurred boundaries. By leveraging contour-derived prior information and enhancing feature consistency across scales, it improves segmentation precision and reliability.

Keywords

sequence guidance multimodal fusion feature calibration image segmentation ultrasound image transformer

1. Introduction

The endometrium is a vital component of the female reproductive system. Accurate evaluation via transvaginal ultrasound is crucial for diagnosing conditions such as infertility, endometrial hyperplasia, endometriosis, and endometrial cancer.¹ According to the World Health Organization, approximately one in six adults worldwide suffers from infertility, with endometrial factors accounting for nearly two thirds of cases.² Transvaginal ultrasound, through comprehensive scanning using a vaginal probe, provides critical information on endometrial length, thickness, morphology, and uterine position, aiding in the detection of abnormalities.³ However, precise segmentation of the endometrium in B-mode ultrasound imaging faces several challenges. First, inherent speckle noise, which results from constructive and destructive interference of coherent echoes scattered at tissue interfaces, often leads to significant noise interference, low overall contrast, considerable boundary variability, and frequent boundary obscuration. Second, as shown in Figure 1, beyond these inherent imaging limitations, ultrasound images of the endometrium are further influenced by menstrual cycle variations. These physiological changes introduce additional speckle noise, acoustic shadowing, and reduced boundary contrast, particularly in scenarios where the demarcation between the endometrium and myometrium is ambiguous.

Figure 1.

Ultrasound endometrial image in a blurred scene. The red arrows indicate blurring in the ultrasound image.

Currently, manual delineation of endometrial contours in ultrasound images is highly subjective and prone to inconsistency, as it relies heavily on clinicians’ experience and may vary among individuals with different levels of seniority.⁴ Therefore, precise and automated segmentation of the endometrial boundary in blurred ultrasound images is crucial for reducing diagnostic subjectivity. Accurate endometrial segmentation can enhance the objectivity and reliability of endometrial receptivity assessment in infertility diagnosis and treatment.⁵ The accurate evaluation of endometrial receptivity depends on key parameters such as endometrial thickness and morphology, which in turn require clear and reliable boundary segmentation. Based on this, clinicians can more objectively determine the implantation window and select the optimal timing for assisted reproductive procedures.

Existing deep learning approaches for this task face significant limitations. Convolutional neural network (CNN)-based methods, while effective in feature extraction, often lose spatial details during down-sampling and struggle with the global context due to their local receptive fields.⁶ Although multi-scale strategies and Transformer-based models have been introduced to capture broader context and improve accuracy, they can still produce jagged boundaries due to detail loss during up-sampling.⁷ To overcome this, several studies have focused on enhancing boundary segmentation. For example, Park et al. proposed a key-point-based framework that incorporates prior key-point information to improve the accuracy and robustness of endometrial region segmentation.³ Nevertheless, in cases of extremely low contrast, these models fail to fully emulate the nuanced reasoning and domain expertise clinicians employ when delineating ambiguous contours. To address these shortcomings, leveraging textual representations of expert knowledge has emerged as a promising direction. In natural image segmentation, sequence-guided methods have demonstrated efficacy by integrating precise linguistic descriptions.^8,9 In medical imaging, Early studies, such as Bhalodia et al., explored the use of radiology report text for weakly supervised lesion localization by extracting keyword-level attributes and employing cross-attention to align visual and textual representations, demonstrating that textual information can effectively guide models toward clinically relevant regions.¹⁰ however, the outputs were limited to coarse bounding boxes rather than pixel-level segmentation or fine boundary modeling. Building on this idea, Tomar et al. proposed TGANet, which introduced structured textual attributes (e.g., polyp size and number) into colorectal polyp segmentation via a label attention mechanism, showing that even simple textual cues can enhance segmentation performance, albeit with limited semantic richness and insufficient boundary guidance.¹¹ Subsequently, Li et al. presented LViT, leveraging richer textual descriptions and BERT embeddings within a dual-branch U-shaped CNN–Transformer architecture to enable deeper language–vision interaction under a semi-supervised framework, but at the cost of increased model complexity and without explicitly addressing high-frequency edge perception.¹² Further, Zhang W. et al. proposed DescriptorMedSAM, which systematically investigated the effect of prompt granularity and demonstrated that structured descriptors incorporating shape and spatial information significantly improve zero-shot and few-shot segmentation performance, while still being constrained by the performance of the underlying foundation model and lacking explicit boundary-aware optimization.¹³

Inspired by these developments, we propose TGMS-UNet, a novel sequence-guided dual-branch segmentation network tailored for ambiguous ultrasound endometrial images. Our core innovation lies in encoding the geometric contour of the endometrium into contour sequence features based on variations in distance and angle from contour points to the centroid. Specifically, contour sequence features are defined as a centroid-based ordered representation of endometrial boundary points, where the manually annotated contour is arranged in a clockwise order starting from the leftmost boundary point and encoded by transforming its coordinates into normalized distance and angular sequences relative to the centroid. These features embed the clinical reasoning process of contour delineation. We design a sequence-image feature alignment module to fuse this prior knowledge with visual features dynamically. Furthermore, to tackle the feature misalignment issue inherent in multi-scale fusion, we introduce a feature correction and adaptive fusion module that incorporates a gating mechanism to selectively integrate and calibrate features. The main contributions of this work are summarized as follows:

1. We propose a novel sequence-guided segmentation framework that encodes endometrial contours into a distance-angle-based sequence, effectively incorporating clinical prior knowledge to resolve ambiguities in ultrasound images.

2. We design a feature correction and adaptive fusion module within the bottleneck to mitigate multi-scale feature misalignment, allowing the model to adaptively adjust fusion ratios and suppress erroneous guidance.

3. Extensive experiments on a private ultrasound endometrial dataset and a public breast ultrasound dataset demonstrate that our TGMS-UNet outperforms state-of-the-art segmentation methods across multiple metrics, offering a robust tool for clinical support.

2. Methodology

2.1. Image datasets

2.1.1. Self-built ultrasound endometrial dataset

This study is a retrospective model development and validation study conducted in Changsha, China. It collated videos of endometrial examinations from 400 patients undergoing TVUS at the Ultrasound Department of the Affiliated Changsha Central Hospital, University of South China from 2023 to 2024, using a LOGIQ E11 system. This retrospective study was approved by the Institutional Review Board of the Affiliated Changsha Central Hospital (approval number KY-2023-156-01) with a waiver of informed consent due to the retrospective nature. The collected videos were filtered according to the following exclusion criteria: (1) images containing severe artifacts or incomplete anatomical structures; (2) videos with poor signal-to-noise ratio or incomplete scanning sequences; (3) cases where the visualization duration of the standard endometrial section was less than 1 second; (4) examination videos with a total duration of less than 10 seconds. To enhance the generalizability of the model, the final included video data covered examination scenes from both the proliferative and secretory phases. Each retained video was approximately 10 seconds in duration, with the standard endometrial section lasting between 1-2 seconds. From these videos, an average of about 25 images were extracted per video, including 2-4 standard sections, resulting in a final dataset of 1063 standard section images.

All images were annotated by two professional physicians to produce real segmentation masks of the endometrial region. The annotation workflow followed a two-reader protocol: inter-rater agreement between the two physicians was first evaluated using the Dice similarity coefficient. The segmentation masks were jointly confirmed by both physicians to form a consensus annotation. For cases with disagreement or large Dice variation, particularly where the endometrial boundary was ambiguous or difficult to delineate in ultrasound images, both experts performed joint re-evaluation and refined the annotations through sufficient discussion, ultimately generating a unified binary consensus mask. Based on these masks, a sequence representation that simulates the doctor’s contour sketching process is generated by encoding the distance and angle variations from each contour point to the centroid. Importantly, to avoid potential data leakage caused by strong intra-patient correlation among frames extracted from the same video, dataset splitting was strictly performed at the patient level rather than the image level. All frames from the same patient were kept within the same subset. The dataset was finally divided into training, validation, and test sets in a ratio of 7:1:2 based on patients.

2.1.2. Breast ultrasound images dataset

To verify the general applicability of the model, additional evaluation was conducted on the public BUSI dataset,¹⁴ collected from Baheya Women’s Hospital, which includes 780 ultrasound images (500 ×500 pixels) from 600 patients aged 25-75, covering 437 benign, 210 malignant, and 133 normal cases, acquired using LOGIQ E9 and E9 Agile systems.

2.2. Two-branch segmentation network based on sequence guidance and multi-scale feature correction (TGMS-UNet)

As illustrated in Figure 2, the proposed network framework, designated TGMS-UNet, accepts ultrasound endometrial standard cross-section images and their corresponding contour sequence data as input. The sequence data contain the “time series” change information of the distance and angle from the endometrial boundary contour points to the center of mass. The model generated segmentation masks in an end-to-end manner. Specifically, the backbone network uses U-Net with residual connections to extract the image features at all levels. The Transformer sequence encoding branch constitutes another backbone branch, which employs the Transformer with self-learning position encoding to extract sequence features. In addition, it designs a sequence feature and image feature alignment fusion module (TIFM). The alignment and fusion of sequence and image features are achieved through the global attention mechanism of the transformer, along with spatial and channel scanning fusion. To enhance the model’s understanding of multi-scale features and global context, the lateral connection was embedded in a multi-scale fusion module (GLFM). The purpose of this is to fuse the global features of the transformer visual branch with the local multi-scale features of the encoder at all levels. Concurrently, a feature correction and adaptive fusion module (FCAM) was integrated into the bottleneck structure to enhance the morphological features of the endometrium, rectify the offset features induced by multi-scale fusion, and regulate the fusion ratio through a gating mechanism to mitigate potential guidance bias. This approach enhances the model’s perception of fuzzy boundaries and improves the overall segmentation performance.

Figure 2.

The general framework of the proposed TGMS-Unet model. The model is based on an enhanced U-Net with residual connections, integrated with a sequence and image feature fusion module, a multi-scale fusion module, and a feature correction and adaptive fusion module.

2.3. Sequence feature extraction branch

As demonstrated in Figure 3, the extraction of features reflecting contour changes in the contour sequence data is achieved using two CNN components specifically designed for one-dimensional contour sequence data.^15,16 In our work, the endometrial contour is transformed into a numerical sequence representation based on its geometric structure. Specifically, the contour is extracted from the binary mask obtained via manual annotation, and the annotated contour points are directly utilized to preserve boundary fidelity. To ensure sequence consistency, the leftmost point is selected as the starting point, and all contour points are arranged in a clockwise order. The centroid of the contour is computed using the region moment method, and each point is represented in polar form relative to the centroid by calculating its Euclidean distance and azimuth angle, with the angle normalized to the range [0,2π). Furthermore, the distance and angle values are normalized to reduce scale variations. Since the annotation process enforces contour closure, the resulting sequence forms a complete and continuous representation of the endometrial shape. This distance-angle pair provides a complete polar coordinate description of the contour, thereby enabling the theoretical reconstruction of the original shape. Each component comprises a 1D convolution layer, a batch normalization layer, and a GELU activation function. The component performs preliminary processing on the sequence data X to extract basic features L reflecting contour changes. Subsequently, these features are input into a multi-head self-attention (MHA) mechanism to learn correlations within the sequence. Each attention head can identify different patterns, thereby capturing the endometrial contour and its variation characteristics. The process can be calculated by the following formulas:

ConvBlock (X) = GELU (B a t c h N o r m (C o n v 1 D (X))),

(1)

L = C o n v B l o c k (C o n v B l o c k (X)),

(2)

Z^{g e n} = M H A (L + P) .

(3)

Figure 3.

Unit module structure of the proposed sequence encoding branch.

Since the positions of node elements in the set lack inherent meaning, a learnable positional encoding is introduced to represent these positions, thereby enhancing the model’s ability to understand contour sequence data. This approach allows the model to capture the changing characteristics of the endometrial contour more accurately and provide precise contour change information for the sequence and image fusion module, which guides the image segmentation process and model attention, ultimately achieving more accurate endometrial contour segmentation.

2.4. Sequence feature and image feature alignment fusion module (TIFM)

Accurate segmentation of the endometrial boundary remains challenging due to artifacts, noise, and physiological variations in ultrasound images. Existing CNN-based models often treat the image uniformly, leading to overly smooth and inaccurate boundaries.¹⁷ To tackle this, we introduce a sequence-image feature fusion module that integrates boundary-related sequence features to enhance boundary precision and attention focus. A key challenge in sequence-guided segmentation lies in aligning cross-modal visual and sequence features. Early methods used RNN/LSTM to encode text and CNN to extract image features, combining them via multimodal LSTM,^18–21 attention mechanisms,^22,23 or cycle consistency.²⁴ More recent approaches employ transformers for deeper fusion, such as MDETR and VLT with transformer decoders,^25,26 LAVT with a Swin Transformer backbone,²⁷ and dual-encoder architectures like ReSTR and CRIS.^28,29

Despite these advances, most existing methods perform modality alignment only after feature encoding, which limits the exploitation of multi-level visual details and may result in information loss. To address this limitation, we adopt a dual-branch encoder architecture and embed TIFM into each encoder layer to enable progressive cross-modal fusion. Specifically, TIFM employs a cross-attention mechanism in which visual features act as queries, while sequence features serve as keys and values, allowing boundary-aware sequence cues to selectively guide visual feature enhancement. Modal alignment is conducted using a single-head cross-attention module at each encoder layer, thereby achieving fusion at a single scale per layer while preserving hierarchical representations. In addition, residual connections and a gating mechanism are introduced to balance gradient propagation between the visual and sequence branches, adaptively regulate the fusion ratio, and alleviate potential bias introduced by noisy or ambiguous sequence priors.

As demonstrated in Figure 4, given the input image features $V_{i} \in R^{C_{i} \times H_{i} \times W_{i}}$ and the sequence language features $L \in R^{C_{t} \times T}$ , TIFM performs multimodal fusion in two steps. In the initial step, at the spatial position of each image feature, TIFM calculates the attention of the sequence features according to the query and key to generate a language feature representation vector related to the local features of the current image.³⁰ This process results in a set of mixed multimodal feature maps $F_{i} \in R^{C_{i} \times H_{i} \times W_{i}}$ .Specifically, the process of first obtaining the sequence mapping feature $G_{i}$ is as follows:

V_{i q} = f l a t t e n (w_{i q} (V_{i}))

(4)

L_{i k} = w_{i k} (L))

(5)

L_{i v} = w_{i k} (L))

(6)

G_{i}^{'} = s o f t m a x (\frac{V_{i q}^{T} L_{i k}}{\sqrt{C_{i}}}) L_{i v}^{T}

(7)

G_{i} = W_{i m} (u n f l a t t e n (G_{i}^{' T}))

(8)

Figure 4.

Sequence and image feature fusion module.

Among them, The implementation of the projection functions $w_{i q}$ 、 $w_{i k}$ 、 $w_{i v}$ and $w_{i m}$ is achieved by $1 \times 1$ convolution, with $w_{i q}$ responsible for query projection, $w_{i k}$ and $w_{i v}$ output channels set to $C_{i}$ for key and value projection, and $w_{i m}$ employed for final sequence feature projection. The module utilizes visual feature $V_{i}$ as the query and sequence language feature $L$ as the key and value. Following the linear transformation of $w_{i q}$ and $W_{i w}$ , and the instance normalisation, the language feature $G_{i}$ is obtained, which has the same shape as $V_{i}$ . Finally, $G_{i}$ and $V_{i}$ are fused by element-by-element multiplication to generate a multimodal feature map $F_{i}$ , Specifically, the formula steps are as follows:

V_{i n} = w_{i n} (V_{i}))

(9)

F_{i} = w_{i o} (V_{i n} ⊙ G_{i})

(10)

Among them, $⊙$ denotes element-wise multiplication. $w_{i n}$ and $w_{i o}$ represent the final projection of the visual features and the projection of multimodal features, respectively. Both functions were implemented using $1 \times 1$ convolutions with ReLU activation.

In the second step, the sequence feature $L \in R^{C_{t} \times T}$ is expanded to a tensor $F_{t}$ of the same size as the image feature through the expansion operation $Expand (.)$ . Then, we concatenate the image feature $V_{i}$ the multimodal feature $G_{i}$ , and the sequence feature $F_{t}$ along the channel dimension to form a mixed feature cube, which is formally expressed as:

F_{t} = E x p a n d (L) \in R^{C_{i} \times H_{i} \times W_{i}}

(11)

F_{c u b e} = [V_{i}, G_{i}, F_{t}] \in R^{3 \times C \times H \times W}

(12)

In order to promote interaction both within and across different modalities, a warping mechanism was designed for the hybrid feature cube. Specifically, this mechanism consists of channel and spatial scanning in sequence, where both scanning operations are implemented via multi-head attention MHA to explicitly model global dependencies rather than local aggregation. Channel scanning regards channels as ordered sequences and learns cross-channel communication to promote modal fusion. In this process, each channel is treated as a token, and self-attention is performed across the channel dimension to dynamically model inter-channel dependencies, allowing information exchange among different modality-related feature groups embedded in the channel space. Then, spatial scanning operates on the spatial dimension and learns to transfer information within a separate spatial dimension slice. In this step, each spatial position is treated as a token, and multi-head attention is applied across all spatial locations to capture long-range spatial relationships and enable contextual propagation across different regions of the feature map, while preserving the channel-refined representations. Formally, this process can be written as:

F_{o u t} = {M H A}_{s p a t i a l} (({M H A}_{c h a n n e l} (F_{c u b e}))

(13)

Among them, ${M H A}_{c h a n n e l}$ is channel scanning, using the multi-head attention mechanism, taking the features on the channel dimension as input for multi-head attention calculation. More specifically, the feature cube is first reorganized along the channel axis so that each channel descriptor participates in global self-attention, enabling adaptive weighting and interaction across different channels and modality-specific representations. And ${M H A}_{s p a t i a l}$ represents spatial scanning, performing multi-head attention calculation along the spatial dimension, where each spatial location interacts with all other spatial positions within the same feature slice to capture global spatial context and enhance structural consistency. After fully integrating the multi-modal mixed features, $F_{o u t}$ is sent to the next module for further processing.

2.5. Feature correction and adaptive fusion module (FCAM)

In ultrasound imaging, the morphology and size of the endometrium vary with the physiological cycle, and down-sampling further aggravates the loss of spatial information. Traditional multi-scale cross-level fusion methods integrate high-resolution spatial features with low-resolution semantic features; however, repeated down-sampling often leads to spatial misalignment and representation discrepancies, ultimately degrading model performance.³¹ To address this, we propose a feature fusion and correction method with two key strategies. As demonstrated in Figure 5, The first is multi-scale fusion, which aligns feature map sizes via down-sampling. Early stages integrate global-local features from the Transformer branch, while later stages fuse multi-scale features across stages, enhancing detail perception and semantic understanding.

Figure 5.

Proposed multi-scale fusion module.

The second strategy involves the incorporation of a feature correction and adaptive fusion module within the bottleneck structure, as shown in Figure 6. This module utilizes a grouping strategy to calculate the offset between multi-scale and multi-modal features, employs sequence-guided boundary morphological attention to correct multi-scale features, enhances boundary attention, and mitigates the issue of spatial feature misalignment. This method optimizes spatial consistency and preserves semantic information. To circumvent the performance degradation that arises from direct concatenation and fusion of features,^32,33 a gating mechanism is introduced to achieve adaptive fusion of the corrected multi-scale features and multi-modal features.³⁴ This mechanism enables dynamic adjustment of the fusion ratio, mitigates the impact of erroneous sequence information, and enhances the robustness of the model. Finally, the corrected features are input into the channel attention module to further focus on key features and improve segmentation accuracy.³⁵

Figure 6.

Feature correction and adaptive fusion module.

Specifically, for the multi-scale feature $F_{m} \in R^{C_{m} \times H_{m} \times W_{m}}$ and the multimodal feature $F_{o} {\in R}^{{(3 C)}_{o} \times H_{o} \times W_{o}}$ , first, we downsample the multi-scale feature to obtain the feature $F_{o}^{'} \in R^{C_{m} \times H_{o} \times W_{o}}$ of the same size as the multimodal feature. Then, we concatenate the two features and feed them into a convolutional module to estimate the offset fields:

Δ_{m}, Δ_{o} = C o n v ([F_{o}^{'}, F_{m}]), Δ_{m}, Δ_{o} \in R^{2 \times G \times H \times W}

(14)

where

G

denotes the unified grouping dimension. The obtained offsets are used for feature calibration to produce the corrected features

{\hat{F}}_{m}

and

{\hat{F}}_{o}

. After that, the gating mechanism is applied to compute the fusion ratio. Specifically, the gating masks are not independent learnable scalar parameters, but are dynamically generated from features via a convolutional transformation followed by a sigmoid activation function:

β = σ (C o n v ([{\hat{F}}_{m}, {\hat{F}}_{o}])), β \in R^{1 \times H \times W}

(15)

where

β_{m} = β

and

β_{o} = 1 - β

, which ensures spatially adaptive and complementary fusion weights. Finally, the corrected features are merged by element-wise weighted addition:

F_{f u s e} = β ⨀ {\hat{F}}_{m} + (1 - β) ⨀ {\hat{F}}_{o}

(16)

where

1 \in R^{1 \times H \times W}

denotes an all-ones tensor. and then input into the channel and spatial attention module to obtain the final output, where channel attention is first applied via global average pooling and global max pooling followed by a shared multilayer perceptron to recalibrate channel-wise feature responses, and spatial attention is subsequently performed by aggregating channel information through average and max pooling along the channel dimension and a convolution operation to highlight salient spatial regions, thereby further refining the fused features in both channel and spatial dimensions.

3. Statistical analysis

3.1. Evaluation metrics

The study used six common metrics to quantitatively compare the performance of different methods for segmentation of standard sections of ultrasound endometrium: Dice, IoU, Acc, Sen, Spe and Hausdorff distance (HD). Dice and IoU evaluate the overlap between the true segmentation mask G and the model predicted segmentation result P. Acc, Pre and Spe are key indicators to measure the performance of the model for pixel classification. TP, FP, TN, and FN stand for true positive, false positive, true negative, and false negative, respectively. They are defined as:

D i c e = \frac{2 | G \cap P |}{| G | + | P |}

(17)

I o U = \frac{| G \cap P |}{| G \cup P |}

(18)

A c c = \frac{T P + T N}{T P + T N + F P + F N}

(19)

S e n = \frac{T P}{T P + F N}

(20)

S p e = \frac{T N}{F P + T N}

(21)

HD measures the maximum distance between the two boundaries of the true segmentation mask and the model predicted segmentation mask. $C_{G}$ and $C_{P}$ are the region boundaries of the true segmentation mask and the predicted segmentation result, $x$ and $y$ are points on the boundaries $C_{G}$ and $C_{P}$ respectively; for any point $j$ on the boundary $C$ , $z - j$ is the Euclidean distance between points z and $j$ , and $d (z, C)$ is the minimum distance between point $z$ and all points on the boundary $C$ . HD is defined as follows:

H D = \max {\begin{array}{l} \max \\ x \in C_{G} \end{array} {d (x, C_{P})}, \begin{array}{l} \max \\ x \in C_{P} \end{array} {d (y, C_{G})}}

(22)

d (z, C) = \begin{array}{l} \max \\ j \in C \end{array} {∥ z - j ∥}

(23)

3.2. Statistical evaluation methods and experimental parameters

All models in this study were implemented using the PyTorch deep learning framework and trained on a workstation equipped with an NVIDIA GeForce RTX 4090 GPU. The models were trained for 100 epochs with the Adam optimizer, using an initial learning rate of 0.0001, a weight decay of $1 \times 10^{- 5}$ , (a) momentum of 0.9, and a batch size of 8. To enhance the generalization ability and robustness of the model, the training set was augmented threefold using the Augmentor library. The augmentation operations included random rotation, elastic deformation, horizontal flipping, and vertical flipping. After data augmentation, the dataset was divided into training, validation, and test sets in a 7:1:2 ratio. All quantitative results were obtained from the independent test set. No additional post-processing was applied to the model outputs to ensure consistency in evaluation.

4. Results

4.1. Comparative experimental results

To validate the effectiveness of the proposed framework and to further assess the auxiliary role of artificial intelligence in disease diagnosis,³⁶ we conducted comparative experiments using a variety of widely adopted segmentation algorithms, including U-Net,³⁷ SegNet,³⁸ Unet++,³⁹ AttUNet,⁴⁰ and the latest segmentation models TransUNet,⁴¹ Swin-Unet,⁴² CASFNet,⁴³ TGANet,¹¹ LViT¹² and WM-unet-v2.⁴⁴ In order to ensure fairness, all comparative models were retrained using their publicly available implementations on the constructed endometrial ultrasound dataset, and the corresponding segmentation metrics were obtained. The quantitative results, as demonstrated in Table 1, illustrate that TGMS-UNet surpasses all comparison models in terms of key segmentation metrics. Specifically, TGMS-UNet achieves a performance enhancement of 4.89% in comparison with the baseline U-Net. In addition, the proposed model has been shown to outperform advanced models such as CASFNet and LViT, with improvements to the Dice coefficient of 1.58% and 2.02%, respectively. The quantitative results indicate that incorporating contour sequence feature-guided attention into TGMS-UNet enables more precise segmentation of the endometrial region in ambiguous ultrasound images than conventional methods, thereby improving overall segmentation accuracy and evaluation performance.

Table 1.

Comparison of our method with state-of-the-art segmentation methods on the ultrasound endometrial standard section dataset. The unit of HD is pixels (px) and the best results are marked in bold text.

Method	Dice	IoU	Acc	Sen	Spe	HD
U-Net	0.8737	0.8046	0.9839	0.8548	0.9848	29.8126
UNet++	0.8785	0.8101	0.9842	0.8548	0.9855	23.0911
AttUNet	0.8533	0.7679	0.9811	0.8431	0.9853	26.3267
TransUNet	0.8801	0.8159	0.9848	0.8624	0.9861	20.9846
Swin-Unet	0.8933	0.8276	0.9878	0.8720	0.9879	15.1206
TGANet	0.8987	0.8338	0.9883	0.8756	0.9921	13.9042
LViT	0.9024	0.8389	0.9889	0.8789	0.9927	13.1120
CASFNet	0.9068	0.8297	0.9895	0.8752	0.9945	11.701
WM-unet-v2	0.8663	0.7959	0.9846	0.8596	0.9857	20.8513
Ours	0.9226	0.8558	0.9906	0.8896	0.9924	6.1887

In order to further compare the segmentation performance of these methods, Figure 7 presents a visual comparison of TGMS-UNet and other approaches on the endometrial ultrasound image dataset. Each row in the figure corresponds to the same input image, while each column represents the segmentation results of different methods. The test images consist of cases of ambiguous ultrasound, where the endometrial boundary is mixed with myometrial echoes and exhibits indistinct boundaries. The findings demonstrate that, through the synergistic effect of the sequence-enhanced guidance and boundary refinement modules, our method is more effective than alternative approaches in handling regions with blurred boundaries or low contrast, and can accurately delineate endometrial boundaries of various shapes. These findings further demonstrate that TGMS-UNet achieves superior segmentation performance compared with other mainstream methods.

Figure 7.

Segmentation visualization results of different models on a dataset of endometrial ultrasound images.

We further retrained the model on the publicly available BUSI breast ultrasound dataset from different sources and evaluated its performance. The quantitative results, as shown in Table 2, indicate that TGMS-UNet outperforms the comparative models across key evaluation metrics. Among the competing methods, TGANet and LViT also achieve strong performance, with Dice scores of 82.65% and 82.52%, respectively. In comparison, TGMS-UNet still improves the Dice coefficient by 0.9% and 1.03%, respectively. The experimental results obtained on datasets with different sources and higher segmentation difficulty demonstrate that the proposed method effectively mitigates the segmentation errors of conventional models in ultrasound images with blurred boundaries and low contrast, and exhibits strong generalization capability across data from diverse sources.

Table 2.

Quantitative comparison of the performance of different methods on the public BUSI dataset, with the best results shown in bold.

Method	Dice	IoU	Acc	Sen	Spe	HD
U-Net	0.7586	0.6591	0.9534	0.7819	0.9703	35.9873
UNet++	0.7794	0.6986	0.9568	0.7893	0.9736	30.5201
AttUNet	0.8027	0.7172	0.9608	0.8011	0.9747	27.3829
TransUNet	0.8142	0.7304	0.9635	0.8022	0.9754	22.9842
Swin-Unet	0.8163	0.7386	0.9649	0.8042	0.9754	19.4526
TGANet	0.8265	0.7437	0.9662	0.8054	0.9761	17.4079
LViT	0.8252	0.7429	0.9659	0.8051	0.9760	17.7823
CASFNet	0.8151	0.7373	0.9645	0.8035	0.9750	20.1784
WM-unet-v2	0.8037	0.7185	0.9609	0.8023	0.9714	23.2765
Ours	0.8355	0.7592	0.9703	0.8146	0.9792	14.8834

To provide a more intuitive evaluation of the actual segmentation performance of different models on the BUSI breast ultrasound dataset, Figure 8 presents a visual comparison of the results produced by various methods. From the visualization, it can be observed that under the challenging conditions of breast ultrasound images, where boundaries are more ambiguous and structural information is insufficient, TGMS-UNet is able to more robustly extract lesion contours and preserve structural integrity through the joint effect of the boundary guidance module and the feature refinement module. Compared with other methods, it demonstrates more pronounced advantages in capturing fine details and maintaining regional coherence.

Figure 8.

Segmentation visualization results on the BUSI ultrasound image dataset with different models.

4.2. Ablation experiment results of the sequence-guided branch

In order to evaluate the effectiveness of the sequence-guided branch in enabling the model to learn the outline process of professional physicians, this study designed the following ablation experiments: 1) using only the sequence data of the distance change from the endometrial contour point to the center; 2) using only the sequence data of the angle change from the contour point to the center; 3) using a combination of distance and angle change sequence data. Additionally, to compare with the sequence-guided strategies, two non-sequence geometric encoding methods were introduced: 4) SMU-Net-based saliency guidance,⁴⁵ which generates a saliency map by fusing multi-level structural information of the foreground and background, representing a contour-map encoding strategy for segmentation enhancement and 5) Keypoint-guided segmentation, which uses four anatomical keypoints of the endometrium (the tip of the endometrial cavity, the cervical os, and the two thickest points between the anterior and posterior basal layers) to guide the segmentation.³ The baseline model removes the sequence-guided segmentation branch and the sequence-image feature fusion module.

The findings presented in Table 3 demonstrate that the incorporation of distance change sequence information into the model enhanced the Dice and IoU indicators by 1.98% and 1.04%, respectively, compared with the baseline model. The performance of the model utilizing solely angle changes was comparable to that of the baseline. In contrast, the non-sequence geometric encoding strategies showed moderate improvements: SMU-Net saliency guidance enhanced the Dice and IoU indicators by 2.23% and 1.90%, respectively, while the keypoint-guided method achieved improvements of 0.57% and 0.18%. Both were consistently outperformed by the distance-only and combined sequence guidance strategies. This indicates that while traditional geometric encodings can provide some structural priors, they lack the expressive capacity and alignment flexibility of the proposed sequence-based “temporal” representation. The strategy that combines distance and angle change sequence information as segmentation guidance enhances the Dice and IoU indicators by 3.53% and 3.46%, respectively, and demonstrates optimal performance in other indicators. Quantitative results substantiate the efficacy of the strategy of combining distance and angle “temporal” change information as sequence guidance for image segmentation, and further justify its advantage over conventional geometric encoding methods in handling blurred and variable endometrial boundaries.

Table 3.

Results of ablation experiments on different sequence-guided segmentation strategies of the model using the ultrasound endometrial standard section dataset.

Method	Dice	IoU	Acc	Pre	Sen	HD
No sequence guidance	0.8848	0.8185	0.9857	0.9261	0.8654	19.7038
Distance sequence guidance	0.9046	0.8289	0.9877	0.9369	0.8783	12.6231
Angle sequence guidance	0.8864	0.8198	0.9848	0.9270	0.8677	18.4336
Keypoint-guided segmentation	0.8905	0.8203	0.9858	0.9305	0.8702	16.8904
SMU-Net saliency guidance	0.9071	0.8375	0.9860	0.9412	0.8715	13.2150
Distance and angle guidance	0.9201	0.8531	0.9893	0.9477	0.8983	8.6914

4.3. Ablation experiment results of the model module

In order to evaluate the role of each component in the segmentation framework, an ablation study was conducted on the ultrasound endometrial dataset. This study involved three main modules: the text–image feature integration module (TIFM), the feature correction and adaptive fusion module (FCAM), and the global–local multi-scale feature fusion module (GLFM). The baseline model is constructed based on U-Net, where the above modules are removed and only residual connections are retained to enhance feature propagation.

In the ablation experiment where FCAM is introduced alone, since FCAM requires two feature inputs for correcting feature misalignment, convolutionally extracted image features are used to replace the fused features generated by TIFM as the input to the module, so as to meet its input configuration requirements. The experimental results are shown in Table 4. Introducing TIFM alone improves the model’s ability to handle ultrasound images with blurred boundaries, particularly in capturing the contour variations of the endometrium, with Dice and IoU increased by 3.17% and 3.57%, respectively, compared with the baseline. Introducing FCAM or GLFM individually further enhances the model’s multi-scale representation and feature correction capabilities, leading to improved segmentation performance, with Dice increases of 1.91% and 1.58%, respectively. Furthermore, the results of the combination experiments show that the modules exhibit good synergistic effects. Compared with adding a single module, their joint use further improves the overall performance of the model. Finally, the complete model integrating TIFM, FCAM, and GLFM achieves the best results in terms of Dice, IoU, and Sen, demonstrating the effectiveness and complementarity of each module in the task of ultrasound endometrial segmentation.

Table 4.

Results of ablation experiments on different modules in the model using the ultrasound endometrial standard section dataset.

Baseline	TIFM	FCAM	GLFM	Dice	IoU	HD
√				0.8624	0.7851	30.7324
√	√			0.8941	0.8208	18.8467
√		√		0.8815	0.8046	21.3925
√			√	0.8782	0.7993	23.9183
√	√	√		0.9026	0.8312	13.6741
√	√		√	0.8992	0.8257	17.2386
√		√	√	0.8958	0.8215	18.1149
√	√	√	√	0.9184	0.8452	10.3741

As demonstrated in Figure 9, the qualitative outcomes provide additional support for the conclusions derived from the quantitative analysis. In this study, endometrial ultrasound images with pronounced boundary ambiguity and noise interference were selected for analysis. A qualitative visual comparison was conducted among the segmentation results of the baseline model, the baseline model with the FCAM module, the baseline model with the GLFM module, and the baseline model incorporating the TIFM module. The results were then overlaid with the corresponding ground truth annotations for comprehensive evaluation. The findings suggest that, in comparison with the baseline model, the incorporation of the FCAM and GLFM modules enhances the completeness of the segmented regions to a certain extent. Nevertheless, limitations persist with regard to boundary continuity and closure. In contrast, the model incorporating the TIFM module demonstrates superior contour consistency and structural integrity when handling blurred boundaries and noisy regions, achieving a higher degree of overlap with the ground truth annotations. These visualization results indicate that all modules contribute to performance improvement. Among them, the TIFM module effectively leverages the sequential prior information of boundary contours to constrain and refine ambiguous regions, thereby guiding the model to achieve more precise boundary segmentation and significantly improving boundary localization accuracy under complex ultrasound imaging conditions.

Figure 9.

Qualitative visualization of segmentation results in blurred regions across different modules.

4.4. Parameter experiment results

To evaluate the computational efficiency of TGMS-UNet, this study compared inference latency, floating-point operations (FLOPs), and parameter count across three representative categories of segmentation networks: (1) conventional CNN-based models (U-Net and UNet++), (2) Transformer-based models (TransUNet and Swin-UNet), and (3) sequence-guided segmentation models (TGANet and LViT). These networks represent the mainstream architectural paradigms for medical image segmentation and provide a comprehensive basis for assessing the trade-off between segmentation accuracy and computational cost. All evaluations were conducted under the same hardware and runtime environment to ensure fairness. Specifically, all models were tested on an NVIDIA RTX 4090 GPU with an input size of 256×256, using the official or publicly available implementations with default configurations. As shown in Table 5, for each model, we measured the average inference latency over 200 runs and reported the number of parameters and FLOPs obtained through static computational graph profiling.

Table 5.

Computational cost comparison of segmentation models.

Method	Params (M)	FLOPs (G)	Latency (ms)
U-Net	31.0	50.3	25.9
UNet++	24.5	42.8	20.8
TransUNet	105.3	105.7	75.9
Swin-UNet	58.2	79.4	40.5
TGANet	52.2	64.6	37.6
LViT	77.2	88.3	49.8
TGMS-UNet	62.2	83.6	45.2

As shown in Table 5, TGMS-UNet has a moderate parameter count (62.2 M), FLOPs (83.6 G), and inference latency (45.2 ms) among the compared models. This may be due to the dual-branch design introducing additional model redundancy. Future work will focus on exploring lightweight strategies to reduce the computational cost while maintaining segmentation performance.

4.5. Clinical validation trial of automated endometrial thickness measurement

In this experiment, the outputs of different segmentation models were combined with an automatic thickness measurement method to evaluate their applicability in quantitative endometrial thickness analysis. For each model’s segmentation result, the largest connected component was first extracted to remove noise and small spurious regions, thereby ensuring the stability and robustness of subsequent measurements. Within this region, an Euclidean distance transform was applied to compute the minimum distance from each pixel to the nearest boundary, resulting in a distance map. The location of the maximum value in the distance map was then identified as the center of the maximum inscribed circle, with the corresponding distance value representing its radius. The endometrial thickness was finally defined as the diameter of this maximum inscribed circle. To ensure consistency across different models and images, all thickness measurements were reported in pixel units.

Figure 10 illustrates the relative errors between the automatically measured thickness and the thickness derived from expert manual contour delineations for representative test samples. The results show that the proposed model maintains a consistently low measurement error across all examples and achieves the smallest error in most cases. Furthermore, Figure 11 presents the mean absolute error between the automatically measured thickness based on segmentation results and the reference thickness obtained from ground-truth contours over the entire test set. The proposed method again demonstrates superior performance compared with the other models.

Figure 10.

Comparison of relative errors between the thickness of the model segmentation results and the thickness annotated by the physician.

Figure 11.

Comparison of mean absolute error of model segmentation thickness.

The results demonstrate that the endometrial ultrasound image segmentation model constructed in this study not only exhibits excellent segmentation performance but also supports highly consistent automatic thickness measurement. This significantly reduces subjective bias introduced by differences in physician operation, improving the repeatability and efficiency of endometrial thickness measurement. This provides a reliable method for the objective and standardized assessment of endometrial thickness, and has practical significance for optimizing clinical examination procedures and assisting in the diagnosis of related diseases.

4.6. GradCAM heatmap experiment results

To enhance the interpretability of TGMS-Net’s boundary segmentation decisions, this study employed the Grad-CAM technique to visualize the regions of interest in the segmentation model and to verify the precision of boundary segmentation with the addition of boundary guidance.⁴⁶ As shown in Figure 12, the experimental results indicate that in ultrasound images with blurred boundaries, models without boundary guidance, such as Swin-Unet and CASFNet, tend to prioritize noise regions rather than true boundary features. Although TGANet and LViT incorporate sequence-guided segmentation, they do not perform effective filtering and correction of the guidance information, which may result in attention being misallocated to non-target regions. Consequently, their segmentation outputs may either exceed the actual contours or entirely overlook the blurred areas where boundaries are discernible, leading to significant segmentation errors. In contrast, the TGMS-UNet model, equipped with boundary guidance branches, can accurately focus on the true endometrial contours within blurred regions by fusing image features with boundary sequence features, achieving precise segmentation. This demonstrates that the sequence-guided attention mechanism effectively corrects the model’s attention allocation in blurred regions, thereby mitigating segmentation errors caused by noise interference and generating more refined and accurate segmentation boundaries.

Figure 12.

Attention heatmap. Red arrows indicate blurred areas in the image.

The experiment comprised four models: two comparison models that achieved the best results on the endometrial ultrasound dataset, the TGMS-UNet model without the sequence boundary guidance branch, and the complete TGMS-UNet model. The test images comprised images in which the endometrial boundary and myometrial echo were mixed and the boundary was unclear, as indicated by the red arrow marking the location of the blurred area. The experimental results demonstrate that the complete TGMS-UNet model, when guided by the sequence boundary, can significantly enhance the focus on areas with blurred boundaries or low contrast, and achieve accurate segmentation of the entire endometrial contour.

4.7. Failure case analysis

As demonstrated in Figure 13, three typical failure or suboptimal segmentation cases of the TGMS-UNet model are presented, with each row corresponding to one specific case. In the first instance, the endometrial region demonstrates notable echogenic similarity with the adjacent myometrial background over a substantial area, leading to indistinct boundaries and consequently impaired segmentation performance. In the second case, the endometrial region appears relatively elongated, and this narrow structure shares similar echogenic characteristics with the myometrium, making it challenging for the model to accurately delineate the complete anatomical boundaries. In the third case, fluid accumulation caused by either device-related factors or endometrial pathology results in the presence of an anechoic dark region within the image, which partially obscures the endometrial contour and reduces segmentation accuracy. In future work, for the aforementioned challenging segmentation cases, we consider incorporating prior information from complete contour sequences together with global structural information to impose consistency constraints, and further refine the segmentation results. It is anticipated that this approach will enhance the structural integrity and segmentation accuracy of the model in complex conditions, such as blurred boundaries, elongated morphologies, and partial occlusions.

Figure 13.

Analysis of three types of failure cases.

5. Discussion

5.1. Comparison and discussion of related ultrasound segmentation methods

In order to further discuss and compare the performance differences between TGMS-UNet and similar models, as well as to validate its cross-organ generalization capability, this study conducts a comparative analysis of related approaches adopting boundary enhancement strategies and CNN–Transformer hybrid architectures. This study places particular emphasis on the architectural designs and the segmentation performance of the subjects on ultrasound datasets, with particular attention being paid to the BUSI dataset.

In the field of boundary enhancement and guidance strategies, Liu et al. proposed Asym-UNet, a multi-branch residual encoder and an external attention module, combined with a deeply supervised boundary detection branch, to strengthen boundary localization. The model attained a Dice score of 79.94% on the BUSI ultrasound dataset, encompassing both benign and malignant cases.⁴⁷ Xue et al. developed GG-Net, which designs a global guidance module along both spatial and channel dimensions, and incorporates a shallow boundary detection branch to impose boundary-aware deep supervision on multi-scale features. This results in a Dice score of 82.1% on the BUSI benign and malignant ultrasound dataset.⁴⁸ Hu et al. proposed BGRA-GSA, which leverages a global-scale adaptive module to capture multi-scale contextual information, and employs a boundary-guided module together with a region-aware module to progressively refine feature representations, reaching a Dice score of 81.43% on the BUSI dataset.⁴⁹ Huang et al. have devised a Boundary-Rendering Network that incorporates a boundary selection module and a graph convolution-based boundary rendering module. Utilizing a graph structure enables the global refinement of contour vertices, thereby achieving a Dice score of 85.4% on the BUS ultrasound dataset.⁸ Boundary enhancement-based methods generally improve segmentation accuracy by introducing boundary supervision branches, edge detection operators, or boundary post-processing modules, which can partially alleviate the issue of blurred boundaries in ultrasound images. However, these approaches fundamentally still rely on image-derived features or treat boundary extraction as an auxiliary learning task, and their performance is limited when lesions exhibit gray-level similarity with surrounding tissues. In contrast, TGMS-UNet does not directly detect boundaries from images; instead, it transforms expert-annotated contours into distance- and angle-based sequential representations and aligns these sequences with image features, thereby embedding structured geometric priors into feature learning and enabling more robust segmentation in low-contrast regions.

In the field of CNN–Transformer hybrid architecture strategies, Feng et al. designed HTBE-Net, which constructs a dual-branch encoder consisting of a CNN and a Swin Transformer. It achieves feature-level deep fusion through a boundary-guided module, a selective feature enhancement module, and a long–short-range attention interaction fusion module, reporting a Dice score of 81.17%.⁵⁰ He et al. developed HCTNet, which interleaves CNN and Transformer encoding blocks within the encoder and introduces a spatial cross-attention module to reduce semantic gaps between the encoder and decoder, achieving a Dice score of 82.00% on the BUSI benign and malignant ultrasound dataset.⁵¹ Zhang et al. proposed HAU-Net, which embeds a Local-Global Transformer module into the skip connections of U-Net. It leverages local window self-attention and global token attention for efficient long-range dependency modeling, and further introduces a cross-attention module in the decoder to fuse multi-scale features, achieving a Dice score of 83.11% on the BUSI dataset.⁵² Yang et al. proposed CSwin-PNet, which constructs a feature pyramid based on Residual Swin Transformer Blocks. It further integrates an interactive channel attention module and a complementary feature fusion module to achieve encoder–decoder feature complementation, obtaining a Dice score of 83.68% on the BUSI dataset.⁵³ These hybrid architectures leverage the self-attention mechanism of Transformers to compensate for the limited receptive field of CNNs, thereby enhancing global contextual modeling and demonstrating advantages in irregular lesion segmentation. However, their global modeling remains confined to semantic relationships within the image space and lacks explicit understanding of holistic geometric structures. Moreover, fusion strategies are mostly based on concatenation or simple attention mechanisms, which do not fundamentally improve boundary awareness. In contrast, although TGMS-UNet also adopts a dual-branch design with cross-modal fusion, its Transformer branch takes contour-derived distance–angle sequences as input, encoding shape priors rather than image semantics. Therefore, its fusion with the CNN branch is essentially prior-guided rather than simple feature complementarity, enabling stronger robustness in boundary-ambiguous regions.

Despite not being specifically designed for breast ultrasound, TGMS-UNet achieves a Dice score of 83.55% on the BUSI benign and malignant segmentation task, reaching or even surpassing most task-specific models. This enhanced performance can be attributed primarily to the contour-sequence guidance strategy, which involves the encoding of geometric information concerning clinician-annotated boundaries into explicit priors. Furthermore, this strategy leverages cross-modal alignment to guide feature learning, thereby enabling the model to maintain robust boundary inference capability even in circumstances where visual information is limited. Concurrently, the feature correction and adaptive fusion modules mitigate spatial misalignment issues in multi-scale feature integration, thereby ensuring stable performance across lesions of varying scales and morphologies. The results obtained demonstrate that cross-modal methods based on contour sequence priors constitute an effective segmentation paradigm with strong generalization ability.

5.2. Limitations of sequence guidance and computational overhead

This study systematically compares TGMS-UNet with representative text and sequence guided segmentation methods including TGANet,¹¹ LViT,¹² LAVT,²⁷ and TVE-Net.^54.All of these methods leverage auxiliary information beyond the image, such as textual descriptions, attribute labels, or contour sequences, to compensate for the insufficient information caused by noise, low contrast, or unclear boundaries in medical images, and align features through cross-modal attention. However, they differ in the form of guidance and fusion strategies. LViT and LAVT use natural language text and rely on pre-trained language models, which introduces high computational cost and a semantic gap that is difficult to fully bridge. TGANet simplifies text into discrete attributes, making the method lightweight but losing boundary shape information. TVE-Net converts textual location descriptions into probabilistic text views, effectively leveraging positional priors but not explicitly modeling continuous contour geometry. In contrast, TGMS-UNet directly converts manually annotated contours into distance and angle sequences, preserving complete geometric information without requiring additional text or pre-trained models. The polar coordinate sequences simulate the cognitive process of clinicians tracing contours, providing strong clinical interpretability and low computational complexity. Our quantitative comparisons, as shown in Tables 1 and 2, demonstrate the model’s superior performance in Dice, IoU, and HD metrics, confirming the effectiveness of sequence guidance for precise boundary segmentation. This feature correction and adaptive fusion strategy is the key reason why TGMS-UNet achieves significantly better HD performance on ultrasound images compared with LViT and TGANet.

Regarding fusion strategies, LViT and LAVT merge features through different forms of cross-modal attention, while TVE-Net employs multi-stage knowledge transfer to propagate textual view information. None of these methods explicitly address spatial misalignment of multi-scale features. TGMS-UNet introduces a sequence-image feature alignment module that fully explores cross-modal interactions through cross-attention and channel and spatial scanning mechanisms. At the bottleneck, a feature correction and adaptive fusion module calculates offsets in groups to correct spatial misalignment across different scales and modalities and applies a gating mechanism to suppress erroneous guidance. This allows the model to perform robustly in ultrasound images with blurred boundaries and severe noise. Nevertheless, the method still relies on high-quality manual contour annotations, which are labor- and time-intensive, and the dual-branch architecture increases computational burden. Future work may explore self-supervised contour learning and lightweight fusion architectures, as well as extending the method to 3D ultrasound or other modalities such as MRI and CT to improve generalizability.

5.3. Limitations in dataset scale, bias, and generalization capability

Although the private endometrial ultrasound dataset constructed in this study consists of 1063 images collected from transvaginal ultrasound videos of 400 patients, the dataset originates from a single institution, a single geographic region in Changsha, China, and a single ultrasound system. This results in significant regional and device-specific biases. In addition, the demographic characteristics of the patients, such as age distribution, body mass index, and ethnic diversity, are relatively limited, making it difficult to fully represent a broader clinical population. The main risks associated with these constraints include domain generalization risk, scanner variability, operator variability, and cycle-phase variability. Specifically, a model trained on a single data distribution may struggle to adapt to new clinical environments. Differences in probe frequency, gain settings, and post-processing algorithms across ultrasound devices can alter speckle noise patterns and boundary clarity. Variations in scanning techniques and standard section selection habits among different operators may also introduce inconsistencies in image quality. Furthermore, although the dataset includes images from both the proliferative and secretory phases, the sample distribution across these phases may still be imbalanced, making it difficult to fully capture the morphological and echogenic variations related to the physiological cycle.

In the field of medical image segmentation, the acquisition of high-quality annotated medical images is inherently constrained by clinical resources, imaging availability, and the limited number of experienced physicians. In this study, we did not explicitly compute a direct quantitative relationship between sample size and model performance. Instead, we adopted multiple indirect validation strategies to ensure that the sample size was sufficient for model development and to verify the reliability and robustness of the proposed method. First, compared with recent studies employing transformer-based models for ultrasound medical image segmentation,^55,56 where sample sizes typically range from approximately 500 to 800 images, the 1063 images in this dataset represent a relatively large scale. Second, extensive data augmentation strategies were applied during training, and additional validation was performed on the public BUSI breast ultrasound dataset. These steps confirmed that the proposed model maintains stable performance and satisfactory generalization across different data sources. Although the dataset in this study may be considered relatively small compared to segmentation projects in natural scenes, this work is part of an ongoing research initiative. We are continuously collecting additional ultrasound and contrast-enhanced ultrasound images from multiple devices and clinical centers, with the goal of further expanding dataset diversity and enhancing the model’s generalizability.

6. Conclusion

This study proposes an innovative visual-language ultrasound medical image segmentation model termed TGMS-UNet. The model uses sequence data derived from changes in distance and angle between the target boundary and centroid to create a multimodal dataset comprising images and sequence The sequence-image feature alignment module addresses boundary blurring caused by noise and artefacts, enabling the model to focus on target regions in unclear conditions. Morphological correction and attention enhancement modules further refine the contour features, and multi-scale fusion effectively integrates global and local context. Extensive experiments on private and public ultrasound datasets demonstrated that the proposed distance-angle-based sequence guidance strategy significantly outperformed existing methods. TGMS-UNet can accurately segment the endometrium in blurred ultrasound images, providing a robust and practical tool for the preliminary evaluation of endometrial function and receptivity, as well as for related clinical applications.

Footnotes

Acknowledgements

The authors express their gratitude to all the institutions and individuals that have provided support for this work.

ORCID iD

Zhiyi Chen

Ethical considerations

This retrospective study was approved by the Institutional Review Board of the Affiliated Changsha Central Hospital (approval number KY-2023-156-01) with a waiver of informed consent due to the retrospective nature.

Author contributions

Qiao Wei: Writing – original draft, Visualization, Validation, Formal analysis, Data curation; Xiaowen Liang: Resources, Investigation, Conceptualization, Data curation; Yanfen Zhang: Resources, Project administration, Supervision; Zhili Guo and Qing Zhang: Resources, Project administration, Data curation; Zhang Xiao: Methodology, Investigation; Yaocheng Xiao: Supervision, Resources, Project administration; Hong Yu: Investigation, Project administration; Xiaoyan Kui: Writing – review & editing, Supervision, Methodology; Zhiyi Chen: Writing – review & editing, Supervision, Methodology, Investigation, Funding acquisition.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Natural Science Foundation of China (82102054), Clinical Research 4310 Program of the Affiliated Changsha Central Hospital of the University of South China (20214310NHYCG06), and Health Research Project of Hunan Provincial Health Commission (W20241010).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

The private dataset used in this project is not applicable to this section. The source code will be available on upon publication.

Guarantor

ZC.

References

Nalaboff

Pellerito

Ben-Levi

. Imaging the Endometrium: Disease and Normal Variants. RadioGraphics 2001; 21: 1409–1424. https://doi.org/10.1148/radiographics.21.6.g01nv211409

Organization WH . Infertility prevalence estimates, 1990–2021. World Health Organization, 2023.

Park

Lee

Kim

, et al. Endometrium segmentation on transvaginal ultrasound image using key point discriminator. Medical Physics 2019; 46: 3974–3984. https://doi.org/10.1002/mp.13677

S-Y

, et al. Deep Learning-Based Automatic Endometrium Segmentation and Thickness Measurement for 2D Transvaginal Ultrasound. In: 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE, 2019, pp. 993–997.

Wang

Bao

Xin

, et al. Automatic evaluation of endometrial receptivity in three-dimensional transvaginal ultrasound images based on 3D U-Net segmentation. Quantitative Imaging in Medicine and Surgery 2022; 12: 4095–4108. https://doi.org/10.21037/qims-21-1155

Chen

Zhao

, et al. CNN-G: Convolutional Neural Network Combined With Graph for Image Segmentation With Theoretical Analysis. IEEE Transactions on Cognitive and Developmental Systems 2021; 13: 631–644. https://doi.org/10.1109/tcds.2020.2998497

Marin

Vajda

, et al. Efficient Segmentation: Learning Downsampling Near Semantic Boundaries. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 2019, pp. 2131–2141.

Huang

Lin

Dou

, et al. Boundary-rendering network for breast lesion segmentation in ultrasound images. Medical Image Analysis 2022; 80: 102478. https://doi.org/10.1016/j.media.2022.102478

Wang

Fang

Wang

, et al. ViTAA: Visual-Textual Attributes Alignment in Person Search by Natural Language. In: European Conference on Computer Vision (ECCV). Springer International Publishing, 2020, pp. 402–420.

10.

Bhalodia

Hatamizadeh

Tam

, et al. Improving Pneumonia Localization via Cross-Attention on Medical Images and Reports. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Springer International Publishing, 2021, pp. 571–581.

11.

Tomar

Jha

Bagci

, et al. TGANet: Text-Guided Attention for Improved Polyp Segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Springer Nature Switzerland, 2022, pp. 151–160.

12.

, et al. LViT: Language Meets Vision Transformer in Medical Image Segmentation. IEEE Transactions on Medical Imaging 2024; 43: 96–107. https://doi.org/10.1109/TMI.2023.3291719

13.

Zhang

Luo

, et al. DescriptorMedSAM: language-image fusion with multi-aspect text guidance for medical image segmentation. Scientific Reports 2026; 16: 1–8. https://doi.org/10.1038/s41598-025-33843-5, Epub ahead of print January 12.

14.

Al-Dhabyani

Gomaa

Khaled

, et al. Dataset of breast ultrasound images. Data in Brief 2020; 28: 104863. https://doi.org/10.1016/j.dib.2019.104863

15.

Foumani

Tan

Webb

, et al. Improving position encoding of transformers for multivariate time series classification. Data Mining and Knowledge Discovery 2023; 38: 22–48. https://doi.org/10.1007/s10618-023-00948-2

16.

Mohammadi Foumani

Wei Tan

Salehi

. Disjoint-CNN for Multivariate Time Series Classification. In: 2021 International Conference on Data Mining Workshops (ICDMW). IEEE, 2021, pp. 760–769.

17.

Min

Wang

. Boundary-aware gradient operator network for medical image segmentation[J]. IEEE Journal of Biomedical and Health Informatics, 2024, 28(8): 4711–4723. https://doi.org/10.1109/jbhi.2024.3404273

18.

Rohrbach

Darrell

. Segmentation from Natural Language Expressions. In: European Conference on Computer Vision (ECCV). Springer International Publishing, 2016, pp. 108–124.

19.

Kuo

, et al. Referring Image Segmentation via Recurrent Refinement Networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 5745–5753.

20.

Liu

Lin

Shen

, et al. Recurrent Multimodal Interaction for Referring Image Segmentation. In: 2017 IEEE International Conference on Computer Vision (ICCV), 2017, pp. 1280–1289.

21.

Margffoy-Tuay

Pérez

Botero

, et al. Dynamic Multimodal Instance Segmentation Guided by Natural Language Queries. In: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 630–645.

22.

Shi

Meng

, et al. Key-Word-Aware Network for Referring Expression Image Segmentation. In: Proceedings of the European Conference on Computer Vision. ECCV), 2018, pp. 38–54.

23.

Rochan

Liu

, et al. Cross-Modal Self-Attention Network for Referring Image Segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 10502–10511.

24.

Chen

Tsai

Wang

, et al. Referring expression object segmentation with caption-aware consistency[J]. In: arXiv preprint arXiv:1910.04748 2019.

25.

Kamath

Singh

LeCun

, et al. MDETR: Modulated Detection for End-to-End Multi-Modal Understanding. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 1780–1790.

26.

Ding

Liu

Wang

, et al. Vision-Language Transformer and Query Generation for Referring Segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 16321–16330.

27.

Yang

Wang

Tang

, et al. LAVT: Language-Aware Vision Transformer for Referring Image Segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. CVPR), 2022, pp. 18155–18165.

28.

Kim

Lan

, et al. ReSTR: Convolution-Free Referring Image Segmentation Using Transformers. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 18145–18154.

29.

Wang

, et al. CRIS: CLIP-Driven Referring Image Segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, pp. 11686–11695.

30.

Seaver

. 11 Attention Is All You Need: Humans and Computers in the Time of Neural Networks. In: Scenes of Attention. Columbia University Press, pp. 230–248.

31.

Yang

Huang

, et al. CLCI-Net: Cross-Level Fusion and Context Inference Networks for Lesion Segmentation of Chronic Stroke. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Springer International Publishing, 2019, pp. 266–274.

32.

Chen

, et al. Learning Implicit Feature Alignment Function for Semantic Segmentation. European Conference on Computer Vision (ECCV). Springer Nature Switzerland, 2022, pp. 487–505.

33.

Huang

Wei

Wang

, et al. AlignSeg: Feature-Aligned Segmentation Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 2021; 44: 1.

34.

Wang

Zhu

Green

, et al. Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation. In: European Conference on Computer Vision (ECCV). Springer International Publishing, 2020, pp. 108–126.

35.

Woo

Park

Lee

, et al. CBAM: Convolutional Block Attention Module. In: Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 3–19.

36.

Liu

Jiao

Liu

. Artificial Intelligence (AI)-aided Disease Prediction. BIO Integration 2020; 1: 130–136. https://doi.org/10.15212/bioi-2020-0017, Epub ahead of print.

37.

Ronneberger

Fischer

Brox

. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). Springer International Publishing, 2015, pp. 234–241.

38.

Badrinarayanan

Kendall

Cipolla

. SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 2017; 39: 2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615

39.

Zhou

Rahman Siddiquee

Tajbakhsh

, et al. UNet++: A Nested U-Net Architecture for Medical Image Segmentation. In: International Workshop on Deep Learning in Medical Image Analysis (DLMIA). Springer International Publishing, 2018, pp. 3–11.

40.

Wang

Zhuang

. AttU-Net: Attention U-Net for Brain Tumor Segmentation. In: International MICCAI Brainlesion Workshop. Springer International Publishing, 2021, pp. 302–311.

41.

Chen

, et al. Transunet: Transformers make strong encoders for medical image segmentation[J]. In: arXiv preprint arXiv:2102.04306, 2021.

42.

Cao

Wang

Chen

, et al. Swin-UNet: U-Net-like Pure Transformer for Medical Image Segmentation. In: European Conference on Computer Vision (ECCV). Springer Nature Switzerland, 2022, pp. 205–218.

43.

Zheng

Liu

Feng

, et al. CASF-Net: Cross-attention and cross-scale fusion network for medical image segmentation. Computer Methods and Programs in Biomedicine 2023; 229: 107307. https://doi.org/10.1016/j.cmpb.2022.107307

44.

Zhang

Jin

, et al. VM-UNet-V2: Rethinking Vision Mamba UNet for Medical Image Segmentation. In: International Symposium on Bioinformatics Research and Applications. Springer Nature Singapore, 2024, pp. 335–346.

45.

Ning

Zhong

Feng

, et al. SMU-Net: Saliency-Guided Morphology-Aware U-Net for Breast Lesion Segmentation in Ultrasound Image. IEEE Transactions on Medical Imaging 2022; 41: 476–490. https://doi.org/10.1109/tmi.2021.3116087

46.

Selvaraju

Cogswell

Das

, et al. Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. In: Proceedings of the IEEE International Conference on Computer Vision. ICCV), 2017, pp. 618–626.

47.

Liu

Shao

, et al. Asym-UNet: An asymmetric U-shape Network for breast lesions ultrasound images segmentation. Biomedical Signal Processing and Control 2025; 99: 106822. https://doi.org/10.1016/j.bspc.2024.106822

48.

Xue

Zhu

, et al. Global guidance network for breast lesion segmentation in ultrasound images. Medical Image Analysis 2021; 70: 101989. https://doi.org/10.1016/j.media.2021.101989

49.

Zhang

Lee

, et al. Boundary-Guided and Region-Aware Network With Global Scale-Adaptive for Accurate Segmentation of Breast Tumors in Ultrasound Images. IEEE Journal of Biomedical and Health Informatics 2023; 27: 4421–4432. https://doi.org/10.1109/JBHI.2023.3285789

50.

Feng

Dong

Liu

, et al. HTBE-Net: A hybrid transformer network based on boundary enhancement for breast ultrasound image segmentation. Displays 2024; 84: 102753. https://doi.org/10.1016/j.displa.2024.102753

51.

Yang

Xie

. HCTNet: A hybrid CNN-transformer network for breast ultrasound image segmentation. Computers in Biology and Medicine 2023; 155: 106629. https://doi.org/10.1016/j.compbiomed.2023.106629

52.

Zhang

Lian

, et al. HAU-Net: Hybrid CNN-transformer for breast ultrasound image segmentation. Biomedical Signal Processing and Control 2024; 87: 105427. https://doi.org/10.1016/j.bspc.2023.105427

53.

Yang

. CSwin-PNet: A CNN-Swin Transformer combined pyramid network for breast lesion segmentation in ultrasound images. Expert Systems with Applications 2023; 213: 119024. https://doi.org/10.1016/j.eswa.2022.119024

54.

Fang

, et al. Driven by textual knowledge: A Text-View Enhanced Knowledge Transfer Network for lung infection region segmentation. Medical Image Analysis 2025; 103: 103625. https://doi.org/10.1016/j.media.2025.103625

55.

Qin

Lin

Gao

, et al. MBE-UNet: Multi-Branch Boundary Enhanced U-Net for Ultrasound Segmentation. IEEE Journal of Biomedical and Health Informatics 2026; 30: 575–585. https://doi.org/10.1109/JBHI.2025.3589293

56.

Wang

Lai

Kong

HTFN

. Transformer Fusion Network for Ultrasound Image Segmentation. In: Asian Conference on Pattern Recognition. Springer International Publishing, 2021, pp. 314–325.