Vision-based detection of bent circular tubes with ball joints

Abstract

Circular tubes and ball joints are widely used in large-span grid structures. Bending of circular tubes is a common form of damage in grid structures, posing a potential risk of global safety. Traditional manual inspections are labor and time cost, and usually need high-altitude operation. To address this issue, an automatic bent tube detection method based on computer vision and deep learning is proposed in this article. In the method, ball joints in the image are identified and localized using YOLOv5s object-detection network, and then the tubes are identified by semantic segmentation conducted by a DeepLab v3+ network enhanced with the contextual transformer block. The axes of the bent tubes are extracted from the segmented masks by the skeletonization algorithms. Finally, bent-tube deflections are assessed as the perpendicular distance from the bent-tube axis to the joint anchor box centerline. The proposed framework is validated by eccentric compression tests on three circular tubes with ball joints. It is shown that the average error in deflection assessment by the proposed framework is less than 2 mm, with a maximum error rate of 3.7% which is deemed acceptable in engineering application.

Keywords

Bent circular tubes vision-based detection deflection assessment computer vision semantic segmentation

Introduction

Grid structures are composed of interconnected members and joints arranged in specific geometric patterns.¹ This structural typology offers several advantages, including spatial efficiency, light weight, superior seismic performance, and the capability to span large distance.² As a result, grid structures are widely used in large-span buildings such as train stations,³ stadiums,⁴ and airport terminals.⁵ In these large-span grid structures, circular tubes are usually used as the main load-bearing members, and ball joints including bolted balls and hollow welded balls are usually used to connect the members. Given the importance of these public facilities, timely and proactively maintenance of in-service grid structures is essential for preventing structural failures and ensuring public safety.⁶

Member bent is one of the main damages observed in in-service grid structures, as shown in Figure 1. Initial imperfections, unnormal loading conditions, and unfavorable support conditions are the possible reasons that cause unrecoverable bending of the members.⁷ The bending deformation reduces the axial load bearing capacity of the member. Moreover, the member’s load-bearing capacity will further decrease due to the P-Delta effect.⁸ Severe member bending may trigger progressive collapse of the whole structural system. Therefore, the timely and accurately detection of bent members is critical for the proactive maintenance and safety evaluation of in-service grid structures.

At current, detection of bent members is mainly relying on manual inspection.⁹ As shown in Figure 2, manual inspection of bent members usually needs to work at height and measure the member deflections by handheld tools. Obviously, the manual inspection is labor and time cost, and exposes the inspectors to safety hazards due to the substantial height clearances, particularly in aging grid structures.^10,11

The emergence of computer vision technology offers a promising way to efficient detection of bent tube. Vision-based methodologies have been extensively investigated and successfully implemented in various structural inspection applications, particularly in damage detection^12–15 and deformation measurement.^17–24 Traditional vision-based methods can be generally classified into three main categories, that is, coded-target photogrammetric methods, surface-point tracking methods, and surface comparison methods.¹⁶ Han et al.¹⁷ developed a deflection measurement method for suspension bridges by combining unmanned aerial vehicle (UAV) photography with a stationary laser spot reference. In a separate study, Xu et al.¹⁸ introduced a feature point-based template matching technique to simultaneously track multiple points on a cable-stayed footbridge, enabling modal frequency estimation for both the deck and cables. Jiang et al.¹⁹ proposed a deep learning-based displacement calculation method incorporating Kalman filter for target trajectory prediction. Yang et al.²⁰ integrated digital image correlation (DIC) techniques with advanced video processing and unsupervised learning algorithms for structural modal analysis. Cheng et al.²¹ utilized a visual sensor system to determine the spatial positions of measurement points during vibration testing of a double suspension system. Ri et al.²² introduced an approach combining a phase-based sampled moiré technique with four-degree-of-freedom geometric modeling, utilizing placed markers on bridge girders to differentiate between actual bridge displacements from apparent displacements induced by camera motion. Sun and Huang²³ implemented a comprehensive deformation monitoring strategy by leveraging DIC techniques with clustering algorithms for large-span steel structures during construction. Wang et al.²⁴ employed advanced subpixel localization techniques to monitor steel truss structure buckling and load identification. However, these traditional approaches exhibit significant limitations for in-service grid structures, primarily due to their dependence on historical deformation data, which are often unavailable in practical applications.

The emergence of deep learning has fundamentally improved vision-based inspection capabilities, enabling deformation assessment from individual images without relying on historical sequential image data.²⁵ Recently, vision-based methods incorporated with deep learning have gained significant attention in structural health monitoring and damage detection.²⁶ Gao and Mosalam²⁷ constructed a multiscale image dataset encompassing pixel, component, and structural levels, proposing a deep learning approach for concurrent various visual tasks, such as component type recognition and damage assessment. Yu et al.²⁸ introduced a deep learning-assisted DIC method for full-field deflection and cracks detection in concrete beams under laboratory conditions. Atha and Jahanshahi²⁹ introduced a CNN-based method for detecting and quantifying surface corrosion on steel structures. Cha et al.³⁰ developed a fast R-CNN-based method for real-time multidamage detection, including cracks, rebar corrosion, and delamination. Choi and Cha³¹ developed a convolutional neural network designed for real-time segmentation of concrete cracks in images, whereas Kang and Cha³² introduced a method for evaluating the complexity of image scenes. Zhai et al.³³ employed cluster segmentation for column edge displacement fitting to monitor dynamic deformation of reinforced concrete (RC) column with fibre reinforced plastics (FRP) tubes in axial compression tests. Several researchers have employed UAVs in conjunction with deep learning algorithms to assess structural damage in indoor environments. Their investigations have encompassed various research directions, including target-based localization methodologies,^34,35 image stabilization acquisition techniques,³⁶ and additional related areas. These advancements have facilitated the detection of bent members in grid structures through visual information extracted from imagery. Although the promising potential of deep learning-enhanced vision-based methods in structural detection, to the best of authors’ knowledge, they are not able to be directly used to detect bent members in in-service grid structures, specifically tailoring being needed.

To fill the gap between the existing method and the demand for efficient and accurate in in-service grid structures, this article develops a novel bent tube detection (BTD) method based on computer vision and deep learning. The outline of the article is as follows: The second section presents the workflow of the proposed BTD method. The third section introduces the dataset building and training details of the networks. In the fourth section, eccentric loading experiments are conducted to verify the effectiveness of the proposed method. Finally, the fifth section concludes the article.

Vision-based framework for bent-tube detection

Currently, there are various computer vision techniques, including object detection, that are able to extract and truncate images of individual members from photographs of grid structures.³⁷ In this article, we consider a target member with ball joints has been truncated from the captured image, as shown in Figure 3.

The main flow of the proposed BTD method is as shown in Figure 4. First, spherical joints are located within the input image. Second, high-fidelity semantic segmentation of the targeted tubes is conducted to obtain their geometric contours. Third, the medial axis of the segmentation results is extracted. Finally, the deflection of the member is calculated by combining the positions of the spherical joints and the tube axis. The details of above procedures are given as follows.

Identification and localization of ball joints

In this study, the lightweight YOLOv5s model which is a state-of-art single stage detector is adopted for its efficient parameterization while maintaining competitive performance.³⁸ As illustrated in Figure 5, the adopted YOLOv5s architecture comprises four principal components: the input layer, backbone network for feature extraction, neck network for feature aggregation, and detection head for final predictions.

In the input of YOLOv5s, the image containing tube with joints undergoes multiple preprocessing steps, including geometric and color transformation, mosaic data augmentation, image size normalization, and adaptive anchor box computation. The backbone architecture implements CSPDarknet architecture³⁹ for feature extraction, comprising three key modules: CBS for standardized convolution operations, C3⁴⁰ for cross-stage partial feature integration, and SPPF⁴¹ for optimized spatial pyramid pooling. This backbone architecture processes the edge and shape information of the joint while implementing dimensional reduction to enhance computational efficiency. The neck architecture of YOLOv5s inherits and optimizes the hybrid feature pyramid structure that integrates FPN and PAN from its predecessors. The detection head architecture integrates convolutional layers, pooling operations, and fully connected layers to enable multiscale target detection within the hierarchical feature pyramid network. This architecture facilities precise joint localization through parametrized bounding boxes. The spatial coordinates and dimensional attributes of each joint are determined within the image coordinate system using the bounding box parameters.

High-fidelity tube semantic segmentation

To facilitate the precise extraction of geometric contour of the target tube, we implemented a semantic segmentation methodology on the acquired imagery data. This deep learning-based approach enables pixelwise classification and boundary delineation of structural components. The DeepLab v3+⁴² establishes a new benchmark in semantic segmentation by integrating the spatial pyramid pooling module with an encoder–decoder structure in deep neural networks.⁴³ It has demonstrated superior performance in detecting structural defects compared to other end-to-end deep convolutional networks.⁴⁴ However, while DeepLab v3+ excels in natural scene understanding and general classification tasks, its original design compromises edge segmentation capability which is a crucial aspect for tube deflection assessment. Specifically, the network’s inherent focus on model generalization limits its performance in precise boundary delineation, particularly for bent tube structures.

To overcome this limitation, a novel contextual transformer (CoT) block⁴⁵ is incorporated into DeepLab v3+ to improve edge detection and feature representation for segmentation of tubular components. As shown in Figure 6, CoT block extends traditional self-attention architectures⁴⁶ by incorporating neighboring contextual information, rather than relying only on isolated query key pairs at each spatial location. The proposed network is named Tb-DeepLab v3+ and illustrated in Figure 7. It is clearly shown that the Tb-DeepLab v3+ maintains the fundamental encoder–decoder framework while introducing specialized components for tube-specific feature learning.

The integration of CoT block into DeepLab v3+ enhances both the feature representation capabilities and network robustness. Through the innovative fusion of static and dynamic attention matrices, the CoT module achieves refined segmentation, particularly in edge details.⁴⁵ This improves the network’s performance in complex engineering scenarios, offering more precise recognition of structural members.

Axis extraction of tube

Following the geometric contour segmentation using the Tb-DeepLab v3+ network, axis extraction is performed to evaluate the tube deflection. The Zhang–Suen thinning algorithm is employed for its capability to generate connected, single-pixelwide skeletons while preserving topological properties.^47,48

As illustrated in Figure 8, the Zhang–Suen thinning algorithm iteratively examines each boundary pixel (P₁) against its neighboring pixels (P₂–P₈) using connectivity-preserving criteria. Specifically, it preserves critical topological features including endpoints, connection points, and internal pixels to maintain structural connectivity. This iterative process continues until no further pixels can be removed, yielding a well-defined centerline that accurately represents the tube’s geometric axis.

Deflection calculation of bent tube

YOLOv5s network is used to detect the joints and their centers in the acquired image. The pixel-level tube axis is then extracted using Zhang–Suen thinning algorithm.⁴⁸ The extracted tube dimensions are calibrated using the actual joint dimensions which could easily be measured to establish the pixel-to-physical conversion ratio. If the shooting angle of the image is orthogonal to the plane where the maximum bending deformation of the tube member occurs, the deflection of the member can be determined as the distance from the axis to the line connecting the centers of the joint, as shown in Figure 9.

In practice, it is almost impossible to ensure that the shooting angle of the image is orthogonal to the maximum bending deformation plane of the target tube member. As a result, biangle images are used to determine the maximum bending deformation, that is, the deflection, of the target tube member.

As shown in Figure 10, the angle θ between two shooting directions can be readily controlled and measured using construction drawings or geometric models of the structure. According to projection principles, the measured deflections represent the projections of the actual deflection vector onto the respective camera incident planes. The true deflection value d can be derived from two measurement values through geometric relationship:

d^{2} = \frac{a^{2} + b^{2} - 2 ab \cos θ}{\sin^{2} θ}

(1)

where a and b are the deflection values obtained from the two image captures, respectively.

In summary, the main procedures of the proposed BTD method can be represented as Figure 11.

Networks establishment for bent-tube detection

Dataset building

A dataset is established for training advanced models of joint object detection and tube semantic segmentation in grid structures. The dataset comprises a total of 438 high-quality images collected from real grid structures in neighborhood and online resources. Typical examples of them are as shown in Figure 12.

The dataset is expanded to 1750 images through data augmentation techniques, including rotation, flipping, and brightness adjustments. This process enhances data volume and diversity, and improves model robustness. Following standard machine learning practices, the dataset is partitioned into 1400 training images (80%) and 350 testing images (20%). Object labeling is conducted using the Roboflow platform to annotate the dataset with bounding boxes and class labels. Segmentation labeling is performed with the LabelMe tool, using contours of bent tubes.

Evaluation metrics

Precision, recall, and mean average precision (mAP) serve as standard metrics for quantitative evaluation of detection model performance.⁴⁹ Precision measures the accuracy of the model’s predictions by calculating the ratio of true positive predictions among all positive predictions made by the model. Recall, on the other hand, evaluates the model’s ability to identify all relevant positive instances by calculating the ratio of true positive predictions among all actual positive instances in the dataset. The precision and recall can be expressed as follows:

Precision = \frac{TP}{TP + FP}

(2)

Recall = \frac{TP}{TP + FN}

(3)

where TP (true positive) represents number of correctly classified positive samples; FP (false positive) denotes the number of negative samples misclassified as positive; and FN (false negative) indicates the number of positive samples misclassified as negative. The AP metric measures the precision rate averaged across all recall levels by integrating the precision-recall curve, while the mAP is the average of AP values across different classes or models.

AP = \int_{0}^{1} P (R) d R

(4)

mAP = \frac{\sum_{i = 1}^{C} A P_{i}}{N_{C}}

(5)

where $N_{C}$ represents number of detected categories; while $P$ and $R$ are the precision and recall, respectively.

The Intersection over Union (IoU) metric is a crucial measure of detection accuracy in segmentation tasks.⁵⁰ It quantifies the overlap between the predicted bounding box and the ground truth. Mathematically, IoU is defined as the ratio of their intersection area to union area:

IoU = \frac{Area of Overlap}{Area of Union}

(6)

In bent-tube detection, a higher IoU suggests more accurate segmentation of the tube member from the image. The mean IoU (mIoU) extends IoU to multiclass scenarios by averaging IoU values across all classes in the dataset. This provides a comprehensive assessment of segmentation accuracy across different bent-tube categories. Similarly, the mAP@0.5: 0.95 metric evaluates detection performance by averaging precision values across multiple IoU thresholds, specifically from 0.5 to 0.95 in steps of 0.05. This approach evaluates model performance under both relaxed (IoU = 0.5) and strict (IoU = 0.95) matching criteria, yielding a comprehensive assessment of the framework’s effectiveness.

Training processes

All our experiments are conducted using PyTorch 3.10 framework on Windows. A workstation equipped with an NVIDIA Tesla V100 (24GB GPU) and an Intel(R) Xeon(R) Silver 4314 CPU is used to run the experiments.

YOLOv5s

Transfer learning is employed to enhance optimization efficiency, utilizing the pretrained weights from YOLOv5s.pt, which are based on the COCO dataset,⁵¹ as initialization parameters. To maintain network stability, a warm-up phase of five epochs is implemented. Training process is monitored through systematic logging weights and performance metrics. The experimental hyperparameters are detailed in Table 1.

Table 1.

Experimental hyperparameter values of YOLOv5s.

Hyperparameters	Values
Batch size	24
Initial learning rate	0.01
Termination learning rate	0.19
Epochs	100
Num-workers	12
IoU threshold	0.8
Adaptive image scaling size	512 × 512

The performance curves depicting loss and mAP@0.5: 0.95 for the YOLOv5s model over 100 epochs, are presented in Figure 13. The transfer learning strategy enables efficient learning of abstract features, leading to rapid convergence within the first 60 epochs with mAP@0.5: 0.95 swiftly reaching 0.85. After 80 epochs, both loss and mAP curves gradually stabilize and maintain consistency until the end of the training process. The mAP@0.5: 0.95 stabilizes after reaching a high value 0.93, which indicates optimal model convergence without overfitting or underfitting.

Tb-DeepLab v3+ network

The hyperparameters utilized for training the Tb-DeepLab v3+ network is presented in Table 2. Training and validation losses are monitored through systematic logging. To evaluate model performance, cross-entropy loss (L_CE) and Dice loss (L_Dice) are selected. L_CE measures the difference between the true labels and the predicted probabilities, guiding the training process, while L_Dice assesses the sample similarity to mitigate the negative impact of foreground–background imbalance. These loss functions which complement each other in the segmentation task are as follows:

L_{CE} = - \frac{1}{N} \sum_{i = 1}^{N} [y_{i} \log ({\hat{y}}_{i}) + (1 - y_{i}) \log (1 - {\hat{y}}_{i})]

(7)

L_{Dice} = 1 - \frac{2 \sum_{i = 1}^{N} y_{i} {\hat{y}}_{i}}{\sum_{i = 1}^{N} y_{i}^{2} + \sum_{i = 1}^{N} {\hat{y}}_{i}^{2}}

(8)

where N is the number of pixels; $y_{i}$ is the true label; and ${\hat{y}}_{i}$ is the predicted probability for the $i$ th pixel.

Table 2.

Experimental hyperparameter values of Tb-DeepLab v3+.

Hyperparameters	Values
Batch size	12
Downsample factor	16
Freeze epochs	50
Total epochs	300
Optimizer	Stochastic Gradient Descent (SGD)
Learning rate decay	Cosine
Initial learning rate	0.007
Momentum	0.9
Adaptive image scaling size	512 × 512

The segmentation performance of the Tb-DeepLab v3+ network is evaluated using the established dataset, as illustrated in Figure 14. During the initial 50 epochs of freeze learning, the network converges rapidly, with loss decreasing swiftly to 0.16, indicating the effectiveness of freeze learning strategy, leveraging through the pretrained model. When transitioning to full network training with all weights unfrozen, a brief rebound in loss occurs, followed by rapid loss recovery after the 55th epoch. The loss curve gradually stabilizes after 150 epochs, with the final mIoU reaching 0.93. These results demonstrate that the improved Tb-DeepLab v3+ network achieves comparable performance to manual labeling, thus validating the effectiveness of the proposed segmentation network.

The effectiveness of the proposed Tb-DeepLab v3+ network is evaluated through an ablation study. The results of different architectures used in the ablation experiments are presented in Figure 15. Identical parameters (e.g., dataset, learning rate) are used across all ablation experiments to ensure a fair comparison. The original DeepLab v3+ network serves as baseline. With the incorporation of the CoT block, the Tb-DeepLab v3+ network yields a 4.2% improvement in mIoU. This improvement demonstrates that the CoT module’s effectiveness in enhancing circular edge segmentation of tube fittings. Consequently, the proposed Tb-DeepLab v3+ network demonstrates superior performance for tube fittings segmenting task.

Comparative experiments are also conducted on multiple advanced CNN-based segmentation networks using the same dataset. To ensure a fair comparison, all models are trained using a transfer learning strategy, initializing the model parameters with pretrained weights from the COCO dataset and trained for 50 epochs under identical conditions. As shown in Table 3, Tb-DeepLab v3+ achieved a high mIoU of 88.7%. This accuracy is competitive with, and in some cases surpasses, that of recent models like Mask2Former (87.1%) and is comparable to segment anything 2 (89.5%). Crucially, “Tb-DeepLab v3+” demonstrates significantly superior computational efficiency. It has substantially fewer parameters (6.72M) and lower floating point operations (FLOPs) (54.87G) compared to Mask2Former (52.3M params, 190G FLOPs) and especially Segment Anything 2 (636M params, 6.4 × 105 G FLOPs). Furthermore, with an inference speed of 51.7 ms, our model is the fastest among those evaluated, which is vital for enabling practical, near real-time inspection. The results demonstrate that our enhanced DeepLab v3+ achieves an optimal balance between the high segmentation accuracy required for precise deflection assessment and the computational efficiency necessary for a deployable engineering solution.

Table 3.

Comparison with state-of-the-art segmentation methods.

Models	Backbone	Input size (pixels)	Para (M)	FLOPs (G)	mIoU (%)	Speed (ms)
Tb-DeepLab v3+	MobileNetv2	512 × 512	6.72	54.87	88.7	51.7
U-Net	ResNet-50	512 × 512	43.93	184.10	75.3	53.3
FCN	VGG16	512 × 512	32.89	91.56	67.8	142.8
PSPNet	ResNet-50	512 × 512	46.70	118.45	67.6	113.6
Segment anything 2	Vision transformer	1024 × 1024	636	6.4 × 10⁵	89.5	220.0
Mask2Former	ResNet-50	512 × 512	52.3	190	87.1	93.8
HRNet-W32	HRNet	480 × 480	30.7	82.1	80.4	87.1

Both ablation studies and comparative analysis validate the effectiveness of the proposed Tb-DeepLab v3+ network for tube segmentation, demonstrating its potential for practical applications.

Experimental validation

Experimental settings

To further validate the proposed BTD method, eccentric compression tests are conducted on three circular tubes with ball joints, as illustrated in Figure 16. Steel with a nominal yield strength of 355 MPa is used for the specimens. Each specimen is made of a 1500-mm long circular tube connected to hollow spherical joints at both ends. The circular tube has an outer diameter of 114 mm and a wall thickness of 4 mm. The hollow spherical joints have a diameter of 300 mm and a wall thickness of 8 mm.

The eccentric compression tests are conducted using an electrohydraulic servo machine with a maximum loading capacity of 10,000 kN. The tests employ a displacement-controlled loading procedure at a rate of 0.2 mm/min. As shown in Figure 16, knife-hinge devices are positioned at both ends of the specimen to provide pin-ended boundary conditions. Prior to testing, specimen alignment is calibrated using laser and a spirit level measurement to ensure that (i) the longitudinal axis of the tube is perpendicular to the knife-hinges and (ii) the mid-axis of the circular tube is parallel to the knife-hinges at prespecified initial loading eccentricity. Five total station reflectors and displacement meters are stuck to the tube to monitor lateral displacements during the loading process. The vertical shortening of the specimen is recorded by the displacement transducer embedded in the loading machine. Tests terminate at 80% postpeak load or unsafe deformation occurring. Lateral displacement measurements and images are collected at 4-mm vertical displacement intervals, beginning at an 8-mm vertical displacement, using a Canon EOS 50D camera.

Experimental results

Due to the slippage of the displacement meter during the experiment, only data from the total station is utilized for comparison with the proposed method. Images from the tests are processed with the YOLOv5s and the Tb-DeepLab v3+ network trained in the third section and then the deflections of the specimens are determined by employing the proposed algorithms of axis extraction, and deflection calculation.

Figure 17 compares deflections from the proposed vision-based detection method with those from the total station measurements. The shape of the line illustrates the deformation of the tube, while the inflection points indicate the deflection measurements at each point along the tube. The loading process encompasses six vertical displacement intervals of 4 mm each. The two curves representing these methods demonstrate general consistency, and the accompanying histograms illustrate that the discrepancies between the two methods are substantially smaller than the deflections. Table 4 summarizes the average maximum deflections of the three specimens at every interval from both measurements, along with the average errors and error rates of the measurements from the proposed method against the total station. It is found that the maximum error and the maximum error rate are 1.72 mm and 3.7%, respectively. This level of error is deemed to be acceptable in practical applications.

Table 4.

Errors between deflection measurements with different methods.

Vertical displacements (mm)	Maximum deflection (mm)		Average errors (mm)	Error rates (%)
Vertical displacements (mm)	Image	Total station	Average errors (mm)	Error rates (%)
8	20.93	21.13	0.61	2.9
12	31.63	32.33	1.21	3.7
16	43.69	45.02	1.32	2.9
20	56.70	56.98	0.53	0.9
24	66.43	67.42	1.52	2.3
28	76.43	75.87	1.72	2.3

Conclusions

This study proposes a novel vision-based method for detection of bent tube with ball joints. An object-detection network using YOLOv5s is trained to identify and localize ball joints. The network convergence with mAP@0.5: 0.95 of 0.93. The DeepLab v3+ network is incorporated with CoT block to enhance its ability in edge segmentation of tube members. The enhanced DeepLab v3+ network, which is named Tb-DeepLab v3+, exhibits a 4.2% improvement in mIoU. Eccentric compression tests on three circular tubes with ball joints are conducted to verify the effectiveness of the propose method. Compared with total station measurements, the proposed method shows error rates within 3.7% which is deemed as acceptable for practical applications.

This study targets the detection of bending in circular tube members with ball joints, which are commonly used in grid structures. The proposed method is not applicable to members with different cross-sections or other types of joints. Additionally, our research currently focuses on deflection detection at the member level and has not been extended to structural-level applications. Images of members taken in real structures often suffer from occlusion by surrounding members, which can significantly limit segmentation accuracy. Moreover, images captured in real structures are constrained by access conditions, posing challenges to acquiring complete visual information of the target members and joints. These practical constraints complicate the acquisition of comprehensive and high-quality data for analysis.

To gain high-quality image from real grid structures, quadruped robots and drones will be considered as vehicles for filming equipment in our future work. In another hand, follow-up work will also focus on upgrading the adopted models based on the current research. While the YOLOv5s and Tb-DeepLab v3+ networks have demonstrated good accuracy and robustness, the rapid advancements in computer vision modeling present opportunities for improvement. We plan to explore and implement customized enhancements on latest models to further improve detection accuracy.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors gratefully acknowledge the support from the National Key Research and Development Program of China (2023YFC3805602) and the Funding of Center for Balance Architecture, Zhejiang University.

ORCID iD

Tingting Shu

References

Huybrechts

Tsai

S W.

Analysis and behavior of grid structures. Compos Sci Technol 1996; 56(9): 1001–1015.

Dong

Zhao

Xing

Application and development of modern long-span space structures in China. Front Struct Civ Eng 2012; 6: 224–239.

Luo

Chen

Wan

, et al. Development of laser-based displacement monitoring system and its application to large-scale spatial structures. J Civ Struct Health Monit 2021; 11: 381–395.

Wang

Luo

Sun

, et al. Field measurement system based on a wireless sensor network for the wind load on spatial structures: design, experimental, and field validation. Struct Control Health Monit 2018; 25(9): e2192.

Liu

Ying

Chen

, et al. Research status of mechanical properties of aluminum alloy grid structure. Structures 2024; 61: 105967.

Wei

Fan

Liu

, et al. Automated inspection and monitoring of member deformation in grid structures. Comput Aided Civ Infrastruct Eng 2022; 37(10): 1277–1297.

Liu

Fan

, et al. Refined safety assessment of steel grid structures with crooked tubular members. Autom Constr 2019; 99: 249–264.

Yao

Taby

Moan

Ultimate strength and post-ultimate strength behavior of damaged tubular members in offshore structures. J Offshore Mech Arct Eng 1988; 110(3): 254–262.

GB 50292-2015. Standard for appraisal of reliability of civil buildings. Beijing: China Architecture and Building Press, 2015 (in Chinese).

10.

Nadhim

Hon

Xia

, et al. Falls from height in the construction industry: a critical review of the scientific literature. Int J Environ Res Public health 2016; 13(7): 638.

11.

Thoft-Christensen

Infrastructures and life-cycle cost-benefit analysis. Struct Infrastruct Eng 2012; 8(5): 507–516.

12.

Cheng

JCP

Wang

. Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques. Autom Constr 2018; 95: 155–171.

13.

Jin Lim

Hwang

Kim

, et al. Steel bridge corrosion inspection with combined vision and thermographic images. Struct Health Monit 2021; 20(6): 3424–3435.

14.

Zhang

Huang

, et al. Robust multitask compressive sampling via deep generative models for crack detection in structural health monitoring. Struct Health Monit 2024; 23(3): 1383–1402.

15.

Wang

Yin

, et al. Automation railway fastener tightness detection based on instance segmentation and monocular depth estimation. Eng Struct 2025; 322: 119229.

16.

Scaioni

Feng

Barazzetti

, et al. Image-based deformation measurement. Appl Geomat 2015; 7: 75–90.

17.

Han

Feng

Vision-based displacement measurement using an unmanned aerial vehicle. Struct Control Health Monit 2022; 29(10): e3025.

18.

Brownjohn

Kong

A non-contact vision-based system for multipoint displacement monitoring in a cable-stayed footbridge. Struct Control Health Monit 2018; 25(5): e2155.

19.

Jiang

Zhang

Gao

Bridge deformation measurement using unmanned aerial dual camera and learning-based tracking method. Struct Control Health Monit 2023; 2023(1): 4752072.

20.

Yang

Dorn

Mancini

, et al. Blind identification of full-field vibration modes from video measurements with phase-based video motion magnification. Mech Syst Signal Process 2017; 85: 567–590.

21.

Cheng

Zhang

Mode shape-aided cable force estimation of a double-hanger system using a vision-based monitoring method. Measurement 2024; 227: 114214.

22.

Toyama

, et al. Drone-based displacement measurement of infrastructures utilizing phase information. Nat Commun 2024; 15(1): 395.

23.

Sun

Huang

Automatic deflection measurement for outdoor steel structure based on digital image correlation and three-stage multi-scale clustering algorithm. Autom Constr 2024; 163: 105416.

24.

Wang

Liu

, et al. Photogrammetry-based bending monitoring and load identification of steel truss structures. Adv Struct Eng 2023; 26(13): 2543–2561.

25.

Gkioxari

Dollár

, et al. Mask r-cnn. In: Proceedings of the IEEE international conference on computer vision, Venice, Italy, 22–29 October 2017. New York: IEEE, pp. 2961–2969.

26.

Cha

Ali

Lewis

, et al. Deep learning-based structural health monitoring. Autom Constr 2024; 161: 105328.

27.

Gao

Mosalam

KM.

Deep transfer learning for image-based structural damage recognition. Comput Aided Civ Infrastruct Eng 2018; 33(9): 748–768.

28.

Zhang

Zhu

, et al. Full-field deformation measurement and cracks detection in speckle scene using the deep learning-aided digital image correlation method. Mech Syst Signal Process 2024; 209: 111131.

29.

Atha

Jahanshahi

MR.

Evaluation of deep learning approaches based on convolutional neural networks for corrosion detection. Struct Health Monit 2018; 17(5): 1110–1128.

30.

Cha

Choi

Suh

, et al. Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Comput Aided Civ Infrastruct Eng 2018; 33(9): 731–747.

31.

Choi

Cha

YJ.

SDDNet: Real-time crack segmentation. IEEE Trans Ind Electron 2019; 67(9): 8016–8025.

32.

Kang

Cha

YJ.

Efficient attention-based deep encoder and decoder for automatic crack segmentation. Struct Health Monit 2022; 21(5): 2190–2205.

33.

Zhai

AHM

Luo

, et al. Dynamic image segmentation and recognition measurement of axial compression experiment based on image clustering and semantic segmentation in RC column with FRP tubes. Measurement 2024; 227: 114207.

34.

Waqas

Kang

Cha

YJ.

Deep learning-based obstacle-avoiding autonomous UAVs with fiducial marker-based localization for structural health monitoring. Struct Health Monit 2024; 23(2): 971–990.

35.

Kang

Cha

YJ.

Autonomous UAVs for structural health monitoring using deep learning and an ultrasonic beacon system with geo-tagging. Comput Aided Civ Infrastruct Eng 2018; 33(10): 885–902.

36.

Ali

Kang

Suh

, et al. Real-time multiple damage mapping using autonomous UAV and deep faster region-based neural networks for GPS-denied structures. Autom Constr 2021; 130: 103831.

37.

Zou

Chen

Shi

, et al. Object detection in 20 years: a survey. Proc IEEE 2023; 111(3): 257–276.

38.

Horvat

Jelečević

Gledec

A comparative study of YOLOv5 models performance for image localization and classification. In: Central European conference on information and intelligent systems, Dubrovnik, Croatia, 21–23 September 2022. Zagreb: University of Zagreb, pp. 349–356.

39.

Wang

Liao

HYM

, et al. CSPNet: a new backbone that can enhance learning capability of CNN. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, 14–19 July 2020. America: IEEE, pp. 390–391.

40.

Park

Yoo

Seo

, et al. C3: concentrated-comprehensive convolution and its application to semantic segmentation. arXiv:1812.04920, 2018. doi: 10.48550/arXiv.1812.04920.

41.

Zhang

Ren

, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE transactions on pattern analysis and machine intelligence 2015; 37(9): 1904–1916.

42.

Chen

Zhu

Papandreou

, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV), Munich, Germany, 8–14 September 2018. Berlin: Springer, pp. 801–818.

43.

Cho

Van Merriënboer

Gulcehre

, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. arXiv:1406.1078, 2014. doi: 10.48550/arXiv.1406.1078.

44.

Wang

Liu

Nie

, et al. Deep convolutional neural networks for semantic segmentation of cracks. Struct Control Health Monit 2022; 29(1): e2850.

45.

Yao

Pan

, et al. Contextual transformer networks for visual recognition. IEEE Trans Pattern Anal Mach Intell 2022; 45(2): 1489–1500.

46.

Vaswani

Attention is all you need. Adv Neural Inf Process Syst 2017; 30: 5998–6008.

47.

Saeed

Tabędzki

Rybnik

, et al. K3M: a universal algorithm for image skeletonization and a review of thinning techniques. Int J Appl Math Comput Sci 2010; 20: 317–335.

48.

Zhang

Suen

CY.

A fast parallel algorithm for thinning digital patterns. Commun ACM 1984; 27(3): 236–239.

49.

Everingham

Van Gool

Williams

CKI

, et al. The pascal visual object classes (voc) challenge. Int J Comput Vis 2010; 88: 303–338.

50.

Uijlings

JRR

Van De Sande

Gevers

, et al. Selective search for object recognition. International journal of computer vision 2013; 104: 154–171.

51.

Lin

Maire

Belongie

, et al. Microsoft coco: common objects in context. In: Computer vision–ECCV 2014: 13th European conference, Zurich, Switzerland, 6–12 September 2014. Berlin: Springer, pp. 740–755.