Abstract
This article proposes a pulse-echo ultrasonic propagation imaging system that is capable of autonomous target recognition and scanning an area of a customizable shape for the non-destructive evaluation of structural defects. The proposed system employs through-the-thickness bulk waves for ultrasonic inspection, which is achieved by joining two laser beams: one each for the ultrasonic wave generation and sensing. Moreover, the system is capable of autonomously suggesting a suitable inspection area for a specimen placed in front of the scanning head. The scan area delimitation algorithm uses the specimen image and ascertains the specimen border by means of edge and contour detection operations, following which a scan area closely conforming to the specimen boundary is suggested. The system can scan an area of any arbitrary shape, thereby preventing any wasteful operations that may result from fixed shape (rectangular or square) scanning. A Q-switched laser is used for generating the aforementioned ultrasonic waves, while a laser Doppler vibrometer is used for sensing these signals. A dual-axis automated translation stage is applied for raster scanning of the specimen, and a data acquisition card is employed for taking measurements. A camera mounted on the scan head is used for imaging the specimen for the scan area detection. Graphical user interface software controls all the individual blocks of the system, while implementing the required image processing, scan area detection, signal acquisition, signal processing, and result display. The graphical user interface is created in C++ using the Qt framework. Moreover, Qt Widgets for Technical Applications is used for the result display, and the Open Source Computer Vision Library is employed for the implementation of basic image processing algorithms. Multi-threading is used for real-time updating of the scan results while the scanning is ongoing.
Keywords
Introduction
Non-destructive evaluation (NDE) techniques are used in a diverse set of applications, ranging from quality control in manufacturing processes to in-service damage inspection. Numerous NDE methods have been developed to date, including simple visual inspection, eddy current, X-ray, and ultrasonic testing (UT). Among these methods, UT offers the advantages of subsurface defect detection without any radiation hazards and has been demonstrated to operate effectively with composite materials. Conventional contact UT necessitates the use of a couplant to maximize the energy transfer from probe to specimen. Even non-contact UT using air-coupled transducers requires the probe to be in close proximity to the specimen. In recent years, laser-based UT has been widely adopted owing to its ability to generate broadband ultrasounds of diverse modes in the medium. 1 This is a non-contact inspection method that does not require the use of a couplant and allows the probe to operate at a distance from the specimen.2–5
Such a laser-based UT method was proposed by Hong et al. 6 for the non-destructive and contactless inspection of structures. The pulse-echo ultrasonic propagation imaging (PE UPI) technique makes use of an excitation and a sensing laser to generate and detect through-the-thickness ultrasounds in the specimen. This approach differs from laser UT methods utilizing in-plane guided waves from either a single source or multiple sources.7–10 In the case of the guided wave-based methods, analysis of the multi-mode Lamb waves requires complex post-processing in order to yield meaningful results, 11 and even then the exact shape of the actual damage cannot be visualized owing to the multi-modal and dispersive nature of the Lamb waves. 12 In contrast, signal processing is simple for through-the-thickness longitudinal waves as a result of their non-dispersive nature during a short travel time. 12 Moreover, through-the-thickness longitudinal ultrasonic waves yield superior results for the detection of thickness-related features, and it is, therefore, easy to locate a defect or damage along the specimen thickness. A compact and mobile PE UPI system operating on this principle has already been developed and deployed for defect evaluation. The developed system uses a two-axis linear translational stage to inspect an area on the target structure. The stage implements a raster scan pattern in order to cover the required scan area effectively and supports rectangular scan area shapes. However, depending on the actual specimen shape or region of interest, the constraints imposed by the rectangular scan area may result in additional scanning outside the area of interest, which naturally leads to a longer scan time and suboptimal resource use for the given inspection.
Scan path planning is an integral part of any NDE system that relies on scanning for inspection. This path planning can be done either offline or online. The offline methods use either fixed scan shape and scan path6,13 or it generates the scan path based on computer-aided design (CAD) model of the specimen; 14 optionally, specimen-sensing techniques may also be used to compliment the CAD driven scan path.15,16 While the former non-flexible approach results in scan paths not conforming to the specimen under test, the later approach adds pre-conditions to the inspections. On the other hand, the online scan path generation based on specimen imaging provides a flexible alternative but presents challenges such as specimen detection and calibration.17,18 This article details the development and operation of a customizable scan area PE UPI system that is capable of performing scans of any arbitrary shape. The scan path and scan shape are calculated online based on the boundaries of the specimen under test. This allows for efficient inspection of areas not conforming to rectangular shapes, while ensuring optimal resource use and scan times. The system can detect the required scan area based on autonomous detection of the specimen boundary.
The remainder of this article is organized into four sections. Section “PE UPI system for autonomous boundary detection and customizable area scanning” details the developed system and its operations. The merits of the customizable area scanning along with autonomous specimen detection are also discussed in this section. Section “Results” includes the inspection results and demonstrates the system savings in terms of both scan time and memory occupation. Finally, concluding remarks are provided in section “Conclusion.”
PE UPI system for autonomous boundary detection and customizable area scanning
The PE UPI system makes use of an excitation and a sensing laser to generate and detect through-the-thickness ultrasounds in a non-contact and couplant-less manner. A Q-switched Nd:YLF laser with a wavelength of 1053 nm is used as the excitation laser, and a continuous-wave laser Doppler vibrometer (LDV) with 633 nm wavelength is used for sensing the ultrasounds. Both the lasers are placed together in a scan head, and their beams are combined to generate and detect the ultrasonic waves in a pulse-echo fashion. As illustrated in Figure 1, this laser head is mounted on a two-axis translation stage system in order to scan a desired inspection area. 6

PE UPI system with camera for customizable area scanning: (a) system block diagram and (b) actual system.
The stage makes a raster scan pattern during the inspection to cover the area effectively. Ultrasonic measurements are performed throughout this scan area, resulting in a grid of such measurements once the scan is completed. Each measurement is initiated by a synchronization signal given by the stage to the excitation laser. This signal triggers the laser to generate a laser pulse. Subsequently, the excitation laser generates another synchronization signal that triggers the data acquisition (DAQ) to record the filtered ultrasonic signal sensed by the LDV. The resolution of measurements in the inspection area is controlled by selection of the interval between two successive measurement points. The interval also dictates the separation between two adjacent scan lines in the raster scan. Intervals of 0.1, 0.25, 0.5, and 1 mm are supported by the system. The stage moves continuously along each scan line in the raster scan pattern and generates synchronization signals along the way. The speed of the stage along with the interval determines the pulse repetition rate of the laser. The translation stage can perform scans at speeds of up to 400 mm/s.
Customizable scan shape PE UPI
In the initially developed PE UPI system, only rectangular scanning shapes were supported. 6 In the system, the user is required to enter the starting point along with the height and width of the intended inspection area, and the scanning starting point is fixed on the bottom left corner of the scan area. Figure 2 illustrates this scanning pattern and the measurements taken along the scan path.

Rectangular area and raster scan pattern of the PE UPI system.
Every measurement taken during the scan is placed in a three-dimensional (3D) array and can be viewed in a frame-wise or point-wise manner during or after the scan. The inspection results can be thought of as originating from a grid of measurement points overlapping the inspection area, where the system takes a pulse-echo measurement at each measurement point. The system can generate an ultrasonic wave propagation imaging (UWPI) movie consisting of all the frames of the 3D array. Figure 3 illustrates the organization of these measurements, with the same data organization as that used in Hong et al. 6

Organization of measurements in the PE UPI system.
The measurements are captured by means of a DAQ card, capable of 16-bit resolution; thus, each element in the measurement array is stored as a 16-bit number. In addition to the storage buffer, an additional buffer is allocated for applying out-of-place post-scan processing algorithms to the data. The data at each measurement point are recorded for approximately 51 μs at a sampling rate of 10 MHz, resulting in the creation of 512 samples per measurement point.
Naturally, the time required by the translational stage to complete a scan is dictated by the stage speed and length of each scan line in the raster scan. Thus, for a given speed, reducing the length of scan lines will shorten the inspection period. Moreover, for a given DAQ configuration and scan resolution, the total number of measurement points in the inspection is also dependent on the scan line lengths. Thus, optimizing the length of individual scan lines will ensure minimum possible memory usage, and the inspection data need less storage space. These storage space savings can have a meaningful impact, especially when the system is in repeated use either in industry for quality assurance or for periodic inspections of structural health monitoring. Shorter scan lines also result in increased operational life for both the detection and generation lasers.
The limitation of the rectangular scan shape results in suboptimal use of time and resources if the desired specimen or inspection area has a different shape. Consider, for example, a triangle-like inspection area. The rectangular shape scanning process will need to scan the bounding rectangle of the triangular area in order to cover the intended area, thereby generating unnecessary measurement points outside the triangular area of interest. This will result in a longer scan time, unnecessary memory occupation, and avoidable laser use.
In order to ensure optimal resource use and scan duration, the PE UPI system has been upgraded to support an inspection area of any arbitrary shape. As illustrated in Figure 1, a camera has been mounted on the scan head to capture an image of the specimen to be evaluated. The system determines the scan shape autonomously by detecting the specimen boundaries via edge detection (this process is covered in further detail in the following section). This scan area is then processed and converted into a grid of vertical scan lines of varying lengths separated by the interval. These variable length scan lines form the scan path to be followed by the laser head mounted on the translational stage. The translation stage has been modified to support raster scanning with the autonomously generated scan path and does not require any intervention from the user for scan path programming. This allows the system to scan specimen of any shape without the need for any downtime for programming the required scan path. Figure 4 presents an example of a triangular scan pattern generated by means of this customizable scan feature. Any arbitrary scan shape can be realized in a similar manner.

Customizable scan shape example: triangular scan pattern and its bounding box.
Autonomous boundary detection
The system selects the scan area based on the shape of the specimen placed in front of the scan head. This feature provides ease of operation from a user point of view and, if required, allows the system to operate in a fully autonomous mode, without user intervention. The fully autonomous operation may be useful in a manufacturing setup in which the system could be employed for quality assurance.
The specimen shape detection can be done through image segmentation, which is the process of portioning an image into different regions based on their attributes. This is a mature field, and a number of different algorithms have been proposed over the years, each with its own merits.19,20 Given the requirements of this system, advanced approaches based on machine learning or pattern recognition are not considered for implementation. We can classify the applicable segmentation algorithms into three categories, namely, threshold based, region based, and edge based. 21 The surface conditions of the specimen may vary depending on its material, but we can control its placement in front of the camera as well as its background. Thus, the edge-based segmentation is the most optimal choice. Contour searching is done after edge detection to ascertain the required inspection boundary.
A detailed flowchart of the developed algorithm is provided in Figure 5.

Algorithm for specimen boundary detection.
First, after capturing the image using the scan head-mounted camera, the specimen mounting mechanisms such as brackets or shelves are removed. Thereafter, the image is converted from a three-channel red green blue (RGB) representation into a single-channel grayscale format, where the values of each pixel are represented within the range of 0–255. Noise removal is then performed using a Gaussian blur smoothing filter, in order to suppress false alarms in edge detection. A Canny detector was selected for detecting the edges in the resulting image.22,23 This is a multi-step edge detection algorithm, which begins by applying a Sobel kernel to obtain the first derivative in both the horizontal (Ih) and vertical (Iv) directions. These two derivative images are used to determine the edge gradient magnitude and direction of each pixel, as illustrated in equation (1)
It should be noted that the gradient direction is always perpendicular to an edge, and it is rounded to one of the four angles representing the horizontal, vertical, and two diagonal directions. The second step of the algorithm comprises edge thinning. This process is also known as non-maximum suppression, in which it is determined whether every pixel is checked and is a local maximum in its neighborhood in the gradient direction. As the gradient is perpendicular to the edges, this results in a binary image of thin edges. As a final step, a hysteresis threshold is applied to filter out the unwanted edges. An upper and a lower threshold are specified, whereby all the edges with a gradient magnitude greater than the upper threshold are marked as strong and included in the final result. The edges with a magnitude less than the lower threshold are promptly marked as weak and discarded. The edges with a magnitude between the two thresholds are only included in the final result if they are connected to a strong edge. This Canny edge detector is favored owing to its resilience against noise interference. The binary image containing the edges detected by the Canny operator is used for contour detection. Finally, the scan edges are marked on the specimen image according to its boundary. It should be noted that the scan head is moved so that the specimen is placed roughly in the center of the camera field of view. In this manner, non-specimen edges such as those from lighting or other background objects can easily be neglected. It is also necessary to capture an image fully indicating the specimen boundaries, as any edges extending outside the image are also discarded. Figure 6 illustrates the results of the boundary detection algorithm on a composite patch pasted on top of an aluminum plate. The boundaries are marked on top of the specimen with a blue line. A shelf is intentionally included in the background along with its shadow in order to showcase the specimen detection ability of the system. The aluminum plate boundaries are not selected by the algorithm, as they are not fully inside the camera field of view.

Autonomous boundary detection of composite patch. The mounting bracket has been masked for specimen detection.
Figure 7 illustrates the results of boundary detection with a circular carbon/carbon brake disk. A border is drawn around both Figures 6 and 7 to highlight the fact that the mounting bracket and shelf have been digitally removed from the bottom of the image, in order to avoid unnecessary edge detection from the mounting brackets. An alternative method would be to physically mask the mounting mechanism, but this can be cumbersome, especially if brackets of varying heights are used for different specimens.

Autonomous boundary detection of carbon/carbon brake disk. The mounting bracket has been masked for specimen detection.
System operation
Following the system assembly and specimen placement, calibration must initially be performed. There are two objectives for this calibration. First, the distance between the camera and specimen should be selected in order to measure the correct size of the specimen in the image. This stand-off distance should be selected by the user from a given set of values. The set is prepared so that all the values maximize the signal-to-noise ratio (SNR) of the measurements conducted by the sensing laser. Second, the true origin of the system should be calculated by measuring the constant offset between the detection laser position in the image and the stage position reported by the stage controller. Calibration is performed automatically by the GUI through detection of the laser pointer in the image. The system calibration should be performed every time the stand-off distance is changed or the scan head is re-assembled or re-mounted on the stage. Following the calibration process, the user can move the scan head to any point within the image by simply clicking. This provides ease of operation for centering the specimen for autonomous boundary detection, as discussed in the previous section.
Figure 8 summarizes the system operation in the form of a flowchart. Once the specimen is at an appropriate position in front of the scan head, the desired scan area can be delimited by means of autonomous boundary detection, as explained in the previous sections. The scan area may have any shape and need not adhere to any fixed or rectangular shapes. The specimen image and delimited scan area are processed, and the lengths and starting point of each vertical scan line in the raster scan are calculated. This information is uploaded to the stage controller, which subsequently performs the scan line-by-line, covering the entire scan area. Moreover, the generating and sensing lasers are configured, and memory buffers for storing the inspection data are declared. The scan then proceeds, and measurements from each point are recorded into the memory buffers until the entire delimited area has been scanned.

Operational flowchart of abstract scan shape PE UPI system.
Results
Several inspections were performed with specimens of different shapes and materials in order to validate the newly developed system operations. Here, we include three results that highlight the system capability of detecting subsurface defects, while also demonstrating the resource savings owing to customizable area scanning.
Aluminum wing skin model
A wing skin model made up of 2-mm-thick aluminum alloy was inspected using this system. Its surface is optically reflective. Figure 9(a) illustrates the image of the plate along with the scan boundary, conforming closely to the actual specimen boundary, delimited prior to the inspection. This image was captured by the camera mounted on the scan head. Figure 9(b) illustrates the plate defects along with the specifications. Two oval-shaped engravings were machined at the back of the plate to mimic the effects of wall thinning. Finally, Figure 9(c) illustrates the UWPI freeze frame, visualizing both defects. The measurement points outside the selected scan area but within its bounding rectangle are initialized to zero, resulting in the distinct light blue region outside the scan area.

Aluminum plate: (a) inspection area selection, with blue line demarking scan boundary, (b) defect map showing defect types and locations, and (c) UWPI freeze frame showing defects.
The details of the specimen, inspection settings, and performance improvements are summarized in Table 1. In order to draw conclusions and quantify the savings, a comparison of the measurement points and scan times is included between the rectangular shape and customizable shape scanning. A rectangular scan shape is assumed according to the bounding box of the delimited customizable shape. In this particular inspection, a reduction of approximately 44% in the measurement points and inspection time is observed. Note that the scan time refers to the actual scan time and does not include the initial system setup time. In addition, the edge detection and delimitation process is running in real time and, thus, does not add any noticeable time to the total scan duration in autonomous mode.
Aluminum plate inspection details and performance improvement.
Composite patch 1
A carbon fiber–reinforced plastic (CFRP) composite patch, pasted on top of an aluminum alloy plate, was inspected using the customizable shape scanning feature. This specimen was prepared in order to simulate repairing a metallic structure with a composite patch. Its surface roughness stood at about 45 µm facing a peel-ply during the manufacturing. The tapered stacking makes non-flat surface, but during the scan, a small change in offset values ±3.4 mm from the LDV to the measured point in the scan has no effect on the SNR of the signal since this change is within the depth of field of the LDV. A number of artificially created defects were embedded into the specimen, mimicking the flaws commonly observed in the curing of such repair patches. Figure 10(a) provides an image of the specimen along with the non-rectangular scan boundary used for the inspection. Figure 10(b) illustrates the specimen defect map, where a 13-ply octagonal patch is bonded onto a 2.25-mm-thick aluminum plate. Composite plies with decreasing areas were pasted on top of one another to achieve maximum thickness at the specimen center. The specimen was cured in an autoclave under pressure, and defects were induced via Teflon film inserts and stainless steel pull tabs. Figure 10(c) illustrates the UWPI freeze frame indicating all the specimen defects. The measurement points outside the selected scan area but within its bounding rectangle are initialized to zero, resulting in the distinct light blue region outside the scan area.

Composite patch 1: (a) inspection area selection with blue line demarking scan boundary, (b) defect map showing defect types and locations, and (c) UWPI freeze frame showing defects.
The details of this specimen, along with the inspection settings and performance improvements, are summarized in Table 2. In order to draw conclusions and quantify the savings, a comparison of the measurement points and scan times is included between the rectangular shape and customizable shape scanning. A rectangular scan shape is assumed according to the bounding box of the delimited customizable shape. In this particular inspection, a reduction of approximately 13% in measurement points and inspection times is observed.
Composite patch 1 inspection details and performance improvement.
CFRP: carbon fiber–reinforced plastic.
Composite patch 2
An additional composite aluminum specimen manufactured with several defects at different layers was tested using the system. Figure 11(a) illustrates an image of the specimen along with the non-rectangular scan boundary. This specimen had an aluminum alloy back plate with a 2.54-mm thickness and total of six plies, each with a 2-mm thickness, cured on top. The composite surface roughness was about 45 µm. Figure 11(b) illustrates the specimen defect map, where all the defects in the specimen were induced by means of Teflon film inserts. Figure 11(c) illustrates the UWPI freeze frame indicating all the specimen defects. The measurement points outside the selected scan area but within its bounding rectangle are initialized to zero, resulting in the distinct light blue region outside the scan area.

Composite patch 2: (a) inspection area selection, with blue line demarking scan boundary, (b) defect map showing defect types and locations, and (c) UWPI freeze frame showing defects.
The details of this specimen, along with the inspection settings and performance improvements, are summarized in Table 3. Similar to the previous results, in order to draw conclusions and quantify the improvement, a comparison of measurement points and scan times is included between the rectangular shape and customizable shape scanning. A rectangular scan shape is assumed according to the bounding box of the delimited customizable shape. In this particular inspection, a reduction of approximately 12% in the measurement points and inspection times was observed.
Composite patch 2 inspection details and performance improvement.
CF: carbon fiber.
Conclusion
This article proposes a PE UPI system that is capable of autonomous target recognition and scanning an area of a customizable shape for the NDE of structural defects. The proposed system can inspect an area of any shape best suited for a certain specimen. Moreover, the system supports the detection of the specimen boundary for autonomous scan area delimitation. This is achieved by imaging the specimen and then determining its borders by means of edge and contour detection operations. Thereafter, a scan area closely conforming to the specimen boundary is suggested. The delimited area is scanned line-by-line in a raster scan pattern, where each line in the raster scan may be of any variable length. The scan path for the inspection is automatically generated and followed by the translational stage without any intervention from the user. It was demonstrated that use of a scan shape adhering to the specimen shape or region of interest, as opposed to a fixed rectangular scan shape, effectively optimizes the time, memory, and laser use by the system. The autonomous operation of scanning is well suited for use in a smart industry with minimum intervention from the user.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Technology Innovation Program (10074278, Development of an SHM system for UAV using CNT/polymer hybrid fiber sensor network and simultaneous proof-of-concept of autonomous flight and diagnosis), funded by the Ministry of Trade, Industry, and Energy (MI, Korea).
