A path planning method for a surface inspection system based on two-dimensional laser profile scanner

Abstract

In this article, a method for two-dimensional scanning path planning based on robot is proposed. In this method, a section division algorithm based on neighborhood search method for scanning orientation determination is firstly produced. The scanning paths which meet constraints of the system are then generated. Finally, the experiment is carried out on robot-based scanning platform. The two-dimensional data from scanner and the robot position are combined to form three-dimensional surface data of measured workpiece. The experiment results verify the effectiveness of proposed method.

Keywords

Two-dimensional laser scanner path planning neighborhood search method measured workpiece

Introduction

Three-dimensional (3-D) information of workpiece can be automatically collected by laser scanning which has been widely used in surface detection, reverse engineering, data visualization, computer-aided design, virtual reality environment, digital cultural relics, and so on.^1,2 Laser scanning is an active and noncontact measurement technology. It has many advantages, such as no damage, high accuracy, fast speed, and easy realization of automatic measurement.³ Laser scanning sensors based on laser triangulation principle have been widely installed in coordinate measuring machines, manipulators, high precision mobile platforms and other devices, and have been combined with stereovision technology to build a simple, fast, flexible, and effective measurement system.⁴ Scanning path planning is one of the key technologies to realize automatic scanning. The results of planning directly determine quality, efficiency, and security of scanning.

Many researchers have studied the model-based scanning path planning algorithms combined with different 3-D measurement technologies, which ensure that the measurement of workpiece can be carried out efficiently and completely.^5,6 EIMaraghy and Yang⁷ introduced a clustering algorithm for laser scanning. The view cone is introduced to analyze visibility, and feature recognition technology based on ray tracing algorithm is used to improve accuracy, and finally the simulation experiment is carried out on the freeform surface. Lee and Park⁸ sampled measured surface along isoparametric line. The workpiece was scanned automatically using a three-axis rotating platform in global scanning direction. Son et al.^9
–11 first sampled the workpiece surface, then proposed an optimal scanning path algorithm for various scanning constraints, and carried out the registration of scanning data, as well as the comparative analysis of measurement data and design data. Some researchers used vision-based system to detect the workpiece surface.¹² Based on computer vision, defect detection technology used industrial camera to obtain the image information of the object surface and used the corresponding image processing method to process the image and complete the detection.^13,14 Chen and Li¹⁵ proposed a shortest scanning path planning method based on hierarchical genetic algorithm for the constraints of visibility, field of view, depth of field, imaging parameters, and kinematic accessibility in the scanning process of digital cameras. Firstly, genetic algorithm is used to calculate the poses of the minimum viewpoints that can scan measured workpiece completely, and then Christofides algorithm is used to obtain the shortest scanning path through these viewpoints. The vision-based detection method has the outstanding advantages of noncontact, high accuracy, and automation, but the system often requires complicated image processing. Fernandez et al.¹⁶ installed line laser on Coordinate measuring machine (CMM) to form an automatic scanning system, analyzed the constraints of scanning system and CMM, and proposed a nonocclusion and collision-free scanning trajectory planning algorithm. Martinez et al.¹⁷ proposed a viewpoint planning scanning algorithm for defect detection of automobile headlamp glass. The view sequence is found by clustering, and genetic algorithm is used to optimize the number of views and their positions and postures in space. Wang¹⁸ studied pose planning algorithm of laser scanner based on clustering algorithm. The minimum required poses and the set of measurement points under each viewpoint are calculated. Li¹⁹ proposed a viewpoint planning algorithm based on minimum bounding box and the scanning cone. Raffaeli et al.²⁰ proposed a combined viewpoint planning method. Different clustering methods of sampling points are selected according to different detection processes, and the optimal scanning view sequence is obtained by iterative method.

Most of scanning path planning algorithms mentioned above are for area scanning system. In the process of area scanning, it is difficult to satisfy the constraint of field depth for measured surface with large curvature change which is in the same grating pattern projection region. In addition, the viewpoint and path generation method in surface scanning algorithm is not suitable for line scanning system. Especially, scanning path based on two-dimensional (2-D) laser profile scanner has its own characteristics. In the process of laser contour scanning, it is necessary to combine the contour data acquired by scanner with position information of actuator to generate 3-D data of workpiece surface. Path planning algorithm based on laser contour scanning studied in this article has important practical and theoretical significance.

The motivation of this work is to develop a path planning algorithm for 2-D laser profile scanning system in which scanning constraints are satisfied and workpiece surface can be scanned efficiently and automatically. The method first calibrates the relationship between the measured surface and the scanned data. In order to achieve high-precision measurement, the scanned surface is divided into sections with meeting several scan constraints. During the process of surface partition, the algorithm based on region growing is proposed. In the divided scanning area, the orientation of scanner is unchanged and the scanning path length is approached by minimizing repetitive scanning area for the high-efficiency measurement.

The rest of this article is organized as follows: system description is presented in the second section; the scanning path planning algorithm is illustrated in the third section; the fourth section describes simulation results of the given method; the experiment results are provided in the fifth section; and finally, the sixth section concludes main points of the research.

System description

The surface of workpiece may not be consistent with CAD model due to errors in production or wear and tear, so it is necessary to scan workpiece to obtain actual data. In this article, a 2-D laser scanning system based on robot is designed to measure the part surface with given CAD model.

Principle of laser scanning

The principle of laser scanning is depicted in Figure 1. The 2-D laser profile scanner uses triangulation principle. It can easily get the location of the laser line which is described as X-axis and the distance between laser scanner and the surface of workpiece which is called Z-axis in the scanner coordinate system. By moving the measured workpiece or scanner, a set of 3-D measurement points are obtained.

Figure 1.

Principle of laser scanning.

During the scanning process, the characteristics of scanner and the relative distance between scanner and measured workpiece determine the quality of measured data. Therefore, the constraints shown in Figure 1 are needed to be considered.

Direction angle

θ _T is the angle between scanner orientation l and the normal vector n of workpiece surface, and the value of angle directly determines whether the surface of workpiece could be gotten or not. When the scanner orientation is parallel to the normal vector of workpiece surface, the maximum point density of surface could be obtained

cos (θ_{T}) < - l \cdot n

Depth of field

The scanner could only measure the surface data which is in the depth of field at one scanning time. Supposing that a point in the scanner coordinate system is (x _s, 0, z _s), the equation (2) needs to be satisfied

H - h / 2 \leq z_{s} \leq H + h / 2

The best distance (H)

The distance H is from laser source to the scanning reference plane which located in the half depth of field. It can enable laser beam to be focused on reference plane.

Width of scanning (w)

Width of scanning is the width of laser beam which locates in the half depth of field of view, which is also the length of scanning line.

Field of view (FOV)

Area is defined by scanning angle δ and scanning width, within which the scanner can scan points.

Angle of field of view (δ)

Angle of laser beam plane.

Collisionless constraints

During scanning process, ensure that the scanner does not collide with robot or workpiece.

Composition of laser scanning system

The laser scanning system is mainly composed of a 2-D laser scanner and a six-degree of freedom industrial robot. The scanner is mounted at the end of the industrial robot. As shown in Figure 2, O_BX_BY_BZ_B is base coordinate system of robot, the O_EX_EY_EZ_E is end coordinate system of robot, the O_SX_SY_SZ_S is scanner coordinate system, and the O_OX_OY_OZ_O is coordinate system of measured workpiece. The line laser profile scanner can only measure the key point information of one scanning line on workpiece surface at a time, so in order to obtain the whole surface information, the scanner is moved by robot. Firstly, the position information of measured workpiece surface on X-axis and Z-axis is obtained, and then, the Y-axis is gained by moving along the planned path driven by robot. The complete 3-D data of measured workpiece surface can be achieved.

Figure 2.

Laser scanning.

Goal analysis of scanning path planning

The goal of scanning path planning is to generate a robot path along which the robot drives laser sensor to scan workpiece surface at different viewpoints to obtain surface data. After scanning has been completed, the scanning data obtained from different viewpoints are joined together to form workpiece surface.

Based on CAD model of workpiece, the pose of viewpoint in scanning process is firstly calculated in workpiece coordinate system and then transformed into base coordinate system of robot to carry out scanning process. The relationship between coordinate systems is shown in Figure 3, and coordinate transformation is shown in equation (3).

Figure 3.

Transformation of coordinate systems.

As shown in Figure 3, ^BT_E is the rotation transformation matrix from O_EX_EY_EZ_E to O_BX_BY_BZ_B , ^ET_S is from O_SX_SY_SZ_S to O_EX_EY_EZ_E , ^BT_O is from O_OX_OY_OZ_O to O_BX_BY_BZ_B , ^OT_S is from O_SX_SY_SZ_S to O_OX_OY_OZ_O

{}^{B}T ​_{E} = {}^{B}T ​_{O} {}^{O}T ​_{S} {}^{E}T ​_{S}^{- 1}

In equation (3), ^BT_E is used to control the motion of robot, which is the goal of scanning path planning and obtained by ^BT_O , ^OT_S , and ^ET_S ; ^BT_O can be obtained by locating measured workpiece; ^OT_S can be obtained by calculating viewpoint in workpiece coordinate system; and ^ET_S is related to the pose of scanner installed at the end of robot, which could be obtained by calibration. Therefore, in order to obtain path of the end-effector of robot in scanning process, the unknown parameters in equation (3) need to be calculated.

Neighborhood search method

The scanned workpiece is based on triangular mesh model which is commonly described by stereolithography (STL) file format. STL file format uses triangular facets to represent 3-D entity model. Half-edge data structure is used in the topology reconstruction of triangular meshes.²¹ After reconstructing the above patches, a patch neighborhood search algorithm is proposed, which can search n-order neighborhood of patches. As shown in Figure 4, f ₀ be a facet in mesh model, and IF ₀ be the index of facet f ₀. IF_fi is a set of adjacent facets of the corresponding facet f_i . The algorithm for obtaining n-order neighborhood of facet f ₀ is as follows:

Figure 4.

Triangular neighborhood search.

Step 1: Find the first-order neighborhood NH ₁ of the facet f ₀. Firstly, three half edges contained in f ₀ are found. Then, the corresponding neighborhood half edge is found. The neighborhood facet set of f ₀ is found, which is called first-order neighborhood face set

N H_{1} = \cup_{I F_{i} \in F L_{f_{0}}} f_{I F_{i}}

Step 2: Find the second-order neighborhood NH ₂ of the facet f ₀. NH ₂ is obtained by searching facets in the first-order neighborhood

N H_{2} = \cup_{I F_{i} \in F L_{N H_{1}}} f_{I F_{i}} - {f_{​_{I F_{0}}}} - {f_{I F_{i}} | I F_{i} \in N H_{1}}

Step 3: Find the n-order neighborhood NH_n of the facet f ₀. NH_n is obtained by searching the facets in the N − 1 order neighborhood

N H_{n} = \cup_{I F_{i} \in F L_{N H_{n - 1}}} f_{I F_{i}} - N H_{n - 2} - {f_{I F_{i}} | I F_{i} \in N H_{n - 1}}

Calibration of system hand–eye parameters

The rotation transformation matrix ^ET_S in equation (3) needs to be calibrated in order to realize the goal of path planning. It is also called hand–eye parameters of scanning system. The calibration is to obtain relationship between scanner and the end of industrial robot

{}^{E}T ​_{S} = [\begin{matrix} n_{x} & o_{x} & a_{x} & p_{x} \\ n_{y} & o_{y} & a_{y} & p_{y}^{​} \\ n_{z} & o_{z} & a_{z} & p_{z} \\ 0 & 0 & 0 & 1 \end{matrix}]

According to the definition of transformation matrix in equation (7), the vectors [n_x , n_y , n_z ]^T, [o_x , o_y , o_z ]^T, and [a_x , a_y , a_z ]^T are unit orthogonal vectors.

For a fixed point in the world coordinate system, such as point p in Figure 5, the position information in the scanner coordinate system varies at each scanning time. Taking base coordinate system of robot as world coordinate system, when laser sensor scans the point in different poses, equation (8) is obtained

{}^{B}T ​_{E} (i + 1) \times {}^{E}T ​_{S} \times {}^{S}P ​_{p} (i + 1) = {}^{B}T ​_{E} (i) \times {}^{E}T ​_{S} \times {}^{S}P ​_{p} (i)

Figure 5.

Diagram of calibration.

In equation (8), ^SP_p (i) is coordinate of space point p in scanner coordinate system at the ith scanning time

{}^{B}T ​_{E} = [\begin{matrix} {n'}_{x} & {o'}_{x} & {a'}_{x} & {p'}_{x} \\ {n'}_{y} & {o'}_{y} & {a'}_{y} & {p'}_{y} \\ {n'}_{z} & {o'}_{z} & {a'}_{z} & {p'}_{z} \\ 0 & 0 & 0 & 1 \end{matrix}]

{}^{S}P ​_{P} = [\begin{matrix} {p'}_{s x} (i) \\ {p'}_{s y} (i) \\ {p'}_{s z} (i) \\ 1 \end{matrix}]

^BT_E , which is in equation (9), can be obtained from robot controller. ^ET_S , which is in equation (10), is the matrix which needs to be computed. ^SP_p is the coordinate of the point achieved from scanner. Equation (11) could be obtained from equation (8)

\begin{array}{l} [\begin{matrix} {n^{'}}_{x} (i + 1) & {o^{'}}_{x} (i + 1) & {a^{'}}_{x} (i + 1) & {p^{'}}_{x} (i + 1) \\ {n^{'}}_{y} (i + 1) & {o^{'}}_{y} (i + 1) & {a^{'}}_{y} (i + 1) & {p^{'}}_{y} (i + 1) \\ {n^{'}}_{z} (i + 1) & {o^{'}}_{z} (i + 1) & {a^{'}}_{z} (i + 1) & {p^{'}}_{z} (i + 1) \\ 0 & 0 & 0 & 1 \end{matrix}] [\begin{matrix} n_{x} & o_{x} & a_{x} & p_{x} \\ n_{y} & o_{y} & a_{y} & p_{y} \\ n_{z} & o_{z} & a_{z} & p_{z} \\ 0 & 0 & 0 & 1 \end{matrix}] [\begin{matrix} {p^{'}}_{s x} (i + 1) \\ {p^{'}}_{s y} (i + 1) \\ {p^{'}}_{s z} (i + 1) \\ 1 \end{matrix}] \\ = [\begin{matrix} {n^{'}}_{x} (i) & {o^{'}}_{x} (i) & {a^{'}}_{x} (i) & {p^{'}}_{x} (i) \\ {n^{'}}_{y} (i) & {o^{'}}_{y} (i) & {a^{'}}_{y} (i) & {p^{'}}_{y} (i) \\ {n^{'}}_{z} (i) & {o^{'}}_{z} (i) & {a^{'}}_{z} (i) & {p^{'}}_{z} (i) \\ 0 & 0 & 0 & 1 \end{matrix}] [\begin{matrix} n_{x} & o_{x} & a_{x} & p_{x} \\ n_{y} & o_{y} & a_{y} & p_{y} \\ n_{z} & o_{z} & a_{z} & p_{z} \\ 0 & 0 & 0 & 1 \end{matrix}] [\begin{matrix} {p^{'}}_{s x} (i) \\ {p^{'}}_{s y} (i) \\ {p^{'}}_{s z} (i) \\ 1 \end{matrix}] (11) \end{array}

Since the scanner can only get 2-D coordinates X and Z of the points, equation (12) can be obtained from equation (11)

[\begin{matrix} {[\begin{matrix} {n'}_{x} (i + 1) \times {p^{'}}_{s x} (i + 1) - {n'}_{x} (i) \times {p^{'}}_{s x} (i) \\ {o'}_{x} (i + 1) \times {p^{'}}_{s x} (i + 1) - {o'}_{x} (i) \times {p^{'}}_{s x} (i) \\ {a'}_{x} (i + 1) \times {p^{'}}_{s x} (i + 1) - {a'}_{x} (i) \times {p^{'}}_{s x} (i) \\ {n'}_{x} (i + 1) \times {p^{'}}_{s z} (i + 1) - {n'}_{x} (i) \times {p^{'}}_{s z} (i) \\ {o'}_{x} (i + 1) \times {p^{'}}_{s z} (i + 1) - {o'}_{x} (i) \times {p^{'}}_{s z} (i) \\ {a'}_{x} (i + 1) \times {p^{'}}_{s z} (i + 1) - {a'}_{x} (i) \times {p^{'}}_{s z} (i) \\ {n'}_{x} (i + 1) - {n'}_{x} (i) \\ {o'}_{x} (i + 1) - {o'}_{x} (i) \\ {a'}_{x} (i + 1) - {a'}_{x} (i) \end{matrix}]}^{T} \\ {[\begin{matrix} {n'}_{y} (i + 1) \times {p^{'}}_{s x} (i + 1) - {n'}_{y} (i) \times {p^{'}}_{s x} (i) \\ {o'}_{y} (i + 1) \times {p^{'}}_{s x} (i + 1) - {o'}_{y} (i) \times {p^{'}}_{s x} (i) \\ {a'}_{y} (i + 1) \times {p^{'}}_{s x} (i + 1) - {a'}_{y} (i) \times {p^{'}}_{s x} (i) \\ {n'}_{y} (i + 1) \times {p^{'}}_{s z} (i + 1) - {n'}_{y} (i) \times {p^{'}}_{s z} (i) \\ {o'}_{y} (i + 1) \times {p^{'}}_{s z} (i + 1) - {o'}_{y} (i) \times {p^{'}}_{s z} (i) \\ {a'}_{y} (i + 1) \times {p^{'}}_{s z} (i + 1) - {a'}_{y} (i) \times {p^{'}}_{s z} (i) \\ {n'}_{y} (i + 1) - {n'}_{y} (i) \\ {o'}_{y} (i + 1) - {o'}_{y} (i) \\ {a'}_{y} (i + 1) - {a'}_{y} (i) \end{matrix}]}^{T} \\ {[\begin{matrix} {n'}_{z} (i + 1) \times {p^{'}}_{s x} (i + 1) - {n'}_{z} (i) \times {p^{'}}_{s x} (i) \\ {o'}_{z} (i + 1) \times {p^{'}}_{s x} (i + 1) - {o'}_{z} (i) \times {p^{'}}_{s x} (i) \\ {a'}_{z} (i + 1) \times {p^{'}}_{s x} (i + 1) - {a'}_{z} (i) \times {p^{'}}_{s x} (i) \\ {n'}_{z} (i + 1) \times {p^{'}}_{s z} (i + 1) - {n'}_{z} (i) \times {p^{'}}_{s z} (i) \\ {o'}_{z} (i + 1) \times {p^{'}}_{s z} (i + 1) - {o'}_{z} (i) \times {p^{'}}_{s z} (i) \\ {a'}_{z} (i + 1) \times {p^{'}}_{s z} (i + 1) - {a'}_{z} (i) \times {p^{'}}_{s z} (i) \\ {n'}_{z} (i + 1) - {n'}_{z} (i) \\ {o'}_{z} (i + 1) - {o'}_{z} (i) \\ {a'}_{z} (i + 1) - {a'}_{z} (i) \end{matrix}]}^{T} \end{matrix}] \times [\begin{matrix} n_{x} \\ n_{y} \\ n_{z} \\ a_{x} \\ a_{y} \\ a_{z} \\ p_{x} \\ p_{y} \\ p_{z} \end{matrix}] = [\begin{matrix} {p^{'}}_{x} (i) - {p^{'}}_{x} (i + 1) \\ {p^{'}}_{y} (i) - {p^{'}}_{y} (i + 1) \\ {p^{'}}_{z} (i) - {p^{'}}_{z} (i + 1) \end{matrix}]

Because of unit orthogonal relation, two equations are generated

{\begin{matrix} n_{x}^{2} + n_{y}^{2} + n_{z}^{2} - (a_{x}^{2} + a_{y}^{2} + a_{z}^{2}) = 0 \\ n_{x} \times a_{x} + n_{y} \times a_{y} + n_{z} \times a_{z} = 0 \end{matrix}

For a fixed point in world coordinate system, equations (12) and (13) can be obtained when scanned with two different poses. Three points at least in space are needed to obtain calibration parameters. In order to improve the accuracy of calculation, five points which are not in the same plane are selected for scanning calibration. Each point is scanned three times. At the same time, the pose parameters of robot end relative to the base coordinate system are recorded.

Scanning path planning algorithm

In this article, a laser sensor is fixed at the end of robot. The robot drives laser sensor to scan measured workpiece. In process of scanning, the constraints must be satisfied. In order to ensure the surface information of workpiece to be collected, it is necessary to plan the pose of scanner in advance. In addition, because the scanner can only detect the length of a scanning line at a time, it is necessary to plan scanning path to ensure that the whole workpiece surface can be completely detected.

Regional division of workpiece surface

When the orientation of scanner (S_z ) is parallel to the normal vector of workpiece surface, the effect of obtaining point cloud is the best. However, in order to reduce additional operation time caused by frequent changes in the orientation of scanner, the surface of measured workpiece is divided to ensure that the scanner orientation is constant in each section. That is to say, the angle between the orientation of scanner (S_z ) and the normal vector of all facets in the section is within the threshold of direction angle.

Steps of regional division are as follows:

Step 1: Traverse all unmarked faces and find out two facets with the largest normal vector angle, as shown in Figure 6, the green and blue facets have the largest normal vector angle θ. If the angle between the two faces satisfies θ < 2θ _T, all faces can be scanned in a viewpoint. If the angle does not satisfy this condition, then go to the next step.

Figure 6.

Selection of seed patch.

Step 2: Two patches with the largest angle between normal vectors are regarded as seed triangle patches f_i and f_j , where i ≠ 0 and j ≠ 0. By using patch neighborhood search method, a scanning area Φ _i is composed of patches, and the angles between these patches and the patch f_i is less than 2θ _T. The scanning area Φ _j is similarly achieved. The planned patches are marked.

Step 3: Calculate the average normal vectors of initial scanning area, then detect direction angle for each face in the area, and cancel the marking of the face that does not satisfy the angle of view constraint, and determine scanning area. If all the patches are marked, the division is end, otherwise go to step 1.

Determination of scanner orientation

The direction of scanner is Z-axis of scanner coordinate system. In the region that satisfies direction angle constraint, the direction of scanner S_z is defined as opposite direction of the mean of normal vectors of all facets in the region, as shown in Figure 7. The orientation of scanner is calculated according to equation (14)

S_{z} = - \sum_{i = 1}^{m} n_{i} / | \sum_{i = 1}^{m} n_{i} |

where m is the number of triangular patches in the scanning area, n_i is the normal vector of each patch, and S_z is the direction of scanner.

Figure 7.

Mean normal vector.

Generation of scanning viewpoint

The vertices of all facets in the region are projected onto S_z , as shown in Figure 8. The coordinates of projection points are calculated according to equation (15). Then the maximum distance between projection points is determined

p_{i} = - (v_{i} \cdot S_{z}) S_{z}

where v_i is the vertex coordinate of the facet.

Figure 8.

Projection of vertices in faces to S_z direction.

Calculation of the maximum distance among projection points in S_z direction is shown in Figure 9. The red points represent projection points from vertices of all facets in the area, and the blue circles represent the highest and lowest points in this direction. ΔH is the maximum distance of all projection points in the direction of S_z , which is calculated according to equation (16), and p_H and p_L are the highest projection point and the lowest projection point in the −S_z direction

Δ H = | p_{H} - p_{L} |

Figure 9.

Distance of projection points in S_z direction.

In Figure 9, if ΔH ≤ h, all facets belong to the same depth range and can be scanned in the same optimal distance plane; if ΔH > h, facets in the same S_z direction need to be classified according to the depth range. The classified data are calculated by equation (17), in which ⌊ ⌋ represents downward rounding

k = ⌊ \frac{Δ Η}{h} ⌋

Generation of scanning paths

The facet which contains point O_j and is vertical to S_z is labeled Π _j , and the point is calculated by equation (18). Firstly, the patches satisfying depth constraints in the scanning area are projected onto the plane Π _j . The projection is calculated by equation (19), and the projection effect is shown in Figure 10. After projection, the polygon formed by the facets in the plane Π _j is shown in Figure 11

O_{j} = p_{L} + (2 t - 1) \frac{Δ H}{2 k} {- S_{z}} t = 1, 2, \dots, k

p_{ij} = v_{i} + (v_{i} O_{j} \cdot S_{z}) S_{z}

where p_ij is coordinate of projection point and v_i represents vertex coordinates of facet.

Figure 10.

Projection of regional patches to plane Π _j .

Figure 11.

Polygons on projection plane Π _j .

In order to obtain scanning path, the convex hull of polygon on the projection plane Π _j is calculated according to the Graham scanning algorithm, and then the minimum outer rectangle²² of convex hull is obtained by rotation method. As shown in Figure 12, the green dotted line represents the minimum convex hull of projected polygon and the red dotted line is the minimum outer rectangle, which is the area to be scanned.

Figure 12.

Minimum enclosing rectangle.

In order to reduce turning times of scanning path, the long direction of outer rectangle is taken as the direction of scanner movement, which is Y-axis direction of scanner coordinate system, and the width direction of rectangle is taken as X-axis. As shown in Figure 12, supposing that four points that make up the smallest outer rectangle are ^OP_r ₁, ^OP_r ₂, ^OP_r _3, and ^OP_r ₄ in the workpiece coordinate system. If | ^OP_r ₁ ^OP_r ₂| > | ^OP_r ₁ ^OP_r ₄|, the posture of scanner in this area is shown in equation (20)

{}^{O}T ​_{S j} = [\begin{matrix} \frac{{}^{O}P ​_{r 1} {}^{O}P ​_{r 4}}{| {}^{O}P ​_{r 1} {}^{O}P ​_{r 4} |} & \frac{{}^{O}P ​_{r 1} {}^{O}P ​_{r 2}}{| {}^{O}P ​_{r 1} {}^{O}P ​_{r 2} |} & S_{z} \end{matrix}]

If | ^OP_r ₁ ^OP_r ₄| ≤ w, the area can be scanned according to one scanning line segment. Otherwise, the width of the smallest rectangle is divided, and the number N of scanning lines in this area is expressed by equation (21), in which ⌈ ⌉ represents upward rounding. The scanning lines in this area are shown in Figure 13. The black line with arrow in the figure is scanning line segment, and each scanning line segment is divided by scanning step, and the scanning viewpoints in projection plane are obtained. Then these points are offset according to the constraints of the best distance, as depicted in equation (22). The global scanning viewpoint ^OP_S in workpiece coordinate system is obtained. The blue line is composed of global scanning viewpoint, and the arrow represents the direction of scanner’s movement and the posture of scanner is constant during the whole scanning process. Finally, different scanning lines are connected to form scanning path of the region

N = ⌈ D_{offset} / d_{w} ⌉

{}^{O}P ​_{S} = {}^{O}P ​_{p} - H S_{z}

Figure 13.

Scanning path in one scanned area.

In the scanning process, collision often occurs between scanner and workpiece, so this article mainly detects whether there is collision between the end tool and scanned workpiece. The path is updated when collision occurs. In order to detect collision, the poses of scanner and workpiece need to transform to the same coordinate system which is the base coordinate system of robot. The box-based algorithm²³ is adopted for collision detection.

Simulation results

The above algorithm is based on facets and requires the edge length of the facets to be less than the width of scanning line, the triangular mesh model of workpiece is refined by dichotomy,²⁴ and then path planning is carried out. The refined model parameters are shown in Table 1, and the scanning constraints are stated in Table 2.

Table 1.

Model parameter.

Model	Facets	Vertices	Edges
Airfoil part	1647	863	2509
Rotary part	1120	609	1728
Freeform part	752	450	1201

Table 2.

Scanning parameters.

Parameters	Value
Direction angle θ _T	45°
Depth of field h	100 mm
The best distance H	240 mm
Width of the scanning w	100 mm
Angle of field of view δ	21.4°

Airfoil part model and scanning path

The model of airfoil part is shown in Figure 14. Firstly, the maximum angle between the normal vectors of triangular surface of airfoil model is traversed, θ _max = 53.4° and θ _max < 2θ _T. It is found that all the facets in the model satisfy the constraints of view angle. Therefore, during the scanning process of airfoil model, the same scanner can be used to scan the whole workpiece surface, which is calculated according to equation (14). Then △H = 124 mm is calculated by equations (15) and (16), and k = 2 is calculated by equation (17). The model is divided into two regions, pink and green, as shown in Figure 15. Finally, the final scan path is generated according to the method in the third section.

Figure 14.

Airfoil part model.

Figure 15.

Scanning paths for airfoil part.

When calculating the scanning path, the facets of pink and green areas in Figure 15 are projected to different projection planes to calculate the minimum enclosing rectangle in different areas, such as the red dotted rectangle. The pose of scanning viewpoint is obtained from equation (20). Then the scanning line segments are calculated according to scanning width. The blue and black lines in the red dotted frame are calculated. Finally, the scanning path composed of scanning viewpoints in the workpiece coordinate system is calculated according to equation (22) as path_a and path_b in Figure 15.

Through the calculation of scanning path, it is obtained that the facets of airfoil part are located in the same direction during scanning, but the scanning paths path_a and path_b are located in two different depth of field areas.

Rotary part model and scanning path

The model of rotary workpiece is shown in Figure 16. The maximum angle between normal vectors of triangular patches in the model is found, and θ _max = 100.3°. For θ _max > 2θ _T, it is necessary to change the orientation of scanner in order to achieve complete scanning of workpiece surface.

Figure 16.

Rotary part model.

Firstly, all unmarked facets of workpiece are traversed, two facets f_i = 1 and f_j = 1049 which are with the largest angle are selected as seed triangle facets, and the neighborhood of seed facets are searched, respectively. The scanning areas Φ _i is constituted by the facets, and the angles between these facets and f_i are smaller than 2θ _T. Φ _j is the same. The facets in the scanning area are marked. Then, the view angle is detected in regions Φ _i and Φ _j , and the facets which do not satisfy the view angle constraints are cancelled. The process of generating scanning paths needs to continue until all facets are marked. Finally, six different scanning areas are generated. Then, the S_z is computed according to equation (14). By equations (15) and (16), the patches in each region satisfy ΔH < h. The final scan path is then generated for each region according to the method in the third section, as shown in Figure 17.

Figure 17.

Scanning paths for rotary part.

Freeform part model and scanning path

The model of freeform workpiece is shown in Figure 18. The maximum angle between normal vectors of triangular patches in the model is found, and θ _max = 95.7°. For θ _max > 2θ _T, it is necessary to change the orientation of scanner in order to achieve complete scanning of workpiece surface.

Figure 18.

Freeform part model.

Firstly, all unmarked facets of workpiece are traversed, two facets (f_i = 8 and f_j = 710) which have the largest angle are selected as seed triangle facets, and the neighborhood of seed facets are searched, respectively. The scanning areas Φ _i is constituted by the facets, and the angles between these facets and f_i are smaller than 2θ _T. Φ _j is the same. The facets in the scanning area are marked. Then, the view angle is detected in regions Φ _i and Φ _j , and the facets which do not satisfy the view angle constraints are unmarked. Then, the S_z is computed according to equation (14). By equations (15) and (16), the patches in each region satisfy ΔH < h. The process of generating scanning paths needs to continue until all facets are marked. Finally, three different scanning areas are generated, and the final scan path is then generated for each region which is shown in Figure 19.

Figure 19.

Scanning paths for freeform part.

Algorithm comparison

The algorithm proposed in the study by Wu et al.²⁴ is used to divide workpiece surface and generate scanning path. The path planning results for rotary part and freeform part are shown in Figures 20 and 21.

Figure 20.

Scanning paths for rotary part by Wu et al.²⁴

Figure 21.

Scanning paths for freeform part by Wu et al.²⁴

The algorithm proposed in this article is compared with that in the study by Wu et al.²⁴ The results are shown in Table 3. By comparing two algorithms, the section number obtained by this algorithm is less than that calculated by Wu et al.,²⁴ and the total path lengths computed are shorter than that produced by Wu et al.²⁴ by 804 mm (for rotary part) and 1182.4 mm (for freeform part), respectively. It can be concluded that the proposed algorithm can reduce the number of sections and the total length of scanning paths.

Table 3.

Comparison.

Model	Method	Section number	Scan path length (mm)
Rotary part	Given method	6	3094.5
Rotary part	Method in Wu et al.²⁴	8	3898.5
Freeform part	Given method	3	4524.9
Freeform part	Method in Wu et al.²⁴	5	5707.3

Scanning experiment

In equation (3), the hand–eye parameters of scanning system need to be computed firstly. The data obtained during calibration process are shown in Table 4. The rotation matrix from the robot end to the robot base coordinate system is expressed by quaternions (q ₁, q ₂, q ₃, q ₄) and the translate vector is expressed by (x _B, y _B, z _B), and they are all obtained from robot controller. The coordinates of the fixed point in the scanner coordinate system are labeled as (x _S, 0, z _S), which are achieved from the scanner. Using data in Table 4 and the above calibration method, the transformation matrix ^ET_S is calculated by the iterative least squares algorithm²⁵

{}^{E}T ​_{S} = [\begin{matrix} 0.2160 & - 0.9754 & 0.0445 & - 53.7943 \\ 0.9524 & 0.2005 & - 0.2297 & - 193.1259 \\ - 0.2151 & - 0.0920 & - 0.9722 & 450.4681 \\ 0 & 0 & 0 & 1 \end{matrix}]

Table 4.

Calibration data.

Point	Robot rotation matrix quaternion				Robot position offset (mm)			Scanner data (mm)
p ₁	q ₁	q ₂	q ₃	q ₄	x _B	y _B	z _B	x _S	z _S
p ₁	0.2182	0.8272	−0.5163	0.0380	1617.0	−287.4	872.1	20.74	214.54
p ₁	0.0768	0.7033	−0.7049	−0.0497	1553.5	−464.0	897.0	−28.09	204.75
p ₂	0.2661	−0.5136	0.8031	0.1433	1291.5	−726.1	737.4	−8.105	204.35
p ₂	0.1335	0.1661	−0.9766	0.0294	1994.7	−178.7	936.6	51.52	255.9
p ₂	0.0189	0.3118	0.9497	0.0212	1913.0	−275.5	963.2	45.86	251.39
p ₃	0.2962	−0.8090	−0.5077	0.0047	2200.7	−714.4	732.9	30.74	210.46
p ₃	0.1484	−0.0129	0.9888	0.0000	1486.9	−21.4	878.3	7.88	241.97
p ₃	0.1735	0.2011	0.9600	0.0889	1491.6	−128.9	894.9	38.27	244.27
p ₄	0.1397	0.8847	−0.3996	0.1952	1400	−69.9	831	19.53	212.98
p ₄	0.2306	0.8399	0.4819	−0.0954	1847.9	−250.7	875.7	−15.65	205.96
p ₄	0.0439	0.9027	−0.4279	0.0082	1613.7	−530.9	925.1	28.2	220.24
p ₅	0.0223	0.9841	0.1750	−0.0226	1819.9	−639.8	950.4	−28.3	250.04
p ₅	0.0189	0.9494	0.3126	−0.0255	1807.8	−330.4	883.3	43.8	217.83
p ₅	0.0858	0.2468	0.9651	−0.0156	1656.8	−61.1	888	27.53	226.59
p ₅	0.0130	−0.5230	0.8518	0.0268	1551.7	−239.0	894.4	27.53	226.59

The method of locating workpiece refers to the method in the study by Wu.²⁶ The scanner installed at the end of robot scans several corners of airfoil part and reads the pose of robot end from robot controller. The coordinates of workpiece corners in the base coordinate system of robot can be obtained by equation (24). The coordinate values of corners in the workpiece coordinate system are known, so that the pose of workpiece in the robot coordinate system can be obtained, and the workpiece can be positioned

[\begin{matrix} {}^{B}x ​_{p} \\ {}^{B}y ​_{p} \\ {}^{B}z ​_{p} \\ 1 \end{matrix}] = [\begin{matrix} {}^{B}R ​_{E} & {}^{B}P ​_{E} \\ 0 & 1 \end{matrix}] [\begin{matrix} {}^{E}R ​_{S} & {}^{E}P_{S} \\ 0 & 1 \end{matrix}] [\begin{matrix} {}^{S}P ​_{p} \\ 1 \end{matrix}]

Experiment of airfoil model and analysis

The scanning system is mainly composed of ABB IRB6640 industrial robot and 2-D laser sensor scanCONTROL 2600-100. The test platform is shown in Figure 22. When the scanner receives trigger command from robot controller, the laser projects a linear laser beam onto the surface profile of workpiece. There are a series of sampling points with approximately equal spacing distribution on the formed scanning line. Then the sensor returns data of sampling points relative to X-axis and Z-axis of scanner coordinate system. Every time the scanner collects data, the robot controller saves pose information of the robot end. In scanning process, the spacing of two continuous laser fringes along the direction of Y-axis in the measuring coordinate system should be set according to requirement of system accuracy.

Figure 22.

Scanning experimental platform.

Scanning path planning based on airfoil model includes six steps: hand–eye parameter calibration of scanning system, triangular mesh model processing, scanning path planning, airfoil workpiece positioning, point cloud collection on workpiece surface, and point clouds registration. The main parameters of experiment are as follows: the length of linear laser beam is about 100 mm, the distance between scanner and scanning plane is 240 mm, the depth of field is 100 mm, and the depth resolution is 12 µm. Each scanning line can collect 640 measuring points at a time, and the scanning frequency is 200 Hz, and the measuring speed is 128,000 points/s. During scanning process, the robot acts as the execution of mechanical movement. Laser sensor installed at robot end is driven to scan, and the repetitive positioning accuracy of the robot is 0.5 mm. The scanning step is 1.0 mm.

When the pose of workpiece is known and the hand–eye relationship matrix has been calibrated, the scanning path points at robot end can be obtained by using equation (3). Scanning paths which are shown in Figure 23 are constituted by connecting path points. Scanners collect point clouds of different sections of workpiece surface at different scanning time. When scanning is completed, the data must be transformed into the same coordinate system in order to achieve point clouds registration. According to equation (25), the scanned data can be unified into the base coordinate system of robot. The scanning results are shown in Figure 24(a)

​^{B} P_{p} =^{B} T_{E} ​^{E} T_{S} ​^{S} P_{p}

where ^SP_p is the position of scanned point in the scanner coordinate system and ^BP_p is the position of scanned point in the robot base coordinate system.

Figure 23.

Scanning paths.

Figure 24.

Airfoil model. (a) Scanned model and (b) designed model.

Due to influence of scanning environment, object surface, illumination, and other factors, point clouds obtained by laser scanner often contain noise and outliers. Firstly, scanned data should be pre-processed by removing noise and outliers. After obtaining the surface data of airfoil part, the scanned model is aligned with CAD model, and then the difference threshold is used to judge whether there are defects.

Geomagic Qualify software, as a computer-aided test software, has the characteristics of simplicity and powerful detection function. First, the scanned point cloud model is imported as testing object, and then CAD model is imported as reference object. According to the minimum deviation, the best fitting and alignment function of the software is used to adjust the matching between point cloud and design model continuously. Then using comparative analysis function of the software, the deviation can be generated by 3-D comparison.

The standard deviation between scanned point cloud model and CAD model is 3.0 mm, and the mean deviation is 2.5 mm. Large deviation occurs at the edge of workpiece which is made of pine wood material. The pine material is easy to crack and deform which is the main reason for deviation. When the airfoil workpiece is used as the placement model in the robotic fiber placement,²⁷ it is necessary to modify the design model according to scanned model to ensure that the subsequent operation of workpiece is based on the actual model. The experimental results show that laser scanner can acquire complete data of workpiece surface according to designed planning path and meet the requirement of surface detection for workpiece.

Conclusions

In this article, a path planning algorithm for 2-D laser scanning system based on robot is produced. Firstly, composition and constraints of scanning system are introduced, and the hand–eye parameters of system are calibrated. Then, based on triangular mesh model, a path planning algorithm for scanning is proposed. In the given method, surface area of workpiece is divided into several regions, and scanning orientations and scanning viewpoints are determined in each region. Then scanning paths are obtained according to minimum enclosing rectangle. During scanning, the scanner will not collide with measured workpiece. Finally, the scanning of airfoil part is carried out on the experiment platform, and scanned model is obtained. Compared with design model, the comparison result shows that the scanning path algorithm is effective.

In the future work, there is a need to consider the relationship between the constantly changing scanning direction and scanning path length, which can make full use of the flexibility of the robot. Additionally, the proposed method assumes that the CAD model of the part is given. Another work is expanding our system to unknown part.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work is partly supported by The National Natural Science Foundation of China (grant no. 51405486) and Shanxi Key Laboratory of Advanced Manufacturing Technology Foundation (grant no. XJZZ201703).

ORCID iD

Lina Li

References

Phan

NDM

Quinsat

Lavernhe

. Scanner path planning with the control of overlap for part inspection with an industrial robot. Int J Adv Manuf Tech 2018; 98(1–4): 629–643.

Wen

. An automatic shoe-groove feature extraction method based on robot and structural laser scanning. Int J Adv Robot Syst 2016; 14(1):1–14.

Yao

AWL

. Applications of 3D scanning and reverse engineering techniques for quality control of quick response products. Int J Adv Manuf Tech 2005; 26(11–12): 1284–1288.

Guo

Zhou

Gao

. Development of a novel high-speed dynamic length measurement system for mobile and large scale cylinder workpiece. P I Mech Eng B-J Eng 2015; 229(5): 823–834.

Yin

Ren

Guo

. Development and calibration of an integrated 3D scanning system for high-accuracy large-scale metrology. Measurement 2014; 54(6): 65–76.

Rahayem

Kjellander

JAP

. Quadric segmentation and fitting of data captured by a laser profile scanner mounted on an industrial robot. Int J Adv Manuf Tech 2011; 52(1–4): 155–169.

EIMaraghy

Yang

. Computer-aided planning of laser scanning of complex geometries. Cirp Ann-Manuf Techn 2003; 52(1): 411–414.

Lee

Park

. Automated inspection planning of free-form shape parts by laser scanning. Robot Cim-Int Manuf 2000; 16(4): 201–210.

Son

Park

Lee

. Computer-aided planning of laser scanning of complex geometries. Int J Mach Tool Manu 2002; 42(8):889–897.

10.

Son

Kim

Lee

. Path planning of multi-patched freeform surfaces for laser scanning. Int J Adv Manuf Tech 2003; 22(5–6): 424–435.

11.

Son

Lee

K H

. Automated scan plan generation using STL meshes for 3D stripe-type laser scanner. In: International conference on computational science and its applications, Heidelberg, Germany, 15–21 May 2003, pp. 424–435.

12.

Hutchinson

Hager

Corke

. A tutorial on visual servo control. IEEE T Robotic Autom 1996; 12(5): 651–670.

13.

McFadyen

Dayoub

Martin

. Assisted control for semi-autonomous power infrastructure inspection using aerial vehicles. In: 2018 IEEE/RSJ International conference on intelligent robots and systems, Madrid, Spain, 1–5 October 2018, pp. 651–670.

14.

Chen

. Automatic sensor placement for model-based robot vision. IEEE T Syst Man Cy B 2004; 34(1): 393–408.

15.

Chen

. A method of automatic sensor placement for robot vision in inspection tasks. In: Proceedings 2002 IEEE international conference on robotics and automation, Washington, DC, USA, 11–15 May 2004, pp. 2545–2550.

16.

Fernandez

Rico

Alvarez

. Laser scan planning based on visibility analysis and space partitioning techniques. Int J Adv Manuf Tech 2008; 39(7–8): 699–715.

17.

Martinez

Ortega

Garcia

. A sensor planning system for automated headlamp lens inspection. Expert Syst Appl 2009; 36(5): 8768–8777.

18.

Wang

FH.

Research on measurement planning of laser-scanning and adaptive geometric modification for aero-engine blades. MA Thesis, Nanjing University of Aeronautics and Astronautics, China, 2016.

19.

Research on fundamental algorithms of inspection planning for area scanning robots. MA Thesis, Hefei University of Technology, China, 2014.

20.

Raffaeli

Mengoni

Germani

. Off-line view planning for the inspection of mechanical parts. IJIDeM 2013; 7(1): 1–12.

21.

Sun

Liu

. Reconstructing accurate boundary representation from STL model. J Comput Aided Des Comput Graph 2004; 16(7): 944–949.

22.

Wang

. An improved Graham algorithm for determining the convex hull of planar points set. Sci Survey Mapp 2010; 35(6): 123–125.

23.

Lin

. Two-stage collision detection technique based on bounding volume hierarchy and ray-tracing algorithm. J Shenyang Univ Technol 2010; 32(5): 574–578.

24.

Zou

. Viewpoint planning for freeform surface inspection using plane structured light scanners. Int J Autom Comput 2016; 13(1): 42–52.

25.

Tan

. A general recursive linear method and unique solution pattern design for the perspective-n-point problem. Image Vision Comput 2008; 26(6): 740–750.

26.

Research on key problems of point cloud based surface inspection and target localization. PhD Thesis, University of Chinese Academy of Sciences, China, 2015.

27.

Wang

. An accurate path planning algorithm based on triangular meshes in robotic fibre placement. Int J Robot Autom 2017; 32(1): 22–32.