Abstract
Unmanned aerial systems (UAS) are experiencing tremendous growth in both the technological advancement of the systems themselves and the expansion of practical uses for the systems. One application for this system includes the use of UAS as a tool in support of bridge inspections. The ability to fly UAS into positions difficult to reach by an inspector has the potential to save time, reduce costs, and improve safety. An increasing number of bridge owners are exploring the use of UAS for bridge inspections through pilot studies and early adoption as an integral part of their inspection processes. This study aims to enhance the industry awareness and knowledge base of stakeholders in bridge inspection processes. This paper presents the aircraft and sensors used to assist or augment inspections, the data needs of the bridge owners that can be provided through the use of UAS, and the means and methods by which the bridge owner, or the organization supporting the bridge owner, can manage the tremendous amount of data that can be collected by sensors deployed on UAS during an inspection.
Unmanned aerial systems (UAS) have been around nearly as long as manned aircraft, making their initial appearance during World War I. Because of advances in materials and aircraft technologies, the miniaturization of computer processors and sensors, the scaling down in system size and cost, and the improvement of flight control systems, UAS have become increasingly popular in various industries, including the transportation industry. The Federal Aviation Administration (FAA) regulates all aircraft in the National Airspace System, including unmanned aircraft. Part 107 of the Federal Aviation Regulations is the current FAA regulation under which most UAS operate for commercial purposes, including, but not limited to, firefighting, search and rescue, agricultural management, construction inspection, wildlife monitoring, land surveys, and infrastructure inspections. For bridge inspections, UAS have the ability to carry high-resolution cameras and other sensors to inspect difficult-to-reach areas around and above a bridge. Bridge members requiring visual inspections that traditionally require the use of specialized equipment to access, placing people at risk, may be more easily viewed using UAS. Additionally, UAS can reach these challenging areas more efficiently while producing live streaming video, allowing the user to use the live video in the field or capture still images or videos of interest or concern—all while the inspector remains in relative safety out of harm’s way. Following the inspection flight, the data collected by UAS is processed into several different formats that can supplement inspection report documentation, better informing decision-makers of the condition of their structures, and improving future inspection planning.
It should be noted that while UAS can provide many advantages to a bridge inspector, they currently cannot replace a person with contact sensors or other inspection methods for hands-on fracture critical inspections. Additionally, UAS are restricted in their application by regulations issued by the FAA and other government agencies.
UAS as a Bridge Inspection Tool
Bridge inspectors have a wide variety of tools available to allow them to adequately meet the objectives of a bridge inspection in accordance with governing National Bridge Inspection Standards (NBIS). The fact is that UAS is a tool that, when successfully employed, may bring improved document quality, and efficiencies in time, cost, and safety of personnel and equipment. The Bridge Inspectors Reference Manual (BIRM) lists seven basic categories for tools used by the inspector at the bridge site during bridge inspections: inspection, cleaning, visual, measuring, documentation, access, and miscellaneous equipment. Of the seven categories of inspection tools, three are currently or projected for use in conjunction with UAS: visual, measuring (estimates from images after post processing), and documentation tools.
UAS can be a cost-effective solution to obtaining either stand-alone, high-quality visual inspection data, or to supplement traditional standard inspection methods and equipment. Although currently available UAS cannot conduct inspections requiring physical contact, their value can still be realized as a way to improve the inspection process. In many inspection scenarios, UAS can reduce or eliminate the need to have traditional inspection resources available to meet these objectives. Current inspection methods are both physical and visual, requiring access equipment suited for the bridge, supplemented with additional tools needed to perform a complete inspection. Depending on the type, size, and condition of the bridge, the inspector may need to utilize several different access methods and tools to complete the inspection, potentially costing thousands of dollars and hundreds of working hours.
UAS can also enhance safety for the inspection team in many cases. Reducing the exposure to hazardous conditions while attempting to access various parts of a bridge, such as traversing rough terrain to view substructure elements, entering fast moving water to view the underside of the structure, or climbing a superstructure at extreme heights, is a key advantage which UAS can offer. UAS can also improve traffic safety for the public by reducing the need for lane closures, and can amount of time that lanes are closed. Some UAS provide inspectors with the ability to examine areas typically difficult to access or in confined spaces, such as a closed spandrel arch or steel box girder. Some areas may be inaccessible through traditional inspections methods. These areas present challenges or heightened risks to personnel. As an example, during a 2018 study performed by the Minnesota Department of Transportation (MnDOT), contractors flying a collision-tolerant UAS captured imagery inside an enclosed steel arch ( 1 ). Using this type of UAS inside the bridge structure potentially removed the requirement for personnel to enter a potentially dangerous confined space—a procedure that requires specific training and may require the receipt of entry permits for a permit-required confined space in accordance with current safety regulations and practices.
UAS do not radically alter the inspection process, rather the use of UAS as a tool can simplify and possibly improve the inspection process. A UAS can be used to capture imagery or live video of the bridge and bridge components with at least the same resolution as if the camera were held by an inspector, whether the inspector be on the ground, in a boat, up close to the bridge structure, or in an aircraft flying above the bridge. It should also be understood that use of a UAS does not remove or change any of the requirements for an inspection. All standards of a full bridge inspection still must be met by the inspector, to include a properly written inspection report with proper narratives of all deficiencies, and assigning the correct National Bridge Inventory (NBI) codes and element condition states where appropriate.
Table 1 presents other advantages and limitations discovered during this research study and the accompanying interviews with bridge owners, inspection service providers, and sensor manufacturers. These factors should be considered when determining if the benefits of using UAS outweigh the costs.
Unmanned Aerial Systems (UAS) Advantages and Limitations/Challenges
When looking at the use of UAS to capture inspection data, it is important that a NBIS qualified inspector and UAS pilot are fully engaged in the inspection planning and actual field operations. It is recommended that the inspector and UAS pilot are different, since the skill sets are different. The team leader shall be on site at all times during the bridge inspection, and will have the specific information requirements and, in most cases, will be responsible for determining the flight path, if the right imagery and the amount of data is collected, and if the imagery is of a high enough quality to meet inspection practices requirements. The team leader will also be involved in interpreting any post-processed information.
Sensors
The visual sensor is the primary payload carried by UAS to support bridge inspections. While the platform and its performance capabilities can provide access to difficult-to-reach parts of a bridge, the sensor captures the imagery data (video as well as still images) which allows inspectors to determine the condition of the bridge. Figure 1a shows an example of a still image captured by visual sensors. Post-processing of imagery data takes the raw data captured by the UAS payload sensor(s) and transforms it into usable information for bridge engineers and owners. Common information products include images, video, orthophotos, ortho-mosaics, 3D models, and surface models.
Example of (a) still image and (b) infrared (IR) image of a bridge deck.
Light detection and ranging (LiDAR) systems, multispectral cameras, and hyperspectral imagers are more advanced types of visual sensors. LiDAR sensors actively emit pulses of light—up to hundreds of thousands of returns per second—to accurately measure the distance between the sensor and a target object (as precise as 1 in.). The main advantages of LiDAR over photogrammetry are the ability to penetrate vegetation with multiple returns, faster imagery processing times, and improved capabilities to resolve fine features. Data captured by LiDAR can be used to produce detailed, dense point clouds of structures. Geospatial software programs and CAD can then be used to create a 3D model of the bridge.
In addition to digital cameras, infrared (IR) sensors are commonly deployed on UAS to view areas of concrete showing signs of deterioration, such as bridge decks, by identifying and viewing temperature gradients. This data may also be used to confirm findings identified using traditional inspection methods such as the chain drag process. Figure 1b shows an IR image of a bridge deck which detects deck delamination.
Additionally, various collision-avoidance sensors, based on computer vision, sonar, laser range finders, and LiDAR, are deployed or in development on some UAS platforms to make semi-autonomous bridge inspections more viable.
Data Needs of Bridge Owners
UAS provide the bridge owner and the inspector the means of remotely positioning a sensor to capture data of a bridge structure. At the current state of UAS and sensor technology, the primary type of data available to the bridge owner is live video, still imagery, and video. Inspectors can utilize this visual information, and the associated geographic position information related to the images, to satisfy inspection requirements. High-resolution images allow the inspector to see deficiencies in an “up-close” or magnified manner without having to physically access the specific area on the bridge. UAS-captured high-resolution images may reveal defects missed in routine visual inspection. High-resolution imagery can be used for various purposes, from solely as a record of surface defects to collecting extract true measurements to track deterioration over time. UAS data captured during an inspection can be used to update the structure inspection records, identify and assess any new deficiencies, and update bridge repair recommendations. In some cases, UAS imagery taken of the entire structure has been used in the creation of bridge “plans” for older bridges where the original structural drawings no longer exist.
UAS provide the bridge owner with the ability to capture multiple images of an entire bridge structure from different views, enabling creation of a 3D computer model of the bridge by utilizing photogrammetric techniques. These 3D models allow the inspector and the bridge owner to view structures from different perspectives and angles and better understand the overall geometry of the bridge in a spatially accurate format, and permit the measurement and quantification of defects post-inspection. These models also provide a very effective means of presenting the information effectively to multiple stakeholders. Figure 2 shows an example of MnDOT’s use of a 3D model. The inspector clicks on a particular point in the model and views images that contain that point. The image can then be inspected for defects.
A digital 3D model of a stone bridge.
Quality and Quantity of Data
High-quality images enhance the inspection results and associated reports by allowing the inspectors to identify small but potentially critical defects. Image quality is affected by sensor specifications, environmental conditions, and the operator’s ability to properly adjust settings to account for ambient lighting conditions and aircraft motion. Image quality can also be affected by improper subject framing; therefore, it is critical that the UAS pilot and inspection team leader work together to properly position the aircraft to capture the appropriate image. When using IR sensors, employing improper techniques could induce false negatives or positives in the image, thus rendering the images useless. Transient thermal conditions are required and can be affected by time of day, weather conditions, and surrounding environment (any non-permanent barriers, cones, or barrels resulting in shaded areas).
A “more is better” operational approach is not necessarily the best approach to collecting UAS data. The quantity of images required will be determined by the desires of the bridge owner. If a 3D model is desired, a greater amount of raw data is necessary than if only high-resolution still images are requested. A review of the bridge’s historical file during inspection planning so the condition of the bridge is well understood will aid in identifying the flight plan and estimating the imagery products required. In cases when time is limited, it may be beneficial to focus on areas of known structural issues, capturing the images needed to track the status of a deficiency and determine its progression. Collecting many images for every bridge component and element may require additional flights and extra time to sort through the data to ensure the correct images were captured.
Cost Impacts
The impact of UAS on the costs associated with bridge inspections has been documented by multiple studies, with both positive and negative impacts found. Various studies identified the cost benefits in monetary savings and personnel time requirements. Some of the specific cost saving case studies include the following:
An analysis conducted by Oregon State University for Oregon DOT identified a monetary saving of around $10,000 per bridge and a 10% saving in personnel time ( 2 ). In one example from their studies, a bridge inspection using UAS saved $3,900 in hourly labor costs, $2,800 in equipment rental, and $3,500 in traffic control costs, for a total saving of $10,200.
A March 2016 survey by the American Association of State Highway and Transportation Officials (AASHTO) found that using UAS resulted in a saving of $4,350, a labor force reduction of two people, and a time reduction of 6 hr for a standard bridge deck inspection ( 3 ).
MnDOT conducted an inspection employing traditional access methods including under bridge inspection truck (UBIT), person lifts, and other access means that cost approximately $59,000 and took 8 days to perform. This was compared with the cost of a UAS contractor conducting the inspection for $20,000 and 5 days on site, resulting in a 66% cost saving and saving 3 days of personnel time ( 4 ). In the final phase of the MnDOT research project, it was determined that there was an average cost saving of 40% and an increase of 2% for personnel time ( 1 ). The increase in personnel time was a result of additional imagery processing and analysis time requirements.
There are cases where using UAS did not save time or money. An inspection conducted on a bridge in Idaho where a hands-on fracture critical member inspection using a UBIT cost $1,564 and took 4 hours to complete; whereas a UAS-assisted inspection was only able to complete approximately one quarter of the inspection in 4.5 hours at cost of $200 per hour. The cost was assumed to be $1,800 to complete the entire bridge ( 5 ). There are other costs that are specific to UAS operations and will vary in amount and frequency of outlay depending on the type and number of flight operations the organization conducts.
Data Management
When employing UAS during bridge inspections, the amount of data captured can be massive, as the data captured by the sensors requires storage, post-processing, analysis, and dissemination. Having a data management plan developed and in place can ensure the inspectors capturing the data have a standard approach for collecting and transferring the data, a known and secure location and structure for storing and retrieving the data, and a well-understood process for sharing the data and inspection products generated by UAS. A data management plan outlines the types of data to be created, the data formats and metadata standards to be used, personnel responsibilities, data security and sharing procedures, duration of data retention, data storage methods, information dissemination methods, and the costs associated with the elements of the plan.
Integration with Existing Bridge Management
In general, an inspector is required to create a standard inspection report for each bridge inspection that conforms to NBIS in accordance with the state DOT policies and standards. UAS facilitate collection of imagery data, and have the ability to record the precise location of defects, which allows for tracking the change in condition of defects over time. Precise location information also leads to more exact work recommendations and cost estimates. Precise location also enters into prioritization for repair, as some locations are more critical than others. UAS also have the ability to record completed work and its location on bridge and members. This will allow for more accurate measurement of effectiveness of work type/treatments (improvement in condition and extended life, etc.) for modeling the cost and benefits of work types.
Supplemental and Future Technologies
As more bridge owners and inspectors incorporate UAS into their processes, the UAS technologies available to improve inspections continue to advance.
First Person View
First person view (FPV) devices or goggles, while not a new technology, are a relatively recent entry to the bridge inspection process. FPV give the user a unique perspective from which to wirelessly view imagery and control the camera. Some FPV systems provide HD 1080p video and allow the user to control the sensor in real time with movements of their head. The image provided has been described as like looking at an 18 ft HD TV from about 9 ft away. Another advantage of some FPV systems is the ability to digitally magnify the image, making it appear as though the object in view were significantly closer, thus allowing a bridge inspector to see hairline cracks in the structure.
This technology, when employed effectively, can aid in reducing the amount of image and video data collected by focusing the inspector’s efforts on a real-time inspection and only capturing images of concern and in need of documentation. This will allow capturing and storing data on those areas that have defects, instead of collecting a large array of images that must be sorted through later.
With the use of FPV goggles, the inspector wearing the goggles will be totally focused on the live video images. This will allow them to perform live inspections of the bridge components but will immerse them in the UAS image and reduce their situational awareness. With this issue being identified, the inspection team will need to consider this in their inspection approach and include it in the safety plan before the inspection.
As FPV and other drone technologies are developed and used in inspection, they open up the bridge inspection work force for those who may have physical limitations, and increase the potential pool of inspection staffing.
Sensor Technologies in Development
The current UAS sensors can provide high-resolution and thermal images that offer immense amounts of visual information about the surface of an object. To acquire information about sub-surface condition, there is ongoing development with sensors like ground penetrating radar (GPR) and ultrasonic sensors that could be deployed on UAS (6, 7). Another technology being explored by researchers at the University of New Mexico is a laser carried by UAS to capture measurements of cracks, as well as the deflection.
In the AErial RObotic System for In-Depth Bridge Inspection (AEROBI) project, underway in Europe, a UAS fitted with robotic arms and computer vision is being designed to conduct non-destructive evaluation of concrete for cracks and spalls, measuring the defect dimensions and automatically processing them to provide structural assessments ( 8 ). Such a system could reduce risks for human inspectors through the use of flying robotics to reach hazardous areas of a bridge.
Artificial Intelligence (AI) in UAS Operations and Data Analysis/Post-Processing
In confined or restricted spaces, where GPS is lost, the UAS enters a mode of flight that requires the pilot to maintain horizontal position control manually. Manual UAS control can be challenging and result in more difficult flying conditions, potentially creating hazardous situations for the aircraft and the personnel on the ground if the pilot is not well trained and proficient. The aircraft becomes more susceptible to winds, and manual corrections may be more exaggerated than those made by the aircraft’s automatic flight controller. This can lead to blurred or imprecise imagery. Flying beneath the bridge deck is when this situation is most commonly encountered. As UAS technology advances, another option will be to augment or remove the pilot from the active control loop by utilizing artificial intelligence (AI). UAS systems with AI technologies incorporated into the system are capable of navigating independently without human input other than instructing the aircraft when and where it supposed to fly, and having override capability in the event of a malfunction.
Other AI technological advances are in development to enhance the data collection processes associated with UAS, and to increase the speed and accuracy of imagery post-processing. As technology advances and machine learning improves, it is possible this technology could be applied to the task of analyzing images of infrastructure for specified defects. Currently AI and machine learning systems can identify, with high levels of accuracy, objects both animate and inanimate, and can categorize objects based on previous analysis. Researchers from Purdue University and the University of Southern California have used machine learning techniques to analyze overlapping images for structural cracks, resulting in detection accuracies as high as 95% ( 9 ).
Conclusion
The speed of technological advances and the improvements discovered in the integration of new technologies is affecting industries and processes of all kinds. The inspection of bridges is no exception. More and more bridge stakeholders are utilizing UAS, and they are exploring new ways to integrate UAS within established inspection guidelines.
Given the current state of UAS and sensor technology, along with existing FHWA and state inspections standards, UAS are best thought of as a means to get a camera into difficult-to-reach areas to help detect and then capture images of structural defects during routine inspections, particularly those for large, long, and complex bridges. Inspectors, engineers, and UAS operators are using UAS to augment traditional inspection techniques, exploring how UAS can replace or reduce the need for large and expensive access equipment, and enhancing safety by keeping inspectors on solid ground and lessening or eliminating the need for traffic control measures. UAS can reduce the cost of an inspection in some cases, but also present data management challenges. Advanced UAS platforms and sensors, along with skilled and experienced operators, may be required to get the maximum benefit. The use of UAS for fracture critical inspections is more challenging at present. Cameras and today’s sensors still have little capability to see through dirt, debris, and corrosion which can hide critical defects. NBIS requires a hands-on inspection of fracture critical members which requires cleaning of the bridge member when necessary to assess condition.
Footnotes
Acknowledgements
The authors acknowledge Futron Aviation for conducting this research study.
Author Contributions
The authors confirm contribution to the paper as follows: study conception and design: H. Azari; draft manuscript preparation: H. Azari, D. O’Shea, and J. Campbell. All authors reviewed the results and approved the final version of the manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was sponsored by the Federal Highway Administration (FHWA).
The contents of this paper do not necessarily reflect the official views or policies of the FHWA.