Structure from motion photogrammetry in physical geography

I Introduction

Over the past few decades advances in surveying technology have revolutionised our ability to record and characterise the Earth’s surface quantitatively: a fundamental requirement of physical geography. Differential Global Positioning Systems (dGPS) and Total Stations (TS) have enabled 3D positioning of observations to millimetre-scale accuracy. More recently, Airborne Laser Scanning (ALS) and Terrestrial Laser Scanning (TLS) have increased the spatial coverage and density of available datasets through non-selective sampling of millions or even billions of survey points to produce 3D point clouds. The impact on the study of physical geography and geomorphology in particular is well documented (Tarolli, 2014). Yet, a limitation to the more widespread uptake of ALS and TLS is the capital outlay and expertise required to acquire and operate these complex instruments.

In the past few years, Structure from Motion (SfM) has been demonstrated to have the potential to democratise 3D topographic survey by offering rapid 3D point cloud acquisition for minimal expense. Geomorphologists, and physical geographers more generally, have been quick to adopt SfM as seen by the recent and increasing proliferation of studies utilising and testing SfM (e.g. James and Robson, 2012; Westoby et al., 2012). SfM has been applied to a wide range of environmental problems, including monitoring glacier movement (Immerzeel et al., 2014; Ryan et al., 2015), quantifying soil loss and gully erosion (Eltner et al., 2014; Frankl et al., 2015), observing and tracking lava movement (Tuffen et al., 2013) and landslide displacement (Lucieer et al., 2013), monitoring coastal recession (James and Robson, 2012), surveying fluvial morphology (Javernick et al., 2014) including submerged surfaces (Woodget et al., 2014), characterising rock outcrops (Favalli et al., 2012) and quantifying aboveground forest biomass (Dandois and Ellis, 2013).

At this point, it is instructive to define what is meant by ‘SfM’. Strictly speaking, Structure from Motion refers to only one element of the SfM workflow. Although there are cases where a SfM-derived point cloud is seen as the end-product (Dandois and Ellis, 2010; Fonstad et al., 2013), the majority of studies then implement Multi-View Stereo (MVS) photogrammetry algorithms to increase the point density by several orders of magnitude. As a result, the combined workflow is more correctly referred to as ‘SfM-MVS’.

To date there is no comprehensive synthesis of the practical options available to physical geographers when planning a SfM survey, errors to be expected from each choice and, perhaps most fundamentally, the underlying processes actually taking place as part of the SfM workflow. In this paper we provide a qualitative comparison of SfM-MVS with other field topographic survey techniques. We detail the origin of SfM-MVS and the workflow applied by many SfM-MVS software packages (Section III) and review the practical details of implementing SfM-MVS by outlining a number of choices (e.g. camera, survey platform, SfM software) to be made by the user (Section IV). Here we provide specific guidance on data acquisition for SfM-MVS users in physical geography. Existing validation approaches are synthesised (Section V) to provide an overview of the achievable quality of the resulting point clouds. We summarise a range of applications of SfM-MVS in physical geography (Section VI) and consider briefly potential future directions and opportunities for SfM-MVS to make important contributions to the discipline (Section VII).

II The place of SfM in topographic survey

In recent decades there has been a gradual increase in our ability to collect topographic survey data, from 10¹–10² measurements per day with a traditional optical level, increasing to 10³–10⁴ with dGPS and to >>10⁶ measurements per day with laser scanning technology. Improvements in precision and accuracy have been notable, but more modest. Each new development has required new technology that is typically very costly (often >£10,000); conversely, SfM-MVS can offer similar data quality to high resolution survey techniques at minimal expense. A broad comparison of SfM-MVS with existing survey techniques is summarised in Figure 1.

OPEN IN VIEWER

Figure 1.

Comparison of digital survey methods with regard to financial cost, data acquisition rate, spatial coverage, resolution and accuracy. Note that photogrammetry and SfM values are dependent on survey range. Colour bars for photogrammetry represent ‘conventional’ aerial photogrammetry, though the range values for alternative implementations of photogrammetry are indicated. Bars derived from information in Brasington et al. (2000), Young (2012), Gallay (2013) and Bangen et al. (2014b).

TS or Electronic Distance Measurement (EDM) surveys offer high-accuracy point data (∼mm 3D accuracy) of the ground surface (Fuller et al., 2003; Milne and Sear, 1997). Where local benchmarks or ground control is available, real-world co-ordinates can be provided, even in conditions where sky-view is limited as no communication with satellites is required. They are typically used in conjunction with a pole-mounted prism or can be used without a reflector. TS points are sampled selectively by the operator(s) and can be strategically positioned to identify notable breaks of slope, providing a high quality digital elevation model (DEM) for relatively small file sizes. The main disadvantage of TS surveys is the point density achievable within a survey campaign, which is limited by the time required to sample points individually. Moreover, the expert judgment of where to locate each survey point may introduce bias towards accessible locations and variability between operators (Bangen et al., 2014a). Such bias is significant because the interpolation that is required to produce a topographic model is affected by the specific survey point geometry.

Data provided by dGPS surveys can be of similar accuracy to that of TS surveys (Bangen et al., 2014b), depending on the length of time a point is occupied and survey mode. Real-time kinetic (RTK) surveys are commonplace in physical geography where a direct radio or GSM (mobile network) link between a surveyor and a base station can provide the surveyor with information on the final accuracy of the solution relative to the base station co-ordinates (Brasington et al., 2000, 2003; Wheaton et al., 2010). As such, dGPS surveys are subject to many of the same advantages (accuracy, precision, selectivity) and disadvantages (low point density) as TS surveys (Bangen et al., 2014b). The accuracy of RTK-dGPS surveys is ∼cm scale (higher accuracies can be achieved on continuously logging static mode) and is dependent on the number and geometry of satellites used to compute a point, therefore a clear sky-view is also required. However, unlike TS surveys, a direct line-of-sight between the instrument and the survey pole is not required.

ALS, also known as airborne Light Detection And Ranging (LiDAR), has become a well-established survey tool in physical geography over the last two decades (e.g. Charlton et al., 2003; Evans and Lindsay, 2010; Thommeret et al., 2010). The remote platform and consequent large survey range (typically 100–4000 m between the sensor and object of interest; Gallay, 2013) means that ALS is naturally suited to landscape scale surveys, resulting in decimetre scale point resolution and accuracy (Bangen et al., 2014b; Gallay, 2013). TLS comprise essentially the same technology mounted statically on a conventional survey tripod (Brasington et al., 2012; Hetherington et al., 2005; Smith, 2015) (though mobile systems are also in use; Alho et al., 2009). The shorter range and static setup of TLS means that mm-scale precision and accuracy can be achieved. The main advantage of LiDAR systems is the acquisition rate, which can be up to hundreds of thousands of points per second. While TLS offers much higher survey accuracy and point density than ALS, this comes at the expense of reduced survey area. Moreover, both ALS and TLS are expensive survey solutions (>£30,000 for a TLS) that produce large quantities of data which must then be decimated to produce useful terrain products. Another disadvantage of TLS is portability; while the TLS instruments themselves typically weigh 5–10 kg, they require much ancillary equipment (e.g. tripods, batteries, cases, targets) and the total weight of many systems restricts their application to areas with good access. Where targets are used to register multiple scans together, surveys must be planned carefully such that they can be viewed from multiple scan locations. Time-limitations determine the number of tripod setups possible meaning that gaps or shadows may be present in the final point cloud owing to occlusion effects. Hand-held mobile laser scanners have been recently used to overcome the limited portability of tripod-mounted TLS (James and Quinton, 2014), though reported errors are centimetre-scale.

Conventional photogrammetry is the closest existing technique to SfM-MVS, and has contributed much to the workflow of SfM-MVS (Micheletti et al., 2015). Conventional photogrammetry uses precise knowledge of the 3D location and pose of cameras, or the 3D location of a set of control points located in the scene of interest, to reconstruct scene geometry. Achievable spatial resolution is a function of the pixel size but digital photogrammetry has been applied across a range of scales from plots of ∼10¹ m² (Butler et al. 2002; Carbonneau et al., 2003) to landscapes or river reaches of >10⁶ m² (Lane, 2000; Westaway et al., 2000). For photogrammetry, once equipment has been set up, data collection requires only several minutes. While vertical (overhead) photographs are required in conventional approaches, oblique images can be used either through the implementation of rotation matrices (Chandler, 1999) or by using more recent photogrammetric software capable of processing oblique images (James et al., 2006). Fine resolution and accurate (mm-scale) topography can be obtained at ranges of several metres; the achievable accuracy is reduced with increasing distance between the camera and object of interest. However, the main disadvantages of conventional photogrammetry are the degree of expertise required, the difficulty and expense of obtaining the large amount of a priori information required and the relative inflexibility of the image geometry (e.g. degree of overlap) which may render it unsuitable for some applications.

SfM-MVS is a non-selective survey method (like ALS, TLS and conventional photogrammetry where each data point is not selected for inclusion in the survey individually by the surveyor) and the resulting data are most similar to those of TLS. Similar (or greater) point densities can be achieved and while point precision and accuracy is mostly determined by survey range, sub-cm scale errors are achievable (Smith and Vericat, 2015). Perhaps the biggest disadvantage of SfM-MVS is the fact that the quality of the resulting surface model depends on many different factors related to an individual survey. SfM-MVS data are typically not as precise as TLS-derived point clouds; however, SfM-MVS is flexible enough to be applied at a wide range of scales. Specifically, to date, it has been used over survey areas from 10^-2–10⁶ m² (Smith and Vericat, 2015) and was even used by Dietrich (2015) to survey downstream morphological patterns in a 32 km river reach. The equipment needed for SfM-MVS is often portable and while scale can be provided using only non-specialist technology (e.g. standard laser rangefinders), accurate scaling and georeferencing requires the use of a TS or dGPS. Occlusion and shadowing remain an issue with SfM-MVS, but it is minimised relative to TLS surveys owing to the greater number of viewpoints that can be integrated into a single SfM-MVS survey.

It is clear that each of the techniques outlined above has different strengths and weaknesses and is better suited to different tasks. Certainly, SfM-MVS is not a complete substitute for these other methods (for example, accurate fluvial bathymetry over small areas is better obtained by TS or dGPS survey than SfM-MVS; Bangen et al., 2014b; Woodget et al., 2014). However, in some circumstances, particularly where plot scale (∼10¹ m²) data is required (Smith and Vericat, 2015) or where decimetre scale accuracy is acceptable over ∼1 km² of bare ground (Javernick et al., 2014), SfM-MVS is an efficient and cost-effective survey method.

III Principles underlying SfM-MVS

The clear similarities between conventional photogrammetry and SfM-MVS may lead to the assumption that SfM-MVS is simply an incremental development in photogrammetry. However, several aspects of SfM-MVS have a different origin entirely, stemming instead from advances in 3D computer vision algorithms. Nonetheless, photogrammetric principles and techniques are embedded in the SfM-MVS workflow (see Micheletti et al., 2015). The typical workflow implemented by many SfM-MVS software packages is presented and summarised in Figure 2. The specifics of this workflow will vary from one software package to another, but there exists a clear commonality. The description here is predominantly qualitative to best communicate a technical workflow to a broad audience. Interested readers are directed to other sources (Hartley and Zisserman, 2003; Lowe, 2004; Snavely, 2008; Szeliski, 2011; Triggs et al., 2000) for specific details of the mathematical operations applied.

OPEN IN VIEWER

Figure 2.

Typical workflow in the production of georeferenced dense point clouds from image sets and ground control points. Inputs and outputs are shown in dark red. In the top right, as a demonstration, matches determined to be valid are shown in red, while matches determined to be invalid are given in blue.

IV SfM in practice

The flexibility of the SfM-approach is advantageous considering the range of problems in physical geography to which it can be applied. However, one consequence of this flexibility is that practitioners are faced with many choices in the design and implementation of a SfM-MVS survey.

1 Platforms

Most imagery can be used for SfM-MVS; indeed the SfM package outlined in Snavely et al. (2008) was designed to use collections of independent images extracted from image searches on the internet. A range of options are available, from ground-based imagery to fully airborne solutions (Figure 3). In general, there are trade-offs between: (i) pixel resolution; (ii) spatial coverage; (iii) image quality; and (iv) cost-effectiveness.

OPEN IN VIEWER

Figure 3.

Examples of platform-types from which SfM-MVS imagery can be applied: (a) lighter-than-air blimp; (b) kite; (c) multicopter-type UAV; (d) camera boom; (e) gyrocopter; (f) trail camera.

Hand-held ground-based solutions are the most inexpensive option (only a camera and georeferencing method is needed), can provide excellent pixel resolution and image quality and are easy to implement. The key disadvantage of hand-held solutions is that they are typically limited to the plot-scale (∼10¹ m²). However, with high relief (e.g. an incised valley or cliffs) areas approaching one hectare can be surveyed from the ground (James and Quinton, 2014; Micheletti et al., 2014; Smith et al., 2014). Matching of oblique ground-based images over large areas can be unreliable on relatively flat terrain; isolated mismatched areas are apparent in the final point clouds, which must be inspected carefully (Smith and Vericat, 2015). Extendable masts, booms (Figure 3d) or mobile inspection poles can be used to raise a camera up to 20 m above ground to reduce the likelihood of such mismatches by gaining a more advantageous viewpoint. However, on-the-ground image swaths are still limited in spatial extent and retaining stability of a pole or mast in adverse weather can be challenging.

Lighter-than-air blimps (Figure 3a) can raise a camera higher still, up to 500 m, though they can become difficult to position if the wind speed exceeds ∼15 km h^-1 (Verhoeven et al., 2009). Mobile blimp systems are a low-cost solution (Vericat et al., 2009) but are limited in their payload capability, constrained to operating loads of between 0.25–0.5 kg per 1 m³ of lighter-than-air gas. They also cannot be deployed in high winds and may require a local source of suitable gas. Conversely, kites (Figure 3b) can carry a payload of several kilograms. Irregular or light winds can make controlling kite-based systems challenging; thus, some systems integrate blimp and kite components to improve the range of suitable weather conditions.

There has been a proliferation in low-cost airborne platforms in recent years; unmanned aerial vehicles (UAVs) have been used extensively for acquisition of SfM-MVS imagery (e.g. Lucieer et al., 2014; Tonkin et al., 2014; Woodget et al., 2014). There exists a wide range of UAVs (see Colomina and Molina, 2014) from self-propelled fixed-wing aircraft to airships. The most popular UAVs have been so-called ‘multicopters’ (including quadcopters, hexacopters or octocopters; Figure 3c) because of their ease of operation, stability in most weather conditions and recent decline in financial cost. Most multicopters can take a payload of several kilograms, though larger systems may take up to 10 kg (Lejot et al., 2007). Flight control software and units can allow pre-programmed routes to be followed (Oúedraogo et al., 2014). On smaller UAVs, image quality can be compromised by blurring from vibrations (Dunford et al., 2009), although use of a gimballed-camera mount can minimise this problem. In many countries legislation limits the range and ground-type over which UAVs can be flown. In general, UAVs have a lower maximum altitude than blimps or kites. In each case, the flying altitude chosen will determine the final pixel resolution for a specific camera.

The broadest spatial coverage can be achieved with a commissioned, piloted overflight in an aircraft, often a helicopter (Dietrich, 2015; Javernick et al., 2014). This removes maximum altitude restrictions (though minimum altitude restrictions apply) and permits a greater flexibility and responsiveness in the survey to ensure full coverage. As with UAVs, higher altitude surveys will improve spatial coverage at the expense of pixel resolution. Issues with vibrations and blurring are circumvented when images are taken manually, though this is not always possible. Cost is a disadvantage of the piloted helicopter approach. Much smaller light aircraft (James and Varley, 2012), microlites (James and Robson (2012) with images from Cecchi et al. (2003)) or piloted gyrocopters (Smith and Vericat, 2015; Figure 3e) offer a more cost-effective solution and can be hired for <£200 per hour.

V Validation of SfM-MVS

As with any emerging technology, quantitative validation of SfM-MVS-derived topographic data against those derived from more conventional methods (e.g. TS, dGPS, ALS, TLS) is prerequisite to confident application to a real-world problem. Only TLS and ALS provide directly comparable topographic data, but concurrent data derived using laser scanning is rarely available. Moreover, issues of beam divergence and ‘mixed pixels’ in laser scan data mean that in some cases there is no a priori reason to treat TLS data as the more correct data set, especially where SfM-MVS survey ranges (i.e. object-camera baselines) are short (Smith and Vericat, 2015).

Numerous studies have validated SfM-MVS-derived data against other methods; we have synthesised them herein to provide a broad overview of the capabilities of SfM-MVS. The dataset is compiled from the findings of Favalli et al. (2012), Harwin and Lucieer (2012), James and Robson (2012), Mancini et al. (2013), James and Quinton (2014), Javernick et al. (2014), Lucieer et al. (2014), Micheletti et al. (2014), Oúedraogo et al. (2014), Ružić et al. (2014), Smith et al. (2014), Thoeni et al. (2014), Tonkin et al. (2014), Smith and Vericat (2015), Stumpf et al. (2015), sub-aerial data from Woodget et al. (2014) and an unpublished result by the authors on ice surface plots. The aggregate dataset consisted of 50 data points covering a wide range of survey scales. Most studies reported the vertical Root Mean Squared Error (RMSE), Mean Error (ME) or Mean Absolute Error (MAE). ME values should be treated with caution, as positive and negative errors will compensate for each other.

Three general validation methods have been identified (see Smith and Vericat, 2015): (i) Point-to-Raster (PR), where a SfM-MVS-derived DEM is validated against point data from TS or dGPS (Javernick et al., 2014; Lucieer et al., 2013) and results depend on both the DEM grid-size and location of the validation point relative to the mean elevation within any cell; (ii) Raster-to-Raster (RR), where two separately derived DEMs are compared (Favalli et al., 2012; James and Robson, 2012) and the method of validation-DEM generation must be considered carefully to ensure comparability; and (iii) Point-to-Point (PP), where two point clouds are compared directly (James and Quinton, 2014) and a limiting assumption is that two points would be exactly concordant under error-free conditions. Analysis of the compiled dataset reveals that PP- and RR-type validation typically results in smaller errors than PR-type validation. Smith and Vericat (2015) showed that even at 0.1 m grid sizes, PR-type validation can yield RMSE vales ∼20% higher than RR-type validation of the same data. This discrepancy is to be expected since RR-type validation compares two rasters that both summarise the topographic variability within a single grid cell statistically, while PR-type validation compares the SfM-MVS-derived raster with a single point measurement taken within that grid cell which will exhibit sub-grid topographic variability. Other validation methods have been described: Favalli et al. (2012) use a mesh-comparison tool (Cignoni et al., 1998) instead of generating raster-based topographic models, while Lague et al. (2013) compute surface normals for each point based on all data points within a pre-specified radius to inform the comparison of points (as applied by Stumpf et al., 2015).

Over half of validation data points (56%) used ground-based imagery, while the remainder are based on imagery from airborne platforms (including manned and unmanned systems). The reported RMSE is an order of magnitude greater for aerial surveys (0.277 m) than ground-based surveys (0.043 m), but aerial surveys typically cover larger areas with the camera positioned at a greater distance from the object of interest (i.e. a longer survey range). Indeed, the main observable effect on SfM-MVS data quality is a result of survey range (Figure 4). A linear degradation of precision with survey range might be expected from the increased pixel size; James and Robson (2012) observed such a linear relationship between the standard deviation of errors and survey range for three validation studies, estimating a ratio between these values as 1:1000. Yet RMSE is reported more commonly, it integrates all sources of error, and absolute accuracy is often required for monitoring studies. Considering RMSE and survey range, Micheletti et al. (2014) observed a ratio of 1:625. Combining the 43 validation data points that report RMSE, our data show a similar median ratio (1:639). A power law better describes the relationship between RMSE and range. The exponent of this relationship (0.88; R² = 0.80) is reasonably close to linear (Smith and Vericat, 2015).

OPEN IN VIEWER

Figure 4.

Variability of SfM-MVS Root Mean Squared Error (RMSE) with survey range (m) for different validation-types. Fitted power law described in the text. Data extracted from: Favalli et al. (2012); Harwin and Lucieer (2012); James and Quinton (2014); James and Robson (2012); Javernick et al. (2014); Lucieer et al. (2014); Mancini et al. (2013); Micheletti et al. (2014); Oúedraogo et al. (2014); Ružić et al. (2014); Smith et al. (2014); Stumpf et al. (2015); Thoeni et al. (2014); Tonkin et al. (2014); sub-aerial data from Woodget et al. (2014) and an unpublished result by the authors on ice surface plots. Adapted from Smith and Vericat (2015).

Figure 4 provides a useful indication of errors expected for SfM-MVS surveys at a given range. However, the effect of validation method can be seen with PR methods exhibiting the highest errors for a given survey range. When survey range is taken into account, there are minimal differences between the accuracy of ground-based and aerial SfM-MVS surveys. Overall, considering the RMSE: range ratio, ∼10–15 mm errors are achievable at ∼10 m survey range suitable for detailed studies, while ∼100–200 mm errors are observed from 100 m range which could be applied over much larger areas. Such errors are similar to those expected from ALS surveys (Gallay, 2013).

Several studies included in the above analysis report results using multiple SfM-MVS workflows (e.g. varying cameras or software) on the same image set or area of interest (James and Robson, 2012; Javernick et al., 2014; Micheletti et al., 2014; Oúedraogo et al., 2014; Smith and Vericat 2015; Smith et al., 2014; Stumpf et al., 2015; Thoeni et al. 2014). In each case, the best-performing SfM-MVS-workflow on each image set or area of interest was selected for inclusion in Figure 4. Through comparison of the best-performing workflow with other workflows, these studies yield further insight into the effects of GCP distribution, processing software used and camera used for image acquisition.

Clearly, the GCP quality will determine overall SfM-MVS survey accuracy. In addition, it is well established that poor GCP distribution has a detrimental effect on survey quality. Javernick et al. (2014) reported an increase in RMSE from 0.23 m to 0.27 m when the study reach was extended beyond the area covered with GCPs. Smith et al. (2014) observed a more pronounced increase (from 0.14 m to 0.47 m). James and Robson (2012) also reported increased survey errors when GCPs were not distributed throughout the survey area.

In a comparison of SfM-MVS processing software, Micheletti et al. (2014) noted a near-doubling of RMSE when the web-based 123DCatch was used instead of Eos PhotoModeler, though this effect was removed when the point clouds were aligned using an iterative closest point algorithm (Besl and McKay, 1992). Oúedraogo et al. (2014) observed that MicMac performed better than Agisoft in a ploughed agricultural catchment as the more sophisticated camera model reduced the observed doming effect and Stumpf et al. (2015) found that MicMac outperformed VisualSfM for similar reasons. In a comparison of sensors, Thoeni et al. (2014) observed that consumer-grade digital cameras performed similarly to professional grade cameras, though GoPro devices performed less well. Micheletti et al. (2014) found that DSLR imagery and that from a camera phone produced models of a similar quality with errors of the same order of magnitude; indeed camera phone imagery even outperformed DSLR imagery in one test. Moreover, Smith and Vericat (2015) found convergent gyrocopter-based images produced a higher quality model than standard vertical UAV-based images at similar survey ranges as the doming effect was removed (James and Robson, 2014a), image blurring from vibrations was reduced and greater user-control over the images from manual camera operation enabled image overlap to be better specified.

Using aerial imagery and applying a basic refraction correction, Woodget et al. (2014) assessed the ability of SfM-MVS to obtain topography through water. Accuracy of exposed and submerged surface reconstructions was compared in four separate surveys. Average water depths were ∼0.15 m, though maximum depths of > 0.5 m were reported. Results were promising: once refraction correction had been applied, MEs increased in only two of the four of the surveys (0.005 m to 0.053 m and 0.004 m to -0.008 m) and were lower than exposed areas in the remaining two surveys (0.044 m to 0.023 m and 0.111 m to -0.029 m) (Woodget et al., 2014).

Overall, the lack of a consistent validation methodology or a systematic campaign to validate SfM-MVS-derived models inhibits precise determination of expected errors. The flexibility of SfM-MVS as a survey tool means that many confounding variables will combine to determine the accuracy of a SfM-MVS terrain model thereby making isolation of individual error sources extremely challenging. There exists insufficient validation data to isolate and quantify the effect of each component. This is in contrast to the standardised validation strategies for MVS algorithms where the performance of each algorithm is compared for a range of freely available, high quality validation data sets (available at: http://vision.middlebury.edu/mview/; Seitz et al., 2006). However, establishing such objective reference data to benchmark approaches is more challenging for the entire SfM-MVS workflow as applied to natural environments. It is encouraging to see images and data being made available by individual studies (James and Robson, 2012); a co-ordinated attempt to host such image sets for which concurrent TLS data is available would provide a useful starting point. The recent proliferation of SfM-MVS validation studies in physical geography-facing publications is testament to the need to improve co-ordination and standardisation of such efforts.

VI Applications of SfM-MVS in physical geography

At present, published applications of SfM-MVS in physical geography are often proof-of-concept studies. As seen in Section V, there are many papers where quantitative validation of SfM-MVS is the primary (or secondary) aim. In general, other aims can be classified into studies that either: (i) extract spatial patterns or planimetric measurements from SfM-MVS-derived orthomosaic photographs; (ii) extract three-dimensional information (volumes or topography) from SfM-MVS point clouds; or (iii) use SfM-MVS to quantify surface change. Each aim is demonstrated in a wide variety of environmental settings and applied to address a range of problems. A brief summary is provided herein.

The ability of SfM-MVS to generate orthophotographs of the scene of interest presents a substantial advantage over TLS as they hold a wealth of information of use to physical geographers. Indeed, orthomosaic photographs created using SfM-MVS have been used to calculate the area of wildlife sampling strips (Lisein et al., 2013), to map active and relict supraglacial drainage pathways and their association with glacier structure (Rippin et al., 2015), to delineate geological faults, joints and fractures (Vasuki et al., 2014; Figure 5a) and outcrops (see Bemis et al., 2014; Figure 5f) and have been used in combination with correlations between image texture and particle grain sizes (Carbonneau et al., 2004) to derive maps of grain size distribution on an alluvial fan (de Haas et al., 2014). Certainly, there is the potential to interrogate these orthophotographs in greater detail, for example, by applying gravelometric analysis in object detection software (e.g. BASEGRAIN; Detert and Weitbrecht, 2012, 2013).

OPEN IN VIEWER

Figure 5.

Example SfM-MVS-derived data products: (a) Faults on a rock surface detected and classified semi-automatically from an SfM-MVS orthophotograph (Vasuki et al., 2014: 29); (b) Orthophoto mosaic of an outer reef covered in live coral draped over a 1 mm resolution SfM-MVS-Digital Terrain Model (Leon et al., 2015: 4); (c) SfM-MVS 3D mesh of a peat slide; (d) Ice-flow velocities of Store Glacier, Greenland, calculated using feature tracking of SfM-MVS-DEMs (Ryan et al., 2015: 5); (d) Displacement of Holme Hill landslide calculated by applying feature tracking algorithms to SfM-MVS-derived orthomosaics (Lucieer et al., 2013: 111); (f) 3D mesh of a sedimentary sequence exposed on the River Mersey near Manchester.

Most commonly, SfM-MVS is applied in the production of fine-resolution terrain data. A large number of studies apply SfM-MVS to coastal environments (Harwin and Lucieer, 2012; James and Quinton, 2014; Mancini et al., 2013; Ružić et al., 2014) to extract the topography of beaches, dunes or cliffs. Implementation of SfM-MVS on cliffs surfaces is particularly straightforward since their near-vertical surfaces offer advantageous viewpoints for ground-based imagery. Beyond the shore, subaqueous topography of coral reefs has been derived by Nicosevici and Garcia (2008) and Leon et al. (2015; Figure 5b). Again, advantageous viewpoints can be achieved with relative ease underwater (images were taken by a snorkeler in Leon et al., 2015); however, obtaining accurate georeferencing data underwater is more problematic, with a basic hand-held GPS device used by Leon et al. (2015). SfM-MVS has also been used to quantify properties of rock outcrops (Favalli et al., 2012) and geological hand samples (James and Robson, 2012) where a range of viewpoints can be easily acquired.

The geometry and environmental setting of fluvial surfaces presents a more challenging test of SfM. Fonstad et al. (2013) used SfM on aerial imagery to produce models of exposed topography and Dietrich (2015) used a SfM-derived orthophotograph and DEM from a helicopter-based survey to describe morphological features of a 32 km river reach. In a braided system, Javernick et al. (2014, 2015) extended this by fusing the SfM-MVS topographic model of exposed surfaces with optical bathymetric mapping also using imagery taken from a helicopter, while Woodget et al. (2014) corrected for refraction effects to extract bathymetry from SfM-MVS data directly. This method relies on the visibility of the river bed and cannot be applied where light does not penetrate to the bed. An ephemeral river setting permitted Smith et al. (2014) to extract bed topography using SfM-MVS once a flood wave had passed. This study coupled a SfM-MVS DEM with flood marks extracted from the imagery used for the SfM-MVS model and both were input into a depth-averaged 2D hydraulic model to evaluate peak flash-flood flood magnitude. Used in this way, SfM-MVS has potential to be used as part of post-flood surveys by increasing the data-acquisition rate of teams in the field and thus enabling greater spatial coverage of flood reconstructions shortly after a flood event.

Other applications of SfM-MVS are increasingly using SfM-MVS data as input into models of surface processes in this way. Javernick et al. (2015) also used SfM topographic data as input to depth-averaged 2D hydraulic model; while Westoby et al. (2015) used SfM-MVS terrain data and a 2D hydraulic model to numerically model the propagation of different glacial outburst flood scenarios as part of a hazard assessment. Meesuk et al. (2015) fuse SfM-MVS data with ALS data, which was shown to improve the performance of urban flood models in predicting flood depth and extents. In a coastal setting Casella et al. (2014) merged SfM-MVS topography with multibeam bathymetry for input to a wave runup model. Predicted maximum wave runup was compared with that observed in SfM-MVS orthophotographs to evaluate the model. Such coupling of the topographic data created from SfM-MVS surveys with data extracted from orthophotographs has the potential expand SfM into more novel physical geography applications. For example, Lucieer et al. (2014) also combined SfM-MVS-derived topographic models and orthophotos to relate the spatial distribution and health of moss beds in East Antarctica to snowmelt drainage pathways.

Topographic data sets are often interrogated further to extract sub-grid roughness metrics (Leon et al., 2015; Rippin et al., 2015; Smith and Vericat, 2015) which are a common requirement in physical geography and related disciplines (see Smith, 2014). The large increase of data density in TLS and SfM-MVS datasets (orders of magnitude above more conventional survey techniques) permits the calculation of roughness at a greater range of scales than before. Such roughness may influence surface processes directly (e.g. ice surface melting; Irvine-Fynn et al., 2014), reflect spatial or temporal differences in process operation (e.g. subglacial erosion and deposition; Li et al., 2010b) or be used as a surrogate for less measureable processes (e.g. flow resistance; Aberle and Smart, 2003). The increased availability of dense topographic datasets resulting from widespread application of SfM-MVS opens up the possibility of more extensive application of roughness metrics to address physical geography problems, particularly in the use of multi-scale roughness metrics (Smith, 2014).

SfM-MVS data are also used for volumetric measurement of features (Figure 5c), including the size of coastal boulders (Gienko and Terry, 2014), eroded gully volumes (Castillo et al., 2012; d’Oleire-Oltmanns et al., 2012; Frankl et al., 2015; Gómez-Gutiérrez et al., 2014) and aboveground forest biomass (Dandois and Ellis, 2013).

Repeat SfM-MVS surveys are used to monitor topographic or positional changes. Changes are quantified through: (i) interrogation of orthophotographs with image correlation software to observe displacements, as applied by Lucieer et al. (2013) to landslide displacement monitoring (Figure 5e) and Immerzeel et al. (2014) and Ryan et al. (2015) to glacier movement (Figure 5d); (ii) differencing resulting topographic models, as applied extensively in physical geography to monitor cliff retreat (James and Robson, 2012), change in periglacial sorted circles (Kääb et al., 2014), braided rivers (Tamminga et al., 2015), erosion rates of badlands (Smith and Vericat, 2015), gully volume change (Kaiser et al., 2014), rill and interrill erosion (Eltner et al., 2014), landslide activity (Stumpf et al., 2015; Turner et al., 2015), glacier thinning (Immerzeel et al., 2014), calving dynamics (Ryan et al., 2015) and lava dome subsidence (James and Varley, 2012); or (iii) by identifying 3D displacement vectors of features observed on two point clouds, as applied to study the mobility of advancing rhyolitic lava flows (Tuffen et al., 2013).

While not comprehensive, this brief overview demonstrates the range of applications for SfM-MVS in the field of physical geography and the high demand for the data provided. Physical geography is not unique in this regard. In related fields, SfM-MVS is used regularly; for example SfM-MVS is now used in archaeology (De Reu et al., 2014; McCarthy, 2014) and as part of efforts to preserve cultural heritage digitally (Gallo et al., 2014; Koutsoudis et al., 2014).

VII Looking forward: The potential of SfM-MVS

There is little doubt that SfM-MVS surveys will play an important role in the study of physical geography into the future. It is unlikely that SfM-MVS has achieved its full potential; recent developments in computer vision and related fields indicate that SfM-MVS can address an even wider range of problems in physical geography.

Aerial platforms suitable for SfM-MVS have developed markedly in the last few years. Previous issues of image stability (Hardin and Hardin, 2010) have largely been resolved in more sophisticated UAVs on the market today. The increasing popularity of UAVs for a range of applications will likely drive down the cost of such systems. Increased battery life is improving the range and performance of UAVs. Developments in pilot-assistance and navigation systems will increase the reliability of UAVs, especially during critical take-off and landing phases. Improvement in on-board GPS systems could increase the accuracy of methods to determine camera position co-ordinates directly and avoid the need for target-based georeferencing. This development would permit SfM-MVS reconstructions of large scale geomorphological features (e.g. alluvial fans, whole valleys) as no physical access to the study site is required. While technical issues remain relating to timing errors and the weight of on-board components required for direct georeferencing (Whitehead and Hugenholtz, 2014), these are largely being resolved. Such developments will enable SfM-MVS data acquisition over much larger areas, useful for capturing topography beyond the hillslope or feature scale and towards the landscape scale. Even routine monitoring of topographic changes over broad areas could be achieved this way. Perhaps the biggest challenge facing UAV-based SfM-MVS surveys is future legislation restricting both flight ranges and the degree of automation of such systems. With such legislation in mind, the deployment of autonomous vehicles for SfM-MVS is likely to be restricted to the ground or underwater settings. Indeed, the development of compact and cheap underwater platforms (e.g. ‘OpenROV’) will likely stimulate underwater applications of SfM-MVS in relatively inaccessible locations.

While the above developments enable larger areas to be incorporated into SfM-MVS surveys, static arrays of cameras can be used to monitor topographic changes over time and improve the temporal resolution of data sets. Discreet trail cameras are available for such networks and could permit truly event-scale monitoring of topographic change, useful to accurately constrain the driving forces of observed change and understand the relevant controls. Images can be transferred over the mobile network and could even be automatically input into SfM-MVS software to produce automated models of topographic change. Using such a network, image capture can be triggered by an external sensor, such as a rain or river stage gauge. Video footage could also be used in this way to ensure individual events are detected automatically thereby avoiding the need to pre-define survey intervals. Developing techniques to create models of dynamic scenes that move during a survey (so-called ‘non-rigid’ SfM; Wang et al., 2014) requires dynamic keypoint correspondences and is currently an active area of research that would have applications in studies of mass movement dynamics, for example. For field deployment, power consumption and data memory storage are both issues that need resolving. Generating topographic models ‘on the fly’ is computationally demanding, but could potentially be used to track the movement of objects through a scene. Video stabilisation algorithms, such as those underlying the ‘hyper-lapse’ videos of Kopf et al. (2014), could improve the quality of video footage taken onboard mobile platforms.

Crowd-sourcing of images for SfM has been already implemented outside of physical geography (Irschara et al., 2012; Snavely et al., 2008) and offers exciting opportunities to develop participatory science projects. In particular, hazard assessment would lend itself to such participatory methods (e.g. post-flood analysis), though issues with georeferencing, reliability, accuracy and precision remain (Boulos et al., 2011). In general, as our ability to acquire input data improves, managing data volumes and identifying efficient methods of generating and intelligently decimating point clouds (Lai et al., 2014; Morales et al., 2011) represents the main bottleneck for the future development of SfM-MVS.

VIII Summary

By removing the necessity for either technical expertise or expensive surveying equipment, SfM has effectively democratised the acquisition of high resolution topographic data (Javernick et al., 2014). The main technical and financial limitation is the need for TS or dGPS equipment for accurate georeferencing of point clouds. The point densities and spatial resolution that are achievable with SfM-MVS are significantly greater than those of TS, dGPS, ALS and comparable to TLS; however, SfM-MVS data are typically less precise at longer ranges (James and Robson, 2012). Reflecting this increased capacity for data capture, there has been an explosion of applications of SfM-MVS in physical geography in recent years. Survey time required in the field is minimal and the technique is sufficiently flexible to be applied over a variety of scales.

Yet, the flexibility of SfM-MVS, in terms of available platforms, sensors, data-acquisition protocols, software packages and DEM generation methods, confounds attempts to robustly validate the practically achievable data quality using SfM-MVS. A general relationship between RMSE-error and survey range has been established from a synthesis of all available quantitative validation data (Smith and Vericat, 2015), but the absence of a standardised or systematic validation strategy complicates further quantification. Indeed, the limited ability to pre-determine data quality in SfM-MVS surveys and the variable success of surveys (which can only be evaluated after field campaigns are complete) is perhaps the biggest weakness in the approach.

From a detailed description of the workflow underlying SfM-MVS, it is clear that the success of SfM-MVS is determined by the specific combination of the properties of the scene and the images captured. There are a large number of complicating factors (e.g. blurring, motion, shadows, angular separation of images) that interact to determine model quality. As the workflow is often not apparent to a typical end-user, reasons for success or failure may be unclear.

Moreover, the workflow demonstrates a large number of quality thresholds that must be specified to implement SfM-MVS. For example, at the keypoint correspondence stage a ‘distance ratio’ criterion is used to ensure the Euclidean distance of the nearest neighbour is sufficiently distinct from the second nearest neighbour; the value specified is variable between implementations of SfM. Other thresholds relate to approximate matching of nearest neighbours for computational efficiency and the identification of geometrically inconsistent matches. At present, these are neither routinely reported nor adjusted individually by SfM-MVS-users in physical geography; rather a qualitative global quality setting is often specified in individual software packages, further encouraging a ‘black box’ approach to SfM-MVS. The specific implementation of SfM-MVS is often more transparent in open-source software; such transparency and ability to adjust specific parameters implemented in the workflow should be encouraged. Critical perspectives of SfM-MVS implementation in physical geography often focus on the stages that are common to conventional photogrammetry (e.g. lens model specification; James and Robson, 2014a). Greater engagement with each step of the SfM-MVS workflow will be of benefit to the physical geography community.

Nevertheless, SfM-MVS has revolutionized topographic data capture in physical geography; it has been applied in a range of environmental settings to address a number of problems. The advent of TLS in the previous decade perhaps primed the physical geography community to be especially receptive to SfM-MVS. Indeed, in common with TLS, greater interrogation and quantification of the rich data sets that arise, combined with further technological advances in sensor design, platforms and real-time topographic monitoring, indicate that the full potential of SfM-MVS in physical geography has yet to be realised.