Techniques for quantifying the accuracy of gridded elevation models and for mapping uncertainty in digital terrain analysis

Abstract

We first provide a critical review of statistical procedures employed in the literature for testing uncertainty in digital terrain analysis, then focus on several aspects of spatial autocorrelation that have been neglected in the analysis of gridded elevation data. When applied to first derivatives of elevation such as topographic slope, a spatial approach using Moran’s I and the LISA (Local Indicator of Spatial Association) allows: (1) georeferenced data patterns to be generated; (2) error hot- and coldspots to be located; and (3) error propagation during DEM manipulation to be evaluated. In a worked example focusing on the Wasatch mountain front, Utah, we analyse the relative advantages of six DEMs resulting from different acquisition modes (airborne, optical, radar, or composite): the LiDAR (2 m), CODEM (5 m), NED10 (10 m), ASTER DEM (15 m) and GDEM (30 m), and SRTM (90 m). The example shows that (apart from the LiDAR) the NED10, which is generated from composite data sources, is the least error-ridden DEM for that region. Knowing error magnitudes and where errors are located determines where corrections to elevation are required in order to minimize error accumulation or propagation, and clarifies how they might affect expert judgement in environmental decisions. Ground resolution issues can subsequently be addressed with greater confidence by resampling the preferred grid to terrain resolutions suited to the landscape attributes of interest. Source product testing is an essential yet often neglected part of DEM analysis, with many practical applications in hydrological modelling, for predictions of slope- to catchment-scale mass sediment flux, or for the assessment of slope stability thresholds.

Keywords

digital elevation model LISA spatial autocorrelation topographic slope uncertainty mapping

I Introduction

The advent of remote sensing technologies has meant that larger areas can be mapped by fewer people in less time and at diminishing cost (Smith and Clark, 2005). The widespread use of digital elevation models (DEMs) is a direct consequence of those advantages. Traditional methods such as field surveying and photogrammetry can yield high accuracy terrain data, but they are time consuming and labour intensive (Liu, 2008). As a result, DEMs have become the most common method for extracting topographic information, also allowing the flow of water across topography to be analysed, with many practical applications in hydrological modelling and for predictions of slope- to catchment-scale mass sediment flux (Desmet and Govers, 1995; Dietrich et al., 1993; Kamp et al., 2003; Kirkby, 1990). Topographic information in a DEM can be represented and stored as: (1) a triangulated irregular network (TIN); (2) discrete landform elements based on the intersection of flow- and contour lines; or (3) a grid. Here we examine gridded DEMs, which despite drawbacks arising from higher demands on computer memory (Claessens et al., 2005: 462; Yue et al., 2007: 161) nevertheless remain the most commonly used data source for digital terrain analysis because of their simple structure and compatibility with other digitally produced data (Wise, 2000: 1910).

Slope gradient and aspect, which are examined in this review, are two of the most basic parameters affecting landscape-scale natural processes. The accuracy of a slope map determines many of the predictions that geomorphologists, engineers, hydrologists, and ecologists, farmers or foresters make about the behaviour of environmental variables, about natural hazards, land-use potential, and for planning. Errors on slope propagate to more sophisticated DEM-derived terrain attributes such as slope curvature, drainage networks, slope channel lengths, slope widths or topographic wetness indices. Although seemingly obvious, the consideration of error distribution and error propagation in DEMs has nonetheless been neglected in routine DEM-based terrain studies, perhaps because the appeal and immediacy of DEM implementation in commonly available software packages overrides any concern for accuracy and error. The imperfections of gridded DEMs, i.e. the fidelity with which they represent the ‘real world’, have been extensively explored in the last decades. Fisher and Tate (2006) have described errors on gridded data sets as the difference between a model value and the true value, i.e. as deviations from reality (called incorrect elevation values, misplaced elevation values, data gaps, or missing values). True or real values are absolute elevation measurements collected on the Earth’s surface with the instruments of terrestrial geodesy, whether satellite-assisted or not. Like all numerical models (Dietrich et al., 2003: 109–110), elevation models are approximations of reality, but this does not mean that sources of error are unknowable, intractable or unquantifiable. Any step in DEM production could be a source of error, and error should be detected early in the batch processes in order to avoid its insidious propagation to subsequent steps and its effects on the quality of the final product (Darnell et al., 2008; Hengl et al., 2010).

The risk of introducing or ignoring distortions is to generate bias in DEM-based quantitative analyses of the Earth’s surface and in the accurate characterization of landforms and processes. The use of terrain models for environmental and engineering applications, or even for heuristic purposes, thus calls for an assessment of error relating to data type, data precision, the way the data acquisition tools operate, and to the comparative advantages of high ground-resolution (g.r.) data such as LiDAR, SpotLight (g.r.: 1 m, TerraSarX), Cosmo-SkyMed 4 (g.r.: 2–4 m) or Alos Prism (g.r.: 2.5 m) (Hengl et al., 2010: 769). At a time when the number and diversity of satellite, airborne and radar platforms is on the increase due to the rapidly evolving technology, this requires at least some knowledge of data processing techniques and appropriate means for meaningfully evaluating the reliability of results used in terrain analysis.

Here we review the variety and relative performance of existing approaches while emphasizing from the start that few studies have embraced the analysis of spatial autocorrelation as a prerequisite for developing error models. Only random patterns exhibit no spatial autocorrelation (Hunter and Goodchild, 1997; Oksanen and Sarjakoski, 2005). DEM error, however, is in essence already spatially dependent because of the interpolation methods used in DEM generation (Carlisle, 2005: 524; Liu and Bian, 2008: 308). This means that redundancy occurs in spatial data and can bias the common assumptions of independence between values in many statistical operations (e.g. least-squares regression analysis). A review of the techniques and applications of spatial autocorrelation to DEM analysis will accordingly occupy the later sections of this paper and will focus around a worked example providing new results for illustrative and didactic purposes. Some recent studies have addressed spatial autocorrelation as a means of calibrating error models, but have exclusively dealt with global autocorrelation rather than with examining patterns of local-scale variation (Hebeler and Purves, 2009; Lindsay and Evans, 2008; Liu and Bian, 2008; Wechsler and Kroll, 2006), and have tended to focus on global data (e.g. GTOPO30, SRTM: Holmes et al., 2000, Wechsler, 1999; Wechsler and Kroll, 2006) and on the interpolation of high accuracy data (e.g. LiDAR: Arrell et al., 2008). Limited attention has been paid to: (1) the potential for analysing local and global spatial autocorrelation conjointly; (2) the generation of continuous spatial representations of error arising from DEM analysis; and (3) the comparative sensitivity to error of DEMs obtained from different sensors (optical, radar, airborne). We will show how local spatial autocorrelation analysis helps to locate the weaknesses of DEM structure, how it highlights the spatial heterogeneity of error, and how, from the use of local indices, it is possible to obtain error maps useful to a wide range of terrain applications relevant to physical geography.

Accordingly, we will focus on three issues that arise when different DEM sources are available for a given geographical area. (1) How can the accuracy of elevation data be assessed in such situations? (2) How can errors affecting primary topographic attributes such as topographic slope be evaluated, and what confidence can be attributed to such data for hydrological or geomorphometric applications? (3) What error mapping techniques are most advantageous for detecting regions susceptible to error propagation and for visually displaying bias in DEM-based scientific predictions? We first present a review of current knowledge about DEM accuracy and the tracking of error propagation in grid processing. The Local Indicator of Spatial Association (LISA: Anselin, 1995), similar to calculating neighbourhood filters in raster analysis, is then introduced and tested as an illustration of how to select the most efficient error model based on clustering results.

We test six DEM grids obtained from different categories of sensors and focus on techniques for mapping the spatial distribution of error affecting topographic slope values. The chosen study area is the Wasatch mountain front, Utah, USA. The data sources are a LiDAR grid, i.e. an airborne source; the composite CODEM, NED10 and GDEM, each compiled from multiple data sources, i.e. maps and/or optical and/or radar; the ASTER DEM (AstD), an optical source; and the SRTM, a radar source. Each possesses initial statistical characteristics relating to the elevation interpolation procedures, to the error corrections relating to physical acquisition parameters, and to altimetric configurations imposed by the initial choice of geoid reference frame. Although the ground resolutions of the DEMs analysed also happen to range from 2 m to 90 m, what we test here is the quality of the initial products. Ground resolution remains a separate issue. Although the complexity of any land surface is likely to decrease with increasing cell size, with ground resolution potentially affecting error distribution as a consequence, the first step in digital terrain analysis still involves choosing among an increasingly wide range of imperfect digital products, whether these are readily available on the market or purpose-generated by stereo-photogrammetry. Whatever the original ground resolution of the digital product, it becomes advantageous to resample or degrade the DEM to the ground resolution best suited to the application required only after quality checks, such as those proposed in this paper, have been performed. After the most satisfactory digital product available has been selected in this way, work on terrain roughness indices, which measure the variability of elevation based on a sliding window specific to each DEM (Grohmann et al., 2009), is one way of addressing the effects of scale on the analytical accuracy affecting the landscape attributes of interest.

II Evaluation of accuracy and error propagation in gridded DEM processing

According to Hunter and Goodchild (1997: 36), the notion of uncertainty covers undefined error (i.e. when a true value is unknown) and structural error. Undefined error covers a range of subcategories defined by Thapa and Bossler (1992: 836), including blunders, gross errors and systematic errors (see Appendix). Often these cannot be removed, whether by repeated observation or by recourse to a deterministic search function, because they tend to evade any clearly detectable pattern. After Burrough (1986) and Wise (1998), Wechsler (1999: 3) has identified three possible sources of structural error: (1) errors due to an incomplete density of observations, or to consequences of spatial sampling; (2) measurement errors such as positional inaccuracy, which might arise from improper orientation of the stereo-model or by neglecting lens distortion in photogrammetric procedures; and (3) random errors, which can normally be detected after removal of gross and systematic errors (see Appendix) and might be due to data entry faults, observer bias, or computer-generated numerical errors, interpolation errors, or problems of classification and generalization.

Finer differentiation among structural errors distinguishes between horizontal (positional accuracy) and vertical components (attributes accuracy), respectively. These may be difficult to separate (Ehlschlaeger and Shortridge, 1996) but an expression both in terms of the metric accuracy of elevation and in terms of planimetric deviation values is possible (Arrell et al., 2008: 945). Planimetric and metric accuracy have long been evaluated and calculated from routine statistical methods such as mean error (ME), mean absolute error (MAE), error in standard deviation units (S), or as the RMSE, which is the square root of the variance of a distribution (e.g. Carlisle, 2005: 522; Erdogan, 2010; Fisher and Tate, 2006: 470; Wechsler and Kroll, 2006: 1082; see Appendix). Within that class of statistical test, RMSE-based comparison of DEM elevation values with sample values from another, more accurate source seems to be the most appropriate way of providing error estimation (e.g. Arrell et al., 2008: 945; Hirano et al., 2003; Wise, 2007: 1353). However, a major drawback of the RMSE is that results merely provide one global value per DEM grid, i.e. they ignore the finer-scale spatial variability of the error (Wechsler, 2003: 58) that often contains a great deal of useful information for terrain-based applications. Global indices are thus insufficient (Li and Wong, 2010: 252) and any useful study of DEM accuracy should also investigate the spatial variation of error values (Carlisle, 2005: 523; Wood, 1994).

Methods of error investigation in DEMs have been widely explored (Fisher and Tate, 2006; Giles and Franklin, 1996: 1165; Wechsler, 2003: 58; Wilson and Gallant, 2000: 15). The simplest methods are based on criteria such as: (1) differences in elevation between adjacent points (e.g. Binh and Thuy, 2008; Hannah, 1981); (2) systematic detection by inspection of anomalous values within a given moving window (e.g. Chaplot et al., 2006; Felicísimo, 1994; Zhou and Liu, 2004b); (3) the minimization of total error, which is the sum of errors originating from the input data sources and operational errors generated by data manipulation (e.g. Walsh et al., 1987; Younan et al., 2002); or (4) spectral (e.g. Tempfli, 1980) and elevation histogram analyses (e.g. Carrara et al., 1997; Holmes et al., 2000), which each focus on the detection of structure in the statistical distribution of elevation values and on the identification of unexpected variations interpreted as indicating systematic or isolated error. More complex methods introduce remote sensing and/or geostatistical processes. Goodchild (1982) and Polidori (1991: 12–20) have investigated the value of Brownian processes in a fractal terrain simulation model for improving DEM accuracy. Brown and Bara (1994) used semi-variogram and fractal dimension to describe the pattern of systematic errors. López (1997, 2000) developed a method based on principal component analysis (PCA) to locate random errors in DEMs and to extract uncorrelated patterns. After comparisons between Felicisimo’s method (1994) and López’s method (López, 1997), López (2000) revised the PCA approach in order to handle the significant spatial autocorrelation observed in the data. By identifying gross and random errors, systematic errors were avoided. Finally, Arrell et al. (2008) identified and removed artifacts associated with gross error using Fast Fourier Transform analysis as a means of discriminating between signal and noise.

Techniques of error location and visualization have also been considered. Fisher and Tate (2006) have reviewed research in this domain and noted that DEM contour maps could be a useful tool for error observation, with the possibility of detecting gross errors and blunders. The reliability of this method, however, is not proven. The method of Hunter and Goodchild (1995) based on the use of the RMSE as a threshold criterion for testing the accuracy of elevation appears more useful. Other more sophisticated methods have focused on error visualization by using TIN or discrete data and by displaying in 3-D the magnitude of local differences and curvature errors from contoured DEM data (Gousie, 2005). Perhaps a more workable method, proposed by Oksanen (2003: 2412), detects and visualizes gross errors in gridded DEMs from relief shading because of its ability to render a 3-D effect on elevation data. Wise (2007: 1352) has argued that visualization techniques used in error identification could also be efficient at demonstrating strong links between error patterns and interpolation-related distortions. Wood (1996: 64) demonstrated this by developing a spline-fitting algorithm on a fractal surface that allowed errors and other impacts of the algorithmic process to be detected and visually displayed. More recent attention has been paid to DEM interpolation algorithms and their error-causing impacts (1) on elevation in the case of high g.r. data (e.g. LiDAR: Arrell et al., 2008: 944; Bater and Coops, 2009; Lindsay and Evans, 2008; Liu, 2008), (2) on the calculation of mathematical derivatives of elevation (e.g. Florinsky, 1998; Jones, 1998; Liu and Bian, 2008; Schmidt et al., 2003; Zhou and Liu, 2004a), but also (3) on flow simulation and its repercussions on drainage network extraction (e.g. Li and Wong, 2010; Lindsay and Evans, 2008).

In terrain analysis, error propagation analyses have usually focused on slope and aspect as test variables. These mathematical derivatives of elevation are recognized to provide a more useful test of error location and propagation, and thus for DEM quality assessment, than elevation itself because of their greater sensitivity to error (Wechsler, 2003; Wise, 2000: 1911). Liu and Jezek (1999: 250) have listed three approaches for treating the propagation of DEM error. These include empirical approaches (e.g. Bolstad and Stowe, 1994; Gatziolis and Fried, 2004; Lindsay, 2006), analytical methods (e.g. Florinsky, 1998; Heuvelink et al., 1989; Leick, 1990; Oksanen and Sarjakoski, 2005) and computer simulations (e.g. Canters et al., 2002; Ehlschlaeger, 1998; Hebeler and Purves, 2009; Oksanen and Sarjakoski, 2005; Wechsler and Kroll, 2006). (1) The empirical approaches rely on a derivation of the spatial properties of error based on a more accurate reference data set, where results for a series of simulations are presented with error fields characterized by a range of spatial properties (Lindsay and Evans, 2008: 1590). This compensates for the often unknown spatial characteristics of error. (2) Analytical methods can involve Taylor series expansion (Heuvelink et al., 1989). Under the assumption that error distribution follows a Gaussian distribution, this approach constitutes the basis for least-squares adjustment techniques (Hunter and Goodchild, 1997: 36). (3) The simulation methods involve stochastic approaches such as Monte Carlo techniques. This requires a number of equally probable outcomes upon which selected statistics are performed. Uncertainty is quantified by evaluating the statistics associated with the range of outputs under the assumption of spatial autocorrelation. A series of random equiprobable error maps, one of which could be the ‘true’ error map, is provided. Such simulations, however, are non-unique and do not ensure that the true error map has been generated by the procedure (Wechsler, 2006: 2360; Wechsler and Kroll, 2006).

The analytical treatment of terrain algorithms is complicated and laborious (Lindsay and Evans, 2008: 1590). Oksanen and Sarjakoski (2005) confronted an analytical approach with Monte Carlo simulations to evaluate the propagation of error in slope and aspect simultaneously. They concluded that the two methods produced similar results even though the data had not been inspected for spatial autocorrelation in the analytical approach. The work of Oksanen and Sarjakoski (2005: 1016) and Liu and Bian (2008: 308) has revived a debate about the value of performing analytical manipulations of gridded DEMs that disregard the spatial autocorrelation of error — the most important consideration being the choice of autocorrelation parameter rather than the shape of the spatial autocorrelation. These aspects are presently examined.

III Spatial autocorrelation and the production of error maps

Spatial autocorrelation characterizes the correlation between a variable and itself (univariate autocorrelation), or between that same variable and another variable’s values (bivariate autocorrelation), in neighbouring cells. Spatial autocorrelation functions are used to quantify the spatial regularity of a phenomenon (a form of spatial complexity) (Aubry and Piégay, 2001). It is based on the assumption that in all data sets near things are more related than distant things (Tobler’s first law of geography). Some approaches avoid the condition of heteroscedasticity, i.e. the necessity to consider variations in error value as a function of variations in another variable. This occurs in the case of spatial autoregressive models (e.g. Hunter and Goodchild, 1997) and of semi-variance analysis (e.g. Carlisle, 2005; Giles and Franklin, 1996). Other approaches, such as pixel swapping and spatial moving averages using low-pass filters (e.g. Fisher, 1998; Wechsler, 2006), introduce spatial correlation without explicitly addressing error patterns or mapping error surfaces (Hebeler and Purves, 2009: 6). Methods such as sequential Gaussian simulation (e.g. Oksanen and Sarjakoski, 2005) use global spatial autocorrelation information without examining the finer structure of the error. Given that the study of spatial autocorrelation and of its potential consequences in gridded DEMs has been unsystematic, here we investigate the definitions of spatial autocorrelation, its uses, how to deal with it, and how error maps displaying local autocorrelation clusters can reveal error propagation phenomena.

Moran’s I (Moran, 1950) is historically the first index to have been used for understanding spatial autocorrelation, and still a standard. In the context of gridded DEMs, Moran’s I is a global indicator of spatial association and is used to measure average deviations in slope or elevation for an entire grid. This global measure is given by the following formula:

I = \frac{N}{\sum_{i} \sum_{j} w_{i j}} \times \frac{\sum_{i} \sum_{j} w_{i j} z_{i} z_{j}}{\sum_{i} z_{i}^{2}}

When there are N units, z_i (

z_{i} = x_{i} - \bar{x}

) is the variable value at particular location, z_j (

z_{j} = x_{j} - \bar{x}

) is the variable value at another location, is the mean of the variable, and w_ij expresses the level of interaction (connectivity) between units i and j. This contiguity matrix compares the sum of the cross-products of values at different locations, two at a time, weighted by the inverse of the distance between the locations (if i is next to j, the interaction receives a weight of 1). Moran’s I ranges between +1 (strongly positive) and –1 (strongly negative spatial autocorrelation). A value of 0 indicates a random spatial pattern.

From this index Anselin (1995) later elaborated the LISA, which can also be used on continuous DEM grid surfaces. Here we use Moran’s I and LISA as techniques for visualizing error propagation, and for adding information about spatial autocorrelation not provided by Moran’s I. LISA is used for estimating the local contribution of each pixel value to the global index. Local indicators are based on a gamma statistic (Getis, 1991) given by the following formula (Oliveau, 2005):

Γ = \sum_{i} \sum_{j} w_{i j} ϑ_{i j}

where ϑ_{i j} = \frac{(z_{i} - \bar{z}) (z_{j} - \bar{z})}{\sum_{i} {(z_{i} - \bar{z})}^{2}} \times \frac{n}{m} and Γ_{i} = \sum_{j} w_{i j} ϑ_{i j}

In these formulae, variables are as in equation (1), n is the total number of subjects in the cross section (

\sum_{i}

), m is the neighbourhood pair number (

\sum_{i}

\sum_{j}

w_ij), and

ϑ_{i j}

is a measure of relationship in attribute space between data sites i and j. LISA measurements take into account spatial and numerical similarities between contiguity matrices. The LISA of each observation gives an indication of the extent of statistically significant spatial clustering of similar values around the observation point. The LISA is obtained by local spatial clustering procedures using the following equation (Anselin, 1995):

I_{i} = z_{i} \sum_{j} w_{i j} z_{j}

with the sum of local indicators:

\sum_{i} I_{i} = \sum_{i} z_{i} \sum_{j} w_{i j} z_{j}

where z_i and z_j are expressed in deviations from the mean, the summation of is such that only neighbouring values

j \in J_{i}

are included, and is the contiguity matrix of weights. The sum of local indices is proportional to Moran’s I from

γ = \sum_{i} \sum_{j} w_{i j} m_{2},

with

m {}_{2}= \sum_{i} z_{i}^{2} / n

. LISA results can be standardized by dividing the LISA by m ₂. The LISA helps to identify local outliers (where negative local spatial autocorrelation shows up on the cluster map as high–low or low–high occurrences, meaning that the variable of interest exhibits dissimilar values among its neighbours); and local clusters (where positive local spatial autocorrelation shows up on the cluster map as high–high or low–low occurrences, meaning that the variable of interest shows values that are similar to its neighbours). Local deviations from the global pattern of spatial autocorrelation can also be delineated.

Bivariate LISA-type cluster analysis explores the spatial distribution of DEM errors affecting elevation and measures its transmission to errors affecting slope gradient. It follows a scheme of pattern recognition (Jacquez, 2008: 400) made up of five components: (1) a null hypothesis, H₀ , which describes the spatial pattern expected when the alternative hypothesis is false (i.e. an absence of significant clustering of errors in slope at sites where errors in elevation occur); (2) an alternative hypothesis, H_A , which states that the clustering of errors in slope closely matches the spatial distribution of errors in elevation; (3) a statistical significance test that quantifies the spatial pattern of the data; (4) a neutral spatial model, which is a numerical device for generating the reference distribution based on a Monte Carlo simulation (achieves randomly operated permutations of values); (5) a reference distribution, which is the distribution of the statistical test scores when the null hypothesis is true, and is set by the p-values, which in statistics measure strength of evidence. For the LISA tests, pseudo p-values are computed as (m+1) / (r+1), where r is the number of replications and m is the number of instances where a statistic computed from the permutations is either equal to or greater than the observed value (in the case of positive LISA scores), or equal to or smaller than the observed value (for negative LISA scores) (Anselin, 1995). Results are filtered with the choice of an arbitrary significance threshold, α (0.05, 0.01, 0.001, or 0.0001), which determines confidence intervals used for assessing results.

IV Patterns of error in gridded DEMs: a worked example from the Wasatch Mountains, Utah, USA

Among the few existing studies to have explored the local structure of spatial autocorrelation in raster analysis, Goovaerts et al. (2005) detected anomalies on high-resolution hyperspectral imagery while testing the significance of LISA values through randomization. Erdogan (2010) investigated spatial variation as well as spatial associations of error by comparing a range of interpolation methods. Monckton (1994) used Moran’s I to simulate the spatial structure of error in elevation data. Results, however, were inconclusive, probably because of the spatial lags (which exceeded 250 m) that were used in dealing with spot heights on Ordnance Survey maps (Carlisle, 2005: 524).

In the following worked example designed to illustrate the power of autocorrelation analysis as a tool in DEM error evaluation, we performed tests on a ~29 km² study area encompassing part of the Weber Segment, one of the longest (61 km) and seismically most active segments of the Wasatch normal fault in Utah (Figure 1). On this segment, faceted spurs and their triangular mountain-front hillslopes are well preserved (McCalpin and Berry, 1996; Nelson and Personius, 1993; Petit et al., 2009; Zuchiewicz and McCalpin, 2000). The geomorphology of fault-generated range fronts is reflected in the drainage basin morphology as well as in the faceted spur geometry. In order to extract precise terrain parameters with minimal bias, DEM evaluation was examined through slope analysis. Using both LISA and Moran’s I, the methodology aims to single out the DEM best suited to geomorphometric or hydrological applications. The best DEM should provide the model with the smallest error, i.e. with the highest accuracy and the lowest level of error clustering.

The grid characteristics of the six DEMs available for the study area (see Introduction), with differing column and row grid-cell sizes, are as follows (Figure 2) – LiDAR (2 m): 2979 × 2370; CODEM (5 m): 1260 × 857; NED10 (10 m): 630 × 474; AstD (15 m): 420 × 316; GDEM (30 m): 210 × 857; SRTM (90 m): 70 × 53. General statistics are usually available for each DEM from supporting documentation, in which major errors and correction processes are mentioned. Out of the six in this study, only the AstD was generated directly from raw data by the authors using digital stereo-photogrammetric techniques summarized in Figure 3. The operational error we use is the difference between the z values provided by each of the five tested DEM grids on the one hand, and the z values provided by the LiDAR data on the other, the latter being closest to any ‘true’ elevation measurement. Any observed difference in z is interpreted as being due to one or several of the error categories defined in the Appendix. Those refer to the various defects originally present in the gridded elevation data. Their distribution pattern across the grid remains unknown until we proceed to the mapping techniques proposed later in this worked example. They depend on the type of sensor from which the data was collected, and how this data was preprocessed before making it available for use by geographers.

1 Vertical accuracy

The first approach to error estimation relied on the RMSE, which integrates errors relating to interpolation methods and to geographical position in x, y and z. Vertical accuracy assessment at pixel scale was first calculated from the LiDAR model (RMSE ≈ 0.086 m) with a regular sample extraction interval of 100 m, each point extracted (3000 for each DEM) being a substitute for ground GPS information (see Appendix for details). Given that, for comparison purposes, the grids could not be used with their original characteristics, DEM resampling was also necessary. Accordingly, for the RMSE calculation resampling procedures were performed on each DEM by transforming cell size to the reference size, i.e. 2 m. To avoid stair-stepping and jagged artifacts in the resampling process, a bicubic spline resampling method was used instead (see Appendix) (Keys, 1981). The resampled grids were used for a comparative assessment of vertical accuracy between DEMs. In order to obtain percent values of transformation from the resampled DEM, a new RMSE value, noted RMSEc, was calculated. The formulae given in the Appendix were applied to the entire DEM grid and assessment of the vertical accuracy of surface objects was carried out using the RMSEc criterion.

Figure 4 and Table 1 illustrate the distribution of error values in elevation among the six DEM grids. Quartile distributions (first and third) describe a band of ±8 m for minimum and ±22 m for maximum deviations from the mean elevation (Table 2). Overall, DEM height distributions exhibit a similar pattern despite the fact that the elevation sources are different (i.e. airborne, radar, optical or compilation data). The more outstanding exception is the AstD, which presents significant offsets with respect to the other DEMs. These offsets attain a difference in values of about ±15 m (roughly equivalent to one AstD pixel) for height and RMSE measurements as well as for statistical moments. This exception is ascribable to the initial AstD extraction process, in which the precision of the DEM tools used was greater than half a pixel (see Figure 3). The initial RMSE for the AstD (11.8 m) was accordingly too large. The global RMSE should be about half a pixel (7.5 m). Additional ground control points were checked in order to newly interpolate the AstD, finally providing it with elevation ranges similar to the other DEMs, i.e. [1355–1298 m]. As a result, the new value obtained for the AstD RMSEc (Table 2) fell into line with the overall RMSEc trend for the other DEMs, i.e. about 12.9 m. Thus RMSEc results (incorporating new values given in brackets for the AstD) yield results similar to those obtained for the other DEMs: the larger the grid cell size, the higher the RMSEc [+5.7; +18.4] and RMSE [+4; +16 m]. The key conclusion at this stage is that whatever the DEM’s data source, DEM generation does not produce large errors in z. Values for the SRTM and GDEM also fall close to the reference LiDAR value and thus perform almost as well as the most accurate reference grids despite their comparatively lower initial resolution.

Table 1.

Distributions of elevation in the six DEMs. Min and Max are the minimum and maximum values; 1st and 3rd Qu. are the first and third quartile. For the ASTER DEM, corrected values (see text) are in brackets. For other DEMs, tabulated values are initial values.

	Elevation value (m)
Data source	SRTM	GDEM	ASTER	NED10	CODEM	LiDAR
Min	1355	1355	1340 (1355)	1357	1360	1351
1st Qu.	1775	1769	1754 (1769)	1769	1776	1771
Median	2076	2073	2057 (2072)	2070	2078	2071
Mean	2073	2069	2051 (2065)	2067	2073	2069
3rd Qu.	2366	2358	2338 (2352)	2358	2363	2359
Max	2897	2884	2871 (2886)	2892	2900	2900

Table 2.

RMSE results for the six DEMs. STD is the standard deviation and ME is the mean error for each DEM grid. RMSE is the initial RMSE with respect to the reference geoids used for the production of each DEM. RMSEc is the new RMSE calculated with respect to the LiDAR reference grid.

	Error values (m)
DEM	Grid cell size	STD	ME	RMSE	RMSEc	RMSEc–RMSE
LiDAR (reference)	2	387.3	Ø	+0.86	Ø	Ø
CODEM	5	408.3	4	+4	+5.6	+1.6
NED10	10	406.5	1.9	≈7	+6.7	+0.3
ASTER DEM	15	403.9	21.8 (3.1)	+11.8 (+9.4)	+25.9 (+12.9)	+14.1 (+3.5)
GDEM	30	405.1	0.5	+9.35	+13.6	+4.25
SRTM	90	407.0	3.7	[+9;+16]	+18.4	[+2.4;+9.4]

As a further test, linear regression analysis was carried out between the elevation data of each DEM and the LiDAR reference grid, respectively (Figure 5). Overall, the DEMs produced a goodness-of-fit, measured by the determination coefficient r² , close to +1, with intervals ranging between +0.9979 for the worst fit and +0.9999 for the best. The CODEM and NED10 show the strongest correlation with the LiDAR, followed by the AstD and the GDEM. Model fits reveal outliers at different locations in the AstD, GDEM and SRTM scatter plots. The outliers reflect some of the error present within the DEM, which appears largest in the case of the AstD. However, neither data scatter in the linear regression models nor vertical accuracy constitute sufficient criteria to achieve a full definition of DEM error because the error is expressed globally (lacks any indication of spatial distribution), and because when untested spatial autocorrelation occurs in data, linear regression models tend to overestimate both strength of correlation and precision.

2 Residual maps and computation of primary topographic attributes

One approach to locating error distribution patterns visually can be achieved through the creation of so-called deterministic error surfaces. These can be generated for a DEM by using reference data assumed to possess higher accuracy than the DEM (Hebeler and Purves, 2009), in this case the LiDAR, but equally ground control points or other surface model data (Holmes et al., 2000). In this way, maps of differences between the reference grid and other grids, termed here residual maps of either elevation or slope gradient, can provide an understanding of error sources and form a basis for further analysis (Van Niel et al., 2004). Residual maps were generated by applying an algebraic operation of pixel by pixel subtraction between each DEM and the LiDAR. In order for these tests to be meaningful, degradation of the LiDAR to the relevant ground resolutions, i.e. from 5 m to 90 m, was necessary. This operation resulted in a series of numerical images representing Re, the pixel-wise residual value (Figure 6).

Figure 6 shows that (1) the range of Re standard deviations is similar (0–11 m) for the three most accurate DEMs, namely CODEM, NED10 and AstD. (2) The SRTM and GDEM show the greatest deviations (up to 41 m) from the reference surface, which is in keeping with the lower ground resolution of these two data sets. Furthermore, the bicubic spline resampling procedure resulted in artificial smoothing of the surfaces, an aspect clearly apparent in the SRTM residual map (Figure 6E). (3) High values of standard deviation tend to cluster around the same sites in the landscape whatever the grid-cell size, with detectable variations in the sharpness of the discontinuities. (4) The highest values, i.e. greater than 9 m, follow narrow valley floors in the mountainous relief. Value distributions differ in the piedmont, where they are located along wider valleys. (5) High deviations mostly affect slopes and valley bottoms. Where hillslopes are affected by anomalous deviations, errors are likely to propagate to higher order topographic attributes (e.g. slope curvature), and hence adversely affect predictions made from the DEM for applications such as hydrology or natural hazards. (6) Flat areas, which predominate in the southern part of the piedmont zone, display values that are close to the minimal deviation, i.e. 0 m. This is most visible with the higher grid-cell sizes in Figure 6, B, C and D. Flat areas, which are often problematic in hydrological network extraction (e.g. Tarboton, 1997), thus seem to be less affected by high deviations. However, this finding does not exclude the possibility that erratic features, such as pits and holes, could be present in this kind of area. Figure 6E (SRTM) shows that regions affected by large error are larger, form continuous patches, and are concentrated in valleys and on elevated ridges whereas they are absent or more discontinuous in the other DEMs.

The following step in the procedure involved the computation of slope gradient values in order to assess error patterns in slope gradient. Slope maps depend on the choice of algorithm and on grid-cell size. We used the Zevenbergen and Thorne (1987) algorithm due to existing consensus over its relative robustness and consistent behaviour in various terrain geometries (e.g. Florinsky, 1998; Jones, 1998; Liu and Bian, 2008; Zhou and Liu, 2004b). Calculations were carried out on each of the six grids (Figure 7). Slope values range from 0° to 89.9° (Table 3). Lower values (0–1°) indicate flat interfluves or valley bottoms, whereas the highest slope angles represent unusual terrain values, typically greater than 35° and located on faceted spurs. Standard deviation and mean slope decrease with grid-cell size, in line with previous research on this topic (e.g. Kienzle, 2004; Warren et al., 2004) but unlike results obtained for DEMs extracted from contour lines (e.g. Ziadat, 2007).

Table 3.

Slope values for the six DEMs. Slope-angle range is the interval between minimum and maximum slope; mean slope angle is the arithmetic mean of the slope value distribution; Std slope is the standard deviation of that distribution.

DEM	Slope-angle range (degrees)	Mean slope angle (degrees)	Std (degrees)
LiDAR	0–89.9	26.33	1.52
CODEM	0–75.8	26.08	1.37
NED10	0–89.7	26.26	1.34
ASTER	0–89.6	23.43	1.29
GDEM	0–89.4	23.14	1.30
SRTM	0.3–42.2	22.37	1.06

The distribution of slope values is representative of the general slope trend for all the DEMs (Figure 8). Values of 0–30° represent 98% of the slope frequency range in the SRTM, whereas other DEMs exhibit a greater proportion of extreme values (for example, slope angles greater than 40° account for 10% of the slope frequency distribution). Field data have revealed that the Weber Segment mountain front is characterized by slope angles never exceeding 43° (Eblen, 1995; Stock et al., 2009), with mean values close to 25–26°. The highest values, between 45° and 90°, are thus anomalous and represent errors due to DEM uncertainty, which were propagated to the slope map during transformation of the elevation data. The geographical location of large errors in slope coincides for each DEM with that of the largest errors in elevation (Figure 7). For angles up to 45°, slope values also increase with cell size. However, anomalous values are fewer for the larger grid-cell sizes than for the smaller ones (CODEM, NED10, AstD). This also means that the larger the grid-cell size, the greater proportion of values <45° and the fewer outliers present.

Although results are so far directly ascribable to misfits originally affecting z, results differ between the mean and lower slope values. The lower 50% of the distribution frequency is marked by several offsets between the highest and smallest grid-cell sizes (Figure 8): for example, contrary to the expectation that error is consistently proportional to grid-cell size, between 0° and 25° the slope distribution values of the CODEM increase faster than for the NED10. Also, in the low range (i.e. 0° to less than 15°) the AstD curve locally crosses the SRTM and GDEM curves. Consequently, lower and mid-range slope values appear to reflect isolated discrepancies in the DEM elevation structure. The standard deviation of residuals (difference between reference and calculated slope) was also calculated (Figure 9), illustrating for each DEM the deviation in mean distribution values for each grid. In the six DEMs, the thalwegs of larger valleys are well marked. Note also that the locations of slope errors are almost identical to the locations of the errors in elevation except on the main central facets of the Weber Segment (Figure 9). The higher the terrain resolution, the more sensitive the gridded data to errors in slope, which are thus also linked to terrain roughness characteristics. Accordingly, slope deviations range between 0° and 14° for the larger grid-cell sizes (SRTM, GDEM) and between 0° and 25° for the smaller (LiDAR, CODEM, NED10, AstD; Figure 9).

The strength of the relationship between errors in slope and errors in elevation (Figures 6 and 9) was tested by performing a linear regression between elevation (Re on elevation) and slope residuals (Re on slope). A non-linear relationship between those two variables was found (Figure 10). However, the form and intensity of the relationship is biased by several outliers that stretch the scatter plot in several directions. Determination coefficients with values <0.3 for the SRTM and CODEM and <0.1 for the AstD, GDEM and NED10, indicate weak to no dependency between the errors in slope and in elevation. In other words, only 10–30% of anomalous slope values (in particular among the lower and mid-range values) can be explained by anomalous elevation values. Such a conclusion, however, is potentially spurious because r² is not a robust criterion for explaining either the numerical or the spatial behaviour of errors affecting slope. Accordingly, as a further tool for evaluating whether higher error affecting elevation was positively correlated with higher error affecting slope, we inspected the data grids for spatial autocorrelation.

3 Spatial autocorrelation analysis

Calculation and 2.5-D mapping of Moran’s I and LISA were used to identify areas with a high risk of error caused by pixels being surrounded by similar areas or by spatial outliers, and as tools for visualizing the location of uncertainty. A spatial weighting matrix was used for each variable, and was defined at varying distance lags corresponding to the respective DEM resolutions. The weighting process was carried out using Rook’s case–contiguity kernel, with symmetrical spatial weights wij = 1 when locations i and j are adjacent, and wij = 0 in other cases. Rook’s case considers the neighbourhood of four locations adjacent to each cell (Elobaid et al., 2009). A Monte Carlo randomization procedure allowed us to assess significance levels by performing 999 permutations (i.e. tests) for each grid (Table 4).

Table 4.

Autocorrelation results for errors on elevation and on slope. Moran’s I elevation and Moran’s I slope are the univariate Moran’s I calculations for the dependent (elevation) and independent (slope) variables, respectively. Bivariate Moran’s I refers to the errors in elevation and in slope and is the result of the multivariate LISA test.

DEM	r ²	Moran’s I elevation error	Moran’s I slope error	Bivariate Moran’s I
CODEM	0.24392	–0.0573	0.9149	0.1403
NED10	0.07332	0.7523	0.9226	0.2772
ASTER	0.01236	0.6729	0.9218	0.1125
GDEM	0.09374	0.7702	0.9261	0.4870
SRTM	0.20950	0.8869	0.9149	0.3048

The randomization test avoids relying on assumptions about data distribution and developing statistical moments under a null hypothesis, and avoids testing all types of statistic analytically (Aubry and Piégay, 2001). Significance levels are given by the p-values (arbitrary threshold), are calculated for each cluster, and allow significant autocorrelation values to be identified. After trial comparisons with other thresholds (0.0001, 0.001, and 0.01) for each DEM, p-value significance levels were set to 0.05 because this configuration yielded the highest and most sharply differentiated levels of data clustering. Table 4 summarizes global clustering levels for bivariate and univariate variables, each time for a run of 999 permutations.

The CODEM grid, with a Moran’s univariate I of about –0.06, illustrates a situation of low negative spatial autocorrelation, implying that neighbours are dissimilar and that error values affecting elevation do not have a unique source or cause. In contrast, error values in the other DEMs obtain a Moran’s index close to +0.9, indicating a strong spatial link between errors in elevation. Error in slope systematically attains values close to +1, with a high level of spatial dependence showing strong positive spatial autocorrelation values. Where high spatial autocorrelation scores were observed for elevation, values likewise increased for slope, thus confirming a mechanism that drives the disproportionate (i.e. non-linear) propagation of errors in elevation to errors in slope gradient. The bivariate Moran’s index, which describes the magnitude and relationship of global error between elevation and slope, is characterized by positive values but none of these are close to +1. The strongest positive relations are observed for the GDEM (+0.4870) and SRTM (+0.3048) grids, whereas the lowest occur in the AstD (+0.1125). Results are thus similar to the univariate values except for the AstD. The SRTM, GDEM and NED10 present a high level of clustering, with the CODEM ranking last.

The LISA helps to bring out local magnitudes of error clustering. Each observation provides a measure of the extent of statistically significant spatial clustering of similar error values around a data point. LISA results reveal a high level of differentiation within each DEM (Figure 11). High–high and low–low clusters define a high error value surrounded by high values and a low value surrounded by low values, respectively. Such situations are present in the SRTM, GDEM and to a lesser extent the NED10 grids. A higher risk of error propagation occurs where high–high values are found. Relative stability, i.e. a low risk of error propagation to higher order topographic derivatives, prevails in low–low situations. High–high clusters are found in the same locations as in the univariate elevation and slope error maps. In those cases, therefore, the LISA maps merely corroborate previous results. Uncertainty in slope is thus highly correlated with initial elevation, and thus with error in elevation, which in turn conditions the spatial distribution of errors.

For grids other than the SRTM, GDEM and NED10, LISA maps show somewhat different results. AstD maps show all the peak occurrences already found in the SRTM and GDEM grids, but low–high and high–low clusters are more prominent between high–high clusters. High–high levels are typical of major thalwegs and low-lying areas. In the case of the NED10, error distribution patterns bring out geomorphological features (particularly ridges and escarpments) more sharply than the SRTM and the GDEM. This arises whatever the level of p-value significance (tested from 0.0001 to 0.05). The CODEM stands as the exception: uncertainty already detected in previous maps (Figures 7 and 9) is well represented in the CODEM LISA map but fails to show the high or low clustering levels encountered in the other DEM grids. Clustering is less intense (note the predominance of half-tones), illustrating uncertainty in the data, i.e. uncertainty not linked to terrain characteristics and presumably caused by other unknown factors which would remain to be investigated.

V Summary and conclusion

The aims of this article were: (1) to review suitable tools for assessing error in remotely sensed data and for calibrating DEM studies that focus on primary topographic attributes; (2) to investigate a statistical tool more often used in ecology and human geography than in digital terrain analysis, namely LISA clustering, which helps to produce and evaluate error maps in order to track DEM error propagation; and (3) to assist in decision-making when selecting among a range of DEM products that are readily available for a chosen study area. In a worked example involving terrain analysis in the Wasatch Mountains of Utah, we were able to use LiDAR data as a surrogate for field control points. We tested five other, freely available, DEM grids against this very high-resolution reference grid.

Despite the existing catalogue of methods for removing various categories of error in gridded data, the overriding conclusion at present is that errors can never be totally erased from DEMs. Empirical approaches work best when higher-quality reference data are available for comparison and allow a range of statistical ranking procedures to be carried out. Initially, basic quantitative tools such as the RMSEc were of little use in determining DEM accuracy. It was shown that the larger the grid-cell size, the lower the accuracy, with nevertheless a fair degree of reliability in z whatever the sensor type and whatever the original data ingredients entered into the DEM. The CODEM, NED10, AstD, GDEM and SRTM could therefore be used almost indiscriminately for basic displays of topography, depending simply on choice of resolution for graphic output.

As an improvement on RMSE procedures we generated a series of residual elevation and residual slope maps (Re), which provided an outline of the spatial distribution of errors based on calculations of the standard deviation of Re. Two main conclusions arose from these findings: (1) errors in slope and in elevation are linked, but with a variable strength of dependency as shown by linear regression results; and (2) the SRTM, GDEM and NED10 stand out for their lower error scores. Irrespective of their ground resolutions, the CODEM and the AstD presented either more extreme error values or more confused results. This finding argues against preferring these data sources for morphometric or hydrological applications. Re results emulated a further approach in error analysis using Moran’s I and LISA maps. This approach has revealed that the intensity of spatial autocorrelation is not linked to grid-cell size. It is more closely related to the intrinsic characteristics of DEM products and to procedures dealing with slope gradient calculation. The LISA maps also help to define error hot- and coldspots, i.e. high–high and low–low clusters. Among the five DEMs analysed in this study, the autocorrelation tests finally allowed the NED10 to stand out as the DEM least affected by the presence of error and its propagation. Like the CODEM, the NED10 is a hybrid DEM compiled from a range of digital and non-digital sources rather than from unique radar or optical sources, and also happens to be of a lower native resolution.

The trends and topics reviewed here have emphasized the value of exploring gridded data structures for detecting spatial error patterns because results can be used for ranking DEM quality. Such procedures seem critical for most practical terrain applications in natural catchments, but also for using the DEM grids as input substitutes for ‘real topography’, i.e. as initial conditions for running predictive surface process models (e.g. Codilean et al., 2006) and observing how the landscape evolves under given conditions (heuristic terrain modelling). Despite its advantages, the LISA still remains a semi-quantitative tool (values express variations in clustering as ‘high’ or ‘low’). A move towards more fully quantitative characterizations of error in DEMs, where, in addition to its location, the intensity of error is provided (deviation of elevation in metres, and/or slope in degrees), could perhaps be achieved by coupling the LISA with other error modelling methods such as geostatistical simulations based on kriging, or geographically weighted regression (Fotheringham et al., 2002).

Footnotes

Acknowledgements

We are grateful to Editor-in-Chief Nicholas Clifford for steering our manuscript through a high-quality reviewing process, with fine reports from several anonymous referees who greatly improved the scope and structure of this contribution.

Appendix

(1) Definitions of error

(a) Artifacts, blunders and gross errors

Reuter et al. (2009) defined artifacts as erratic features that appear on a DEM as holes, ghost lines, stripes and so on. Artifacts and associated errors could even have been invisible in the input DEM but can be inferred from unrealistic values occurring on elevation derivative maps. Blunders are vertical errors associated with data collection. They can be mistakes caused by misreading contours, by transposing numerical values, by erroneous correlations, or by careless observations (Wechsler, 2003). Gross errors occur when a measurement process is subject to occasionally large inaccuracies and generates data outliers. It includes topographical misinterpretations, and long linear features along some boundaries of the calculation window used in DEM interpolation (Oksanen, 2003). Some techniques such as ‘data snooping’ are efficient at detecting them (Karras and Petsa, 1993), and inspection of a shaded relief map can also help (Gousie, 2005).

(b) Systematic errors

Sytematic errors result from deficiencies in measurement or processing and affect the accuracy of the final data, i.e. the degree to which the final data agree with reality. Systematic errors are also identified as residual systematic error surfaces caused by slightly inaccurately estimated lens distortion parameters (Wise, 2000), by translation effects, or false features such as phantom tops, ridges, benches or striations (Raaflaub and Collins, 2006). In general, they reflect bias inherent in the data collection method (Reuter et al., 2009).

(c) Random errors

Random errors result from mistakes such as inaccurate surveying or improper recording of elevation information. Often associated with signal noise, random errors are extracted after known blunders and systematic errors have been removed (Podobnikar, 2009; Reuter et al., 2009; Wechsler, 1999, 2006; Zhang and Cen, 2008).

(d) Uncertainty

Uncertainty is a term employed when a modelled value deviates from its true value, with a confidence interval (Hebeler and Purves, 2009). Lack of knowledge about measurement reliability and about its representation of true values is referred to as its uncertainty and is a measure of what we do not know (Wechsler, 1999, 2006). DEM uncertainty thus covers lack of knowledge about parameters, model assumptions, and scenarios, i.e. spatial evolution models (Liu and Bian, 2008).

(2) Equations and formulas

(a) Root mean square error and mean error

R M S E (z) = \sqrt{\frac{1}{n} \sum_{i = 1}^{n} {(z_{d e m} - z_{r e f})}^{2}}

M E = \frac{1}{n} \sum_{i = 1}^{n} (z_{d e m} - z_{r e f})

where z_dem is DEM elevation, z_ref is reference elevation, and n is the number of test points.

(b) Bicubic spline formula

p (x, y) = \sum_{i = 0}^{3} \sum_{j = 0}^{3} a_{i j} x^{i} y^{i}

where p is the interpolated surface, x and y are rectangular coordinates of point (i,j), and a is the polynomial coefficient. This interpolation maintains the continuity of the function and its first derivatives (both normal and tangential) across cell boundaries.

(c) Residual maps and standard deviation algorithms Pixel by pixel algebraic subtraction of elevation values provides a series of residual values (Re) that can be mapped:

{Re}_{(i, j)} = z_{(i, j) d e m} - z_{(i, j) r e f}

where i and j are lines and columns of the grid, and and are the elevations of the DEM and the reference grid, respectively. The standard deviation of Re is calculated as follows:

S t d = \sqrt{\frac{\sum_{i = 1}^{n} {[z_{d e m} - z_{r e f} - M E]}^{2}}{n - 1}}

where ME is the mean error of the DEM. Standard deviations were calculated each time for the entire image, cell by cell, yielding a map of Re values in which deviations in elevation values from those of the reference model could be located.

References

Anselin

(1995) Local indicators of spatial association – LISA. Geographical Analysis 27: 93–115.

Arrell

Wise

Wood

Donoghue

(2008) Spectral filtering as a method of visualizing and removing striped artifacts in digital elevation data. Earth Surface Processes and Landforms 33: 943–961.

Aubry

Piégay

(2001) Pratique de l’analyse de l’autocorrélation spatiale en géomorphologie: Définitions opératoires et test. Géographie Physique et Quaternaire 55: 111–126.

Bater

Coops

(2009) Evaluating error associated with lidar-derived DEM interpolation. Computers and Geosciences 35: 289–300.

Binh

Thuy

(2008) Assessment of the influence of interpolation techniques on the accuracy of digital elevation model. VNU Journal of Science, Earth Sciences 24: 176–183.

Bolstad

Stowe

(1994) An evaluation of DEM accuracy: Elevation, slope and aspect. Photogrammetric Engineering and Remote Sensing 60: 1327–1332.

Bolstad

Stowe

(1994) An evaluation of DEM accuracy: Elevation, slope and aspect. Photogrammetric Engineering and Remote Sensing 60: 1327–1332.

Brown

Bara

(1994) Recognition and reduction of systematic error in elevation and derivative surfaces from 7.5-minute DEMs. Photogrammetric Engineering and Remote Sensing 60: 189–194.

Burrough

(1986) Principles of Geographical Information Systems for Land Resources Assessment. New York: Oxford

University

Press.

10.

Canters

Genst

Dufourmont

(2002) Assessing effects of input uncertainty in structural landscape classification. International Journal of Geographical Information Science 16: 129–149.

11.

Carlisle

(2005) Modelling the spatial distribution of DEM Error. Transactions in GIS 9: 521–540.

12.

Carrara

Bitelli

Carla

(1997) Comparison of techniques for generating digital terrain models from contour lines. International Journal of Geographical Information Science 11: 451–472.

13.

Chaplot

Darboux

Bourennane

Leguédois

Silvera

Phachomphon

(2006) Accuracy of interpolation techniques for the derivation of digital elevation models in relation to landform types and data density. Geomorphology 77: 126–141.

14.

Claessens

Heuvelink

GBM

Schoorl

Veldkamp

(2005) DEM resolution effects on shallow landslide hazard and soil redistribution modeling. Earth Surface Processes and Landforms 30: 461–477.

15.

Codilean

Bishop

Hoey

(2006) Surface process models and the links between tectonics and topography. Progress in Physical Geography 30: 307–333.

16.

Darnell

Tate

Brunsdon

(2008) Improving user assessment of error implications in digital elevation models. Computers, Environment and Urban Systems 32: 268–277.

17.

Desmet

PJJ

Govers

(1995) GIS-based simulation of erosion and deposition patterns in an agricultural landscape: A comparison of model results with soil map information. Catena 25: 389–401.

18.

Dietrich

Bellugi

Sklar

Stock

Heimsath

Roering

(2003) Geomorphic transport laws for predicting landscape form and dynamics. In: Wilcock

Iverson

(eds) Prediction in Geomorphology, AGU Geophysical Monograph Series 135. Washington, DC: American Geophysical Union, 103–132.

19.

Dietrich

Wilson

Montgomery

McKean

(1993) Analysis of erosion thresholds, channel networks, and landscape morphology using a digital terrain model. Journal of Geology 101: 259–278.

20.

Eblen

(1995) A probabilistic investigation of slope stability in the Wasatch Range, Davis County, Utah. Utah Geological Survey Contract Report 95-3. Utah: Department of Geological Survey – Department of Natural Resources.

21.

Ehlschlaeger

(1998) The stochastic simulation approach: Tools for representing spatial application uncertainty. Unpublished doctoral thesis, University of California, Santa Barbara.

22.

Ehlschlaeger

Shortridge

(1996) Modeling elevation uncertainty in geographical analyses. Proceeding of the International Symposium on Spatial Data Handling, Delft, Netherlands, 9B.15–9B.25. Available at: http://chuck.ehlschlaeger.info/older/SDH96/paper.html.

23.

Elobaid

Shitan

Ibrahim

Ghani

ANA

Daud

(2009) Evolution of spatial correlation of mean diameter: A case study of trees in natural dipterocarp forest. Journal of Mathematics and Statistics 5: 267–269.

24.

Erdogan

(2010) Modelling the spatial distribution of DEM error with geographically weighted regression: An experimental study. Computers and Geosciences 36: 34–43.

25.

Felicísimo

(1994) Parametric statistical method for error detection in digital elevation models. ISPRS Journal of Photogrammetry and Remote Sensing 49: 29–33.

26.

Fisher

(1998) Improved modeling of elevation error with geostatistics. GeoInformatica 2: 215–233.

27.

Fisher

Tate

(2006) Causes and consequences of error in digital elevation models. Progress in Physical Geography 30: 467–489.

28.

Florinsky

(1998) Accuracy of local topographic variables derived from digital elevation models. International Journal of Geographical Information Science 12: 47–62.

29.

Fotheringham

Brunsdon

Charlton

(2002) Geographically Weighted Regression, the Analysis of Spatially Varying Relationships. Chichester: Wiley.

30.

Gatziolis

Fried

(2004) Adding gaussian noise to inaccurate digital elevation models improves fidelity of derived drainage networks. Water Resources Research 40: W02508.

31.

Getis

(1991) Spatial interaction and spatial autocorrelation: A cross-product approach. Environment and Planning 23: 1269–1277.

32.

Giles

Franklin

(1996) Comparison of derivative topographic surfaces of a DEM generated form stereoscopic SPOT images with field measurements. Photogrammetric Engineering and Remote Sensing 62: 1165–1171.

33.

Goodchild

(1982) The fractal Brownian process as a terrain simulation model. Modelling and Simulation 13: 1133–1137.

34.

Goovaerts

Jacquez

Marcus

(2005) Geostatistical and local cluster analysis of high resolution hyperspectral imagery for detection of anomalies. Remote Sensing of Environment 95: 351–367.

35.

Gousie

(2005) Digital elevation model error detection and visualization. In: Gold

(ed.) The 4th Workshop on Dynamic and Multi-dimensional GIS, Pontypridd, Wales, UK. International Society for Photogrammetry and Remote Sensing (ISPRS), 42–46.

36.

Grohmann

Smith

Riccomini

(2009) Surface roughness of topography: A multiscale analysis of landform elements in Midland Valley, Scotland. Proceeding of Geomorphometry 2009, 31 August–2 September. Zürich: Zürich University, 140–148.

37.

Hannah

(1981) Error detection and correction in digital terrain models. Photogrammetric Engineering and Remote Sensing 47: 63–69.

38.

Hebeler

Purves

(2009) The influence of elevation uncertainty on derivation of topographic indices. Geomorphology 111: 4–16.

39.

Hengl

Heuvelink

GBM

Van Loon

(2010) On the uncertainty of stream network derived from elevation data: The error propagation approach. Hydrology and Earth Sciences Discussions 7: 767–799.

40.

Heuvelink

GBM

Burrough

Leenaers

(1989) Propagation of errors in spatial modelling with GIS. International Journal of Geographical Information Systems 3: 303–322.

41.

Hirano

Welch

Lang

(2003) Mapping from ASTER stereo image data: DEM validation and accuracy assessment. ISPRS Journal of Photogrammetry and Remote Sensing 57: 356–370.

42.

Holmes

Chadwick

Kyriakidis

(2000) Error in a USGS 30-meter elevation model and its impact on terrain modeling. Journal of Hydrology 333: 154–173.

43.

Hunter

Goodchild

(1995) Dealing with error in spatial databases: A simple case study. Photogrammetric Engineering and Remote Sensing 61: 529–537.

44.

Hunter

Goodchild

(1997) Modeling the uncertainty of slope and aspect estimates derived from spatial databases. Geographical Analysis 29: 35–49.

45.

Jacquez

(2008) Spatial cluster analysis. In: Fotheringham

Wilson

(eds) The Handbook of Geographic Information Science. Oxford: Blackwell, 395–416.

46.

Jones

(1998) A comparison of algorithms used to compute hill slope as a property of the DEM. Computer and Geosciences 24: 315–323.

47.

Kamp

Bolch

Olsenholler

(2003) DEM Generation from ASTER satellite data for geomorphometric analysis of Cerro Sillajhuay, Chile/Bolivia. Proceedings Annual Meeting Imaging and Geospatial Information Society (ASPRS), 5–9 May, Anchorage, USA.

48.

Karras

Petsa

(1993) DEM matching and detection of deformation in close-range photogrammetry without control. Photogrammetric Engineering and Remote Sensing 59: 1419–1424.

49.

Keys

(1981) Cubic convolution interpolation for digital image processing. IEEE Transactions on Signal Processing, Acoustics, Speech, and Signal Processing 29: 1153–1160.

50.

Kienzle

(2004) The effect of DEM raster resolution on first order, second order and compound terrain derivatives. Transactions in GIS 8: 83–111.

51.

Kirkby

(1990) The landscape viewed through models. Zeitschrift für Geomorphologie NF, Supplement Band 79: 63–81.

52.

Leick

(1990) GPS Satellite Surveying. New York: Wiley.

53.

Wong

(2010) Effect of DEM sources on hydrologic applications. Computers, Environment and Urban Systems 34: 251–261.

54.

Lindsay

(2006) Sensitivity of channel mapping techniques to uncertainty in digital elevation data. International Journal of Geographical Information Science 20: 669–692.

55.

Lindsay

Evans

(2008) The influence of elevation error on the morphometrics of channel networks extracted from DEMs and the implication for hydrological modeling. Hydrological Processes 22: 1588–1603.

56.

Liu

Jezek

(1999) Investigating DEM error patterns by directional variograms and Fourier analysis. Geographical Analysis 3(3): 249–266.

57.

Liu

(2008) Airborne LiDAR for DEM generation: Some critical issues. Progress in Physical Geography 32: 31–49.

58.

Liu

Bian

(2008) Accuracy assessment of DEM slope algorithms related to spatial autocorrelation of DEM Errors. In: Zhou

Lees

Tang

(eds) Advances in Digital Terrain Analysis, Lecture Notes in Geoinformation and Cartography. Berlin: Springer, 307–322.

59.

López

(1997) Locating some types of random errors. International Journal of Geographical Information Science 11: 677–698.

60.

López

(2000) Improving the elevation accuracy of digital elevation models: A comparison of some error detection procedures. Transactions in GIS 4: 43–64.

61.

McCalpin

Berry

(1996) Soil catenas to estimate ages of movements on normal fault scarps, with an example from the Wasatch fault zone, Utah, USA. Catena 27: 265–286.

62.

Monckton

(1994) An investigation into the spatial structure of error in digital elevation data. In: Worboys

(ed.) Innovations in GIS 1. London: Taylor and Francis, 201–211.

63.

Moran

PAP

(1950) Notes on continuous stochastic phenomena. Biometrika 37: 17–23.

64.

Nelson

Personius

(1993) Surficial geologic map of the Weber segment, Wasatch fault zone, Weber and Davis Counties, Utah. US Geological Survey Miscellaneous Investigations. Series Map I-2199, pamphlet, scale 1:50,000, 22 pp.

65.

Oksanen

(2003) Tracing the gross errors of DEM – visualization techniques for preliminary quality analysis. Proceedings of the 21st International Cartographic Conference, 10–16 August, Durban, South Africa, 2410–2416.

66.

Oksanen

Sarjakoski

(2005) Error propagation of DEM-based surface derivatives. Computers and Geosciences 31: 1015–1027.

67.

Oliveau

(2005) Les indicateurs locaux d’association spatiale (LISA) comme méthode de régionalisation. Septièmes Rencontres de Théo Quant, January. Available at: http://thema.univ-fcomte.fr/theoq/pdf/2005/TQ2005%20ARTICLE%2029.pdf.

68.

Petit

Gunnell

Gonga-Saholiariliva

Meyer

Séguinot

(2009) Faceted spurs at normal fault scarps: Insights from numerical modeling. Journal of Geophysical Research 114: B05403.

69.

Podobnikar

(2009) Methods for visual quality assessment of a digital terrain model. S.A.P.I.EN.S 2. Available at: http://sapiens.revues.org/738.

70.

Polidori

(1991) Validation de modèles numériques de terrain, application à la cartographie des risques géologiques. Unpublished doctoral thesis, Université Paris Diderot.

71.

Raaflaub

Collins

(2006) The effect of error in gridded digital elevation models on the estimation of topographic parameters. Environmental Modelling and Software 21: 710–732.

72.

Reuter

Hengl

Gessler

Soille

(2009) Preparation of DEMs for geomorphometric analysis. In: Hengl

Reuter

(eds) Geomorphometry – Concepts, Software, Applications, Developments in Soil Science 33. Amsterdam: Elsevier, 87–119.

73.

Schmidt

Evans

Brinkmann

(2003) Comparison of polynomial models for land surface curvature calculation. International Journal of Geographical Information Science 17: 797–814.

74.

Smith

Clark

(2005) Methods for the visualization of digital elevation models for landform mapping. Earth Surface Processes and Landforms 30: 885–900.

75.

Stock

Frankel

Ehlers

Schaller

Briggs

Finkel

(2009) Spatial and temporal variations in denudation of the Wasatch Mountains, Utah, USA. Lithosphere 1: 34–40.

76.

Tarboton

(1997) A new method for the determination of flow directions and upslope areas in grid digital elevation models. Water Resources Research 33: 309–319.

77.

Tempfli

(1980) Spectral analysis of terrain relief for the accuracy estimation of digital terrain models. ITC Journal 3: 478–510.

78.

Thapa

Bossler

(1992) Accuracy of spatial data used in geographic information systems. Photogrammetric Engineering and Remote Sensing 58: 835–841.

79.

Van Niel

Laffan

Lees

(2004) Effect of error in the DEM on environmental variables for predictive vegetation modeling. Journal of Vegetation Science 15: 747–856.

80.

Walsh

Lightfoot

Butler

(1987) Recognition and assessment of error in geographic information systems. Photogrammetric Engineering and Remote Sensing 53: 1423–1430.

81.

Warren

Hohmann

Auerswald

Mitasova

(2004) An evaluation of methods to determine slope using digital elevation data. Catena 58: 215–233.

82.

Wechsler

(1999) Digital elevation model (DEM) uncertainty: Evaluation and effect on topographic parameters. Proceedings 19th Annual ESRI User Conference, San Diego, California, USA. Available at: http://gis.esri.com/library/userconf/proc99/proceed/papers/pap262/p262.htm.

83.

Wechsler

(2003) Perceptions of digital elevation model uncertainty by DEM users. Urban and Regional Information Systems Association Journal 15: 57–64.

84.

Wechsler

(2006) Uncertainties associated with digital elevation models for hydrologic applications: A review. Hydrology and Earth System Sciences Discussions 3: 2343–2384.

85.

Wechsler

Kroll

(2006) Quantifying DEM uncertainty and its effect on topographic parameters. Photogrammetric Engineering and Remote Sensing 72: 1081–1090.

86.

Wilson

Gallant

(2000) Digital terrain analysis. In: Wilson

Gallant

(eds) Terrain Analysis: Principles and Applications. New York: Wiley, 1–28.

87.

Wise

(1998) The effect of GIS interpolation errors on the use of digital elevation models in geomorphology. In: Lane

Richards

Chandler

(eds) Landform Monitoring, Modelling and Analysis. Chichester: Wiley, 139–164.

88.

Wise

(2000) Assessing the quality for hydrological applications of digital elevation models derived from contours. Hydrological Processes 14: 1909–1929.

89.

Wise

(2007) Effect of differing DEM creation methods on the results from a hydrological model. Computers and Geosciences 33: 1351–1365.

90.

Wood

(1994) Visualising contour interpolation accuracy in digital elevation models. In: Hearnshaw

Unwin

(eds) Visualization in Geographical Information Systems. Chichester: Wiley, 168–180.

91.

Wood

(1996) The geomorphological characterisation of digital elevation models. Unpublished doctoral thesis, University of Leicester.

92.

Younan

Lee

King

(2002) DTM error minimization via adaptive smoothing (LiDAR forest measurements). Geoscience and Remote Sensing Symposium. IGARSS’02.2002 IEEE International 6: 3611–3613.

93.

Yue

Song

Gong

(2007) A new method of surface modeling and its application to DEM construction. Geomorphology 91: 161–172.

94.

Zevenbergen

Thorne

(1987) Quantitative analysis of land surface topography. Earth Surface and Landforms 12: 47–56.

95.

Zhang

Cen

(2008) Robust DEM co-registration method for terrain changes assessment using least trimmed squares estimator. Advances in Space Research 41: 1827–1835.

96.

Zhou

Liu

(2004a) Error analysis on grid-based slope and aspect algorithms. Photogrammetric Engineering and Remote Sensing 70: 957–962.

97.

Zhou

Liu

(2004b) Analysis of errors of derived slope and aspect related to DEM data properties. Computers and Geosciences 30: 369–378.

98.

Ziadat

(2007) Effect of contour intervals and grid cell size on the accuracy of DEMs and slope derivatives. Transactions in GIS 11: 67–81.

99.

Zuchiewicz

McCalpin

(2000) Geometry of faceted spurs on an active normal fault: Case study of the central Wasatch Faul t, Utah, USA. Annales Societatis Geologorum Poloniae 70: 231–249.