Lidar Accuracy Measures


Lidar vertical accuracy measures can be confusing. Someone looked at one of our lidar records and pointed out that it said the data had an accuracy of 10 cm RSME on the website but 19.6 cm at 95% confidence in the metadata. He wanted to know which one was right. We’re going to look at several different measurements that are commonly used and how they relate.

95% Confidence Level

The NSSDA provides guidance for reporting geospatial accuracy in the U.S. government. It specifies reporting the 95% confidence level. This means the value of error at which we expect 95% of the errors to be at or below that value. If I assess the error of a data set in 1000 locations, sort the errors by magnitude, and take the value of the 50th largest error, that will be my 95% confidence number. We don’t usually have 1000 error estimates to work with, but you do need at least 20.

Root Mean Square Error (RMSE)

One difficulty with directly estimating the 95% confidence level is that we rarely have the right number of error estimates so we can pick a particular error as the 95%. You’d have to have a multiple of 20 error estimates. However, if the errors are normally distributed, you can compute the root mean square error and multiply by 1.96. You’ll see this a lot in lidar data reports. We typically expect the non-vegetated areas will have normally distributed errors, so this works well. Note that it means that 10 cm RMSE is the same as 19.6 cm at 95% confidence, so that answers the original question. The formula for the RMSE is:


where xi is the error for each N observations.
Normal distribution and some of its properties.

Standard Deviation

The standard deviation is another one you may remember from an introductory stats class. It is similar to, but a bit more complicated than the RMSE with a formula:

where the x with a bar over it () is the mean value of all the errors.

It is worth noting that if the mean value of all the errors is zero, then the RMSE and the standard deviation converge with a sufficiently large number of observations.

One drawback to the standard deviation is that it doesn’t stand alone very well. You might have a point cloud with a standard deviation of 5 cm, which sounds pretty good, but if the mean error is 1 meter you may have to reconsider. In contrast, the RMSE for such a dataset would be near 1 meter.

Depth Dependent

Finally, there are the accuracy standards used by the IHO. It’s a 95% confidence level for total vertical uncertainty (TVU), but incorporates a depth component such that the uncertainty increases with depth as:

where a is the depth independent uncertainty portion, b is the depth dependent uncertainty portion, and d is the depth.

You’ll most often see this sort of nomenclature with bathymetric data collected for charting purposes that need to meet the IHO S-44 standards. If you think of this for topography, where d = 0, then a is simply the 95% confidence level.

Conclusion

That’s all the systems I tend to come across. Let me know if there is something else you see a lot. You can learn a lot more about making lidar accuracy measurements from the ASPRS Positional Accuracy Standards document. It includes discussion of vegetated (VVA) versus non-vegetated (NVA) accuracy measures that you’ll see in many lidar reports, as well as the older FVA, CVA, and SVA measures.

Leave a Reply. Comments are moderated.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.