Lidar vertical accuracy measures can be confusing. Someone looked at one of our lidar records and pointed out that it said the data had an accuracy of 10 cm RSME on the website but 19.6 cm at 95% confidence in the metadata. He wanted to know which one was right. We’re going to look at several different measurements that are commonly used and how they relate.
95% Confidence Level
The NSSDA provides guidance for reporting geospatial accuracy in the U.S. government. It specifies reporting the 95% confidence level. This means the value of error at which we expect 95% of the errors to be at or below that value. If I assess the error of a data set in 1000 locations, sort the errors by magnitude, and take the value of the 50th largest error, that will be my 95% confidence number. We don’t usually have 1000 error estimates to work with, but you do need at least 20.
Root Mean Square Error (RMSE)
One difficulty with directly estimating the 95% confidence level is that we rarely have the right number of error estimates so we can pick a particular error as the 95%. You’d have to have a multiple of 20 error estimates. However, if the errors are normally distributed, you can compute the root mean square error and multiply by 1.96. You’ll see this a lot in lidar data reports. We typically expect the non-vegetated areas will have normally distributed errors, so this works well. Note that it means that 10 cm RMSE is the same as 19.6 cm at 95% confidence, so that answers the original question. The formula for the RMSE is:
where xi is the error for each N observations.
The standard deviation is another one you may remember from an introductory stats class. It is similar to, but a bit more complicated than the RMSE with a formula:
It is worth noting that if the mean value of all the errors is zero, then the RMSE and the standard deviation converge with a sufficiently large number of observations.
One drawback to the standard deviation is that it doesn’t stand alone very well. You might have a point cloud with a standard deviation of 5 cm, which sounds pretty good, but if the mean error is 1 meter you may have to reconsider. In contrast, the RMSE for such a dataset would be near 1 meter.
Finally, there are the accuracy standards used by the IHO. It’s a 95% confidence level for total vertical uncertainty (TVU), but incorporates a depth component such that the uncertainty increases with depth as:
where a is the depth independent uncertainty portion, b is the depth dependent uncertainty portion, and d is the depth.
You’ll most often see this sort of nomenclature with bathymetric data collected for charting purposes that need to meet the IHO S-44 standards. If you think of this for topography, where d = 0, then a is simply the 95% confidence level.
That’s all the systems I tend to come across. Let me know if there is something else you see a lot. You can learn a lot more about making lidar accuracy measurements from the ASPRS Positional Accuracy Standards document. It includes discussion of vegetated (VVA) versus non-vegetated (NVA) accuracy measures that you’ll see in many lidar reports, as well as the older FVA, CVA, and SVA measures.