Mistakes People Make—Common Data Access Viewer Errors


[April 2016 update: The interface has been changed, so the screen shots are out of date, but generally the issues still happen.]

A lot of the geospatial data hosted on the Digital Coast is made available through the Data Access Viewer (aka DAV). The DAV lets you customize a lot of things about your data, particularly the lidar data. Sometimes it gives you enough rope to hang yourself. Today I’m going to cover a few of the common errors we see with lidar data requests.

Contours: Single Versus Interval

Example of interval contour options
This example interval output would provide contour lines set at every 3 feet.

The default mode is to pick a contour interval. For instance, in the case where you want a contour line every 2 feet. However, the system has a limit on how fine an interval you can pick and it’s based on the data accuracy. Essentially, you can’t pick a contour interval that is closer than three times the RMSEz of the data, which follows the national map accuracy standards.

The other option is to pick a single contour. This is for the case where you want a single isoline at some arbitrary elevation. For example, if mean high water for your little area is at 1.23 feet NAVD88, you might ask for the 1.23 foot contour line to get something akin to a shoreline. All your lines will be the ones at that height.

Example of single contour option
This example of a single contour option would provide contour lines only at 0.23 feet.

The trouble comes when someone realizes they can’t get the 1-foot contours from data that only support 2-foot contours using the interval setting, so they pick the single contour setting where they can put any arbitrary number instead. One of two things will happen. Either they will get a single line, which isn’t what they want, or they will get nothing—also not what they want. The nothing case happens because the value they picked is not in the range of the data. They may have wanted 1-foot contours of a hill that has elevations from 10 to 30 feet, but they asked the system to give them the 1-foot contour line, which doesn’t exist for that area.

All Points for Derived Products

Example of options with all points selected for contours.
The options in this example would result in contours every three feet, but they would go up and down trees, buildings, etc. That tends to be very messy and there is a good chance they would break the size limit of a shapefile.

This happens for both raster DEMs and contours, though we see the system failures with the contours. Lidar data is stored as a point cloud and (usually) the individual points are given a classification (ground, water, etc) to indicate what kind of surface the laser reflected from. When you make a derived product, such as a DEM, from all the points, you generally end up with an uninterpretable mess. Each cell in your DEM has a single height value and that height may come from averaging together a ground point at 2 feet and a vegetation point at 10 feet, resulting in a value of 6 feet that doesn’t represent anything useful.

For contours, the same basic idea applies with potentially worse results (assuming failure to generate output is worse than being useless, but I’m not so sure). The contour generation currently uses a TIN process. With all points included, it will try to contour up and down trees. That’s bad enough, but with the ground points thrown in that might be within the footprint of a tree, the contours become a real mess. Enough of a mess that all those vertices exceed the limits of the shapefile format that is the initial output. In theory, we could rework the system to output to something that doesn’t have a 4Gb limit, but it seems better to have it fail than for us to store and for you to download an enormous useless file.

Usually you want to use only the ground points for derived products. Sometimes you do want to create a first surface product though. In that case, you want to make sure you limit it to first return points as detailed in a previous GeoZone post.

ASCII XYZ with All Points

asciiXYZallPoints
Options set to all points for ASCII X,Y,Z – probably not what you want.

Continuing the theme of all points, be careful when ordering points in ASCII. You’ll only get x,y, and z, so any information on the point classification will be lost. So will any other point by point information such as the return intensity, return number, etc. Even if you wanted to test out your own classification routines, you’d be better off getting all the info using LAS or LAZ format and resetting all the classifications. Luckily, the default is to only get ground points, but every once in awhile, someone will change it to all points.

LAS or LAZ with Ground Only

This might not be a mistake if you know you only care about the ground. However, you are throwing away all the other information that might be useful for understanding the landscape. Unfortunately, the default is to get ground only, which is something we need to change when we update the DAV. Personally, I’d also recommend getting LAZ instead of LAS to save yourself some transfer time with the smaller files.

Missing the Page with All the Options

editChoicesCheckbox
Make sure to check the edit choices box (pointed to by red arrow). Also note the blue question mark for help. That brings up a pdf with more info.

[Update: The new version doesn’t have this step so you can no longer make this mistake]

My guess is that this is the most frequent error, but it’s impossible to tell if it was an error. The DAV’s current checkout page with all the lidar options is mostly grayed out and there is a little checkbox you have to hit to let you edit everything. One of our developers was trying out the DAV and blew right past that page by hitting “next” because it looked like there was nothing to do. He’s probably not the only one. As you can tell from the other errors, we give you enough rope to hang yourself, but only if you check the box. There is no perfect default though, so I’m sure some people are ending up a bit disappointed because they didn’t get what they wanted. Instead, they got a GeoTiff DEM of the ground points in geographic coordinates. Likely they wanted something in projected coordinates.