Digital Coast Data Changes

We’ve finally updated the interface to our Data Access Viewer (DAV) that provides custom downloads to lidar, imagery, and land cover data on Digital Coast. It went live today, so I thought I’d give a little overview of the changes as well as some tips and tricks.

The biggest change is how the footprint of each data set is shown. The old viewer gave you all footprints for all data types at once as an outline. That could be a confusing mess. The new viewer shows one data type at a time, and the tabs are used to select which type you want to view. On the technology side, we switched from sending the vector form of each footprint to the client and now make an image on the server side. This speeds things up considerably. The figure below shows the viewers after you’ve drawn a search box in the southeast.

Screen capture of the version 3 Data Access Viewer.

Screen capture of new version 4 Data Access Viewer. Only the part of the footprints within your search box are shown.

While all of the functionality is still there, some has moved around a bit and a few new things have been added. First, I’ll take you on a screenshot tour to give a sense of things to look for in the user interface. Next, I’ll cover some of the changes to help current users find their way around. Then, I’ll talk about some of the new features you might appreciate and, lastly, provide you with some tips and tricks.

Quick Screen Grab Tour

Below are a few screen grabs of the new interface with annotation on some of the points of interest. These may be enough to get the TL;DR crowd going and I’ll refer back to them later.

Figure 1. Opening screen of the DAV highlighting: 1) tabs to switch data type; 2) search box with draw tool; 3) initial footprints showing all imagery in the view; 4) buttons to change the basemap; and 5) the “about” button. The cart is also circled but not numbered.

Figure 2. After searching for Charleston, SC, we see: 1) resulting footprints with grey lines showing borders; 2) tools to filter or sort the results; 3) an example search result; 4) tool to add to cart; and 5) a data set that is too big with this AOI for custom download.

Figure 3. Clicking on a title in the results provides the details view for that data set. This includes: 1) the blue outline of the data and a view of the data when available; 2) a button to add the data to the cart for custom processing; 3) a link to bulk download all the base files; 4) some of the important attributes of the data; 5) links to the metadata and, if available, image services; and 6) a slider to change the opacity of the image services when available.

Searching

The first step has always been to search for data in your area of interest (AOI). The new DAV has all the familiar ways to search for data: searching by the name of a location, the geographic coordinates of a location, or using the map and drawing a custom AOI to search. They all start in the upper left area (Figure 1, #2).

The name lookup is functioning again, so you can type in something like “Charleston” into the text box and pick from the various Charlestons that show up. The yellow box around your feature isn’t always perfect. In many cases it’s just a 10 mile box around a point, so you’ll want to refine it.

The little pencil circled in red is the tool you need to draw an area of interest.

To refine your area or simply start by drawing a box, use the pencil tool in the search box to draw the area. This will kick off a search for the data sets within the box and also sets the extent limits for any requests you make from the search.

If you need to zoom to an area before drawing a search box, the usual pan and zoom mouse tools will work. You can also hold down the shift key and draw a zoom box.

Searching for a name isn’t the only way to use the text box. You can also put in a geographic latitude, longitude point or the corners for a box. The box corners will allow you to perform multiple requests of the same dataset without worrying about gaps or overlap. They are also useful for areas without landmarks. Make sure you enter them as longitude_min, latitude_min, longitude_max, latitude_max, which is the opposite of entering a point.

Sometimes you’ll need to be able to see reference points to draw your AOI, and you can change the basemap using the little boxes that say Streets and Satellite in the lower left (Figure 1, #4).

Once you search, you’ll get a list of datasets that are in your area on the right side panel. If you need to change what data type you’re searching, just hit one of the Imagery, Land Cover, or Elevation tabs at the top (Figure 1, #1) and it will switch while retaining your AOI. To add a data set to the cart for custom processing, select the shopping cart icon (Figure 2, #4) that goes with that data set. You’ll see a number appear below the icon indicating how many areas you’ve added to the cart for that data set, and the number in the cart in the upper right will increment too. Moving your mouse over a search result will show a blue polygon showing the dataset extent.

Most datasets will also have a link in the search results that will take you to a bulk download (also linked at Figure 3, #3). You’ll lose the custom processing ability, but if you need a full county of imagery or lidar and can do your own data processing, it may be your preferred option. You can also get more information about a data set by clicking on its name (Figure 3).

Checkout

Image showing the cart button in the upper right (circled in red) to take you to checkout.

Once you’ve chosen the various datasets and areas you want, select the cart button in the upper right. That will take you to the checkout. We’ve always tried to lead people by using an e-commerce pattern (e.g. the cart icon and checkout), so naturally people tend to ask about the cost. Well, they’ve already been covered by the American citizens, and there is no additional cost.

Within the checkout, things should look pretty familiar to frequent DAV users. One significant change is that we removed the need to check a box in order to edit the choices as many people were overlooking it anyway.

New Features

Is there new stuff? Of course there is! I already mentioned the ability to enter the corners for an AOI and search by geographic feature name. Here’s some other features:

• For elevation data that is already a DEM, you can select tidal datums (e.g. mean sea level) as long as your data set and AOI intersect the VDatum grids. It does use VDatum to do the transforms, and that will add additional uncertainty to the result. Several caveats go with using VDatum, including the fact that you’ll only get data where there are valid transform grids, and they don’t go very far inland. We do plan to add this capability to the point clouds in the future.
• New file formats: Imagery can now support JPEG2000 output. Point cloud lidar can now support Imagine format when creating a DEM.
• Geoid model selection: Previously, the latest NGS Geoid model was applied to the data when ordering point cloud data in NAVD88. You’ll now have the option to pick an older model that might match data you already have. Check the advanced options to see your choices. The latest model (currently GEOID12B) is always the default.
• TIN interpolation of point clouds is now available when creating DEMs from point clouds. The TIN’ing is done using las2dem from LAStools.
• The system is designed to use data services (OData 4) for job submission. That means you can skip our new user interface entirely and do your own programming. We don’t have the API written up yet, but that will be coming and I’ll write another blog post about it then.

Tips and Tricks

Now I want to talk a bit about some tips and tricks and things you might miss.

• Many of the data sets now have image services available using the ESRI Image Server. This lets you see the data before you download. If you select the data set name in the search results to get the additional details about the data, the image service will display if it exists. You’ll also see a slider bar to change the image opacity at the bottom of the details and there will be a link to the services REST page. We’re working to get more services up. Most of the imagery and land cover data are there, but elevation has a little ways to go. The services also take a bit longer to develop than we’d like, so a little patience may be required.

  

  Example of image services for a 4-band imagery data set. You can see that the blue footprint boundary that shows when you mouse over the search results isn’t always right on.

• Getting multiple data sets or areas (jobs) in the same request? Now you can set the parameters you want for the job and then copy those settings to the other jobs. There are some limits to this method as some options may not make sense for all data sets. Be sure to check each data set after the copy.
• If you return to the viewer for another request after submitting one, the checkout will remember your settings.
• You can no longer paste your email address into the “confirm email” box as too many people were managing to paste an incorrect email. You can blame them (including our staff) for the inconvenience. If your email is wrong you won’t get the notices that your data is ready, and we’ll have no way to let you know, so double check that.
• Adding something to the cart may take a little longer than you expect because the app is getting more information about that dataset from the server. For example, is there an intersection between your AOI, the dataset footprint, and the VDatum grids that would make a tidal transformation possible? Watch the blue progress line near the top fill from left to right to show status.
• For some datasets, a lidar point cloud and a pre-made DEM from the points exist as two different datasets. In the old viewer, they had different colors in the results panel. The new viewer has them both on the elevation tab. Look at the dataset title to differentiate. A pre-made DEM will have “DEM” or “Digital Elevation Model” in the title and will have fewer options for output, but it may have had breaklines applied and artifacts removed.
• Some data has been removed from the DAV. These data are typically already in zip files or otherwise not appropriate for custom processing. You can still find them on the Digital Coast data registry.

I hope you like the improvements in the new viewer. If you’d like to send us some feedback, select the little blue question mark in the lower left and you’ll get an “about” panel that includes a feedback button. Of course, you can also comment here and I’ll be happy to answer any questions. That way everyone benefits. We’d love to hear from you.