Good ol’ Rockyweb


Elevation data users know that one of the best places to access elevation data (in addition to the Digital Coast) is the U.S. Geological Survey (USGS). Not only are the lidar point and digital elevation model (DEM) data available via the National Map VIewer (TNM), they are also available for direct download via the USGS rockyweb site. If you are wondering what this rockyweb thing is, it is the new rockyftp site.  Because several popular browsers stopped supporting ftp protocol, USGS made the switch from ftp to https to continue to provide user access to the data. While the structure of rockyweb is pretty much the same as rockyftp, folder organization and evolving data set naming standardization can still make data discovery a challenge. In addition to rockyweb, USGS also hosts the data, organized in the same way, but on a cloud platform. This site can provide faster downloads and allow data processing in the cloud without the need to download the data. While I wouldn’t consider myself a rockyweb expert, I would like to share some of what I have learned about the site in the hopes it can help other elevation data users navigate and access the data available from this very useful resource. 

What types of data are available?

The rockyweb site has a wealth of data available for the elevation enthusiast. This includes lidar points, original product resolution (OPR) DEMs, hydro breaklines, project and survey reports, USGS determined vertical accuracy values, and tile index and control point shapefiles. If you know the name of the data set you are looking for, you can probably just dive into the site. If you don’t know the name and just want to find out what data is available, a good place to start is the work unit extent spatial metadata (WESM) GeoPackage file.

Work Unit Extent Spatial Metadata (WESM) GeoPackage

In my frequent rockyweb stalking (er, researching), I have learned that new data is added to rockyweb almost daily. USGS is adding data to the site in work units (which are just subsets), not waiting until the entire data set has completed processing and validation, before making the data set available. To view the work units and all the data available, USGS provides the WESM. This is the spatial metadata that lists the data sets and their work units and is a good source of information for current lidar data availability and basic information about the lidar projects, including quality level, acquisition dates, and links to the project level metadata. This file is currently accessible from this rockyweb folder: https://rockyweb.usgs.gov/vdelivery/Datasets/Staged/Elevation/metadata. You’ll need software that understands an OGC GeoPackage, many of which are listed on the GeoPackage implementation page.

Image illustrating where the WESM.gpkg file appears in a browser.
Screenshot of the rockyweb metadata folder with the WESM.gpkg highlighted.

Lidar Points

The lidar points are the workhorse of the elevation products and the top need for many elevation data users. If you want to download the laz files in their delivered parameters, access is provided from this rockyweb folder: https://rockyweb.usgs.gov/vdelivery/Datasets/Staged/Elevation/LPC/Projects/.  The LPC in the folder name stands for lidar point cloud. As mentioned earlier, there is some variability in how the data sets are named. Some are listed with the state 2 letter abbreviation at the beginning of the folder name. Some are listed with ARRA or USGS_LPC at the beginning of the folder name, but the state two letter abbreviation follows. If you have downloaded the WESM.gpkg and found the name of the data set you are interested in, that will be a help in looking through the LPC/Projects folder to find your data.

Image illustrating one of the naming conventions used by USGS.
Screenshot of the rockyweb LPC/Projects folder highlighting the folders named with USGS_LPC.

Once you have found your desired data set, you can either download all the laz files in the data set using wget or uGet or, if you just need a single or few specific files, you will need to download the tile index.shp (discussed below) to determine which files you need to download. You might see a TL in the lidar point cloud data set folder. The TL is short for tribal lands and indicates that the files in that folder fall in a tribal land area.

Image illustrating a naming convention with TL for tribal lands.
Screenshot of the rockyweb LPC/Projects/CA_NoCAL_3DEP_Supp_Funding_2018_D18 folder highlighting the TL in the work unit folder names.

If you want to download an entire data set, you’ll generally want some sort of download manager. An internet search will give you lots of options. Linux users probably already have a free command line tool called wget that works well.  Another free tool that might be more Windows friendly is uGet. The USGS has directions on how to download their data using uGet. 

Digital Elevation Models (DEMs)

If the lidar points are considered the workhorse product, the DEMs are the polished, easy to understand, regularly spaced grids that can be used for visualization and computing. When elevation data is contracted for collection, most of the time, DEMs are also delivered along with the lidar points. USGS calls these the Original Product Resolution or OPR DEMs and access to them is provided via this rockyweb folder: https://rockyweb.usgs.gov/vdelivery/Datasets/Staged/Elevation/OPR/. The organization is pretty much the same as the lidar points, with the state two letter abbreviation at the beginning of the folder name (with some exceptions), however, data that was added to rockyweb since December 2019 are located a folder down in the Projects folder. It is available here: https://rockyweb.usgs.gov/vdelivery/Datasets/Staged/Elevation/OPR/Projects/

If you are unsure of when the data was likely added to rockyweb, the best approach would be to just check both locations to determine if the OPR DEMs are available.

Hydro Breaklines

The hydro breaklines are also often a requested deliverable when lidar data is collected. These data are lines that show where water features or shorelines are located. These are used to help create the hydro-flattened DEMs. These datasets are usually provided in ESRI File GeoDatabase (gdb) format and can be accessed from the metadata/project/work unit/breaklines/ folder. 

Image illustrating the location and naming of an example breakline geodatabase.
Screenshot of the breaklines gdb in the rockyweb metadata/project/work unit/breaklines folder.

Reports, Metadata, Tile Index

The last, but certainly not the least, important type of data available on rockyweb is all of the supporting information about the data. This accompanying information includes the metadata, reports, and shapefiles. As anyone who has worked with great data, the lack of any descriptive metadata about the data can limit its usability. Fortunately, USGS provides access to all this information so the user can make an informed decision about the suitability of the data for their project. All of the reference information is available in the metadata/project/work unit folder here:

https://rockyweb.usgs.gov/vdelivery/Datasets/Staged/Elevation/metadata/.

USGS has recently started adding their project (also known as work package, or WP) report which details their vertical accuracy value, as well as the dates of collection, work unit information, list of point classes, sensor used, las version, etc. This file can be identified by the WP_Report in the file name. 

Image illustrating the location and naming of an example work package report.
Screenshot of the work package report in the rockyweb metadata/project folder.

Also, in the same location as the WP report is the vertical accuracy folder. In this folder you can find both the USGS and the contractor’s shapefiles for the control points and the checkpoints, as well as the USGS las checkpoint report which details the results of the Non-Vegetated Vertical Accuracy (NVA) and the Vegetated Vertical Accuracy (VVA) value determinations.

Image illustrating where the vertical accuracy reports can be found.
Screenshot of the USGS and contractor_provided folders in the metadata/project/vertical_accuracy folder.

If the contractor collection, survey, or processing reports are needed, they can be found in the metadata/project/work unit/reports folder.

Image illustrating where project reports can be found.
Screenshot of the different reports in the rockyweb metadata/project/work unit/reports folder.

If the project, lidar point, DEM, or breakline metadata record from the original vendor is needed, those records can be found in the metadata/project/work unit/reports/vendor_provided_xml folder.

Image illustrating where the vendor provided xml metadata records can be found.
Screenshot of the different metadata records  in the rockyweb metadata/project/work unit/reports/vendor_provided_xml folder.

As mentioned earlier, if you want just a specific file or files, you will need to first download the USGS tile index shapefile. The LPC index is usually available as a shapefile and GeoPackage (gpkg) format.  These files are available in the metadata/project/work unit/spatial_metadata/USGS folder.

Image illustrating the location and naming of a project's tile index files.
Screenshot of the tile index shp in the rockyweb metadata/project/work unit/spatial_metadata/USGS folder.

Cloud Access

As mentioned at the top, all of this data is also in the cloud and that is where USGS points you to for direct data access. That means you can do some data processing in the cloud without having to download the data. It’s in the AWS cloud, so you need to know the s3 bucket name to access it. The key is to recognize that web address https://prd-tnm.s3.amazonaws.com/index.html?prefix=StagedProducts/Elevation matches bucket location s3://prd-tnm/StagedProducts/Elevation. However, the point cloud data isn’t there. Instead it is in a requester pays bucket. You can find the addresses to use on the AWS open data registry page for 3DEP.

Summary

While the data available from the rockyweb site may be a bit of a challenge to navigate, the wealth of elevation information that USGS has made available for free download, from the lidar point data, to the DEMs, hydro breaklines, and the vital metadata, you too, may find yourself becoming a rockyweb creeper.

Leave a Reply. Comments are moderated.

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.