Once upon a time, we kept all the vertices we had for the footprints shown in the U.S. Interagency Elevation Inventory (USIEI). As the app became slower and slower, we realized we would need to simplify the footprints for drawing purposes. Those simplified footprints are the ones that show in the app.
As federal agencies and others have tried to use the inventory to find gaps in coverage, the simplified footprints have become a problem. They can incorrectly show a sliver of a gap where none exists or vice-versa. This can waste a lot of time and effort as everything has to be double checked.
An additional problem is that the footprints can change during the process of data acquisition and there isn’t an indication of what the footprint represents. One footprint might be planned, but it might be changed during the contracting phase when you find out just how far your money can stretch. It might change again with the final delivered data if, for example, persistent cloud cover made it impossible to collect an area.
To address these issues, we’re making a change to what we store and serve with the inventory. The app will still use the simplified footprints for viewing and won’t look any different. However, we will have a second spatial table that has the detailed footprints. This table will be used when generating the GDB and GeoPackage download files. That means the download data will be a little different than the app. This change will happen slowly. The detailed footprint table is initialized with the simplified footprints, but we’ll start to change them to more detailed footprints during updates.
That leads us back to the issue of knowing what the footprint you’re looking at represents. The download GIS files will have an additional column in the NationalElevationInventory table for the FootprintType. This is just a text field and will contain entries like “Planned Footprint”, “Contracted Footprint”, “Completed Footprint”, “Simplified Web Footprint”, and “Unknown”. I’m hoping to never use “Unknown”. Other types may be added as we find the need. As you probably surmised, they are almost all “Simplified Web Footprint” as I write this.
Extra details/weeds
Now that the real news is out of the way, I wanted to provide some details on how all this came about and why it didn’t happen before. The addition of the detailed footprints was an unexpected opportunity that resulted from something else. While the inventory is held at NOAA, we send a replica of the database to USGS to get their updates. The database is managed by ArcGIS, which allows us to create the replica and sync the data changes made by USGS back into the primary enterprise geodatabase.
One issue we found is that updating a footprint in the database was not as simple as it should be. We couldn’t figure out a way in ArcPro or ArcMap to take the feature in a shapefile and use it to replace the feature in a specific row of the database using the GUI. Instead, staff would add a new row with the new shape, copy all the information from the old row, and delete the old row. As we expanded from a single table to a relational model to accommodate multiple entries for a few things, these updates became much more painful and time consuming. I’ll take the blame for causing that.
While there may be a way to do that update more efficiently in the GUI and we just missed it, it looked pretty straightforward to do it using arcpy and a toolbox. The basic approach is to reproject the new feature class if necessary, simplify it for the web, extract the shape with an arcpy.da.SearchCursor, and then use an arcpy.da.UpdateCursor to replace the shape for the correct row in the database. No more copying of all those attributes. There are some weird requirements though. In our setup, you have to be in an edit session for that UpdateCursor and if you have the attribute table open you open an edit session. The basics of the update code are shown below.
# pull the shape we need from the feature class with the shape to update
simp_fc = "simplifed_dissolved_shape" # really our output from dissolve and simplify processes not shown here
shps=[] # a list, but there should only be one since we flattened
with arcpy.da.SearchCursor(simp_fc,["SHAPE@"]) as s_cursor:
for row in s_cursor:
shps.append(row[0])
shp = shps[0]
# whereclause has the condition to pick only the records we want
# database_table is the table to be updated.
# UpdateCursor is a wrapper class for arcpy.da.UpdateCursor
# adapted from the solution answer at https://community.esri.com/t5/python-questions/versioned-data-multiple-updatecursors-runtimeerror/td-p/1043525
# we have a versioned table, so we need to have an edit session. This makes sure we do.
with UpdateCursor(database_table,['SHAPE@'], whereclause) as u_cursor:
for row in u_cursor:
row[0] = shp
u_cursor.updateRow(row)
Once I had a toolbox to do the updates, it made sense to add tools to create or delete a record in general. This gives us most of the pieces of CRUD. Now the tasks of simplifying and reprojecting could be done in the tool and make the process easier for staff. It could also provide more consistency to the process.
Detailed Footprints
How does this lead us to detailed footprints? We certainly could have made a separate spatial table for the detailed footprints before. However, we would have had to make sure everything stayed synchronized between the tables in a manual process. That appeared to be a recipe for disaster. Once we had a toolbox, and therefore a scripted consistent process, we could take the new or updated shape and put it in the detailed footprint table before simplifying it for the web app. Thus, the opportunity to finally capture the full detail was there and we’re going down that path. New entries will automatically get this treatment, but we’ll have to update the older records a bit at a time.
One last thing on the footprints. You might be wondering how the tables were joined so that the GDB and GeoPackage have the detailed footprints and the attributes from the original simplified table. While it would make sense to use the AddJoin function in arcpy, this resulted in field names that included the origin table. The solution was to do the join in the database to make a view with the desired fields and names, and then export (arcpy.conversion.FeatureClassToFeatureClass) into the GDB.
Digital Coast GeoZone 