Scaling an Overlay Analysis
This project was featured as an ArcGIS StoryMap at the 2021 GIS in the Rockies conference and served as the focus of my Harvard REU Fellowship.
Objective:
The primary task is to overlay US block group boundaries with BLM PLSS section boundaries to determine each section or part of a section that is contained within a block group. The results need to be reported in one field per block group in a delimited string variable, where the delimiter is a hashtag (CO060090N0530W0SN270#CO060090N0530W0SN280#CO060090N0530W0SN290). The unique identifier for block groups is the FIPS code, and the unique identifier for sections is the First Division ID.
Visualization of a small subsection of US block groups overlaying PLSS sections. The cartographic boundary of block groups can be seen in black, and sections can be seen in grey."
Let me know if you need any modifications or further processing!
Initial Mockup of the desired result.
Methodology:
The Tabulate Intersection tool in ArcGIS Pro was used primarily as a means to summarize the section or part of a section that is contained within each block group. This tool cross tabulates the area, length, or count of the intersecting features which completed a large portion of this project and what can be considered it's GIS component. Due to the large size of the input data sets (US block groups where n=219,000 and BLM PLSS section boundaries where n=30,000,000), this geoprocessing tool offered the fastest processing time of the available options.
Illustration of the tool from ESRI's Tool Reference catalog. In the case of this overlay, zones can be considered block groups and colors can be considered sections.
Sample output of the Tabulate Intersection tool. The tool helped rearrange the data so that multiple First Division ID's were associated with a single FIPS code.
Formatting the results according to the project specifications is where the Jupyter Notebook script became essential. The driving force behind this script was a cursor, which was used to iterate through the table produced by the Tabulate Intersection tool, store unique values in a list, and then append them back appropriately.
These two cells in the Jupyter Notebook accomplished most of the task at hand.
One essential preprocessing steps to ensure the script would run without error was removing all null values from the FIPS column of the US block group layer. It should also be noted that PLSS First Division townships were not created for the entire country. Some states have gaps in township data or do not contain any township data. Therefore, only 115,389 block groups out of 219,831 were included in the calculations performed by the Jupyter Notebook script.
Results:
The overlay analysis was successfully completed per the project specifications. Initially, due to the large size of the datasets in this overlay analysis, the processing time while running the script was a significant barrier. The Tabulate Intersection tool therefore was a crucial discovery. Additionally, the Jupyter Notebook script reconciled the inability of ArcGIS Pro to compile the final dataset in our desired format. ArcPy scripting with Jupyter Notebooks was advantageous in this case as it allowed for a closer monitoring of processing time for each major component of the process and helped suggest how scaling the process to progressive larger datasets would go.
H3 Cell Biodiversity Index Study
In this project I analyzed the biodiversity of tree species across San Francisco using the Shannon Index, a well-known metric that combines species richness and evenness. By grouping trees within H3 spatial cells, I assessed the diversity of tree species in each area, normalizing the data (on a 0 to 1 scale) to account for varying tree densities across regions. This analysis provides valuable insights into the ecological health of urban green spaces, helping to identify areas with high species diversity and those that may require further conservation efforts or planting initiatives.