Data Processing

Gathering Geospatial Datasets

In order to create an impactful index, we gathered data from a variety of sources (detailed in the citations below). The data we used captures the indicators that were specified in the driver-indicator structure for drivers of water reuse described on the Motivation page.

The data we used was meant to represent the United States on a national level as much as possible, though of course certain variables were more sparse in certain areas. We acknowledge the bias and the impact that a lack of data availability creates in our calculated index and mapped representation.

Most of the data we used was publically available, and anything that was not publically available was provided by other researchers to use with permission.

Merge Data to Target Scale (“Crosswalk”)

Because we used data on various geographic scales, for example census data on county or census tract level in combination with environmental data on a watershed level, we had to scale values up and down based on area or population weighting, depending on the variables and if they were best represented by a mean (e.g. population density) or a sum (e.g. total population) value, respectively. We also had to match boundary lines of overlapping areas by calculating the intersecting areas, bounded by their coordinate geometries. These types of procesesses are common in GIS tasks. Our final dataframe for the Water Reuse Index is on the drinking water service area level.

We decided to create functions that allow for “crosswalking” data between different geospatial units, sometimes based on a common ID column (for nested geometries) and sometimes based on more complicated area calculations. We allowed for point (longitude, latitude), raster, polygon, and tabular (with an ID column) data in these functions. The functions are detailed in this tutorial and the associated R scripts (complete with doc strings) are located in this folder. If you are looking for examples of how we used these functions in our data cleaning and crosswalking process, you can take a look at our cleaning files here.

Additional resources and tips for GIS processing have been compiled here.

Notes and examples: For the Provisional Water Reuse Index, we imputed null values to 0. We converted all values to percentiles prior to performing a Principal Component Analysis. We dropped any water service areas with no associated geometry that could be mapped.

Assign Variables (“Indicators”) to Categories (“Drivers”)

We knew that it was important to maintain the conceptual indicator/driver structure shown on the Motivation. We did this by creating a list mapping the drivers to the indicator values (columns) they are measured by. This impacts both how the index is calculated and the menu options that appear on the final website. This is detailed in following pages and in the create_website.Rmd.

Data Citations