Accuracy assessment/validation methodology and results of 2010–11 land-cover/land-use data for Pools 13, 26, La Grange, and Open River South, Upper Mississippi River System

Abstract

Introduction/Background The U.S. Geological Survey (USGS)-Upper Midwest Environmental Sciences Center (UMESC) was responsible for development of several land cover/land use (LCU) systemic datasets of the Upper Mississippi River System (UMRS). These efforts (1989 and 2000) were funded by the U.S. Army Corps of Engineers’ Upper Mississippi River Restoration Program (UMRR) Long Term Resource Monitoring (LTRM) element. Development of systemic datasets includes the acquisition, processing, and serving of high-resolution aerial photography and land cover/land use spatial datasets ( http://www.umesc.usgs.gov/data_library/land_cover_use/land_cover_use_data.html ). In 2008, the UMRR reached a collaborative agreement with the U.S. Fish and Wildlife Service-Region 3 to collect high-resolution digital imagery of the entire UMRS floodplain during 2010–11 for LTRM. The UMESC helped acquire, process, and serve this imagery, as well as produce and serve the 2010–11 LCU systemic dataset of the UMRS floodplain. Digital imagery for Pools 13, 26, La Grange, and Open River South was collected using an Applanix DSS 439 digital sensor system with a 40 millimeter lens and Color Infrared (CIR) filter. The imagery was collected at a resolution of 20 centimeters/pixel (8 inches/pixel) for Pool 13 and 40 centimeters/pixel (16 inches/pixel) for Pools 26, Open River South, and La Grange. All imagery was projected to Universal Transverse Mercator (UTM) Zone 15, North American Datum of 1983 (NAD 83). The General Wetland Vegetation Classification (GWVC) system used for mapping is hierarchical, and its 31 classes can be collapsed into broader categories using either a 15- or 7-class level. While the 1989 and 2000 LCU systemic datasets have not gone through a traditional thematic accuracy assessment (AA) in the past, nor have they undergone a validation analysis, the end products are of high quality. For each systemic dataset produced (1989, 2000, 2010–11), extensive field reconnaissance is performed before photointerpretation. The intent of this field reconnaissance is to learn, test, and verify image signatures as they relate to vegetation types. Questionable areas on the imagery are visited, and the plants or land features observed in the area are recorded for reference. This procedure verifies vegetation signatures on the imagery with those on the ground. In addition, once the photointerpretation is complete, the final LCU dataset undergoes extensive quality assurance/quality control to ensure the imagery is mapped correctly. Since the 2000 LCU systemic dataset was developed, there has been a growing interest in completing thematic AAs for the LTRM LCU spatial datasets. The objective of an AA is to measure the probability that a particular location has been assigned its correct vegetation class. An AA estimates thematic (map class) errors in the data, giving users information needed to determine data suitability for a particular application. At the same time, data producers are able to learn more about the nature of errors in the data. Thus, the two attributes of an AA are “producers’ accuracy,” which is the probability that an AA point has been mapped correctly (also referred to as an error of omission); and “users’ accuracy,” which is the probability that the map actually represents what was found on the ground (also referred to as error of commission). Producers’ and users’ accuracies can be obtained from the same set of data by using different analyses. Accuracy assessment is an extensive effort that requires seasonal field personnel and equipment, data entry, analyses, and post processing—tasks that are costly and time consuming. The geospatial team at the UMESC has suggested a validation process for understanding the accuracy of the spatial datasets, which will be tested on at least some areas of the UMRS. Validation is not a true verification of map-class type in the field; however, it can provide the user of the map with useful information that is similar to a field AA. Similar to an AA, validation involves generating random points based on the total area for each map class. However, instead of collecting field data, two or three individuals not involved with the photo-interpretative mapping separately review each of the points onscreen and record a best-fit vegetation type(s) for each site. Once the individual analyses are complete, results are joined together and a comparative analysis is performed. The objective of this initial analysis is to identify areas where the validation results were in agreement (matches) and areas where validation results were in disagreement (mismatches). The two or three individuals then perform an analysis, looking at each mismatched site, and agree upon a final validation class. (If two vegetation types at a specific site appear to be equally prevalent, the validation team is permitted to assign the site two best-fit vegetation types.) Following the validation team’s comparative analysis of vegetation assignments, the data are entered into a database and compared to the mappers’ vegetation assignments. Agreements and disagreements between the map and validation classes are identified, and a contingency table is produced. This document presents the AA processes/results for Pools 13 and La Grange, as well as the validation process/results for Pools 13 and 26 and Open River South.