Statistical analysis of water-quality data containing multiple detection limits II: S-language software for nonparametric distribution modeling and hypothesis testing

L. Lee; D. Helsel

doi:10.1016/j.cageo.2006.09.006

Home/Archive/USGS

USGS · 70030209

Statistical analysis of water-quality data containing multiple detection limits II: S-language software for nonparametric distribution modeling and hypothesis testing

L. Lee·D. Helsel

View original source ↗Export metadata ↓

Abstract

Analysis of low concentrations of trace contaminants in environmental media often results in left-censored data that are below some limit of analytical precision. Interpretation of values becomes complicated when there are multiple detection limits in the data-perhaps as a result of changing analytical precision over time. Parametric and semi-parametric methods, such as maximum likelihood estimation and robust regression on order statistics, can be employed to model distributions of multiply censored data and provide estimates of summary statistics. However, these methods are based on assumptions about the underlying distribution of data. Nonparametric methods provide an alternative that does not require such assumptions. A standard nonparametric method for estimating summary statistics of multiply-censored data is the Kaplan-Meier (K-M) method. This method has seen widespread usage in the medical sciences within a general framework termed "survival analysis" where it is employed with right-censored time-to-failure data. However, K-M methods are equally valid for the left-censored data common in the geosciences. Our S-language software provides an analytical framework based on K-M methods that is tailored to the needs of the earth and environmental sciences community. This includes routines for the generation of empirical cumulative distribution functions, prediction or exceedance probabilities, and related confidence limits computation. Additionally, our software contains K-M-based routines for nonparametric hypothesis testing among an unlimited number of grouping variables. A primary characteristic of K-M methods is that they do not perform extrapolation and interpolation. Thus, these routines cannot be used to model statistics beyond the observed data range or when linear interpolation is desired. For such applications, the aforementioned parametric and semi-parametric methods must be used.

Explore related subjects

Computers & Geosciences Article

Keep this discovery

Explore connections, maps & timelines

BibTeX RIS

L. Lee, D. Helsel. 2007. Statistical analysis of water-quality data containing multiple detection limits II: S-language software for nonparametric distribution modeling and hypothesis testing. https://doi.org/10.1016/j.cageo.2006.09.006

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related USGS reports

USGS2026

3D semantic mapping of surface geological features

Semantic mapping in 3D is fundamental to a wide range of geoscientific studies and applications, including geomorphology, hazard assessment, and environmental monitoring. However, automatically segmenting geological features from large-scale photogrammetric datasets remains a significant challenge. We present a methodology to address this gap. Using overlapping images collected over environments of interest, Structure-from-Motion (SfM) produces georeferenced point clouds and estimates camera poses. Existing large vision models, such as Segment Anything Model, segment objects in the images, generating pixel-segmentation associations. To produce pixel-point associations, we project the points back onto the camera image planes. As objects are independently segmented across multiple images with different perspectives, we develop a segmentation mosaicking algorithm to build probabilistic point-segmentation associations that combines the pixel-segmentation associations and pixel-point associations. Our methodology is validated using both synthetic data generated by Kubric and real-world UAV-SfM data. The implementation is designed to be compatible with existing SfM software, including Agisoft and OpenDroneMap, for photogrammetry mapping in geoscience studies. As a case study, we apply our method to the semantic mapping of precariously balanced rocks (PBRs), which provide upper-bound constraints on historical ground motion shaking intensity. To support object-level identification of PBRs, we additionally integrated Grounding DINO, enabling text-prompted segmentation of features of interest within UAV imagery. This case study demonstrates the effectiveness of our method in generating a 3D semantic map of PBRs, enabling spatial distribution of PBR fragility for earthquake hazard analysis.

Computers & Geosciences↗

USGS2022

gTOOLS, an open-source MATLAB program for processing high precision, relative gravity data for time-lapse gravity monitoring

gTOOLS is an open-source software for the processing of relative gravity data. gTOOLS is available in MATLAB and as a compiled executable to be run under the free MATLAB Runtime Compiler. The software has been designed for time-lapse (temporal) gravity monitoring. Although programmed to read the Scintrex CG-5 and CG-6 gravimeters output data files, it can be easily modified to read data files from other gravimeters. The software binds together single-task processing modules within a very simple user interface that is based on one text file. Gravity processing involves three modules: (a) gravimeter calibration; (b) automatic processing of gravity data to find adjusted gravity differences; and (c) post processing of results. Each module is optional and runs independently from the others. Data processing includes (a) averaging out the measurements noise, and correction for solid Earth tides, and ocean loading, and residual instrumental drift, and (b) calculate the residual instrumental drift and gravity differences between the base station and monitoring sites, and their uncertainties, by a weighted least square analysis of the gravity data. The software allows the automatic processing of a gravity campaign spanning multiple days in a single run. The software is tested on gravity data from 2015 eruption at Cotopaxi volcano, Ecuador.

Computers & Geosciences↗

USGS2022

MTH5: An archive and exchangeable data format for magnetotelluric time series data

Magnetotellurics (MT) is a passive electromagnetic geophysical method that measures variations in subsurface electrical resistivity. MT data are collected in the time domain and processed in the frequency domain to produce estimates of a transfer function representing the Earth’s electrical structure. Unfortunately, the MT community lacks metadata and data standards for time series data. As the community grows and findability, accessibility, interoperability, and reuse of digital assets (FAIR) data principles are enforced by government and funding agencies, a standard is needed for time series data. Presented here is a hierarchical data format (MTH5) that is logically formatted to how MT data are collected. Open-source Python packages are also described to read, write, and manipulate MTH5 files. These include a package to deal with metadata ( mt_metadata ) based on standards developed by the Working Group for Magnetotelluric Data Handling and Software assembled by the Incorporated Research Institutions for Seismology (IRIS), and mth5 : a package to interact with MTH5 files that uses mt_metadata . Example code and workflows are presented.

Computers & Geosciences↗