Why soil-data normalization is difficult¶
Soil data is fragmented across survey databases, laboratory archives, sensor networks, modeled rasters, research releases, and regulatory systems. The hard part is not only finding a number. It is preserving enough context to know what that number means.
Names and units¶
The same property can have many names, abbreviations, and database columns. Two values may share a name while using different extraction methods. Conversely, different unit spellings may represent an identical quantity. OpenSoil maps aliases to a canonical vocabulary but retains the original name and unit.
Depth intervals¶
A 0–5 cm sample is not interchangeable with a 0–30 cm value. Survey horizons have variable boundaries; raster products often publish fixed standard depths; laboratory profiles may contain irregular horizons. OpenSoil always returns a top and bottom depth and does not silently interpolate across intervals.
Methods¶
pH depends on the solution and ratio. Available phosphorus is operationally defined by its extraction. Bulk density depends on sample treatment and water tension. Method metadata can be incomplete, so a missing method becomes a warning—not permission to assume compatibility.
Observed, modeled, mapped, derived, and supplied¶
- Observed data comes from a sample or sensor under a stated method.
- Modeled data is a prediction with spatial and statistical support.
- Mapped survey data describes representative map-unit components or horizons.
- Derived data results from a recorded calculation or transformation.
- User-supplied data was entered or uploaded by a user and retains that origin.
These categories are not rankings. They answer different questions and must not be collapsed.
Spatial support and uncertainty¶
A coordinate can intersect a polygon without identifying the actual soil at that point. A raster pixel is an area estimate, not a pinpoint measurement. A laboratory pedon is precise at its sampled location but may not represent a field. OpenSoil retains spatial resolution or accuracy when a provider exposes it and returns null when it does not.
Reliability, licensing, and attribution¶
Government and research services can be interrupted, change schemas, or impose distribution terms. Provider adapters use timeouts, a limited retry, bounded caches, and structured errors. Source-data licensing and citation remain provider-specific; the OpenSoil code license does not overwrite them.
What normalization can and cannot do¶
Normalization can make fields predictable, record unit conversions, and attach provenance. It cannot make unlike methods scientifically equivalent, recover missing uncertainty, or turn representative survey values into laboratory observations. OpenSoil surfaces those limits in every relevant response.