🎓 Lesson 21
D5
Troubleshooting Failed Predictions: Diagnosing Model-Data Discrepancies
When a geochemical model predicts something different from what actually happens in the field, this lesson teaches you how to figure out why—and fix it.
🎯 Learning Objectives
- ✓ Analyze discrepancies between modeled and measured pore water chemistry using residual plots and error metrics (e.g., RMSE, % bias)
- ✓ Explain how incorrect mineralogical assumptions or kinetic constraints lead to systematic prediction failures in acid rock drainage (ARD) modeling
- ✓ Apply sensitivity analysis to identify which input parameters (e.g., pyrite content, O₂ diffusion rate, carbonate buffering) most strongly influence model output uncertainty
- ✓ Design targeted field or lab experiments (e.g., humidity cell tests, sequential extraction) to test hypothesized causes of model failure
- ✓ Evaluate model performance against industry validation benchmarks (e.g., EPA MM21, ICMM Good Practice Guidance)
📖 Why This Matters
In mine waste management, a failed prediction—like underestimating sulfate release or overpredicting neutralization capacity—can lead to unanticipated environmental impacts, regulatory non-compliance, or costly remediation retrofits. Real-world cases (e.g., Mount Polley tailings failure follow-up, Chuquicamata ARD misprediction) show that 60–70% of early-stage geochemical model failures stem not from software flaws, but from undiagnosed data-model mismatches. Learning to troubleshoot these discrepancies is essential for building defensible, auditable, and adaptive waste characterization strategies.
📘 Core Principles
Diagnosing model-data discrepancies follows a hierarchical logic: (1) Verify data integrity—check for analytical outliers, detection limit handling, and temporal/spatial representativeness; (2) Audit model structure—assess whether the conceptual model includes all dominant controls (e.g., O₂ ingress pathways, reactive surface area, microbial kinetics); (3) Evaluate parameterization—determine if inputs (e.g., mineral mass fractions, rate constants) are derived from appropriate methods (XRD vs. Leco S, batch vs. column kinetic tests); (4) Test boundary and initial conditions—confirm they reflect actual hydrological and geochemical drivers (e.g., infiltration rate vs. evaporation, redox zonation). Each layer must be ruled out before attributing failure to 'model limitations'—a common diagnostic shortcut that masks avoidable errors.
📐 Normalized Prediction Error (NPE)
NPE quantifies the relative magnitude and direction of model-data mismatch per observation, enabling pattern recognition across species and time. It supports root-cause clustering (e.g., consistent overprediction of Fe²⁺ suggests missing oxidation kinetics; underprediction of SO₄²⁻ points to incomplete sulfide inventory). Use NPE before applying global metrics like RMSE to avoid masking systematic biases.
Normalized Prediction Error (NPE)
NPE_i = (C_{pred,i} - C_{meas,i}) / C_{meas,i}Quantifies relative model over- or under-prediction for individual species/timepoints to reveal systematic bias.
Variables:
| Symbol | Name | Unit | Description |
|---|---|---|---|
| C_{pred,i} | Predicted concentration | mg/L or mol/kgw | Model output for species i at time/depth point |
| C_{meas,i} | Measured concentration | mg/L or mol/kgw | Laboratory-analyzed value with documented QA/QC |
Typical Ranges:
Validated ARD models: -0.5 to +0.5
Early-stage screening models: -3.0 to +5.0
💡 Worked Example
Problem: A PHREEQC model predicts dissolved Cu = 1.8 mg/L at month 12 of a humidity cell test; measured value = 0.45 mg/L. Detection limit = 0.02 mg/L.
1.
Step 1: Confirm measurement is above detection limit (0.45 > 0.02 → valid)
2.
Step 2: Compute NPE = (Predicted − Measured) / Measured = (1.8 − 0.45) / 0.45 = 3.0
3.
Step 3: Interpret: NPE = +3.0 indicates 300% overprediction — strongly suggests missing sorption or precipitation sink (e.g., tenorite or Cu-Cl mineral phases not included in phase database)
Answer:
The result is NPE = +3.0, which exceeds the action threshold of |±1.5| defined in ICMM 2021 Validation Protocol, triggering review of secondary mineral controls.
🏗️ Real-World Application
At the Antamina mine (Peru), initial PHREEQC simulations predicted rapid neutralization of ARD within 3 years due to assumed calcite abundance. Field pore water data showed persistently low pH (>5 years). Discrepancy diagnosis revealed: (1) XRD underestimated reactive carbonate by 70% due to fine-grained dolomite intergrowths; (2) model used equilibrium dissolution—ignoring slow dolomite kinetics; (3) no O₂ diffusion limitation was imposed despite saturated, clay-rich cover. Revised model incorporated kinetic dolomite dissolution (k = 1.2×10⁻¹³ mol/m²·s) and diffusion-limited O₂ transport, reducing prediction error from RMSE = 1.9 pH units to 0.3 pH units.
🔧 Interactive Calculator
🔧 Open Mine Waste Characterization & Geochemical Modeling Calculator📋 Case Connection
📋 Copper Mine Waste Rock Stockpile ARD Mitigation at Escondida Extension
High-pyrite waste rock (>3.2% S) stockpiled without cover; predicted ARD onset within 5 years
📋 Gold Tailings Geochemical Stabilization at Granny Smith Mine (WA)
Arsenic-rich tailings (up to 120 mg/kg As) exhibiting elevated As leaching under oxidizing conditions
📋 Limestone Mine Neutral Drainage Management at Mount Read Complex (Tasmania)
Historic waste dumps containing carbonate-hosted Pb-Zn mineralization generating neutral metal leachate (Zn >15 mg/L, Cd...
📋 Iron Ore Mine Waste Rock Long-Term Stability at Brockman 4 (Pilbara)
Massive hematite-goethite waste rock (low sulfide but high Mn/Al) showing delayed acidity and Al leaching post-construct...
📋 Coal Mine Spoil Geochemical Capping at Hunter Valley Reclamation Project
Spoil with pyritic shale interbeds generating ARD despite initial alkaline overburden; inconsistent capping led to local...