🎓 Lesson 16
D5
Cross-Validation Strategies for Spatial ML Models
Cross-validation is a way to test how well a machine learning model predicts ore grades at unseen locations by repeatedly splitting real drillhole data into training and testing parts.
🎯 Learning Objectives
- ✓ Explain why standard k-fold cross-validation fails for spatial data and how spatial blocking addresses this
- ✓ Design a spatially aware cross-validation scheme (e.g., block, leave-one-group-out) for a 3D drillhole dataset
- ✓ Analyze cross-validation metrics (RMSE, MAE, R²) to diagnose model overfitting or spatial bias
- ✓ Apply Moran’s I to quantify residual spatial autocorrelation in cross-validated predictions
📖 Why This Matters
In grade control, deploying an ML model that looks accurate on paper—but fails at the next blast hole—can cost millions in misclassified ore/waste, dilution, or missed high-grade zones. Standard validation assumes data points are independent; but drillholes are spatially clustered and geologically correlated. Using naive cross-validation leads to inflated confidence and catastrophic field failures. This lesson teaches you how to *trust* your model—not just because it fits well, but because it generalizes reliably across space.
📘 Core Principles
Spatial cross-validation departs from i.i.d. assumptions by enforcing spatial separation between training and test sets. Key strategies include: (1) Block CV—partitioning the domain into non-overlapping geographic blocks (e.g., 50 × 50 m grid cells), holding out entire blocks; (2) Leave-One-Group-Out (LOGO)—holding out all samples from a single drillhole or collar location; and (3) Spatially constrained k-fold—using distance-based clustering (e.g., DBSCAN) to group proximal samples before folding. Each method controls for spatial autocorrelation, ensuring test points are genuinely independent in a geological sense—not just statistically shuffled. Critically, the choice affects bias-variance tradeoff: smaller blocks increase variance but reduce leakage; larger blocks improve stability but risk underestimating error if geological domains are undersampled.
📐 Moran’s I for Residual Autocorrelation
After cross-validation, residuals (observed − predicted grades) must be tested for spatial structure. Significant positive Moran’s I indicates systematic spatial error—e.g., model smoothing out high-grade pockets—revealing structural model inadequacy, not just noise.
Moran’s I Statistic
I = \frac{n}{\sum_{i}\sum_{j} w_{ij}} \cdot \frac{\sum_{i}\sum_{j} w_{ij} e_i e_j}{\sum_i e_i^2}Measures global spatial autocorrelation in cross-validation residuals; values near 0 indicate no spatial pattern, >0 indicate clustering, <0 indicate dispersion.
Variables:
| Symbol | Name | Unit | Description |
|---|---|---|---|
| n | Number of samples | dimensionless | Total count of cross-validated drillhole composites or assays |
| w_ij | Spatial weight | dimensionless | Row-standardized weight between sample i and j (e.g., inverse distance or binary adjacency) |
| e_i | Residual | g/t or % | Observed grade minus predicted grade at location i |
Typical Ranges:
Well-calibrated spatial ML model: -0.1 to +0.1
Severe spatial leakage: +0.25 to +0.6
💡 Worked Example
Problem: Given 120 cross-validated drillhole residuals (g/t Au) across a 400 × 300 m pit area, with row-standardized spatial weights matrix W (based on inverse distance < 25 m), ΣΣw_ij = 1842, Σe_i = 0, Σe_i² = 362.4, and ΣΣw_ij e_i e_j = 47.9.
1.
Step 1: Compute mean-centered residuals — already satisfied (Σe_i = 0).
2.
Step 2: Plug into Moran’s I formula: I = (n / ΣΣw_ij) × (ΣΣw_ij e_i e_j / Σe_i²).
3.
Step 3: Substitute values: I = (120 / 1842) × (47.9 / 362.4) = 0.065 × 0.132 ≈ 0.0086.
Answer:
The result is I = 0.0086, which is near zero and not statistically significant (p > 0.1), indicating no detectable residual spatial autocorrelation — a sign the model captures spatial structure adequately.
🏗️ Real-World Application
At the Red Lake Mine (Ontario), a Random Forest model for gold grade prediction initially achieved R² = 0.84 using standard 5-fold CV. However, block CV (50 × 50 m blocks) dropped R² to 0.61—and revealed systematic underprediction in NW fault-bounded zones. Re-training with lithology-stratified blocks and adding structural orientation features improved block-CV R² to 0.73 and reduced RMSE from 1.8 to 1.1 g/t Au. The model was then deployed only after passing LOGO-CV on 3 newly drilled validation holes—confirming <15% grade misclassification at cutoff (0.5 g/t Au).
🔧 Interactive Calculator
🔧 Open AI-Powered Orebody Delineation & Grade Control Calculator📋 Case Connection
📋 Copper Mine Block Model Refinement Using Neural Kriging
Traditional kriging over-smoothed high-grade chalcocite zones, causing 8.2% reserve underestimation
📋 Gold Mine Real-Time Grade Control at Development Drift Face
Manual chip sampling caused 24–48 hr delay in stope boundary decisions, leading to 14% dilution
📋 Limestone Mine Digital Twin for Karst-Related Grade Uncertainty
Solution cavities caused unpredictable grade drops (CaCO₃ purity <85%) in otherwise uniform deposits, resulting in 11% p...