Beyond the Performance Illusion: Structure-Aware Stratified Partitioning and Curriculum Distributionally Robust Optimization for Spatially Correlated Domains

2026-07-02

AI Summary

This work addresses the critical issue of data leakage and hidden stratification in spatiotemporal domains—such as aerial surveillance, precision agriculture, and medical imaging—where conventional random data splits lead to distorted model evaluation. To mitigate these problems, the authors propose a unified training and evaluation framework that integrates Structure-Aware Stratified Partitioning (SASP) with Curriculum Distributionally Robust Optimization (CDRO). SASP constructs rigorously separated validation sets based on spatiotemporal structure to minimize leakage while preserving class balance, whereas CDRO enhances model robustness to distributional shifts through curriculum-based optimization. Empirical results across multiple benchmarks demonstrate substantial improvements in both generalization performance and confidence calibration, while also uncovering failure modes previously obscured by standard evaluation protocols.

Abstract

Performance evaluation in AI systems commonly assumes that random dataset splits produce independent and identically distributed (i.i.d.) subsets. We show that this assumption often breaks down in spatiotemporally correlated domains such as aerial surveillance, precision agriculture, and medical imaging, leading to two systematic failures: data leakage, where correlated samples span training and validation splits and inflate performance estimates, and hidden stratification, where errors on minority subpopulations are obscured by aggregate metrics. To address these issues, we propose a unified evaluation and training framework for spatially correlated data. We introduce Structure-Aware Stratified Partitioning (SASP), which constructs validation splits that reduce spatiotemporal leakage while preserving meaningful class balance, and Curriculum Distributionally Robust Optimization (CDRO), a curriculum-based relaxation of distributionally robust training that stabilizes optimization under these stricter splits. Across multiple benchmarks, this combination yields consistently improved generalization, more reliable confidence calibration, and exposes failure modes that remain hidden under conventional random-split evaluation.

Problem

• spatiotemporal correlation
• data leakage
• hidden stratification
• non-i.i.d. data
• performance evaluation

Innovation

• Structure-Aware Stratified Partitioning
• Curriculum Distributionally Robust Optimization
• spatiotemporal correlation
• data leakage
• distributional robustness