Conditional Temporal Neural Processes with Covariance Loss
Neural processes often fail to accurately model dependencies between inputs and targets under noisy observations. To address this, we propose the Covariance Loss—a novel objective that explicitly incorporates second-order statistical dependencies among target variables into the end-to-end training of conditional neural processes for the first time. By regularizing the covariance structure of the predictive distribution, our loss enhances the model’s ability to recover missing or degraded dependencies and improves robustness to observation noise. The method is architecture-agnostic and can be seamlessly integrated into mainstream neural process frameworks. Extensive experiments across multiple real-world time-series and regression benchmarks demonstrate consistent and significant improvements over state-of-the-art methods in three key aspects: predictive accuracy, fidelity of dependency structure recovery, and robustness to observational noise.