Martingale-Consistent Self-Supervised Learning

2026-05-12

AI Summary

This work addresses the instability and poor calibration of representations in self-supervised learning under scenarios where information is progressively revealed, often due to inconsistent predictions across refinement stages. The paper introduces martingale consistency into self-supervised learning for the first time, proposing a novel framework that enforces martingale properties in both prediction and latent spaces. This ensures that coarse-grained predictions remain consistent in expectation with their refined versions, thereby preserving predictive stability during information updates. Unlike conventional invariance-based objectives, the approach permits meaningful prediction evolution while preventing systematic drift. An unbiased two-sample Monte Carlo estimator based on stochastic refinements is devised to implement this constraint. Experiments demonstrate that the framework significantly enhances robustness and calibration across partial observation benchmarks in time-series, tabular, and image domains under semi-supervised and fully unlabeled settings.

Abstract

Self-supervised learning (SSL) is often deployed under changing information, such as shorter histories, missing features, or partially observed images. In these settings, predictions from coarse and refined views should be coherent: before refinement, the coarse-view prediction should match the average prediction expected after refinement. Martingales formalize this coherence principle, but standard SSL objectives do not enforce it. Unlike invariance objectives that pull views together, martingale consistency constrains only the expected refined prediction, allowing predictions to update as information is revealed while preventing systematic drift. We introduce a martingale-consistent SSL framework that closes this gap, with practical prediction- and latent-space variants and an unbiased two-sample Monte Carlo estimator based on stochastic refinement. We evaluate the approach on synthetic and real time-series, tabular, and image benchmarks under partial-observation regimes, in both semi-self-supervised and fully label-free settings. Across these experiments, our framework improves robustness and calibration under partial observation, yielding more stable representations as information is revealed.

Problem

• self-supervised learning
• martingale consistency
• partial observation
• prediction coherence
• representation stability

Innovation

• martingale consistency
• self-supervised learning
• partial observation
• stochastic refinement
• coherent prediction