Learning Energy-Based Models from Stochastic Interpolants using Spatiotemporal Differences

2026-05-26

AI Summary

Existing energy-based modeling approaches based solely on spatial or temporal differences each suffer from inherent limitations. This work proposes a spatiotemporal joint differencing framework that constructs a unified spatiotemporal density via random interpolation and introduces spatiotemporal Noise Contrastive Estimation (stNCE) as a cohesive training objective. This formulation effectively integrates and surpasses prior methods by jointly leveraging spatial and temporal information. Empirical evaluations demonstrate that the proposed approach achieves density estimation performance on par with state-of-the-art models across both image and molecular datasets, thereby confirming its effectiveness and broad applicability.

Abstract

Learning an energy-based model from data samples is a central problem in machine learning. Many recent and popular methods, such as denoising score matching for training energy-based diffusion models, use stochastic interpolants to corrupt data samples at different noise levels indexed by a time variable. This defines a joint density over both the data space and time, and most methods learn its energy through either spatial or temporal differences. We identify distinct failure modes for both of these approaches. To solve them, we propose Spatiotemporal Noise-Contrastive Estimation (stNCE), a framework for learning the energy through joint spatiotemporal differences. stNCE unifies many existing methods and leads to new training objectives. Experiments on images and molecules demonstrate performance competitive with state-of-the-art density estimation methods.

Problem

• energy-based models
• stochastic interpolants
• spatiotemporal differences
• denoising score matching
• density estimation

Innovation

• energy-based models
• stochastic interpolants
• spatiotemporal differences
• noise-contrastive estimation
• denoising score matching