Constant-Target Energy Matching: A Unified Framework for Continuous and Discrete Density Estimation

2026-05-09

AI Summary

Existing density estimation methods struggle to handle continuous, discrete, and mixed-type variables within a unified framework and often suffer from unstable training objectives in discrete settings. This work proposes a general energy matching framework that optimizes via bounded energy difference transformations and a constant sample-dependent target (fixed at 1), thereby circumventing unbounded ratio regression and partition function estimation. The approach achieves, for the first time, a unified formulation for both continuous and discrete density estimation by leveraging energy-based models and scalar potential functions, significantly enhancing training stability and generalization. Empirical results demonstrate consistent superiority over existing methods across benchmark tasks in diverse data spaces, along with the generation of higher-quality samples.

Abstract

Density estimation is a central primitive in probabilistic modeling, yet continuous, discrete, and mixed-variable domains are often treated by separate objectives, limiting the ability to exploit a common statistical structure across data types. Continuous score-based methods rely on log-density gradients, while discrete extensions typically use concrete score whose unbounded targets become unstable near low-probability states. We introduce Constant-Target Energy Matching (CTEM), a unified energy-based framework for density estimation on general state spaces. CTEM replaces ordinary density-ratio regression with a bounded energy-difference transform and derives from it a sample-only training objective with the constant target 1. The learned scalar potential recovers log p without partition-function estimation or explicit unbounded ratio regression. Across continuous, discrete, and mixed-variable benchmarks, CTEM substantially improves density estimation over competitive baselines and yields higher-quality samples under standard sampling procedures.

Problem

• density estimation
• continuous variables
• discrete variables
• mixed-variable domains
• score-based methods

Innovation

• Constant-Target Energy Matching
• energy-based modeling
• density estimation
• unified framework
• bounded energy-difference