Analytic DAG Constraints for Differentiable DAG Learning

2025-03-24

AI Summary

This work addresses directed acyclic graph (DAG) structure learning from observational data, tackling two core challenges in differentiable DAG optimization: gradient vanishing and the difficulty of enforcing the DAG constraint via differentiable formulations. We establish, for the first time, a theoretical connection between analytic function families and the DAG constraint, proposing a closed class of analytic functions—specifically, positive-coefficient convergent power series—that supports differentiation, addition, and multiplication, thereby enabling fully differentiable DAG regularization. Integrating graph-structured priors with an efficient automatic differentiation scheme, our approach ensures stable gradient propagation and exact DAG constraint satisfaction. Extensive experiments across diverse benchmark tasks demonstrate significant improvements over existing state-of-the-art methods, with superior constraint fidelity, computational efficiency, and robustness. The implementation is publicly available.

Technology Category

• Machine Learning: Structured Learning
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Reasoning under Uncertainty: Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Recovering the underlying Directed Acyclic Graph (DAG) structures from observational data presents a formidable challenge, partly due to the combinatorial nature of the DAG-constrained optimization problem. Recently, researchers have identified gradient vanishing as one of the primary obstacles in differentiable DAG learning and have proposed several DAG constraints to mitigate this issue. By developing the necessary theory to establish a connection between analytic functions and DAG constraints, we demonstrate that analytic functions from the set ${f(x) = c_0 + sum_{i=1}^{infty}c_ix^i | forall i>0, c_i>0; r = lim_{i ightarrow infty}c_{i}/c_{i+1}>0}$ can be employed to formulate effective DAG constraints. Furthermore, we establish that this set of functions is closed under several functional operators, including differentiation, summation, and multiplication. Consequently, these operators can be leveraged to create novel DAG constraints based on existing ones. Using these properties, we design a series of DAG constraints and develop an efficient algorithm to evaluate them. Experiments in various settings demonstrate that our DAG constraints outperform previous state-of-the-art comparators. Our implementation is available at https://github.com/zzhang1987/AnalyticDAGLearning.

Problem

• Overcoming gradient vanishing in differentiable DAG learning
• Connecting analytic functions to DAG constraint formulation
• Developing efficient algorithms for DAG structure recovery

Innovation

• Analytic functions for DAG constraints
• Functional operators create new constraints
• Efficient algorithm evaluates DAG constraints