Extremely Greedy Equivalence Search

2025-02-26

AI Summary

GES suffers from premature convergence to local optima under limited data, particularly in learning dense DAGs. To address this, we propose XGES—a novel algorithm that introduces the “early edge removal, late edge addition” heuristic within the established equivalence-class–based greedy search framework of GES, while strictly preserving the BIC scoring criterion. This strategy effectively mitigates local optimality issues without compromising theoretical consistency. Furthermore, we design an efficient hybrid implementation leveraging both C++ and Python, tightly integrating DAG acyclicity constraints into the optimization routine. Extensive experiments on standard synthetic benchmarks demonstrate that XGES consistently achieves higher structural recovery accuracy than GES, with an average runtime speedup of approximately 10×. The source code is publicly available.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Machine Learning: Graph-based Machine Learning

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

The goal of causal discovery is to learn a directed acyclic graph from data. One of the most well-known methods for this problem is Greedy Equivalence Search (GES). GES searches for the graph by incrementally and greedily adding or removing edges to maximize a model selection criterion. It has strong theoretical guarantees on infinite data but can fail in practice on finite data. In this paper, we first identify some of the causes of GES's failure, finding that it can get blocked in local optima, especially in denser graphs. We then propose eXtremely Greedy Equivalent Search (XGES), which involves a new heuristic to improve the search strategy of GES while retaining its theoretical guarantees. In particular, XGES favors deleting edges early in the search over inserting edges, which reduces the possibility of the search ending in local optima. A further contribution of this work is an efficient algorithmic formulation of XGES (and GES). We benchmark XGES on simulated datasets with known ground truth. We find that XGES consistently outperforms GES in recovering the correct graphs, and it is 10 times faster. XGES implementations in Python and C++ are available at https://github.com/ANazaret/XGES.

Problem

• Improves GES's local optima issue
• Proposes XGES for better graph recovery
• Enhances search efficiency and speed

Innovation

• Enhanced GES heuristic
• Edge deletion prioritization
• Efficient algorithmic formulation