In Search of Trees: Decision-Tree Policy Synthesis for Black-Box Systems via Search

2024-09-05

arXiv.org

AI Summary

Synthesizing optimal, interpretable controllers for black-box dynamical systems—without access to analytical models or expert demonstrations—remains a fundamental challenge. Method: This paper proposes a model- and policy-agnostic decision-tree controller synthesis method. Given input-output trajectory data, a discrete predicate set, and an initial state set, it employs a trajectory-driven pruning search algorithm to construct decision trees that are provably step-optimal and structurally minimal in the black-box setting. Key technical components include predicate-space discretization, combinatorial trajectory evaluation, domain-specific pruning rules, and black-box interaction-based verification. Results: Evaluated on multiple control benchmarks, the synthesized policies are compact, formally verifiable, and achieve 10–100× speedup in synthesis time over MILP-based and reinforcement learning baselines. The approach overcomes longstanding limitations of white-box modeling assumptions and imitation learning dependencies, enabling scalable, certifiable control synthesis for unknown dynamics.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Decision trees, owing to their interpretability, are attractive as control policies for (dynamical) systems. Unfortunately, constructing, or synthesising, such policies is a challenging task. Previous approaches do so by imitating a neural-network policy, approximating a tabular policy obtained via formal synthesis, employing reinforcement learning, or modelling the problem as a mixed-integer linear program. However, these works may require access to a hard-to-obtain accurate policy or a formal model of the environment (within reach of formal synthesis), and may not provide guarantees on the quality or size of the final tree policy. In contrast, we present an approach to synthesise optimal decision-tree policies given a deterministic black-box environment and specification, a discretisation of the tree predicates, and an initial set of states, where optimality is defined with respect to the number of steps to achieve the goal. Our approach is a specialised search algorithm which systematically explores the (exponentially large) space of decision trees under the given discretisation. The key component is a novel trace-based pruning mechanism that significantly reduces the search space. Our approach represents a conceptually novel way of synthesising small decision-tree policies with optimality guarantees even for black-box environments with black-box specifications.

Problem

• Black-box Systems
• Decision Tree Optimization
• Dynamic System Control

Innovation

• Black-box Systems
• Optimal Decision Tree Generation
• Efficient Search Algorithm