Actively Learning Combinatorial Optimization Using a Membership Oracle

2024-05-23

arXiv.org

AI Summary

This work addresses combinatorial optimization problems subject to unknown linear constraints. We propose an active learning framework for feasible region approximation that operates under a limited membership oracle query budget. Instead of explicitly modeling constraints, our method employs a mixed-integer quadratic programming (MIQP)-driven optimal sampling strategy, jointly leveraging support vector machines (SVMs) and a convex-optimization-inspired linear separation mechanism to efficiently identify the feasible boundary. Compared to conventional SVM-based margin sampling, our approach significantly improves both query efficiency and boundary estimation accuracy. Experiments on knapsack and university course scheduling problems demonstrate accelerated objective convergence—by 37%–62%—and an average 12.4% improvement in final solution quality. The core contribution lies in integrating MIQP into the active learning loop, yielding a theoretically interpretable and computationally tractable paradigm for constraint discovery.

Technology Category

• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Machine Learning: Active Learning
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

We consider solving a combinatorial optimization problem with an unknown linear constraint using a membership oracle that, given a solution, determines whether it is feasible or infeasible with absolute certainty. The goal of the decision maker is to find the best possible solution subject to a budget on the number of oracle calls. Inspired by active learning based on Support Vector Machines (SVMs), we adapt a classical framework in order to solve the problem by learning and exploiting a surrogate linear constraint. The resulting new framework includes training a linear separator on the labeled points and selecting new points to be labeled, which is achieved by applying a sampling strategy and solving a 0-1 integer linear program. Following the active learning literature, one can consider using SVM as a linear classifier and the information-based sampling strategy known as simple margin. We improve on both sides: we propose an alternative sampling strategy based on mixed-integer quadratic programming and a linear separation method inspired by an algorithm for convex optimization in the oracle model. We conduct experiments on the pure knapsack problem and on a college study plan problem from the literature to show how different linear separation methods and sampling strategies influence the quality of the results in terms of objective value.

Problem

• Support Vector Machines
• Linear Rules
• Oracle Queries

Innovation

• Support Vector Machines
• Membership Oracle
• Mathematical Programming