Computation of Least Trimmed Squares: A Branch-and-Bound framework with Hyperplane Arrangement Enhancements

2026-04-13

AI Summary

This work addresses the NP-hard problem of penalized least trimmed squares (LTS) regression in robust statistics, for which existing mixed-integer optimization approaches struggle to scale to large datasets. The authors propose a novel formulation that explicitly embeds the arrangement structure of hyperplanes into a perspective reformulation and develop a tailored branch-and-bound algorithm that leverages first-order methods to efficiently solve node relaxations. When the feature dimension is fixed, the method guarantees that the size of the branch-and-bound tree grows polynomially with the sample size, substantially improving both theoretical and practical scalability. Experiments on synthetic data with 5,000 samples and 20 features demonstrate that the proposed approach achieves a 1% optimality gap within one minute, whereas current methods fail to converge within an hour, thereby significantly expanding the tractable scale of exact robust regression.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

We study computational aspects of a key problem in robust statistics -- the penalized least trimmed squares (LTS) regression problem, a robust estimator that mitigates the influence of outliers in data by capping residuals with large magnitudes. Although statistically attractive, penalized LTS is NP-hard, and existing mixed-integer optimization (MIO) formulations scale poorly due to weak relaxations and exponential worst-case complexity in the number of observations. We propose a new MIO formulation that embeds hyperplane arrangement logic into a perspective reformulation, explicitly enforcing structural properties of optimal solutions. We show that, if the number of features is fixed, the resulting branch-and-bound tree is of polynomial size in the sample size. Moreover, we develop a tailored branch-and-bound algorithm that uses first-order methods with dual bounds to solve node relaxations efficiently. Computational experiments on synthetic and real datasets demonstrate substantial improvements over existing MIO approaches: on synthetic instances with 5000 samples and 20 features, our tailored solver reaches a 1% gap in 1 minute while competing approaches fail to do so within one hour. These gains enable exact robust regression at significantly larger sample sizes in low-dimensional settings.

Problem

• Least Trimmed Squares
• Robust Regression
• Outlier Detection
• NP-hard Optimization
• Mixed-Integer Optimization

Innovation

• Least Trimmed Squares
• Branch-and-Bound
• Hyperplane Arrangement
• Mixed-Integer Optimization
• Robust Regression