Bundle Network: a Machine Learning-Based Bundle Method

2025-09-29

AI Summary

Traditional bundle methods for convex nonsmooth optimization suffer from low efficiency and poor generalizability due to manual parameter tuning and iterative subproblem solving for search directions. Method: This paper proposes the first end-to-end trainable neural optimizer that integrates bundle methods with deep learning: a recurrent neural network with attention replaces the iterative subroutine, enabling data-driven automatic learning of regularization parameters; unfolded computational graphs and automatic differentiation facilitate gradient backpropagation through the entire optimization process. Contribution/Results: The method eliminates grid search and human intervention entirely. Evaluated on Lagrangian dual relaxation tasks for multicommodity network design and generalized assignment problems, it significantly outperforms conventional bundle methods in both solution quality and convergence speed, while demonstrating strong cross-dataset generalization capability.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Optimization
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This paper presents Bundle Network, a learning-based algorithm inspired by the Bundle Method for convex non-smooth minimization problems. Unlike classical approaches that rely on heuristic tuning of a regularization parameter, our method automatically learns to adjust it from data. Furthermore, we replace the iterative resolution of the optimization problem that provides the search direction-traditionally computed as a convex combination of gradients at visited points-with a recurrent neural model equipped with an attention mechanism. By leveraging the unrolled graph of computation, our Bundle Network can be trained end-to-end via automatic differentiation. Experiments on Lagrangian dual relaxations of the Multi-Commodity Network Design and Generalized Assignment problems demonstrate that our approach consistently outperforms traditional methods relying on grid search for parameter tuning, while generalizing effectively across datasets.

Problem

• Automatically learns regularization parameters from data
• Replaces iterative optimization with recurrent neural model
• Outperforms traditional methods on network design problems

Innovation

• Learns regularization parameter automatically from data
• Replaces iterative optimization with recurrent neural model
• Uses attention mechanism and end-to-end training