Efficient Decoder Scaling Strategy for Neural Routing Solvers

2026-02-27

AI Summary

This study investigates the impact of decoder scaling strategies—specifically depth versus width expansion—on the performance of neural routing solvers. Building upon an encoder-decoder architecture, the authors systematically construct twelve models ranging from 1M to 150M parameters and evaluate their efficacy on vehicle routing problems across three dimensions: parameter efficiency, data efficiency, and computational efficiency. The findings reveal that model performance cannot be reliably predicted by parameter count alone, with depth expansion consistently outperforming width expansion. Based on these insights, the work proposes a “depth-first” design principle for decoders, which significantly enhances both solution quality and resource utilization efficiency in neural combinatorial optimization.

Technology Category

• Search and Optimization: Combinatorial Optimization
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling
• Constraint Satisfaction and Optimization: Search

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Construction-based neural routing solvers, typically composed of an encoder and a decoder, have emerged as a promising approach for solving vehicle routing problems. While recent studies suggest that shifting parameters from the encoder to the decoder enhances performance, most works restrict the decoder size to 1-3M parameters, leaving the effects of scaling largely unexplored. To address this gap, we conduct a systematic study comparing two distinct strategies: scaling depth versus scaling width. We synthesize these strategies to construct a suite of 12 model configurations, spanning a parameter range from 1M to ~150M, and extensively evaluate their scaling behaviors across three critical dimensions: parameter efficiency, data efficiency, and compute efficiency. Our empirical results reveal that parameter count is insufficient to accurately predict the model performance, highlighting the critical and distinct roles of model depth (layer count) and width (embedding dimension). Crucially, we demonstrate that scaling depth yields superior performance gains to scaling width. Based on these findings, we provide and experimentally validate a set of design principles for the efficient allocation of parameters and compute resources to enhance the model performance.

Problem

• neural routing solvers
• decoder scaling
• model depth
• model width
• parameter efficiency

Innovation

• decoder scaling
• neural routing solvers
• model depth vs width
• parameter efficiency
• vehicle routing problem