Path-Adaptive Matting for Efficient Inference Under Various Computational Cost Constraints

2025-03-05

AI Summary

To address the challenge of balancing accuracy and efficiency in image matting under diverse FLOP constraints, this paper proposes a novel single-network dynamic path adaptation paradigm. Methodologically, it reformulates matting as a bi-level optimization problem: the upper level optimizes path selection, while the lower level refines pixel-wise alpha predictions. Key contributions include: (1) the first formulation of matting as a bi-level optimization task; (2) a performance-aware online path learning mechanism and learnable connection layers that enable fine-grained, context- and FLOP-budget-driven computational path adaptation; and (3) FLOP-aware training coupled with online path sampling for evaluation. Extensive experiments on five benchmark datasets demonstrate significant improvements in accuracy–efficiency trade-offs across varying FLOP budgets, achieving state-of-the-art cross-constraint generalization performance.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Learning to Search
• Machine Learning: Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

In this paper, we explore a novel image matting task aimed at achieving efficient inference under various computational cost constraints, specifically FLOP limitations, using a single matting network. Existing matting methods which have not explored scalable architectures or path-learning strategies, fail to tackle this challenge. To overcome these limitations, we introduce Path-Adaptive Matting (PAM), a framework that dynamically adjusts network paths based on image contexts and computational cost constraints. We formulate the training of the computational cost-constrained matting network as a bilevel optimization problem, jointly optimizing the matting network and the path estimator. Building on this formalization, we design a path-adaptive matting architecture by incorporating path selection layers and learnable connect layers to estimate optimal paths and perform efficient inference within a unified network. Furthermore, we propose a performance-aware path-learning strategy to generate path labels online by evaluating a few paths sampled from the prior distribution of optimal paths and network estimations, enabling robust and efficient online path learning. Experiments on five image matting datasets demonstrate that the proposed PAM framework achieves competitive performance across a range of computational cost constraints.

Problem

• Efficient image matting under FLOP constraints
• Dynamic network path adjustment for cost constraints
• Online path learning for robust matting performance

Innovation

• Dynamic network path adjustment for matting
• Bilevel optimization for cost-constrained training
• Online path learning with performance awareness