Extremal Contours: Gradient-driven contours for compact visual attribution

2025-11-03

AI Summary

Dense perturbation masks in visual attribution often suffer from fragmentation and overfitting, necessitating complex post-processing. To address this, we propose a training-free attribution method that replaces dense masks with smooth, star-shaped convex contours parameterized by Fourier series. These contours are optimized via gradient-based methods to either preserve or remove extremal features, enabling explicit area control and multi-object localization. Our approach is the first to introduce differentiable contour modeling into extremal attribution frameworks. It achieves high fidelity—matching dense-mask performance on ImageNet—while significantly improving inter-run consistency and relevance quality (+15% or more). Notably, it preserves positive fidelity–relevance correlation even on self-supervised models such as DINO. By unifying interpretability, compactness, and stability, our method advances the practicality and reliability of visual explanations.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Transparent, Interpretable, Explainable ML
• Search and Optimization: Non-convex Optimization

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Faithful yet compact explanations for vision models remain a challenge, as commonly used dense perturbation masks are often fragmented and overfitted, needing careful post-processing. Here, we present a training-free explanation method that replaces dense masks with smooth tunable contours. A star-convex region is parameterized by a truncated Fourier series and optimized under an extremal preserve/delete objective using the classifier gradients. The approach guarantees a single, simply connected mask, cuts the number of free parameters by orders of magnitude, and yields stable boundary updates without cleanup. Restricting solutions to low-dimensional, smooth contours makes the method robust to adversarial masking artifacts. On ImageNet classifiers, it matches the extremal fidelity of dense masks while producing compact, interpretable regions with improved run-to-run consistency. Explicit area control also enables importance contour maps, yielding a transparent fidelity-area profiles. Finally, we extend the approach to multi-contour and show how it can localize multiple objects within the same framework. Across benchmarks, the method achieves higher relevance mass and lower complexity than gradient and perturbation based baselines, with especially strong gains on self-supervised DINO models where it improves relevance mass by over 15% and maintains positive faithfulness correlations.

Problem

• Replaces fragmented dense masks with smooth contours for visual explanations
• Optimizes star-convex regions using classifier gradients for compact attribution
• Enables multi-object localization while maintaining high fidelity and consistency

Innovation

• Uses smooth contours instead of dense masks
• Optimizes star-convex regions with Fourier series
• Generates single connected masks with explicit area control