Selective Synergistic Learning for Video Object-Centric Learning

2026-06-13

AI Summary

Existing video object-centric learning methods perform indiscriminate dense alignment between encoder attention maps and decoder object maps, which often introduces noise, blurs object boundaries, and incurs high computational costs. This work proposes Selective Synchronous Learning (SSync), a lightweight and plug-and-play framework that employs selective knowledge distillation to leverage the encoder for refining boundaries and the decoder for denoising internal regions. To avoid exhaustive spatial comparisons, SSync introduces a linear-complexity spatiotemporal activation consistency mechanism for pseudo-label fusion. This approach effectively mitigates the propagation of module-specific weaknesses and slot redundancy, significantly enhancing object decomposition quality and robustness across diverse slot configurations while remaining readily integrable into existing systems.

Abstract

Typical video object-centric learning (VOCL) approaches employ slot-based frameworks that rely on reconstruction-driven encoder-decoder architectures, where learning is mediated by two spatial maps: attention maps from the encoder and object maps from the decoder. As these two distinct maps exhibit different properties, a recent dense alignment strategy attempted to reconcile this discrepancy by enforcing agreement across all spatio-temporal patches via contrastive learning. However, this indiscriminate alignment inadvertently propagates the inherent weaknesses of each module, such as noisy encoder predictions and blurred decoder boundaries. Moreover, computing dense similarities across all pairs incurs a computational cost quadratic in the total number of spatio-temporal patches, severely limiting scalability. Motivated by this, we propose Selective Synergistic Learning (SSync). Instead of exhaustive patch-to-patch alignment, SSync prevents error propagation by selectively distilling only the most reliable cues: leveraging the encoder strictly for boundary refinement and the decoder for interior denoising. This is realized via a pseudo-labeling with linear complexity, eliminating the need for quadratic spatial comparisons. Also, to prevent the reinforcement of architectural biases like slot redundancy, we introduce a transitive pseudo-label merging that consolidates overlapping slots based on spatio-temporal activation consistency. Extensive studies demonstrate that SSync improves decomposition quality and serves as a versatile, plug-and-play module while also exhibiting exceptional robustness to slot configurations. Code is available at github.com/wjun0830/SSync.

Problem

• video object-centric learning
• slot-based framework
• dense alignment
• error propagation
• computational scalability

Innovation

• Selective Synergistic Learning
• Video Object-Centric Learning
• Pseudo-labeling
• Slot Redundancy
• Linear Complexity