🤖 AI Summary
This work addresses unsupervised multi-object motion segmentation: pixel-wise, real-time separation of dynamic objects in images without semantic annotations, prior knowledge of object count or location. We propose a conditional adversarial Slot Attention framework grounded in optical flow reconstruction—using the input image as a condition and modeling only motion dynamics to avoid redundant appearance encoding. A novel context-separation adversarial loss explicitly enforces disjoint attention slots for distinct objects. The architecture supports dynamic slot cardinality and variable-resolution processing, eliminating the need for regularization or label-based slot ordering. Our method integrates an enhanced Slot Attention mechanism, optical-flow-guided representation learning, and a min-max optimization strategy. On multiple benchmarks, it achieves performance close to fully supervised methods (within 12% gap), operates at 104 FPS inference speed, and demonstrates strong generalization without fine-tuning.
📝 Abstract
We introduce a method to segment the visual field into independently moving regions, trained with no ground truth or supervision. It consists of an adversarial conditional encoder-decoder architecture based on Slot Attention, modified to use the image as context to decode optical flow without attempting to reconstruct the image itself. In the resulting multi-modal representation, one modality (flow) feeds the encoder to produce separate latent codes (slots), whereas the other modality (image) conditions the decoder to generate the first (flow) from the slots. This design frees the representation from having to encode complex nuisance variability in the image due to, for instance, illumination and reflectance properties of the scene. Since customary autoencoding based on minimizing the reconstruction error does not preclude the entire flow from being encoded into a single slot, we modify the loss to an adversarial criterion based on Contextual Information Separation. The resulting min-max optimization fosters the separation of objects and their assignment to different attention slots, leading to Divided Attention, or DivA. DivA outperforms recent unsupervised multi-object motion segmentation methods while tripling run-time speed up to 104FPS and reducing the performance gap from supervised methods to 12% or less. DivA can handle different numbers of objects and different image sizes at training and test time, is invariant to permutation of object labels, and does not require explicit regularization.
