TSA: Temporal Slot Activation for Persistent Object-Centric Video Representation

2026-06-10

AI Summary

Existing video slot attention methods unconditionally update and decode all slots even when objects are occluded or absent, leading to state drift, reconstruction interference, and degraded temporal consistency. To address this, this work proposes Temporal Slot Activation (TSA), the first unsupervised, slot-level activation mechanism that gates slot state updates and decoding participation via per-frame learned activation scores. TSA further incorporates activation-dependent attention logit biases and temporal memory to effectively suppress interference from invisible objects. Built upon standard slot attention architectures, TSA significantly improves object decomposition and temporal identity preservation on MOVi-C/E, YT-VIS, and OVIS benchmarks, achieving substantial gains—particularly in long-duration sequences with severe occlusions.

Abstract

Unsupervised video object-centric learning aims to decompose dynamic scenes into temporally persistent entity representations. Existing recurrent video slot-attention methods propagate a fixed set of slots across frames, but typically assume unconditional slot propagation: every slot is updated and decoded at every frame, regardless of whether its corresponding object is visible. We show that this design violates a basic lifecycle requirement for persistent slots: when an object is absent or fully occluded, its slot should preserve its previous state and avoid explaining unrelated visible content. Instead, unconditional propagation creates two failure pathways: update-induced state drift, where current-frame evidence overwrites the absent object's representation, and decoder-induced reconstruction interference, where the inactive slot remains coupled to reconstruction through decoder attention. We propose Temporal Slot Activation (TSA), a mechanism that learns a per-slot, per-frame activation score $α_{k,t} \in (0, 1)$ without visibility supervision. TSA uses this activation as a shared latent control variable for slot lifecycle modeling. When a slot is inactive, TSA anchors its state to the previous slot via activation-gated updating and suppresses its decoder participation through an activation-dependent additive bias on attention logits before softmax normalization. This jointly reduces state drift and reconstruction-driven interference. To improve decisions under partial occlusion and gradual reappearance, TSA further conditions activation prediction on a per-slot temporal memory produced by a Temporal Context Encoder. We evaluate TSA on MOVi-C/E, YT-VIS, and OVIS benchmarks using both standard and tracking-based metrics (FG-ARI, mBO, IDF1, HOTA). TSA consistently improves object decomposition and temporal identity preservation, with large gains on long, heavily occluded videos.

Problem

• object-centric learning
• temporal persistence
• slot attention
• occlusion handling
• unsupervised video decomposition

Innovation

• Temporal Slot Activation
• object-centric learning
• slot attention
• temporal persistence
• unsupervised video decomposition