PRISM: Predictive Recomposition via Semantic Latent Decomposition for View-invariant Video Representation Learning

2026-08-31

AI Summary

为解决跨视角视频表征学习中视点不变与视点变化语义纠缠问题，提出PRISM方法，通过语义潜分解及重组实现更有效的视点不变视频特征学习。

Abstract

Cross-view video representation learning aims to capture viewpoint-invariant action semantics despite substantial appearance changes across egocentric and exocentric videos. However, existing methods encode each video as a unified embedding, where view-invariant and view-variant semantics inevitably entangle under co-occurrences - a failure mode we show persists even in cross-view methods explicitly trained for view-invariance. Our key insight is that a view-invariant feature is truly disentangled when it can be sufficiently recomposed with an arbitrary view-variant feature while preserving their independent semantics. Building on this, we propose PRISM, that decomposes video into view-invariant and view-variant latents and recompose them under language supervision encouraging clean decomposition of the two streams. PRISM achieves state-of-the-art results on EgoExo4D, EgoExoLearn, AE2, even surpassing in-domain models under zero-shot setting. Code is available at https://github.com/litcoderr/prism.

Problem

• Cross-view video representation
• view-invariant action semantics
• view-variant semantics

Innovation

• view-invariant
• semantic latent decomposition
• recomposition
• language supervision
• cross-view video representation