Problem
Research questions and friction points this paper is trying to address.
event cameras
event representation learning
large-scale annotated event data
modality discrepancies
semantic collapse
Hyper-RED: Scalable Event Pre-training via Semantic Hypergraph Distillation
🤖 AI Summary
为解决事件数据标注不足的问题，Hyper-RED通过使用语义超图蒸馏方法从图像到事件传递高阶语义结构，实现跨模态知识迁移。
📝 Abstract
Event cameras have shown great potential for robust visual perception, yet scaling event representation learning remains challenging due to the scarcity of large-scale annotated event data. Pretrained image models provide scalable semantic supervision, but existing image-to-event methods rely on rigid pixel-wise or token-wise alignment that overlooks modality discrepancies in texture, density, and appearance, potentially causing semantic collapse and limiting transferability. To address this issue, we propose Hyper-RED, a simple, painless, and scalable image-to-event pretraining framework that transfers high-order semantic structures from images to events. Hyper-RED uses hypergraphs to model and align high-order semantic associations among multiple image and event tokens, enabling cross-modal knowledge transfer while accommodating modality-specific differences rather than enforcing rigid one-to-one correspondence. Specifically, given a paired event--image sample, Hyper-RED leverages DINOv3 to extract spatial token representations and constructs image, event, and cross-modal semantic hypergraphs, where each hyperedge connects multiple semantically correlated tokens. We further introduce a hypergraph relational distillation loss that imposes complementary intra- and cross-modal constraints, enabling the event encoder to inherit image-derived semantic organization while preserving local relational consistency and event-specific characteristics. Experiments on three tasks across five event datasets demonstrate consistent scaling from ViT-S to ViT-L and state-of-the-art performance (Fig.1). The code is available at: https://github.com/meisenwang/Hyper--RED.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Hyper-RED
Semantic Hypergraph Distillation
Cross-modal Knowledge Transfer
Event Representation Learning
Hypergraph Relational Distillation
🔎 Similar Papers
No similar papers found.
