OnPoint: Offline-to-Online Multi-Level Distillation for Point-Supervised Online Temporal Action Localization

2026-06-30

AI Summary

This work addresses the challenging problem of pointly supervised online temporal action localization, where only sparse point-level annotations are available without full video supervision or dense labels. We formally define this task for the first time and introduce OnPoint, a novel framework that efficiently transfers knowledge from an offline teacher model to an online student model through a three-tier distillation mechanism: pseudo-segment instance distillation, class activation sequence distillation, and prospective window-level distillation. To enhance localization robustness, we further incorporate activeness-guided attention calibration. Extensive experiments demonstrate that OnPoint significantly outperforms strong existing baselines across five benchmark datasets, establishing a new paradigm for streaming action localization under point supervision.

Abstract

Temporal Action Localization (TAL) typically relies on segment annotations or offline access to full videos, limiting scalability and online use. We introduce Point-Supervised Online TAL (POTAL), which localizes actions in streaming videos using only one temporal point per instance. To solve POTAL, we propose OnPoint, an offline-to-online multi-level distillation framework that transfers knowledge from a point-supervised offline teacher to an online student via (i) pseudo-segment instance distillation, (ii) class-activation sequence distillation, and (iii) anticipatory window-level distillation. We further improve robustness by incorporating the original point labels into student training and by refining anchor decoding with actionness-guided attention calibration. Experiments on five datasets show OnPoint consistently outperforms strong baselines, establishing a solid foundation for POTAL.

Problem

• Temporal Action Localization
• Point Supervision
• Online Video
• Streaming Video
• Weak Supervision

Innovation

• point-supervised learning
• online temporal action localization
• multi-level distillation
• actionness-guided attention
• pseudo-segment distillation