Problem
Research questions and friction points this paper is trying to address.
Human Activity Recognition
Novelty Adaptation
Robustness
Human Activity Recognition in an Open World
🤖 AI Summary
Open-world human activity recognition (HAR) faces persistent emergence of task-relevant (e.g., novel actions) and task-irrelevant (e.g., environmental shifts, sensor variations, noise) novelties—challenging model robustness and adaptability. Method: We formally define task-relevant/irrelevant novelty in HAR and propose an incremental open-world learning (OWL) protocol. We introduce KOWL-718, a new benchmark built upon Kinetics and Kinetics-AVA, and release the first containerized, reproducible, and extensible OWL evaluation framework. Contribution/Results: Through systematic evaluation of state-of-the-art models under diverse novelty types, we characterize their performance degradation patterns and identify key robustness bottlenecks. Our plug-and-play evaluation toolkit supports open-world analysis over arbitrary data subsets and future updates. This work establishes foundational methodology and benchmarking infrastructure for HAR to operate reliably in dynamic real-world environments.
📝 Abstract
Managing novelty in perception-based human activity recognition (HAR) is critical in realistic settings to improve task performance over time and ensure solution generalization outside of prior seen samples. Novelty manifests in HAR as unseen samples, activities, objects, environments, and sensor changes, among other ways. Novelty may be task-relevant, such as a new class or new features, or task-irrelevant resulting in nuisance novelty, such as never before seen noise, blur, or distorted video recordings. To perform HAR optimally, algorithmic solutions must be tolerant to nuisance novelty, and learn over time in the face of novelty. This paper 1) formalizes the definition of novelty in HAR building upon the prior definition of novelty in classification tasks, 2) proposes an incremental open world learning (OWL) protocol and applies it to the Kinetics datasets to generate a new benchmark KOWL-718, 3) analyzes the performance of current stateof-the-art HAR models when novelty is introduced over time, 4) provides a containerized and packaged pipeline for reproducing the OWL protocol and for modifying for any future updates to Kinetics. The experimental analysis includes an ablation study of how the different models perform under various conditions as annotated by Kinetics-AVA. The code may be used to analyze different annotations and subsets of the Kinetics datasets in an incremental open world fashion, as well as be extended as further updates to Kinetics are released.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Human Activity Recognition
Incremental Open-Set Learning
Kinetics Dataset
🔎 Similar Papers
No similar papers found.
