Unsupervised Discovery of Long-Term Spatiotemporal Periodic Workflows in Human Activities

2025-11-18

AI Summary

This paper addresses the unsupervised discovery of long-term, low-contrast, structurally complex spatiotemporal periodic workflows in human activities—such as manufacturing assembly, sports motions, and daily tasks. To support this research, we introduce a novel benchmark comprising 580 multimodal sequences, covering three core tasks: periodicity detection, task tracking, and procedural anomaly identification. Methodologically, we propose a training-free, lightweight unsupervised baseline that jointly models spatiotemporal patterns and performs multimodal sequence analysis. We further conduct zero-shot large language model (LLM) comparative experiments to demonstrate their limitations in modeling weak periodic signals. Empirical results show that our baseline significantly outperforms existing unsupervised approaches across all tasks, while achieving deployment efficiency comparable to supervised models—validating both the challenge and practical utility of the proposed benchmark.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Activity and Plan Recognition
• Computer Vision: Video Understanding & Activity Analysis

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Periodic human activities with implicit workflows are common in manufacturing, sports, and daily life. While short-term periodic activities -- characterized by simple structures and high-contrast patterns -- have been widely studied, long-term periodic workflows with low-contrast patterns remain largely underexplored. To bridge this gap, we introduce the first benchmark comprising 580 multimodal human activity sequences featuring long-term periodic workflows. The benchmark supports three evaluation tasks aligned with real-world applications: unsupervised periodic workflow detection, task completion tracking, and procedural anomaly detection. We also propose a lightweight, training-free baseline for modeling diverse periodic workflow patterns. Experiments show that: (i) our benchmark presents significant challenges to both unsupervised periodic detection methods and zero-shot approaches based on powerful large language models (LLMs); (ii) our baseline outperforms competing methods by a substantial margin in all evaluation tasks; and (iii) in real-world applications, our baseline demonstrates deployment advantages on par with traditional supervised workflow detection approaches, eliminating the need for annotation and retraining. Our project page is https://sites.google.com/view/periodicworkflow.

Problem

• Unsupervised discovery of long-term periodic workflows in human activities
• Modeling low-contrast spatiotemporal patterns without annotations
• Detecting procedural anomalies in multimodal activity sequences

Innovation

• Unsupervised multimodal benchmark for long-term periodic workflows
• Training-free baseline modeling diverse periodic workflow patterns
• Outperforms LLM and unsupervised methods without annotation requirements