Sequential keypoint density estimator: an overlooked baseline of skeleton-based video anomaly detection

2025-06-23

AI Summary

This paper addresses pose-based anomaly detection in safety-critical scenarios (e.g., healthcare monitoring, industrial safety). We propose SeeKer—the first method to model skeletal sequences via autoregressive density estimation at the keypoint level. Its core innovations are: (i) causal Gaussian prediction for pose-level anomaly discrimination, and (ii) a detection-confidence-weighted log-likelihood scoring mechanism that significantly enhances localization robustness. SeeKer reveals the strong baseline potential of lightweight generative models for this task. Experiments demonstrate that SeeKer achieves state-of-the-art performance across UBnormal and MSAD-HR benchmarks, while attaining competitive results on ShanghaiTech—validating both the effectiveness and generalizability of keypoint-level density modeling.

Technology Category

• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Biometrics, Face, Gesture & Pose

Application Category

• Security and Privacy: Large-scale security measurements
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Detecting anomalous human behaviour is an important visual task in safety-critical applications such as healthcare monitoring, workplace safety, or public surveillance. In these contexts, abnormalities are often reflected with unusual human poses. Thus, we propose SeeKer, a method for detecting anomalies in sequences of human skeletons. Our method formulates the skeleton sequence density through autoregressive factorization at the keypoint level. The corresponding conditional distributions represent probable keypoint locations given prior skeletal motion. We formulate the joint distribution of the considered skeleton as causal prediction of conditional Gaussians across its constituent keypoints. A skeleton is flagged as anomalous if its keypoint locations surprise our model (i.e. receive a low density). In practice, our anomaly score is a weighted sum of per-keypoint log-conditionals, where the weights account for the confidence of the underlying keypoint detector. Despite its conceptual simplicity, SeeKer surpasses all previous methods on the UBnormal and MSAD-HR datasets while delivering competitive performance on the ShanghaiTech dataset.

Problem

• Detects anomalous human poses in video sequences
• Estimates skeleton sequence density via keypoint autoregression
• Flags skeletons with low-density keypoints as anomalies

Innovation

• Autoregressive factorization at keypoint level
• Causal prediction of conditional Gaussians
• Weighted sum of per-keypoint log-conditionals