Seeing the Needle in the Haystack: Towards Weakly-Supervised Log Instance Anomaly Localization via Counterfactual Perturbation

2026-05-09

AI Summary

This work addresses the challenge of fine-grained anomaly localization in large-scale network systems, where massive log volumes and the high cost of instance-level annotations hinder effective diagnosis. To overcome this, the authors propose LogMILP, a novel framework that, for the first time, integrates prototype-guided modeling with counterfactual perturbation consistency regularization within a weakly supervised setting. Operating solely on bag-level labels, LogMILP simultaneously achieves high-accuracy bag-level anomaly detection and reliable instance-level localization by leveraging multi-instance learning. The approach substantially enhances both localization accuracy and model interpretability in the absence of instance-level annotations. Experimental results demonstrate that LogMILP delivers competitive detection performance across three public datasets and significantly outperforms existing methods in terms of instance-level localization reliability.

Abstract

Log anomaly detection is a critical task for system operations and security assurance. However, in networked systems at scale, log data are generated at massive scale while instance-level annotations are prohibitively expensive, posing great difficulties to fine-grained anomaly localization. To address this challenge, we propose LogMILP (Log anomaly localization based on Multi-Instance Learning enhanced by prototypes and Perturbation), a weakly supervised framework that enables both bag-level anomaly detection and instance-level anomaly localization using only bag-level labels. Our method guides the model to pinpoint the critical log entries using prototype-guided structural modeling with counterfactual perturbation consistency regularization, thereby improving localization reliability and interpretability under coarse-grained supervision. Experimental results on three public datasets demonstrate that LogMILP achieves competitive detection performance while yielding significantly more reliable instance-level localization. Our code is open-sourced at https://github.com/YUK1207/LogMILP.

Problem

• log anomaly localization
• weakly-supervised learning
• instance-level annotation
• multi-instance learning
• coarse-grained supervision

Innovation

• weakly-supervised learning
• log anomaly localization
• multi-instance learning
• counterfactual perturbation
• prototype-guided modeling