Generality Is Not Enough: Zero-Label Cross-System Log-Based Anomaly Detection via Knowledge-Level Collaboration

2025-11-08

AI Summary

To address the challenge of jointly leveraging general knowledge and target-system-specific knowledge in zero-label cross-system log anomaly detection, this paper proposes GeneralLog. Its core innovation is a knowledge-level collaboration mechanism between large language models (LLMs) and small models: logs are dynamically routed to either a specialized or a general branch based on their semantic knowledge attributes—not uncertainty—enabling separate modeling of system-specific patterns and generalized regularities. This routing mechanism operates without any labeled data from the target system, effectively balancing specificity capture and generalization capability. Evaluated on three public log datasets, GeneralLog achieves an average F1-score of 90.2%, significantly outperforming existing zero-shot and few-shot methods. The results demonstrate the effectiveness of knowledge-aware dynamic routing for unsupervised cross-system transfer.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Multiagent Systems: Multiagent Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Log-based anomaly detection is crucial for ensuring software system stability. However, the scarcity of labeled logs limits rapid deployment to new systems. Cross-system transfer has become an important research direction. State-of-the-art approaches perform well with a few labeled target logs, but limitations remain: small-model methods transfer general knowledge but overlook mismatches with the target system's proprietary knowledge; LLM-based methods can capture proprietary patterns but rely on a few positive examples and incur high inference cost. Existing LLM-small model collaborations route'simple logs'to the small model and'complex logs'to the LLM based on output uncertainty. In zero-label cross-system settings, supervised sample complexity is unavailable, and such routing does not consider knowledge separation. To address this, we propose GeneralLog, a novel LLM-small model collaborative method for zero-label cross-system log anomaly detection. GeneralLog dynamically routes unlabeled logs, letting the LLM handle'proprietary logs'and the small model'general logs,'enabling cross-system generalization without labeled target logs. Experiments on three public log datasets show that GeneralLog achieves over 90% F1-score under a fully zero-label setting, significantly outperforming existing methods.

Problem

• Addresses zero-label cross-system log anomaly detection without target system labels
• Resolves knowledge mismatch between general models and proprietary system patterns
• Enables cost-effective anomaly detection through dynamic LLM-small model collaboration

Innovation

• LLM-small model collaboration for zero-label anomaly detection
• Dynamic routing separates proprietary and general logs
• Achieves cross-system generalization without labeled data