CodeAD: Synthesize Code of Rules for Log-based Anomaly Detection with LLMs

2025-10-27

AI Summary

Existing log anomaly detection (LogAD) methods suffer from poor interpretability, high inference overhead, or prohibitive manual rule maintenance costs. To address these challenges, this paper proposes CodeAD: a framework leveraging large language models (LLMs) to automatically generate lightweight, executable Python rules for LogAD. Its core innovations include (i) constructing contrastive log windows via hierarchical clustering and anchor-driven sampling, and (ii) orchestrating an agent-based workflow for iterative rule generation, automated testing, and self-repair. CodeAD synthesizes highly interpretable rules directly from raw logs—requiring no complex preprocessing. Evaluated on BGL, Hadoop, and Thunderbird datasets, CodeAD achieves an average 3.6% F1-score improvement over state-of-the-art baselines, attains 4× higher throughput, and incurs less than $4 in LLM invocation cost per dataset. The approach thus significantly advances the joint optimization of accuracy, efficiency, and interpretability in LogAD.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Multiagent Systems: Adversarial Agents

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Log-based anomaly detection (LogAD) is critical for maintaining the reliability and availability of large-scale online service systems. While machine learning, deep learning, and large language models (LLMs)-based methods have advanced the LogAD, they often suffer from limited interpretability, high inference costs, and extensive preprocessing requirements, limiting their practicality for real-time, high-volume log analysis. In contrast, rule-based systems offer efficiency and transparency, but require significant manual effort and are difficult to scale across diverse and evolving environments. In this paper, We present CodeAD, a novel framework that automatically synthesizes lightweight Python rule functions for LogAD using LLMs. CodeAD introduces a hierarchical clustering and anchor-grounded sampling strategy to construct representative contrastive log windows, enabling LLMs to discern discriminative anomaly patterns. To ensure robustness and generalizability, CodeAD employs an agentic workflow that iteratively generates, tests, repairs, and refines the rules until it meets correctness and abstraction requirements. The synthesized rules are interpretable, lightweight, and directly executable on raw logs, supporting efficient and transparent online anomaly detection. Our comprehensive experiments on three public datasets (BGL, Hadoop, Thunderbird) demonstrate that CodeAD achieves an average absolute improvement of 3.6% F1 score over the state-of-the-art baselines, while processing large datasets up to 4x faster and at a fraction of the cost (total LLM invocation cost under 4 USD per dataset). These results highlight CodeAD as a practical and scalable solution for online monitoring systems, enabling interpretable, efficient, and automated LogAD in real-world environment.

Problem

• Automating rule generation for log anomaly detection using LLMs
• Addressing interpretability and efficiency issues in existing LogAD methods
• Synthesizing lightweight executable rules for real-time log analysis

Innovation

• Synthesizes Python rule functions using LLMs
• Uses hierarchical clustering and anchor-grounded sampling
• Employs iterative agentic workflow for rule refinement