A novel framework for quantifying nominal outlyingness

2024-08-14

AI Summary

Nominal data anomaly detection lacks a formal definition of anomalies and interpretable, quantitative measures. Method: This paper proposes the first theoretically grounded definition of nominal anomaly degree and a general quantification framework based on association rule mining. Departing from distance- or density-based assumptions, it adopts a multinomial distribution as the generative model and introduces variable contribution scores and anomaly depth to enable fine-grained anomaly attribution and enhanced interpretability. A hyperparameter self-adaptation mechanism further improves robustness. Results: Extensive experiments on synthetic and real-world datasets demonstrate that the method achieves detection performance competitive with or superior to state-of-the-art frequent-pattern-based approaches. It uniquely supports quantitative anomaly severity assessment and diagnostic root-cause tracing. The framework thus establishes a new paradigm for nominal anomaly analysis—rigorous in theory and effective in practice.

Technology Category

• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Machine Learning: Calibration & Uncertainty Quantification
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Outlier detection is an important data mining tool that becomes particularly challenging when dealing with nominal data. First and foremost, flagging observations as outlying requires a well-defined notion of nominal outlyingness. This paper presents a definition of nominal outlyingness and introduces a general framework for quantifying outlyingness of nominal data. The proposed framework makes use of ideas from the association rule mining literature and can be used for calculating scores that indicate how outlying a nominal observation is. Methods for determining the involved hyperparameter values are presented and the concepts of variable contributions and outlyingness depth are introduced, in an attempt to enhance interpretability of the results. The proposed framework is evaluated on both synthetic and real-world data sets, demonstrating comparable performance to state-of-the-art frequent pattern mining algorithms and even outperforming them in certain cases. The ideas presented can serve as a tool for assessing the degree to which an observation differs from the rest of the data, under the assumption of sequences of nominal levels having been generated from a Multinomial distribution with varying event probabilities.

Problem

• Defining nominal outlyingness for categorical data
• Quantifying outlier scores using association rule mining
• Enhancing interpretability through variable contributions and depth

Innovation

• Framework quantifying nominal outlyingness using association rules
• Hyperparameter determination methods for enhanced interpretability
• Outperforms frequent pattern mining in certain cases