🤖 AI Summary
This study addresses the limited effectiveness of unsupervised anomaly detection in identifying fraudulent transactions under label-scarce conditions by proposing an enhanced Isolation Forest method. The approach innovatively integrates the silhouette coefficient into the Isolation Forest framework, leveraging tree path lengths as distinctive “fingerprints” for clustering and fusing silhouette scores with original anomaly scores to improve discriminative power. Designed with adjustable structure and ease of deployment, the method also explicitly delineates its applicability boundaries. Evaluated on the real-world IEEE-CIS dataset, it achieves an average AUC-PR improvement of 0.0080 and consistently outperforms baseline methods across five random trials (p = 0.046). Its limitations are further validated on the Sparkov synthetic dataset.
📝 Abstract
Unsupervised anomaly detection is widely used in transaction fraud detection where labels are scarce. Isolation Forest (IF) is among the most popular classical methods due to its scalability and ease of deployment. We propose SilIF, an augmentation of Isolation Forest that adds a silhouette-based scoring layer computed in a representation space induced by the trees of the forest. For each point, we extract a vector of per-tree path lengths, cluster these "fingerprints" into structural groups, and compute a silhouette score that measures how well the point fits its assigned group versus the nearest alternative. The silhouette signal is combined with the base IF score via a single hyperparameter alpha. On the IEEE-CIS Fraud Detection benchmark (~590K transactions, 3.5% fraud), SilIF with alpha=1.0 improves over plain Isolation Forest by +0.0080 AUC-PR on average across five seeds, with SilIF winning on all five seeds (paired t-test p=0.046). We also report results on a synthetic credit-card dataset (Sparkov) where the silhouette augmentation does not improve over plain IF, and we characterize the conditions that distinguish the two outcomes. The paper presents SilIF as a tunable, easy-to-deploy enhancement to Isolation Forest with honest reporting of when it helps and when it does not. Code at https://github.com/venkat15vk/silif-anomaly-detection.