Multi-layer Radial Basis Function Networks for Out-of-distribution Detection

2025-01-05

AI Summary

Existing out-of-distribution (OOD) detection methods typically rely on post-hoc, classification-agnostic anomaly scoring, resulting in redundant, multi-stage pipelines. This work proposes the Multi-Layer Radial Basis Function Network (MLRBFN), the first architecture to jointly perform classification and OOD detection within a single forward pass. Its key contributions are: (1) an end-to-end trainable multi-layer RBF architecture that overcomes the limited expressivity and scalability of conventional RBF networks; (2) an embedded feature-space suppression mechanism that intrinsically couples classification confidence with OOD discriminability; and (3) seamless compatibility with pretrained feature extractors, enabling plug-and-play deployment. Evaluated on standard OOD benchmarks, MLRBFN achieves performance on par with or superior to state-of-the-art methods—despite its significantly simpler architecture—while maintaining high in-distribution classification accuracy and robust OOD detection. These results empirically validate the efficacy and generalizability of single-stage joint modeling for unified classification and OOD detection.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Existing methods for out-of-distribution (OOD) detection use various techniques to produce a score, separate from classification, that determines how ``OOD'' an input is. Our insight is that OOD detection can be simplified by using a neural network architecture which can effectively merge classification and OOD detection into a single step. Radial basis function networks (RBFNs) inherently link classification confidence and OOD detection; however, these networks have lost popularity due to the difficult of training them in a multi-layer fashion. In this work, we develop a multi-layer radial basis function network (MLRBFN) which can be easily trained. To ensure that these networks are also effective for OOD detection, we develop a novel depression mechanism. We apply MLRBFNs as standalone classifiers and as heads on top of pretrained feature extractors, and find that they are competitive with commonly used methods for OOD detection. Our MLRBFN architecture demonstrates a promising new direction for OOD detection methods.

Problem

• Anomaly Detection
• Out-of-Distribution Detection
• Neural Network Architecture

Innovation

• Multi-Layer Radial Basis Function Network
• Out-of-Distribution Detection
• Depression Mechanism