Building Bridges between Regression, Clustering, and Classification

2025-02-05

AI Summary

Mean squared error (MSE) loss in neural network regression often yields suboptimal predictions, particularly under long-tailed or multimodal target distributions, where robustness and calibration are compromised. Method: We propose a novel encoder–decoder regression paradigm that reformulates continuous scalar prediction as a discrete cluster-index classification task. A target encoder performs clustering-based discretization of continuous targets, while a prediction decoder refines estimates within each cluster; joint representation learning unifies the optimization of classification and intra-cluster regression objectives. Contribution/Results: This is the first end-to-end framework to systematically integrate regression, classification, and clustering paradigms. Extensive experiments on multiple real-world regression benchmarks demonstrate significant accuracy improvements over state-of-the-art methods. Moreover, the approach exhibits superior generalization under distribution shift, confirming enhanced robustness to target distribution irregularities such as heavy tails and multimodality.

Technology Category

• Machine Learning: Classification and Regression
• Search and Optimization: Mixed Discrete/Continuous Search
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text

Abstract

Regression, the task of predicting a continuous scalar target y based on some features x is one of the most fundamental tasks in machine learning and statistics. It has been observed and theoretically analyzed that the classical approach, meansquared error minimization, can lead to suboptimal results when training neural networks. In this work, we propose a new method to improve the training of these models on regression tasks, with continuous scalar targets. Our method is based on casting this task in a different fashion, using a target encoder, and a prediction decoder, inspired by approaches in classification and clustering. We showcase the performance of our method on a wide range of real-world datasets.

Problem

• Improve neural network regression training
• Use target encoder and prediction decoder
• Apply method on real-world datasets

Innovation

• Target encoder improves regression training
• Prediction decoder inspired by classification
• Method enhances neural network performance