Counterfactual Residual Data Augmentation for Regression

2026-06-26

AI Summary

This work addresses the challenges of scarce samples, high data acquisition costs, and substantial observation noise commonly encountered in real-world regression tasks by proposing a model-agnostic counterfactual residual data augmentation method. The approach uniquely integrates counterfactual perturbations with residual invariance, generating high-fidelity synthetic samples through minimal perturbations applied to key features—without requiring additional real data. The proposed framework is compatible with diverse regressors, including MLPs and XGBoost, and demonstrates consistent performance gains across multiple benchmark datasets, reducing mean squared error by 22.9% for MLPs and 6.4% for XGBoost on average, thereby outperforming existing data augmentation techniques.

Abstract

Data-driven modeling in real-world regression tasks often suffers from limited training samples, high collection costs, and noisy observations. Inspired by the impact of data augmentation in vision and language, we propose a novel Counterfactual Residual Data Augmentation (CRDA) technique for tabular regression. Our key insight is that once a regressor has modeled the systematic component of the data, the remaining noise can be viewed as an invariant residual that remains stable under small perturbations of carefully selected features. We exploit this residual invariance to generate new, yet realistic, training samples, effectively expanding the dataset without requiring additional real data. Our method is model-agnostic and readily applicable to various types of regressors. In experiments across datasets from a variety of benchmark repositories, on average, CRDA reduces an MLP Regressor's MSE by 22.9% and an XGBoost Regressor's MSE by 6.4%. When compared to existing state-of-the-art data generators and augmentation techniques, CRDA consistently outperforms in MSE reduction. By adding principled counterfactual variations to the training data, our method offers a simple and efficient remedy for noise-prone, small-sample regression settings.

Problem

• data augmentation
• regression
• limited data
• noisy observations
• tabular data

Innovation

• Counterfactual
• Residual Invariance
• Data Augmentation
• Tabular Regression
• Model-Agnostic