CAWI: Copula-Aligned Weight Initialization for Randomized Neural Networks

2026-05-12

AI Summary

This work addresses the limitation of conventional random neural network initialization, which neglects complex dependency structures among input features—such as correlations, asymmetries, and tail dependencies—leading to poor conditioning and degraded performance. To overcome this, the authors propose CAWI, a novel framework that introduces copula theory into random weight initialization for the first time. By mapping inputs through empirical cumulative distribution functions, fitting multivariate copulas (including Gaussian, t, Clayton, Frank, and Gumbel families), and applying inverse marginal transformations, CAWI generates dependency-aware weights between input and hidden layers. This approach preserves the closed-form training mechanism of the output layer while ensuring that frozen random projections faithfully retain the original feature dependencies. Compatible with both elliptical and Archimedean copula families, CAWI flexibly captures diverse dependency patterns and demonstrates consistent performance gains over standard initialization methods across 83 classification benchmarks and biomedical datasets such as BreaKHis and schizophrenia diagnosis tasks.

Abstract

Randomized neural networks (RdNNs) enable efficient, backpropagation-free training by freezing randomly initialized input-to-hidden weights, which permits a closed-form solution for the output layer. However, conventional random initialization is blind to inter-feature dependence, ignoring correlations, asymmetries, and tail dependence in the data, which degrades conditioning and predictive performance. To the best of our knowledge, this limitation remains unaddressed in the RdNN literature. To close this gap, we propose CAWI (Copula-Aligned Weight Initialization), a framework that draws input-to-hidden weights from a data-fitted copula that matches empirical dependence, ensuring the frozen projections respect inter-feature dependence without sacrificing the closed-form solution. CAWI (i) maps each feature to the unit interval using empirical CDFs, (ii) fits a multivariate copula that captures rank-based dependence among features, and (iii) samples each weight column w_j from the fitted copula and applies a fixed inverse marginal transform to set scale. The objective, solver, and "freeze-once" paradigm remain unchanged; only the sampling law for W becomes dependence-aware. For dependence modeling, we consider two copula families: elliptical (Gaussian, t) and Archimedean (Clayton, Frank, Gumbel). This enables CAWI to handle diverse dependence, including tail dependence. We evaluate CAWI across 83 diverse classification benchmarks (binary and multiclass) and two biomedical datasets, BreaKHis and the Schizophrenia dataset, using standard shallow and deep RdNN architectures. CAWI consistently delivers significant improvements in predictive performance over conventional random initialization. Code is available at: https://github.com/mtanveer1/CAWI

Problem

• randomized neural networks
• weight initialization
• feature dependence
• copula
• tail dependence

Innovation

• Copula
• Weight Initialization
• Randomized Neural Networks
• Feature Dependence
• Tail Dependence