🤖 AI Summary
This study addresses the discrepancy between nominal and effective ranks in foundation model adapters and the lack of finite-sample statistical inference. We propose an auditing framework based on joint spectral gap calibration and empirical null hypotheses. By integrating high-dimensional spectral analysis, Monte Carlo testing, and multiple correction, we establish precise chi-square divergence and Le Cam bounds to calibrate spectral evidence. Empirical evaluation across 26 adapters demonstrates that the calibrated effective rank is significantly lower than the nominal rank and distinct from energy retention rates. These findings reveal the intrinsic low-dimensional structure of adapters, providing a rigorous statistical certification and novel evaluation paradigm for parameter-efficient fine-tuning.
📝 Abstract
Nominal LoRA rank is a design parameter; calibrated spectral evidence is a separate inferential quantity. This article develops a finite-sample framework for inferring effective rank structure in public foundation-model adapters. The theoretical core is an exact chi-square divergence for the fixed-dimensional Gaussian rank-one reference experiment, with an unknown signal direction integrated under a rotation-invariant reference prior. The resulting series yields a computable finite-sample Le Cam bound at concrete layer sizes, an explicit remainder bound for numerical truncation, and the rectangular Baik-Ben Arous-Peche (BBP) limit. A compact-manifold Laplace expansion shows that finite-sample likelihood evidence also depends on leading spectral gaps through the factor $s_1^{|m-n|}\prod_{i\ge2}(s_1^2-s_i^2)$, motivating joint calibration of clustered singular values. Building on these results, we introduce an empirical-null workflow for PEFT LoRA adapters: factor reconstruction, Monte Carlo $p$-values, stagewise and block testing, and module-wise and corpus-level BH reporting. In an audit of 26 public adapters, 684 modules, six architecture families, and 31,770 public-checkpoint spectra rows, calibrated effective rank is typically much smaller than nominal rank and differs systematically from 95\% energy retention. A measured RoBERTa-RTE slice on $n=24$ examples illustrates the measurement path from calibrated ranks to task evaluation, without treating the slice as a utility study. The main empirical finding is that calibrated effective rank is usually far below nominal rank, and that energy retention and statistical surprise answer different questions.