Selected work

Representative Papers

Schauder Bases for $C[0, 1]$ Using ReLU, Softplus and Two Sigmoidal Functions

Jun 09, 2025

Prior work on neural activation functions has primarily focused on universal approximation, without establishing their capacity to form Schauder bases in classical function spaces. Method: This paper rigorously constructs four unconditional Schauder bases for the Banach space $C[0,1]$—the space of continuous real-valued functions on $[0,1]$—using the ReLU, Softplus, and two sigmoidal variants, via functional-analytic arguments and piecewise smooth approximation techniques. Contribution/Results: It is the first work to prove that each of these four widely used activation functions generates a Schauder basis for $C[0,1]$, thereby providing a rigorous functional-analytic foundation for corresponding neural networks. Specifically, any continuous function on $[0,1]$ admits a unique, norm-convergent series expansion in terms of the associated basis functions. This result extends the theoretical expressive power of activation functions beyond mere approximation capability and establishes, for the first time, functional-space completeness guarantees for networks built upon them—significantly advancing their foundational role in approximation theory and representation learning.

0 citationsRead paper

Generative Data Mining with Longtail-Guided Diffusion

Feb 04, 2025

This work addresses the limited generalization of deployed models on rare or challenging samples. We propose Long-Tailed Guidance (LTG), a post-deployment data augmentation method that requires no retraining or model updates. LTG computes differentiable long-tailed signals—such as epistemic uncertainty—in a single forward pass and leverages them to steer diffusion models in latent space, generating semantically rich, conceptually targeted samples that explicitly address model blind spots. We introduce the first coupled “long-tailed signal–diffusion generation” framework, offering both interpretability and controllability. LTG is model-agnostic, enabling long-tailed instance discovery, visual attribution, and targeted model remediation. Evaluated on image classification benchmarks, LTG significantly improves generalization performance; generated samples precisely localize and fill conceptual gaps in the target predictor’s decision boundary.

0 citationsRead paper

Fine Tuning without Catastrophic Forgetting via Selective Low Rank Adaptation

Jan 26, 2025

To address catastrophic forgetting, degraded out-of-distribution (OOD) generalization, and high computational overhead in large-model domain adaptation, this paper proposes a parameter-efficient fine-tuning method based on selective activation of LoRA modules. Our core innovation is a learnable binary gating function that enables fine-grained, task-aware sparsity in LoRA updates, integrated within the Task Adaptive Parameter Sharing (TAPS) framework and low-rank decomposition. The method updates only ~5% of parameters. Evaluated on CLIP and DINO-ViT, it reduces trainable parameters by over 95% compared to standard LoRA, maintains or improves OOD accuracy, and significantly mitigates forgetting of prior-task knowledge. To our knowledge, this is the first work within the parameter-efficient fine-tuning (PEFT) paradigm to systematically enhance both OOD robustness and long-term knowledge retention.

0 citationsRead paper

On the Approximability of Stationary Processes using the ARMA Model

Aug 20, 2024arXiv.org

Existing research on ARMA approximation of stationary stochastic processes lacks rigorous theoretical foundations and precise error characterizations. Method: This work adopts a novel analytical framework centered on the process generating function—rather than the conventional spectral measure—and employs the sup-norm on the unit circle to quantify approximation error. It introduces the spectral lemma for quantitative error analysis of ARMA approximation, transforming heuristic rational approximation arguments into rigorous theorems. Contribution/Results: We prove that Padé approximants are not universally optimal for fixed-order ARMA approximation and construct explicit counterexamples of non-approximable stationary processes. Furthermore, we characterize the class of stationary processes admitting ARMA approximation, derive exact asymptotic approximation bounds for canonical processes (e.g., AR, MA, fractional Gaussian noise), and extend beyond traditional prediction-error-centric paradigms to establish a comprehensive theory of spectral approximation fidelity.

0 citationsRead paper

Recent publications

Latest Papers

Schauder Bases for $C[0, 1]$ Using ReLU, Softplus and Two Sigmoidal Functions

Jun 09, 2025

Prior work on neural activation functions has primarily focused on universal approximation, without establishing their capacity to form Schauder bases in classical function spaces. Method: This paper rigorously constructs four unconditional Schauder bases for the Banach space $C[0,1]$—the space of continuous real-valued functions on $[0,1]$—using the ReLU, Softplus, and two sigmoidal variants, via functional-analytic arguments and piecewise smooth approximation techniques. Contribution/Results: It is the first work to prove that each of these four widely used activation functions generates a Schauder basis for $C[0,1]$, thereby providing a rigorous functional-analytic foundation for corresponding neural networks. Specifically, any continuous function on $[0,1]$ admits a unique, norm-convergent series expansion in terms of the associated basis functions. This result extends the theoretical expressive power of activation functions beyond mere approximation capability and establishes, for the first time, functional-space completeness guarantees for networks built upon them—significantly advancing their foundational role in approximation theory and representation learning.

0 citationsRead paper

Generative Data Mining with Longtail-Guided Diffusion

Feb 04, 2025

This work addresses the limited generalization of deployed models on rare or challenging samples. We propose Long-Tailed Guidance (LTG), a post-deployment data augmentation method that requires no retraining or model updates. LTG computes differentiable long-tailed signals—such as epistemic uncertainty—in a single forward pass and leverages them to steer diffusion models in latent space, generating semantically rich, conceptually targeted samples that explicitly address model blind spots. We introduce the first coupled “long-tailed signal–diffusion generation” framework, offering both interpretability and controllability. LTG is model-agnostic, enabling long-tailed instance discovery, visual attribution, and targeted model remediation. Evaluated on image classification benchmarks, LTG significantly improves generalization performance; generated samples precisely localize and fill conceptual gaps in the target predictor’s decision boundary.

0 citationsRead paper

Fine Tuning without Catastrophic Forgetting via Selective Low Rank Adaptation

Jan 26, 2025

To address catastrophic forgetting, degraded out-of-distribution (OOD) generalization, and high computational overhead in large-model domain adaptation, this paper proposes a parameter-efficient fine-tuning method based on selective activation of LoRA modules. Our core innovation is a learnable binary gating function that enables fine-grained, task-aware sparsity in LoRA updates, integrated within the Task Adaptive Parameter Sharing (TAPS) framework and low-rank decomposition. The method updates only ~5% of parameters. Evaluated on CLIP and DINO-ViT, it reduces trainable parameters by over 95% compared to standard LoRA, maintains or improves OOD accuracy, and significantly mitigates forgetting of prior-task knowledge. To our knowledge, this is the first work within the parameter-efficient fine-tuning (PEFT) paradigm to systematically enhance both OOD robustness and long-term knowledge retention.

0 citationsRead paper

On the Approximability of Stationary Processes using the ARMA Model

Aug 20, 2024arXiv.org

Existing research on ARMA approximation of stationary stochastic processes lacks rigorous theoretical foundations and precise error characterizations. Method: This work adopts a novel analytical framework centered on the process generating function—rather than the conventional spectral measure—and employs the sup-norm on the unit circle to quantify approximation error. It introduces the spectral lemma for quantitative error analysis of ARMA approximation, transforming heuristic rational approximation arguments into rigorous theorems. Contribution/Results: We prove that Padé approximants are not universally optimal for fixed-order ARMA approximation and construct explicit counterexamples of non-approximable stationary processes. Furthermore, we characterize the class of stationary processes admitting ARMA approximation, derive exact asymptotic approximation bounds for canonical processes (e.g., AR, MA, fractional Gaussian noise), and extend beyond traditional prediction-error-centric paradigms to establish a comprehensive theory of spectral approximation fidelity.

0 citationsRead paper