BandRC: Band Shifted Raised Cosine Activated Implicit Neural Representations

2025-05-16

AI Summary

Implicit Neural Representations (INRs) suffer from spectral bias, poor noise robustness, imbalanced local-global feature modeling, and strong hyperparameter dependence—largely attributable to conventional activation functions. To address these issues, we propose the Band-Shifted Raised Cosine (BandRC) activation function, which introduces *learnable frequency-domain priors* for the first time, enabling adaptive spectral response modulation and end-to-end, task-driven optimization. BandRC is trained jointly across multiple vision tasks—including image reconstruction, denoising, super-resolution, inpainting, and 3D shape reconstruction. Experimental results demonstrate consistent and significant improvements: +8.93 dB PSNR in image reconstruction, +0.46 dB PSNR in denoising, +1.03 dB PSNR over prior state-of-the-art in 6× super-resolution, and leading performance in both inpainting and 3D reconstruction. These gains validate BandRC’s effectiveness in mitigating fundamental limitations of INR activations while enhancing generalization and robustness across diverse downstream tasks.

Technology Category

• Computer Vision: Representation Learning for Vision
• Machine Learning: Active Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

In recent years, implicit neural representations(INRs) have gained popularity in the computer vision community. This is mainly due to the strong performance of INRs in many computer vision tasks. These networks can extract a continuous signal representation given a discrete signal representation. In previous studies, it has been repeatedly shown that INR performance has a strong correlation with the activation functions used in its multilayer perceptrons. Although numerous activation functions have been proposed that are competitive with one another, they share some common set of challenges such as spectral bias(Lack of sensitivity to high-frequency content in signals), limited robustness to signal noise and difficulties in simultaneous capturing both local and global features. and furthermore, the requirement for manual parameter tuning. To address these issues, we introduce a novel activation function, Band Shifted Raised Cosine Activated Implicit Neural Networks extbf{(BandRC)} tailored to enhance signal representation capacity further. We also incorporate deep prior knowledge extracted from the signal to adjust the activation functions through a task-specific model. Through a mathematical analysis and a series of experiments which include image reconstruction (with a +8.93 dB PSNR improvement over the nearest counterpart), denoising (with a +0.46 dB increase in PSNR), super-resolution (with a +1.03 dB improvement over the nearest State-Of-The-Art (SOTA) method for 6X super-resolution), inpainting, and 3D shape reconstruction we demonstrate the dominance of BandRC over existing state of the art activation functions.

Problem

• Addressing spectral bias in implicit neural representations
• Improving robustness to signal noise in INRs
• Enhancing local and global feature capture simultaneously

Innovation

• BandRC activation function enhances signal representation
• Deep prior knowledge adjusts activation functions adaptively
• Superior performance in image and 3D reconstruction tasks