ArcGate: Adaptive Arctangent Gated Activation

2026-05-14

AI Summary

This work addresses the limitations of conventional fixed-shape activation functions in remote sensing image tasks, which struggle to capture the dynamic nonlinearities of multi-level features and exhibit insufficient robustness under noise. To overcome these challenges, the authors propose ArcGate, an adaptive arctangent-gated activation function that introduces, for the first time, a learnable activation mechanism with depth-dependent evolution characteristics. ArcGate employs three cascaded nonlinear transformations, each governed by seven learnable parameters, enabling autonomous optimization of activation shapes per layer and enhancing gating strength in deeper layers to improve signal propagation. Experiments on ResNet-50 and ViT-B/16 demonstrate that ArcGate significantly outperforms mainstream baselines across three remote sensing benchmarks, achieving 99.67% accuracy on PatternNet and surpassing ReLU by 26.65% under Gaussian noise with a standard deviation of 0.1.

Abstract

Activation functions are central to deep networks, influencing non-linearity, feature learning, convergence, and robustness. This paper proposes the Adaptive Arctangent Gated Activation (ArcGate) function, a flexible formulation that generates a broad spectrum of activation shapes via a three-stage non-linear transformation. Unlike conventional fixed-shape activations such as ReLU, GELU, or SiLU, ArcGate uses seven learnable parameters per layer, allowing the neural network to autonomously optimize its non-linearity to the specific requirements of the feature hierarchy and data distribution. We evaluate ArcGate using ResNet-50 and Vision Transformer (ViT-B/16) architectures on three widely used remote sensing benchmarks: PatternNet, UC Merced Land Use, and the 13-band EuroSAT MSI multispectral dataset. Experimental results show that ArcGate consistently outperforms standard baselines, achieving a peak overall accuracy of 99.67% on PatternNet. Most notably, ArcGate exhibits superior structural resilience in noisy environments, maintaining a 26.65% performance lead over ReLU under moderate Gaussian noise (standard deviation 0.1). Analysis of the learned parameters reveals a depth-dependent functional evolution, where the model increases gating strength in deeper layers to enhance signal propagation. These findings suggest that ArcGate is a robust and adaptive general node activation function for high-resolution earth observation tasks.

Problem

• activation function
• adaptivity
• non-linearity
• robustness
• remote sensing

Innovation

• adaptive activation
• gated non-linearity
• learnable parameters
• robustness to noise
• remote sensing classification