Resonant Sparse Geometry Networks
This work proposes a brain-inspired neural architecture to address the high computational complexity, large parameter count, and lack of biological plausibility in Transformers when handling long-range dependencies and hierarchical classification tasks. The approach uniquely integrates hyperbolic space embeddings, input-dependent dynamic sparse connectivity, and Hebbian structural learning, modulating connection strengths via geodesic distance decay and incorporating a dual-timescale learning mechanism with fast and slow components. The resulting model reduces computational complexity to O(n·k), achieving 96.5% accuracy on long-range dependency tasks with only 1/15 the parameters of a standard Transformer. On a 20-class hierarchical classification benchmark, it attains 23.8% accuracy using just 41,672 parameters—approximately five times the random baseline—demonstrating substantial gains in both efficiency and biological plausibility.