Local Pheromone Network: Sparse Local Learning with Multi-Scale Synaptic Trails, Consolidation, and Replay

2026-06-22

AI Summary

This work addresses the limitations of conventional dense neural networks—namely, their susceptibility to catastrophic forgetting under task interference, strong parameter coupling, and reliance on backpropagation—by introducing a novel sparse, locally connected neural network prototype that operates without automatic differentiation. The model employs handcrafted updates guided by multiscale pheromone-like traces and local Hebbian rules, integrating short- and long-term memory mechanisms, dynamic synaptic budgets, structural plasticity, local replay, and target-free contrastive learning to enable compartmentalized memory storage and robust resistance to forgetting. Experimental results demonstrate that the proposed architecture effectively preserves local linear regularities, substantially mitigates catastrophic forgetting, and handles task conflicts reliably across synthetic regression, conflicting memory, and long-context mixed tasks.

Abstract

Backpropagation-trained dense neural networks are powerful function approximators, but they couple learning across many parameters and can overwrite previous associations when tasks conflict. This paper describes Local Pheromone Network, a small research prototype for sparse, local, manually updated neural networks. In Local Pheromone Network, each output unit reads only a fixed local neighborhood of input units subject to geometric distance and molecular-tag compatibility. Each synapse stores a weight, a short-term pheromone trace, a long-term pheromone trace, and an optional consolidation state. Training does not call automatic differentiation. Instead, every layer performs a pheromone-weighted Hebbian-style update on a budgeted subset of local synapses selected from local error and co-activity. The update budget adapts online: it shrinks when loss improves and expands toward recently active neighborhoods when loss worsens. Optional mechanisms add structural plasticity, local replay, output masks for partitioned learning, and a target-free local contrastive step. We present the implementation, learning rule, and preliminary experiments on synthetic regression, partitioned memory, conflicting memory, consolidated conflict, structural plasticity, replay, and a synthetic long-context hybrid memory task. The prototype learns local linear rules, preserves partitioned memories through tags and masks, reduces forgetting under consolidation, and uses replay under conflict.

Problem

• catastrophic forgetting
• sparse learning
• local plasticity
• memory consolidation
• conflicting tasks

Innovation

• Local Pheromone Network
• sparse local learning
• pheromone-weighted Hebbian update
• synaptic consolidation
• local replay