ChaosProbe: A Neurochaotic Lens on Frozen Transformer Input-Embedding Spaces
This work addresses the challenge of characterizing the input embedding space structure of frozen Transformers without relying on downstream tasks or contextual computations. It introduces ChaosProbe, a novel method that pioneers the application of neural chaos dynamics to embedding analysis: by applying deterministic chaotic trajectory transformations to input embeddings and combining neuronal firing rates with entropy responses, it generates fixed-length structural fingerprints. This approach requires no training or task-specific adaptation, yet effectively reveals macroscopic relationships among embedding spaces. Experiments across four pretrained models and 80 neutral prompts demonstrate that multiple similarity metrics consistently recover both intra-family nearest neighbors and inter-family pairings, confirming the stability and validity of the proposed fingerprints.