Prime Fourier Embeddings: A Principled Basis for Modular Arithmetic

2026-06-22

AI Summary

Standard neural embeddings struggle to capture the algebraic structure of integers, particularly in modular arithmetic, where relational patterns must be learned from scratch. This work proposes Prime Fourier Embeddings (PFE), the first approach that integrates group representation theory with harmonic analysis to encode integers as cosine–sine pairs indexed by prime numbers, thereby reducing modular operations to selection over corresponding prime-indexed channels. Theoretically, PFE exhibits a block-diagonal structure under linear equivariant mappings, with channel selection guided by the Chinese Remainder Theorem. Empirically, PFE achieves 100% in-distribution accuracy across all square-free composite modulus tasks, and demonstrates a specialization ratio exceeding 500× between task-relevant and task-irrelevant channels.

Abstract

Numbers have algebraic structure that standard neural embeddings often fail to expose. We introduce Prime Fourier Embeddings (PFE), which encode integers as prime-indexed (cos, sin) pairs derived from the harmonic analysis of Q, providing a pre-structured representation in which modular arithmetic reduces to selecting the relevant prime channel rather than discovering algebraic structure from scratch. We prove that any linear map equivariant with respect to the product group action on PFE must be block-diagonal with one independent block per prime -- a consequence of Schur's lemma applied to the resulting character decomposition. For square-free composite moduli, the Chinese Remainder Theorem predicts which prime channels are task-relevant. Both predictions are confirmed empirically: ablation studies show specialization ratios exceeding 500x between task-relevant and task-irrelevant channels, with perfect in-distribution test accuracy across all square-free composite moduli tested.

Problem

• modular arithmetic
• neural embeddings
• algebraic structure
• integer representation
• harmonic analysis

Innovation

• Prime Fourier Embeddings
• modular arithmetic
• equivariant representation
• Chinese Remainder Theorem
• harmonic analysis