All You Need Is Non-Commutative Words

📅 2026-08-29
📈 Citations: 0
Influential: 0
📄 PDF
🤖 AI Summary
该研究通过将词汇表示为幺正矩阵并利用其非交换性来编码句子,无需位置编码,实现了高效的文本分类和持续学习。
📝 Abstract
We represent lexical tokens as unitary matrices and encode each sentence as their ordered product. The noncommutativity of matrix product captures word order without positional encodings (PEs). The same algebra yields several capabilities, including antisymmetric self-attention with no query, key, or value projections, and parallel composition of variable-length text chunks at a reduced attention cost. Furthermore, it provides a canonical-coset readout layer that encodes all true unitary degrees of freedom compactly, while supporting continual learning through nested group extensions that enlarge the operator space with each new task preserving prior representations exactly. Across standard text-classification benchmarks, the method matches or exceeds bag-of-words baselines. Achieving higher accuracy on IMDB and comparable performance on AG News. Notably, this is accomplished by replacing the conventional $\sim$30,000-dimensional vocabulary space with a dense, 64-parameter real-valued encoding, highlighting the expressive efficiency of our parameterization.
Problem

Research questions and friction points this paper is trying to address.

Non-Commutative Words
Matrix Product
Attention Cost
Continual Learning
Vocabulary Space
Innovation

Methods, ideas, or system contributions that make the work stand out.

non-commutative words
unitary matrices
antisymmetric self-attention
parallel composition
canonical-coset readout
🔎 Similar Papers
2024-03-15arXiv.orgCitations: 2
C
Carla M. Quispe Flores
Department of Physics, Colorado School of Mines, Golden, Colorado 80401, USA
S
Stanley Salvatierra
Lookia MX, OR 0401, BO
R
Renan Cabrera
Yardley, PA 19067, USA