A Pāninian Foundation for Indic Language Processing

2026-06-23

AI Summary

This work addresses the fragmentation in Indo-Aryan language processing caused by the absence of a unified architectural foundation, which has led to redundant development efforts. For the first time, it leverages Pāṇini’s Aṣṭādhyāyī—a formal grammatical system from classical Sanskrit linguistics—as a shared basis for modern natural language processing, establishing a cross-lingual unified framework. By integrating ancient Indian linguistic theory with contemporary computational methods, the study introduces a metalanguage resource fusion paradigm and designs a benchmark suite comprising four tasks to evaluate models’ capacity to acquire Pāṇinian structural knowledge. Experimental results demonstrate that this approach substantially improves model accuracy, data efficiency, and cross-lingual generalization, while also offering an interpretable pathway to investigate whether neural models spontaneously learn classical grammatical categories.

Abstract

More than a billion people communicate in Indic languages, yet the natural language processing infrastructure serving them remains fragmented and underdeveloped. The cause is structural: the field organizes its tools and benchmarks around individual languages or small subsets of genealogical language families, building separate analyzers, parsers, and datasets for each language and starting over for the next. This overlooks a deep regularity. Through more than two millennia of convergence around Sanskrit, Indic languages came to share a morphosyntactic architecture formalized in Pānini's grammar, the Astādhyāyī. This cuts across genealogical lines, uniting languages through a common framework. We argue that this Pāninian framework supplies a unifying computational architecture the field has lacked, and that benchmarks grounded explicitly in it would make Indic language systems more accurate, more data-efficient, and more transferable, effectively merging many apparently disparate and sparse Indic language resources into a single high-resource metalanguage bedrock. We propose a four-part benchmark suite to render this shared architecture explicit, measurable, and ready to be leveraged for practical applications. Moreover, we underscore the question it raises for interpretability research: whether neural models trained on these languages come to represent Pānini's categories on their own.

Problem

• Indic languages
• natural language processing
• Pāṇinian grammar
• morphosyntactic architecture
• language convergence

Innovation

• Pāṇinian grammar
• Indic languages
• morphosyntactic architecture
• cross-lingual transfer
• computational linguistics