Programming by Examples Meets Historical Linguistics: A Large Language Model Based Approach to Sound Law Induction

2025-01-27

AI Summary

This work addresses sound change induction (SLI) in historical linguistics—the automatic inference of phonological regularities from proto-language–descendant word pairs. We reformulate SLI as a programming-by-examples (PBE) problem, modeling each pair as an ordered string transformation. We propose a formal PBE framework, design four synthetic data generation strategies embodying distinct inductive biases, and introduce a fine-grained distribution alignment technique to enhance large language model generalization. Our lightweight, open-source model achieves a 6% absolute improvement in pass rate on standard SLI benchmarks while using only one-third the parameters of the second-best model. Methodologically, this work advances computational historical linguistics by enabling interpretable, rule-based phonological generalization; theoretically, it demonstrates the efficacy of the PBE paradigm for structured symbolic reasoning tasks. The approach provides a scalable, explainable framework for modeling language evolution—bridging formal linguistics and neural sequence modeling.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Evolutionary Learning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Web Mining and Content Analysis: Models for Web evolution
• Semantics and Knowledge: Methods, algorithms and applications for the development of semantic models, knowledge graphs and other forms of structured data models with machine-interpretable semantics
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Historical linguists have long written"programs"that convert reconstructed words in an ancestor language into their attested descendants via ordered string rewrite functions (called sound laws) However, writing these programs is time-consuming, motivating the development of automated Sound Law Induction (SLI) which we formulate as Programming by Examples (PBE) with Large Language Models (LLMs) in this paper. While LLMs have been effective for code generation, recent work has shown that PBE is challenging but improvable by fine-tuning, especially with training data drawn from the same distribution as evaluation data. In this paper, we create a conceptual framework of what constitutes a"similar distribution"for SLI and propose four kinds of synthetic data generation methods with varying amounts of inductive bias to investigate what leads to the best performance. Based on the results we create a SOTA open-source model for SLI as PBE (+6% pass rate with a third of the parameters of the second-best LLM) and also highlight exciting future directions for PBE research.

Problem

• Computational Phonology
• Machine Learning Efficiency
• Historical Linguistics

Innovation

• Similarity Assessment
• Example Generation Strategies
• Speech Pattern Learning