Holographic Neural PCFG for Unsupervised Parsing

2026-07-08

AI Summary

This work addresses the challenge of accurately inferring syntactic tree structures from raw text under unsupervised conditions by proposing a neural probabilistic context-free grammar (PCFG) model based on holographic embeddings. It introduces holographic embeddings into unsupervised constituency parsing for the first time, explicitly modeling the algebraic relationships among parent nodes, left and right children, and lexical emissions through circular correlation operations under toroidal constraints, which yields closed-form rule probabilities. The model maintains high interpretability while reducing the number of rule-scoring parameters by 99.94%, significantly enhancing training stability and cross-lingual generalization. It achieves state-of-the-art performance in unsupervised parsing across six languages and, notably, enables the first character-level direct parsing for Japanese without word segmentation, attaining results comparable to token-level approaches.

Abstract

Unsupervised constituency parsing aims to accurately induce latent tree structures from raw text alone. Recent neural parameterizations of PCFGs achieve strong performance in both supervised and unsupervised parsing, yet rely on high-capacity black-box networks for rule scoring -- as exemplified by the Neural PCFG family -- leaving rule probabilities without an interpretable mathematical form. In this paper, we propose Holographic Neural PCFG (Hol-PCFG), which recasts PCFG rule scoring as algebraic relation modeling among grammar-symbol embeddings. Hol-PCFG adapts Holographic Embeddings (Nickel et al., 2016), which scores knowledge-graph triples via circular correlation, to the left-child, right-child, and lexical-emission relations over torus-constrained embeddings, giving every rule probability a closed form that carries the intrinsic structure of grammar rules by construction. Hol-PCFG achieves state-of-the-art parsing performance in six languages while cutting rule-scoring parameters by 99.94% relative to the baseline model and training more stably. Additionally, we demonstrate that Hol-PCFG can parse Japanese directly from characters without any morphological segmentation, retaining nearly the same morpheme-level performance.

Problem

• unsupervised parsing
• PCFG
• rule scoring
• interpretability
• grammar induction

Innovation

• Holographic Neural PCFG
• unsupervised parsing
• algebraic relation modeling
• circular correlation
• torus-constrained embeddings