🤖 AI Summary
This work investigates how to effectively leverage pretrained encoder-decoder models to enhance constituent parsing performance, with a focus on both continuous and discontinuous syntactic structures. The authors formulate constituent parsing as a sequence-to-sequence task and present the first systematic exploration of architectures such as BART, mBART, and T5 for this purpose, evaluating multiple tree linearization strategies. Experimental results demonstrate that the proposed approach outperforms existing seq2seq models on continuous constituent parsing, achieving performance on par with state-of-the-art specialized parsers. Moreover, it exhibits strong generalization capabilities on benchmarks involving discontinuous structures, underscoring the potential of pretrained encoder-decoder frameworks to uniformly handle both structural paradigms within a single model.
📝 Abstract
To achieve deep natural language understanding, syntactic constituent parsing plays a crucial role and is widely required by many artificial intelligence systems for processing both text and speech. A recent approach involves using standard sequence-to-sequence models to handle constituent parsing as a machine translation problem, moving away from traditional task-specific parsers. These models are typically initialized with pre-trained encoder-only language models like BERT or RoBERTa. However, the use of pre-trained encoder-decoder language models for constituency parsing has not been thoroughly explored. To bridge this gap, we extend the sequence-to-sequence framework by investigating parsers built on pre-trained encoder-decoder architectures, including BART, mBART, and T5. We fine-tune them to generate linearized parse trees and extensively evaluate them on different linearization strategies across both continuous treebanks and more complex discontinuous benchmarks. Our results demonstrate that our approach outperforms all prior sequence-to-sequence models and performs competitively with leading task-specific constituent parsers on continuous constituent parsing.