Is neural semantic parsing good at ellipsis resolution, or isn't it?

2025-07-31

AI Summary

This work systematically evaluates neural semantic parsers’ capability to handle English verb phrase ellipsis (VP ellipsis), a highly context-sensitive linguistic phenomenon. To address the prevalent deficiency in semantic copying during ellipsis resolution, we construct the first challenge set comprising 120 high-quality, manually annotated VP-ellipsis instances and conduct rigorous evaluation using fully resolved semantic representations. Experimental results reveal that while state-of-the-art neural parsers achieve strong performance on standard benchmarks, they consistently fail on this challenge set—exposing fundamental limitations in modeling long-range semantic dependencies and cross-sentential information copying. This study provides the first diagnostic, systematic assessment of VP ellipsis in neural semantic parsing, identifying data sparsity and structural bias as critical bottlenecks. Furthermore, we empirically demonstrate that targeted data augmentation specifically designed for ellipsis structures significantly improves resolution accuracy. Our findings offer a novel pathway toward enhancing the contextual robustness of neural semantic parsers.

Technology Category

• Natural Language Processing: Lexical Semantics and Morphology
• Knowledge Representation and Reasoning: Common-Sense Reasoning
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Neural semantic parsers have shown good overall performance for a variety of linguistic phenomena, reaching semantic matching scores of more than 90%. But how do such parsers perform on strongly context-sensitive phenomena, where large pieces of semantic information need to be duplicated to form a meaningful semantic representation? A case in point is English verb phrase ellipsis, a construct where entire verb phrases can be abbreviated by a single auxiliary verb. Are the otherwise known as powerful semantic parsers able to deal with ellipsis or aren't they? We constructed a corpus of 120 cases of ellipsis with their fully resolved meaning representation and used this as a challenge set for a large battery of neural semantic parsers. Although these parsers performed very well on the standard test set, they failed in the instances with ellipsis. Data augmentation

Problem

• Evaluating neural parsers on ellipsis resolution
• Assessing performance on context-sensitive linguistic phenomena
• Testing semantic parsers with verb phrase ellipsis cases

Innovation

• Neural semantic parsers tested on ellipsis
• Corpus of 120 ellipsis cases constructed
• Data augmentation used for improvement