Zero-Shot Complex Question-Answering on Long Scientific Documents

2025-03-04

AI Summary

Social science researchers lack zero-shot complex question-answering tools capable of directly processing lengthy scholarly papers to perform multi-span extraction, multi-hop reasoning, and long-answer generation—without requiring machine learning expertise. Method: We propose the first zero-shot framework for this task, integrating extractive (SpanBERT) and generative (T5/LLaMA) models into an end-to-end pipeline explicitly optimized for long scientific documents. Contribution/Results: Evaluated on MLPsych—a newly constructed benchmark dataset in social psychology—the framework achieves state-of-the-art performance in both multi-span localization accuracy and long-answer coherence, significantly outperforming existing baselines. It effectively bridges the critical gap in zero-shot deep understanding of social science literature, enabling domain experts to engage with complex texts without model fine-tuning or annotation effort.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

With the rapid development in Transformer-based language models, the reading comprehension tasks on short documents and simple questions have been largely addressed. Long documents, specifically the scientific documents that are densely packed with knowledge discovered and developed by humans, remain relatively unexplored. These documents often come with a set of complex and more realistic questions, adding to their complexity. We present a zero-shot pipeline framework that enables social science researchers to perform question-answering tasks that are complex yet of predetermined question formats on full-length research papers without requiring machine learning expertise. Our approach integrates pre-trained language models to handle challenging scenarios including multi-span extraction, multi-hop reasoning, and long-answer generation. Evaluating on MLPsych, a novel dataset of social psychology papers with annotated complex questions, we demonstrate that our framework achieves strong performance through combination of extractive and generative models. This work advances document understanding capabilities for social sciences while providing practical tools for researchers.

Problem

• Addresses zero-shot question-answering on long scientific documents
• Handles complex questions with multi-span extraction and reasoning
• Provides tools for social science researchers without ML expertise

Innovation

• Zero-shot pipeline for complex QA tasks
• Integrates pre-trained language models
• Combines extractive and generative models