From Reasoning to Learning: A Survey on Hypothesis Discovery and Rule Learning with Large Language Models

2025-05-28

AI Summary

This paper investigates whether large language models (LLMs) can transcend information retrieval and instruction following to achieve genuine novel knowledge discovery. Method: Grounded in Peirce’s abductive–deductive–inductive triadic logic, it establishes the first unified analytical framework for LLM-driven hypothesis generation toward AGI, systematically characterizing critical pathways and fundamental bottlenecks in generative knowledge discovery. It proposes a closed-loop “hypothesis generation–application–validation” technical architecture integrating prompt engineering, self-verifying reasoning, rule distillation, and empirical evaluation. Contribution/Results: Synthesizing over 100 state-of-the-art studies, the work identifies key advances—including transferable hypothesis modeling, domain-adaptive validation, and enhanced causal interpretability—while revealing six persistent challenges: weak falsifiability, poor cross-domain generalization, among others. The framework provides both theoretical grounding and methodological foundations for evolving LLMs into scientific innovation engines.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Since the advent of Large Language Models (LLMs), efforts have largely focused on improving their instruction-following and deductive reasoning abilities, leaving open the question of whether these models can truly discover new knowledge. In pursuit of artificial general intelligence (AGI), there is a growing need for models that not only execute commands or retrieve information but also learn, reason, and generate new knowledge by formulating novel hypotheses and theories that deepen our understanding of the world. Guided by Peirce's framework of abduction, deduction, and induction, this survey offers a structured lens to examine LLM-based hypothesis discovery. We synthesize existing work in hypothesis generation, application, and validation, identifying both key achievements and critical gaps. By unifying these threads, we illuminate how LLMs might evolve from mere ``information executors'' into engines of genuine innovation, potentially transforming research, science, and real-world problem solving.

Problem

• Can LLMs discover new knowledge beyond deductive reasoning
• Need for models to generate novel hypotheses for AGI
• Survey examines LLM-based hypothesis discovery and validation gaps

Innovation

• LLMs evolve from information executors to innovation engines
• Survey synthesizes hypothesis generation, application, validation
• Guided by Peirce's abduction, deduction, induction framework