Dolphin: Closed-loop Open-ended Auto-research through Thinking, Practice, and Feedback

2025-01-07

AI Summary

Scientific research remains heavily reliant on manual intervention, limiting automation and scalability. Method: This paper introduces the first closed-loop, open-domain autonomous scientific research framework. It integrates literature-driven hypothesis generation, anomaly-guided structured code synthesis and debugging, and result-feedback-driven iterative method redesign—enabling end-to-end automation from problem formulation and algorithm invention to implementation and experimental validation. Contribution/Results: The framework establishes the first self-evolving research paradigm in open domains, enabling cross-task continuous generation of high-quality novel methods without human intervention while approaching state-of-the-art (SOTA) performance. Empirical evaluation on 2D image classification and 3D point cloud classification demonstrates that autonomously discovered algorithms match or rival current SOTA methods. Extensive benchmarking across multiple domains validates its sustained innovation capability and robust closed-loop execution.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Search and Optimization: Metareasoning and Metaheuristics
• Computer Vision: Vision for Robotics & Autonomous Driving

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Search and Retrieval-Augmented AI: Vertical and domain-specific search

Abstract

The scientific research paradigm is undergoing a profound transformation owing to the development of Artificial Intelligence (AI). Recent works demonstrate that various AI-assisted research methods can largely improve research efficiency by improving data analysis, accelerating computation, and fostering novel idea generation. To further move towards the ultimate goal (i.e., automatic scientific research), in this paper, we propose Dolphin, the first closed-loop open-ended auto-research framework to further build the entire process of human scientific research. Dolphin can generate research ideas, perform experiments, and get feedback from experimental results to generate higher-quality ideas. More specifically, Dolphin first generates novel ideas based on relevant papers which are ranked by the topic and task attributes. Then, the codes are automatically generated and debugged with the exception-traceback-guided local code structure. Finally, Dolphin automatically analyzes the results of each idea and feeds the results back to the next round of idea generation. Experiments are conducted on the benchmark datasets of different topics and results show that Dolphin can generate novel ideas continuously and complete the experiment in a loop. We highlight that Dolphin can automatically propose methods that are comparable to the state-of-the-art in some tasks such as 2D image classification and 3D point classification.

Problem

• Automated Scientific Research
• Artificial Intelligence
• Data Analysis and Experiment Design

Innovation

• Dolphin
• Automated Scientific Research
• Iterative Self-improvement