Evolving Deeper LLM Thinking

2025-01-17

AI Summary

Large language models (LLMs) often exhibit insufficient reasoning depth in natural language planning tasks and rely heavily on manually designed pipelines or formal solvers. Method: This paper proposes Mind Evolution—a self-supervised, inference-time optimization framework that leverages LLMs’ intrinsic capabilities to autonomously generate, recombine, and iteratively evolve responses without explicit task-structure modeling. Contribution/Results: The approach introduces the first response evolution mechanism grounded entirely in LLMs’ internal reasoning capacities, integrating an evaluator-driven unsupervised selection strategy. Evaluated on natural language planning benchmarks (TravelPlanner and Natural Plan), it achieves >98% success rate on Gemini 1.5 Pro—substantially outperforming Best-of-N and Sequential Revision baselines—while maintaining controllable inference cost and scalability.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We explore an evolutionary search strategy for scaling inference time compute in Large Language Models. The proposed approach, Mind Evolution, uses a language model to generate, recombine and refine candidate responses. The proposed approach avoids the need to formalize the underlying inference problem whenever a solution evaluator is available. Controlling for inference cost, we find that Mind Evolution significantly outperforms other inference strategies such as Best-of-N and Sequential Revision in natural language planning tasks. In the TravelPlanner and Natural Plan benchmarks, Mind Evolution solves more than 98% of the problem instances using Gemini 1.5 Pro without the use of a formal solver.

Problem

• Large Language Models
• Deep and Efficient Reasoning
• Natural Language Planning Tasks

Innovation

• Mind Evolution
• Natural Language Processing
• Enhanced Search Strategy