Evolutionary Context Search for Automated Skill Acquisition

2026-02-17

AI Summary

This work addresses the challenge of enabling large language models to acquire new skills post-deployment by effectively leveraging external textual resources. Traditional retrieval-augmented approaches, constrained by semantic similarity, often fail to identify context that genuinely enhances task performance. To overcome this limitation, the authors propose Evolutionary Context Search (ECS), a method that automatically discovers non-intuitive yet effective context combinations through evolutionary optimization on a small development set, using task accuracy as the objective. ECS requires no model weight updates and operates solely through inference calls and task-specific feedback, rendering it model-agnostic and supporting cross-model context transfer. Experimental results demonstrate that ECS improves performance by 27% on BackendBench and 7% on the τ-bench airline task, with contexts optimized for Gemini-1.5-Flash successfully transferring to Claude Sonnet and DeepSeek.

Technology Category

• Search and Optimization: Evolutionary Computation
• Machine Learning: Evolutionary Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Large Language Models cannot reliably acquire new knowledge post-deployment -- even when relevant text resources exist, models fail to transform them into actionable knowledge without retraining. Retrieval-Augmented Generation attempts to bridge this gap by surfacing relevant documents at inference time, yet similarity-based retrieval often fails to identify context that actually improves task performance. We introduce Evolutionary Context Search (ECS), an evolutionary method that searches context combinations using accuracy on a small development set, requiring only inference calls without weight updates. ECS moves beyond semantic similarity to discover non-obvious context pairings that significantly boost performance. Our empirical results show that ECS improves BackendBench by 27\% and $τ$-bench airline by 7\%. The evolved contexts are model-agnostic, as those evolved with Gemini-3-Flash transfer effectively to Claude Sonnet and DeepSeek. This suggests that ECS opens a path toward automated context discovery for skill acquisition -- an efficient alternative to manual prompt engineering or costly fine-tuning.

Problem

• skill acquisition
• context retrieval
• large language models
• knowledge updating
• retrieval-augmented generation

Innovation

• Evolutionary Context Search
• Retrieval-Augmented Generation
• automated skill acquisition
• model-agnostic context
• inference-time optimization