RAGSmith: A Framework for Finding the Optimal Composition of Retrieval-Augmented Generation Methods Across Datasets
Traditional RAG systems suffer from suboptimal performance due to tight coupling among retrieval, reranking, prompt rewriting, and generation modules, hindering holistic optimization. Method: This paper proposes the first end-to-end RAG architecture search framework, modeling the RAG configuration space as an evolvable search problem and employing genetic algorithms to jointly optimize multi-objective metrics—including recall@k/nDCG for retrieval and LLM-Judge/semantic similarity for generation. The framework encompasses nine component types across vector retrieval, reranking, and prompt rewriting. Contribution/Results: Evaluated across six domains, the framework achieves an average 3.8% performance gain (up to +12.5% in retrieval, +7.5% in generation) while converging after exploring only 0.2% of the configuration space. It identifies robust architectural patterns transferable across datasets and quantifies how domain characteristics and question types systematically influence optimal configurations.