🤖 AI Summary
This work addresses the challenge of generating personalized, factually accurate, and topically relevant reading materials tailored to user-specified queries and target readability levels. To this end, the authors propose a four-module system that integrates retrieval-augmented generation (RAG) with large language models (LLMs), uniquely combining RAG with multiple prompting strategies—including Chain-of-Thought, zero-shot, and few-shot prompting—for personalized reading recommendations. The system further incorporates an LLM-as-a-Judge mechanism to automatically evaluate the factual accuracy, relevance, and readability alignment of generated content. Experimental results demonstrate that RAG consistently enhances the performance of diverse models—including LLaMA 4 Scout, LLaMA 3.1 8B, and Gemma2 9B—across all prompting strategies, yielding improvements of up to 26–35 percentage points in relevance and factual accuracy, thereby enabling high-quality customized reading material generation.
📝 Abstract
This work presents the design, implementation, and evaluation of a system for generating personalized reading content using Large Language Models (LLMs) combined with Retrieval-Augmented Generation (RAG). The proposed architecture consists of four modules: Input, RAG, Generation, and Judging and enables users to specify both a question and a target reading content complexity. RAG is employed to retrieve relevant information from the Internet, enriching and grounding the content produced by three modern LLMs: Meta LLaMA 4 Scout, LLaMA 3.1 8B Instant, and Google Gemma2 9B. Reading materials are generated using three prompting strategies (Chain-of-Thought, zero-shot, and few-shot), and the LLM-as-a-Judge module automatically evaluates answer quality and alignment with the desired readability level. Experimental results show that RAG consistently improves system performance across all models and prompting techniques, increasing relevance and particularly groundedness by up to 26-35 percentage points. Overall, the findings demonstrate that the RAG-augmented architecture effectively produces reading content tailored to user queries and desired textual complexity.