Inject or Navigate? Token-Efficient Retrieval for LLM Analysis of Transactional Legal Documents
This study addresses the challenge of high token consumption and performance degradation in transactional legal document question answering when naively injecting entire corpora into large language models. The authors propose a structure-aware chunking strategy and comparatively evaluate three approaches: full corpus injection, embedding-based retrieval (NAVEMBED), and a novel large model navigation framework leveraging a compact structural index (NAVINDEX). NAVINDEX introduces an innovative navigation mechanism coupled with a cached intersection cost model, achieving substantial reductions in token usage—1.61× fewer total tokens and a 56× smaller context window—alongside a 25% cost reduction, while maintaining answer quality on par with full injection. Experimental results show that NAVEMBED matches full injection on 16 out of 18 tasks with 17.3× fewer tokens, whereas NAVINDEX consistently matches performance across all tasks, demonstrating superior efficiency and cost-effectiveness.