Qwen Goes Brrr: Off-the-Shelf RAG for Ukrainian Multi-Domain Document Understanding
This work addresses the challenge of automatically answering multiple-choice questions from Ukrainian multi-domain PDF documents while simultaneously locating supporting evidence. The authors propose a retrieval-augmented question-answering pipeline that integrates context-aware PDF chunking, dense retrieval informed by both questions and answer options, and a reranking stage enhanced with awareness of the answer space. A constrained decoding mechanism is introduced during answer generation to ensure correctness under strict competition constraints, preserving document structure without relying on complex post-processing. The system leverages Qwen3-Embedding-8B for retrieval, a fine-tuned Qwen3-Reranker-8B for reranking, and Qwen3-32B for final answer selection. Reranking improves Recall@1 from 0.6957 to 0.7935, and using the top two retrieved passages boosts accuracy from 0.9348 to 0.9674, achieving a private leaderboard score of 0.9598.