PetQA: Benchmarking Veterinary Knowledge and Clinical Reasoning

2026-09-03

AI Summary

研究通过创建PetQA基准，使用多种模型评测方法解决兽医知识和临床推理在大语言模型中的应用问题。

Abstract

We introduce PetQA, a Korean long-form question-answering (QA) benchmark for evaluating veterinary knowledge and clinical reasoning in large language models (LLMs) and large vision-language models (LVLMs). PetQA contains 10,076 text-only and 8,751 multimodal QA pairs derived from real-world questions about dogs and cats, paired with answers from expert veterinarians. Its test split, PetQA-Bench, further includes annotations for question types and clinical conditions. We evaluate eighteen models using ROUGE, BERTScore, and LLM-as-a-judge metrics for factuality and helpfulness under three settings: zero-shot inference, retrieval-augmented generation (RAG), and supervised fine-tuning (SFT). The benchmarking results provide an overview of the strengths and limitations of current models in addressing veterinary clinical queries and highlight the need for more effective adaptation methods to develop clinically reliable AI systems for veterinary care. To facilitate broader use, we additionally provide translated versions of PetQA-Bench in five languages.

Problem

• veterinary knowledge
• clinical reasoning
• large language models
• multimodal QA

Innovation

• PetQA
• veterinary knowledge
• clinical reasoning
• multimodal QA
• large language models