OntologyBench: Can Dense Retrieval Satisfy Structured Biomedical Constraints?

2026-09-08

AI Summary

研究通过OntologyBench评估密集检索在生物医学约束下的表现，发现现有方法在关系和组合任务上效果不佳，需要改进以更好地结合结构化知识。

Abstract

We introduce OntologyBench, a tiered biomedical retrieval benchmark comprising 471,854 training and 125,744 evaluation query-document relevance pairs across concept grounding, relational retrieval, and compositional phenotype-based retrieval. Although these tasks can be tractable using ontology-aware reference methods, across task tiers, embedding performance is generally lower on relational and compositional tasks than on concept-grounding tasks. Fine-tuning on ontology-derived supervision improves performance on several relational and compositional tasks, whereas the evaluated reranking and LLM-based candidate-scoring methods provide little or no end-to-end improvement. Errors frequently reflect diseases matching only subsets of the phenotype evidence. These findings indicate that the evaluated embedding and reranking configurations do not reliably recover the compatibility encoded by the selected ontology relations and phenotype combinations and motivate retrieval systems that better integrate learned representations with structured biomedical knowledge.

Problem

• biomedical retrieval
• ontology relations
• phenotype combinations
• embedding performance

Innovation

• OntologyBench
• dense retrieval
• biomedical constraints
• ontology-derived supervision
• phenotype-based retrieval