ADAM: A Diverse Archive of Mankind for Evaluating and Enhancing LLMs in Biographical Reasoning

2025-09-26

AI Summary

This paper addresses the pervasive hallucination problem and lack of standardized evaluation in biographical reasoning by multimodal large language models (MLLMs), proposing Adam—the first systematic framework for this task. Methodologically, Adam comprises: (1) AdamDB, a multilingual biographical knowledge base covering over 4 million individuals; (2) AdamBench, a hierarchical evaluation benchmark grounded in Bloom’s taxonomy, which—through empirical analysis—first uncovers a strong positive correlation between entity popularity and model accuracy; (3) AdamRAG, a biographical-scenario-specific retrieval-augmented generation method that effectively mitigates hallucinations, yielding significant gains in low-level reasoning on open-source models and robust improvements on closed-source models; and (4) an ablation study demonstrating that multimodal (image-text joint) input offers limited and unstable benefits, whereas structured retrieval provides more universally effective hallucination suppression.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We introduce ADAM (A Diverse Archive of Mankind), a framework for evaluating and improving multimodal large language models (MLLMs) in biographical reasoning. To the best of our knowledge, this is the first work to systematically examine LLM capabilities in biography, a critical yet underexplored dimension of factual knowledge. At its core, AdamDB is a multilingual and multimodal dataset covering over 4 million individuals across geography, time, and profession, while AdamBench provides cognitively structured evaluations based on Bloom's taxonomy, spanning six reasoning levels in both English and native languages. To address hallucinations, particularly for lesser-known individuals, we propose AdamRAG, a retrieval-augmented generation system tailored to biographical contexts. Experiments show that AdamRAG substantially improves open-source models and modestly benefits closed-source ones, with the largest gains on lower-order reasoning. Popularity strongly mediates accuracy, and multimodal input via face images offers smaller, less consistent improvements than retrieval. ADAM establishes the first benchmark and framework for cognitively, culturally, and multimodally grounded biographical evaluation, advancing the development of multilingual, accurate, and hallucination-resistant MLLMs.

Problem

• Evaluating multimodal LLMs' biographical reasoning across cultures and languages
• Addressing hallucinations in biographical knowledge for lesser-known individuals
• Establishing first benchmark for cognitively-grounded biographical evaluation

Innovation

• Multimodal multilingual dataset covering 4 million individuals
• Retrieval-augmented generation system for biographical contexts
• Cognitively structured evaluation benchmark using Bloom's taxonomy