Beyond Single Character: Evaluating MLLMs for Sentence-Level Oracle Bone Inscription Understanding

2026-06-30

AI Summary

This study addresses the limitation of existing oracle bone script research, which predominantly focuses on individual character recognition while neglecting the semantic coherence and contextual dependencies inherent in divinatory inscriptions. To bridge this gap, the authors introduce S-OBI, the first sentence-level benchmark for oracle bone script understanding. S-OBI employs glyph substitution and compositional synthesis techniques to generate clear, standardized sentence-level images, thereby eliminating low-level visual noise and enabling focused evaluation of high-level semantic reasoning capabilities. The benchmark encompasses three core tasks: semantic matching, slot filling, and contextual inference, providing a systematic assessment of multimodal large language models. Experimental results reveal that current models remain inadequate in sentence-level comprehension, with visual perception errors frequently disrupting reasoning chains—highlighting their overreliance on isolated character recognition and underscoring the necessity and value of S-OBI in advancing deep semantic understanding of oracle bone script.

Abstract

Existing AI-assisted oracle bone inscription (OBI) visual recognition and understanding studies mainly focus on character-level, ignoring the long-form textual coherence and contextual dependencies embedded in complete divination charges. Recently, the powerful visual perception capabilities of multimodal large language models (MLLMs) have opened new possibilities for OBI information processing. In this work, we introduce S-OBI, a novel benchmark for evaluating MLLMs in Sentence-level OBI understanding. Instead of using noisy and incomplete rubbings as the visual input, S-OBI synthesizes clear and standardized sentence-level OBI instances through glyph substitution and composition. According to 95 original rubbings with translations that have been identified, corrected, and verified by experts, we replace characters in the original rubbings with corresponding clean glyph samples sourced from existing OBI datasets while preserving the overall inscriptional structure and semantic organization. This mitigates the influence of low-level distortions and enables a more focused evaluation of sentence-level OBI understanding. Based on this, we design semantic matching, semantic slot extraction, and contextual reasoning tasks and obtain 695 question-answer pairs. Experiments reveal the inferiority of contemporary MLLMs on sentence-level OBI understanding. In particular, visual perception errors in unmasked regions propagate through the reasoning chain, leading to erroneous predictions for masked characters, which indicates that sentence-level OBI understanding in current models remains strongly dependent on character-level recognition. Overall, S-OBI provides a diagnostic benchmark for evaluating whether MLLMs can move beyond isolated character recognition toward structured inscription-level understanding.

Problem

• oracle bone inscription
• sentence-level understanding
• multimodal large language models
• contextual dependencies
• textual coherence

Innovation

• sentence-level understanding
• multimodal large language models
• oracle bone inscription
• glyph synthesis
• structured semantic evaluation