Visual Search Augmented Chain-of-Thought Reasoning for Attribute Value Extraction from Product Videos

2026-09-06

AI Summary

为解决视频属性值提取问题，提出视觉搜索增强思维链推理方法（ViS-CoT），通过视觉聚类和语义相似产品知识检索来丰富属性线索，并迭代优化推理。

Abstract

Existing approaches to visual attribute value extraction (AVE) primarily rely on static product images, failing to capture temporal cues, multi-angle views and fine-grained visual details. Directly applying video vision-language models (VLMs) to product AVE results in limited performance due to the lack of domain knowledge, and fine-tuning them requires extensive high-quality data and substantial computational resources. Thus, we propose visual search augmented chain-of-thought reasoning (ViS-CoT), a training-free, plug-and-play pipeline that can be easily applied to any open-source video VLM for video-to-text AVE in e-Commerce. Specifically, ViS-CoT employs visual clustering to identify representative frames, followed by visual search to retrieve semantically similar product knowledge that can enrich attribute cues. Next, an interleaved CoT reasoning module iteratively refines reasoning through visually-aligned auxiliary texts derived from captioning and automatic speech recognition. Finally, the integrated information guides the model toward accurate and fine-grained attribute predictions. Extensive experiments across 14 product categories on the VideoAVE dataset show that ViS-CoT consistently enhances multiple state-of-the-art video VLMs, achieving an average improvement of 17.91 percentage points in micro-F1.

Problem

• Visual Attribute Value Extraction
• Video Vision-Language Models
• Temporal Cues
• Multi-angle Views
• Fine-grained Visual Details

Innovation

• Visual Search Augmented Chain-of-Thought Reasoning
• Video Vision-Language Models
• Attribute Value Extraction
• Plug-and-Play Pipeline
• Visual Clustering