Valley3: Scaling Omni Foundation Models for E-commerce

2026-05-02

AI Summary

This work addresses the challenge of unified understanding and reasoning across text, images, video, and audio in global e-commerce settings by proposing a native multilingual multimodal large language model that inherently supports audio. Through a four-stage progressive pre-training strategy, the model incrementally acquires capabilities in audio comprehension, cross-modal instruction following, domain-specific e-commerce knowledge, and long-context reasoning. During post-training, it incorporates a controllable reasoning mechanism—featuring non-thinking and three levels of depth-of-thought modes—and an active search agent architecture. Evaluated on both proprietary and public e-commerce benchmarks, the model significantly outperforms strong baselines while maintaining competitive performance on general multimodal tasks, demonstrating its effectiveness and generalization capacity in complex e-commerce applications.

Abstract

In this work, we present Valley3, an omni multimodal large language model (MLLM) developed for diverse global e-commerce tasks, with unified understanding and reasoning capabilities across text, images, video, and audio. A key feature of Valley3 is its native multilingual audio capability for e-commerce, developed by extending vision-language models to better support crucial audio-visual tasks, particularly in short-video scenarios. To achieve this, we carefully design a four-stage omni e-commerce continued pre-training pipeline, through which Valley3 progressively acquires audio understanding, cross-modal instruction-following, e-commerce domain knowledge, and long-context reasoning capabilities, ultimately evolving into an omni model for diverse e-commerce scenarios. Then, we further improve Valley3 through post-training to encourage long-chain reasoning with controllable reasoning modes, enabling one non-thinking mode and three distinct levels of thinking, thereby balancing inference efficiency in simple scenarios with deep reasoning for complex applications. Moreover, we equip Valley3 with agentic search capabilities to proactively invoke search tools and acquire task-relevant information for e-commerce deep research tasks. To comprehensively assess the capabilities of Valley3, we construct an omni e-commerce benchmark spanning 6 tasks. Experimental results show that Valley3 consistently outperforms strong baselines on our in-house and open-source e-commerce benchmarks, while remaining competitive on general-domain benchmarks.

Problem

• omni foundation model
• e-commerce
• multimodal understanding
• multilingual audio
• cross-modal reasoning

Innovation

• omni multimodal
• audio-visual understanding
• controllable reasoning
• agentic search
• e-commerce foundation model