Kwai Keye-VL 1.5 Technical Report

2025-09-01

AI Summary

Video understanding faces dual challenges of dynamic content and high information density, where existing models struggle to balance spatial resolution with temporal coverage. To address this, we propose a Slow-Fast video encoding strategy that dynamically allocates computational resources between key frames and static frames. We design a four-stage progressive pretraining paradigm, extending context length to 128K tokens. Furthermore, we introduce a comprehensive post-training framework integrating inference enhancement, human preference alignment, and progressive prompting, augmented by a five-step chain-of-thought data construction pipeline and GSPO-driven iterative reinforcement learning. Our method achieves significant improvements over state-of-the-art models on public video understanding benchmarks and internal human evaluations, while maintaining leading performance across general multimodal tasks.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

In recent years, the development of Large Language Models (LLMs) has significantly advanced, extending their capabilities to multimodal tasks through Multimodal Large Language Models (MLLMs). However, video understanding remains a challenging area due to the dynamic and information-dense nature of videos. Existing models struggle with the trade-off between spatial resolution and temporal coverage when processing video content. We present Keye-VL-1.5, which addresses fundamental challenges in video comprehension through three key innovations. First, we introduce a novel Slow-Fast video encoding strategy that dynamically allocates computational resources based on inter-frame similarity, processing key frames with significant visual changes at higher resolution (Slow pathway) while handling relatively static frames with increased temporal coverage at lower resolution (Fast pathway). Second, we implement a progressive four-stage pre-training methodology that systematically extends the model's context length from 8K to 128K tokens, enabling processing of longer videos and more complex visual content. Third, we develop a comprehensive post-training pipeline focusing on reasoning enhancement and human preference alignment, incorporating a 5-step chain-of-thought data construction process, iterative GSPO-based reinforcement learning with progressive prompt hinting for difficult cases, and alignment training. Through extensive evaluation on public benchmarks and rigorous internal human assessment, Keye-VL-1.5 demonstrates significant improvements over existing models, particularly excelling in video understanding tasks while maintaining competitive performance on general multimodal benchmarks.

Problem

• Addressing video understanding challenges in multimodal language models
• Balancing spatial resolution and temporal coverage trade-offs
• Enhancing long-context video processing and human alignment

Innovation

• Slow-Fast video encoding strategy for dynamic resource allocation
• Progressive four-stage pre-training for extended context length
• Comprehensive post-training pipeline for reasoning enhancement