EventFlash: Towards Efficient MLLMs for Event-Based Vision

2026-02-03

AI Summary

This work addresses the inefficiency of existing event-based multimodal large language models (MLLMs), which adopt dense image processing paradigms and neglect the inherent spatiotemporal sparsity of event streams, leading to computational redundancy and suboptimal inference speed. To overcome this limitation, we propose EventFlash, the first sparse MLLM architecture tailored for event vision. EventFlash leverages spatiotemporal token sparsification, adaptive temporal window aggregation, and density-guided sparse attention to efficiently model long event sequences. Integrated with a curriculum training strategy and EventMind—a large-scale, multi-granularity instruction dataset—EventFlash supports inputs up to 1,000 time bins, achieving a 12.4× higher inference throughput than the baseline EventFlash-Zero while maintaining comparable performance, thereby substantially surpassing prior methods like EventGPT that are restricted to only 5 time bins.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Event-based multimodal large language models (MLLMs) enable robust perception in high-speed and low-light scenarios, addressing key limitations of frame-based MLLMs. However, current event-based MLLMs often rely on dense image-like processing paradigms, overlooking the spatiotemporal sparsity of event streams and resulting in high computational cost. In this paper, we propose EventFlash, a novel and efficient MLLM to explore spatiotemporal token sparsification for reducing data redundancy and accelerating inference. Technically, we build EventMind, a large-scale and scene-diverse dataset with over 500k instruction sets, providing both short and long event stream sequences to support our curriculum training strategy. We then present an adaptive temporal window aggregation module for efficient temporal sampling, which adaptively compresses temporal tokens while retaining key temporal cues. Finally, a sparse density-guided attention module is designed to improve spatial token efficiency by selecting informative regions and suppressing empty or sparse areas. Experimental results show that EventFlash achieves a $12.4\times$ throughput improvement over the baseline (EventFlash-Zero) while maintaining comparable performance. It supports long-range event stream processing with up to 1,000 bins, significantly outperforming the 5-bin limit of EventGPT. We believe EventFlash serves as an efficient foundation model for event-based vision.

Problem

• event-based vision
• multimodal large language models
• spatiotemporal sparsity
• computational efficiency
• event streams

Innovation

• event-based vision
• spatiotemporal sparsification
• multimodal large language models
• adaptive temporal aggregation
• sparse attention