Efficient Universal Perception Encoder

2026-03-23

AI Summary

This work addresses the challenge of achieving efficient multi-task visual perception on resource-constrained edge devices by balancing model compactness with representational generality. The authors propose a two-stage knowledge distillation framework: first, multiple domain-specific expert models are fused to construct a high-capacity proxy teacher; then, a unified lightweight universal visual encoder (EUPE) is distilled from this proxy. This approach overcomes the limitations of conventional direct compression from multiple teachers by integrating multi-teacher knowledge distillation, proxy teacher modeling, and cross-task representation learning. The resulting encoder matches or exceeds the performance of similarly sized task-specific models across diverse downstream tasks and outperforms existing aggregated encoders. Code and models will be made publicly available.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Running AI models on smart edge devices can unlock versatile user experiences, but presents challenges due to limited compute and the need to handle multiple tasks simultaneously. This requires a vision encoder with small size but powerful and versatile representations. We present our method, Efficient Universal Perception Encoder (EUPE), which offers both inference efficiency and universally good representations for diverse downstream tasks. We achieve this by distilling from multiple domain-expert foundation vision encoders. Unlike previous agglomerative methods that directly scale down from multiple teachers to an efficient encoder, we demonstrate the importance of first scaling up to a large proxy teacher and then scaling down from this single teacher. Experiments show that EUPE achieves on-par or better performance than individual domain experts of the same size on diverse task domains and also outperforms previous agglomerative encoders. We will release the full family of EUPE models and the code to foster future research.

Problem

• edge AI
• vision encoder
• model efficiency
• universal representation
• multi-task learning

Innovation

• Efficient Universal Perception Encoder
• knowledge distillation
• proxy teacher
• multi-task vision encoder
• edge AI