Rasterized Steered Mixture of Experts for Efficient 2D Image Regression

2025-10-07

AI Summary

To address the high computational cost and deployment challenges of Steered Mixture of Experts (SMoE) in image regression tasks, this paper introduces rasterization into the SMoE framework for the first time, proposing an efficient 2D image reconstruction method. Our approach replaces conventional global iterative optimization with Gaussian-kernel-based rasterization rendering; incorporates an edge-aware gating mechanism to enable spatially adaptive sparse expert activation; and employs sparse parameter updates alongside memory-efficient model representation. The method natively supports super-resolution and denoising. It achieves reconstruction quality on par with the original SMoE while accelerating parameter updates by at least 3.2× and reducing GPU memory consumption by 47%. Experiments demonstrate competitive accuracy, inference speed, and sparsity across image reconstruction, super-resolution, and denoising tasks—effectively overcoming key practical deployment bottlenecks of SMoE.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Computer Vision: Multi-modal Vision
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

The Steered Mixture of Experts regression framework has demonstrated strong performance in image reconstruction, compression, denoising, and super-resolution. However, its high computational cost limits practical applications. This work introduces a rasterization-based optimization strategy that combines the efficiency of rasterized Gaussian kernel rendering with the edge-aware gating mechanism of the Steered Mixture of Experts. The proposed method is designed to accelerate two-dimensional image regression while maintaining the model's inherent sparsity and reconstruction quality. By replacing global iterative optimization with a rasterized formulation, the method achieves significantly faster parameter updates and more memory-efficient model representations. In addition, the proposed framework supports applications such as native super-resolution and image denoising, which are not directly achievable with standard rasterized Gaussian kernel approaches. The combination of fast rasterized optimization with the edge-aware structure of the Steered Mixture of Experts provides a new balance between computational efficiency and reconstruction fidelity for two-dimensional image processing tasks.

Problem

• Accelerating 2D image regression while preserving quality
• Reducing computational cost of Steered Mixture of Experts
• Enabling efficient super-resolution and image denoising

Innovation

• Rasterized optimization accelerates Steered Mixture of Experts
• Combines Gaussian kernel efficiency with edge-aware gating
• Replaces global iterative optimization for faster parameter updates