Vitality-Aware Compression for Efficient Image-to-Shape Diffusion Transformers

2026-06-30

AI Summary

This work addresses the high computational cost of existing diffusion Transformer models for image-to-3D shape generation, which hinders their deployment in resource-constrained environments, and the inadequacy of generic compression techniques in preserving geometric fidelity. To overcome these limitations, we propose the first geometry-aware compression framework tailored for such models. Our approach introduces a saliency-aware mechanism to evaluate the importance of each layer in geometric generation, guiding structured pruning, adaptive quantization, and direction-specific fine-tuning. The resulting method is plug-and-play and achieves up to 66% model size reduction across multiple state-of-the-art image-to-3D generation models while maintaining geometric fidelity comparable to that of the original large models.

Abstract

We propose the first compression approach for image-to-shape Diffusion Transformers (DiTs) that substantially reduces model size while preserving geometric fidelity. Despite remarkable progress in 3D shape generation, large DiT-based models remain computationally prohibitive in resource-constrained settings. Furthermore, it is difficult to directly transfer existing diffusion model compression strategies developed for different domains to 3D generation, and prior 3D efficiency approaches focus primarily on inference speed rather than backbone compression. To address this limitation, we build a geometry-aware compression framework tailored to image-to-shape DiTs. Guided by the observation that 3D DiT layers exhibit non-uniform importance for geometry synthesis, we introduce a vitality-guided framework integrating structured pruning, adaptive quantization, and targeted fine-tuning. Our method achieves up to 66% model-size reduction across state-of-the-art image-to-3D models while maintaining synthesis fidelity comparable to full-sized counterparts. This highlights the potential of our framework as a plug-and-play solution for efficient 3D shape generation across diverse models.

Problem

• Diffusion Transformers
• model compression
• 3D shape generation
• resource-constrained settings
• geometric fidelity

Innovation

• Diffusion Transformers
• model compression
• structured pruning
• adaptive quantization
• 3D shape generation