HomeDiffusion: Zero-Shot Object Customization with Multi-View Representation Learning for Indoor Scenes

2026-06-29

AI Summary

This work addresses the limitations of existing zero-shot object customization methods, which often fail to preserve fine-grained details—such as texture and curvature—and struggle with pose coherence, particularly when integrating asymmetric objects into indoor scenes. To overcome these challenges, the authors propose a novel diffusion-based approach that introduces multi-view representation learning for the first time in this context. By leveraging multi-view images of the same reference object, the method extracts high-fidelity details in the latent space and integrates them with noisy latents through a cross-attention mechanism. Trained on a newly curated dataset of multi-view furniture items within indoor environments, the model significantly improves pose plausibility and detail consistency. Experimental results demonstrate that the proposed method outperforms current zero-shot and few-shot customization approaches both qualitatively and quantitatively, yielding more photorealistic and scene-coherent generations.

Abstract

Recently, zero-shot object customization generation methods have rapidly developed and shown tremendous potential for applications. For instance, in the e-commerce domain, consumers can observe the visual effect of furniture placed within their personal living spaces or clothes worn on their own bodies. Many existing approaches perform object customization generation based on diffusion models and extracted reference object features. However, the generated object significantly diverges from the original reference object in details such as patterns and curves. Particularly for asymmetrical reference objects, the absence of comprehensive multi-viewpoint information prevents the generation of object poses that harmonize with the background scene. To address these shortcomings, we have constructed a novel dataset comprising multi-angle images of furniture and indoor scenes. Based on diffusion models, we introduce HomeDiffusion, which can leverage multi-viewpoint images of the same reference object to accurately generate visually harmonious object poses within specified areas of the background scene. During the diffusion process, we further extract high-fidelity details of the reference object and perform cross-attention with the noise latents in the latent space, thereby ensuring the preservation of details in the customized object generation. Extensive qualitative and quantitative experiments demonstrate that our method achieves superior performance over other existing zero-shot as well as few-shot object customization approaches.

Problem

• zero-shot object customization
• multi-view representation
• indoor scene generation
• detail preservation
• object-scene harmonization

Innovation

• Zero-shot object customization
• Multi-view representation learning
• Diffusion models
• Cross-attention mechanism
• Indoor scene generation