MeshOn: Intersection-Free Mesh-to-Mesh Composition

2026-04-09

AI Summary

This work addresses the common challenges of geometric interpenetration and semantically implausible configurations in 3D mesh composition by proposing a multi-stage optimization framework. The approach first leverages a vision-language model to guide semantic alignment within user-specified regions, followed by joint optimization incorporating a geometric attraction loss, a physics-inspired barrier loss to prevent penetration, and deformation guided by diffusion priors. This method uniquely integrates vision-language models, physical constraints, and diffusion-based shape priors into a unified pipeline, achieving robust, high-fidelity, and intersection-free mesh assembly across diverse materials and target regions. It significantly outperforms existing generative and traditional registration techniques and seamlessly integrates into digital content creation workflows.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

We propose MeshOn, a method that finds physically and semantically realistic compositions of two input meshes. Given an accessory, a base mesh with a user-defined target region, and optional text strings for both meshes, MeshOn uses a multi-step optimization framework to realistically fit the meshes onto each other while preventing intersections. We initialize the shapes'rigid configuration via a structured alignment scheme using Vision-to-Language Models, which we then optimize using a combination of attractive geometric losses, and a physics-inspired barrier loss that prevents surface intersections. We then obtain a final deformation of the object, assisted by a diffusion prior. Our method successfully fits accessories of various materials over a breadth of target regions, and is designed to fit directly into existing digital artist workflows. We demonstrate the robustness and accuracy of our pipeline by comparing it with generative approaches and traditional registration algorithms.

Problem

• mesh composition
• intersection-free
• 3D shape fitting
• geometric alignment
• physical realism

Innovation

• intersection-free composition
• mesh registration
• physics-inspired barrier loss
• diffusion prior
• vision-to-language alignment