SARe: Structure-Aware Large-Scale 3D Fragment Reassembly

2026-03-23

AI Summary

This work addresses the challenging problem of reassembling large-scale, unordered 3D fragments with unknown target shapes and weak semantic cues. Existing approaches often suffer from cascading failures as fragment count increases, primarily due to unreliable contact reasoning. To overcome this, we propose SARe, a structure-aware two-stage framework. In the generation stage (SARe-Gen), a query-point mechanism jointly predicts fracture labels and a contact graph. The refinement stage (SARe-Refine) then verifies geometric consistency to identify reliable substructures and resamples ambiguous regions. Notably, SARe requires no pretraining on structural priors, explicitly models contact relationships, and dynamically refines assemblies, substantially improving robustness. Our method achieves state-of-the-art performance across synthetic, scan-simulated, and physically fractured datasets, maintaining high success rates and graceful performance degradation even as fragment counts grow.

Technology Category

• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Computer Vision: Segmentation
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

3D fragment reassembly aims to recover the rigid poses of unordered fragment point clouds or meshes in a common object coordinate system to reconstruct the complete shape. The problem becomes particularly challenging as the number of fragments grows, since the target shape is unknown and fragments provide weak semantic cues. Existing end-to-end approaches are prone to cascading failures due to unreliable contact reasoning, most notably inaccurate fragment adjacencies. To address this, we propose Structure-Aware Reassembly (SARe), a generative framework with SARe-Gen for Euclidean-space assembly generation and SARe-Refine for inference-time refinement, with explicit contact modeling. SARe-Gen jointly predicts fracture-surface token probabilities and an inter-fragment contact graph to localize contact regions and infer candidate adjacencies. It adopts a query-point-based conditioning scheme and extracts aligned local geometric tokens at query locations from a frozen geometry encoder, yielding queryable structural representations without additional structural pretraining. We further introduce an inference-time refinement stage, SARe-Refine. By verifying candidate contact edges with geometric-consistency checks, it selects reliable substructures and resamples the remaining uncertain regions while keeping verified parts fixed, leading to more stable and consistent assemblies in the many-fragment regime. We evaluate SARe across three settings, including synthetic fractures, simulated fractures from scanned real objects, and real physically fractured scans. The results demonstrate state-of-the-art performance, with more graceful degradation and higher success rates as the fragment count increases in challenging large-scale reassembly.

Problem

• 3D fragment reassembly
• large-scale
• contact reasoning
• shape reconstruction
• fragment adjacency

Innovation

• 3D fragment reassembly
• structure-aware modeling
• contact graph prediction
• geometric consistency refinement
• query-based representation