EPO: Boosting 3D Foundation Models with Edge-based Pose Optimization

2026-07-01

AI Summary

Existing 3D foundation models often sacrifice geometric accuracy for improved inference speed, while conventional post-processing pipelines rely on feature trajectory extraction, undermining their efficiency advantages. This work proposes a novel geometry optimization framework that eliminates the need for explicit feature matching by introducing edge map alignment as a differentiable proxy objective. By directly optimizing camera poses without feature extraction or track construction, the method achieves reconstruction accuracy on par with or surpassing traditional bundle adjustment across multiple datasets. Moreover, it significantly reduces both runtime and memory consumption, enabling efficient execution on consumer-grade hardware.

Abstract

We introduce \textbf{Edge-based Pose Optimization (EPO)}, a trackless geometric optimization framework specifically designed to boost the Structure-from-Motion reconstructions generated by 3D Foundation Models. These models achieve rapid inference by bypassing the time-consuming feature extraction and matching stages of traditional pipelines, where explicit correspondences between each 3D point and multiple images, referred to as tracks, are established. However, their geometric accuracy currently falls short of traditional pipelines. While this can be addressed in a post-processing step via Bundle Adjustment-like refinement, doing so requires extracting feature tracks, thus defeating the original speed advantage. Instead, our fully differentiable framework uses edge map alignment as a proxy for geometric optimization, avoiding feature extraction and track construction entirely. Through extensive evaluation across multiple datasets and tasks, we demonstrate that EPO matches or outperforms Bundle Adjustment-like methods while requiring significantly lower runtime and memory. Notably, its reduced memory footprint makes EPO suitable for consumer-grade hardware, where competing refinement methods cannot run.

Problem

• 3D Foundation Models
• Structure-from-Motion
• geometric accuracy
• feature tracks
• pose optimization

Innovation

• Edge-based Pose Optimization
• 3D Foundation Models
• Structure-from-Motion
• Trackless Optimization
• Differentiable Rendering