FOCUS - Multi-View Foot Reconstruction From Synthetically Trained Dense Correspondences

2025-02-10

AI Summary

To address the high multi-view image requirements, strong annotation dependency, and insufficient geometric accuracy in human foot 3D reconstruction, this paper proposes an unsupervised few-view reconstruction method. Our approach comprises three key components: (1) SynFoot2—the first large-scale synthetic foot dataset with dense 3D annotations; (2) an uncertainty-aware dense correspondence predictor that robustly estimates pixel-level correspondences under sparse views; and (3) a dual-path reconstruction framework jointly optimizing Structure-from-Motion (SfM)-inspired geometric fusion and a differentiable FIND (Flexible Implicit Neural Deformation) parameterization model. The method achieves high-fidelity reconstruction from only 2–4 RGB images, setting new state-of-the-art performance under few-view settings and matching top-tier multi-view methods while significantly accelerating inference. Both the code and the SynFoot2 dataset are publicly released.

Technology Category

• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Surface reconstruction from multiple, calibrated images is a challenging task - often requiring a large number of collected images with significant overlap. We look at the specific case of human foot reconstruction. As with previous successful foot reconstruction work, we seek to extract rich per-pixel geometry cues from multi-view RGB images, and fuse these into a final 3D object. Our method, FOCUS, tackles this problem with 3 main contributions: (i) SynFoot2, an extension of an existing synthetic foot dataset to include a new data type: dense correspondence with the parameterized foot model FIND; (ii) an uncertainty-aware dense correspondence predictor trained on our synthetic dataset; (iii) two methods for reconstructing a 3D surface from dense correspondence predictions: one inspired by Structure-from-Motion, and one optimization-based using the FIND model. We show that our reconstruction achieves state-of-the-art reconstruction quality in a few-view setting, performing comparably to state-of-the-art when many views are available, and runs substantially faster. We release our synthetic dataset to the research community. Code is available at: https://github.com/OllieBoyne/FOCUS

Problem

• Reconstruct 3D foot from multi-view images
• Train dense correspondence on synthetic data
• Improve reconstruction quality and speed

Innovation

• Extends synthetic foot dataset SynFoot2
• Uses uncertainty-aware dense correspondence predictor
• Proposes two 3D surface reconstruction methods