QuadSync: Quadrifocal Tensor Synchronization via Tucker Decomposition

2026-02-26

AI Summary

This work proposes the first multi-camera synchronization framework based on the quadrifocal tensor, addressing the limitation of traditional methods that rely solely on two-view geometric constraints and fail to exploit higher-order information. By constructing a block-wise quadrifocal tensor and leveraging its Tucker decomposition structure, the approach directly links the factor matrices to stacked camera projection matrices, enabling efficient synchronization. The method reveals intrinsic relationships among quadrifocal, trifocal, and bifocal tensors and supports their joint optimization. Employing an ADMM-based solver combined with iteratively reweighted least squares (IRLS), the framework demonstrates significant improvements in both accuracy and robustness on standard benchmarks, underscoring the critical role of higher-order geometric constraints in multi-view synchronization.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

In structure from motion, quadrifocal tensors capture more information than their pairwise counterparts (essential matrices), yet they have often been thought of as impractical and only of theoretical interest. In this work, we challenge such beliefs by providing a new framework to recover $n$ cameras from the corresponding collection of quadrifocal tensors. We form the block quadrifocal tensor and show that it admits a Tucker decomposition whose factor matrices are the stacked camera matrices, and which thus has a multilinear rank of (4,~4,~4,~4) independent of $n$. We develop the first synchronization algorithm for quadrifocal tensors, using Tucker decomposition, alternating direction method of multipliers, and iteratively reweighted least squares. We further establish relationships between the block quadrifocal, trifocal, and bifocal tensors, and introduce an algorithm that jointly synchronizes these three entities. Numerical experiments demonstrate the effectiveness of our methods on modern datasets, indicating the potential and importance of using higher-order information in synchronization.

Problem

• quadrifocal tensor
• structure from motion
• camera synchronization
• Tucker decomposition
• multiview geometry

Innovation

• quadrifocal tensor
• Tucker decomposition
• camera synchronization
• structure from motion
• multilinear rank