Unsupervised Pixel-Level Semantic Left-Right Understanding of In-the-Wild Images

2026-07-06

AI Summary

This work addresses the challenge of pixel-level semantic left-right understanding in single-view wild images, where factors such as missing 3D information, occlusions, and pose variations hinder accurate reasoning. To tackle this, the authors propose an unsupervised learning framework that jointly leverages limited 3D shape priors—restricted to humans and quadrupeds—and in-the-wild image data. This approach enables, for the first time, cross-category pixel-level semantic left-right prediction without requiring ground-truth annotations. By modeling semantic left-right correspondence at the vertex level in 3D, the method generalizes effectively to unseen categories such as cars and trains. Experiments demonstrate substantial improvements over state-of-the-art methods on both rendered and real-world images.

Abstract

While various works address reflective symmetry understanding in 3D data and images, pixel-level semantic left-right prediction of in-the-wild images remains challenging, due to certain difficulties including the lack of 3D information, occlusion, object pose variation, partiality, etc. In this work, we propose an unsupervised learning framework to tackle this challenge. Leveraging recent advances in vertex-wise semantic left-right understanding of 3D data, our unsupervised learning method jointly utilises 3D shape and image datasets to infer pixel-wise semantic left-right predictions in single-view images. In particular, we show that a medium-scale 3D shape dataset comprising mainly of human- and quadruped animal-like shapes, combined with diverse in-the-wild image data, are sufficient to achieve high-quality semantic left-right prediction in images, even for entirely unseen 3D object categories, such as cars or trains. Overall, our approach achieves superior performance in dense pixel-wise semantic left-right predictions on both rendered and in-the-wild image datasets when compared to existing state-of-the-art methods.

Problem

• unsupervised learning
• pixel-level semantic
• left-right understanding
• in-the-wild images
• reflective symmetry

Innovation

• unsupervised learning
• pixel-level semantic left-right understanding
• 3D shape priors
• in-the-wild images
• cross-domain generalization