Vision-Reasoning-Guided Occlusion Removal from Light Fields

2026-06-18

AI Summary

This work addresses the challenge of limited visibility in natural scenes caused by dense foreground occlusions, such as vegetation, by proposing a novel approach that integrates light field integration with vision-language models (VLMs). The method first leverages multi-view light field integration to suppress occlusions and then employs a VLM as a semantic prior to guide the recovery of structural details. By incorporating physical imaging constraints and a multi-hypothesis fusion strategy, the framework effectively mitigates hallucination artifacts and enhances reconstruction consistency. To the best of our knowledge, this is the first study to harness the semantic reasoning capabilities of VLMs for light field de-occlusion. The approach achieves state-of-the-art SSIM performance on the 4-Syn dataset and demonstrates strong generalization across both structured and unstructured real-world scenes, showing promise for applications in search-and-rescue and robotic exploration.

Abstract

Occlusion-robust scene recovery remains a major challenge in computational imaging, particularly in natural environments where dense foreground vegetation severely limits visibility. We propose a vision-reasoning-guided light field occlusion removal framework that combines the visibility recovery capability of light field integration (LFI) with the semantic reasoning capacity of vision-language models (VLMs). Multi-view observations are first integrated via LFI to suppress foreground occlusions and produce an initial visibility-enhanced representation. A VLM is then incorporated as a conditional semantic prior to restore degraded structures and recover fine details, guided by the observed measurements. To improve recovery consistency and reduce hallucination artifacts, we introduce a multi-sample fusion strategy that aggregates multiple generated hypotheses into a unified estimate. Experimental results on synthetic and real-world datasets demonstrate state-of-the-art performance, achieving the highest average SSIM across four synthetic light field benchmark scenes (4-Syn) and strong generalization across structured and unstructured acquisition settings. These results highlight the effectiveness of combining physical imaging constraints with vision-language reasoning for robust perception under severe occlusion, with applicability to search-and-rescue and exploratory robotic navigation.

Problem

• occlusion removal
• light fields
• scene recovery
• computational imaging
• visibility enhancement

Innovation

• light field occlusion removal
• vision-language model
• multi-view integration
• semantic prior
• multi-sample fusion