🤖 AI Summary
To address the challenge of coexisting large disparities and non-Lambertian effects (e.g., depth-induced blur, complex reflectance) in light field rendering, this paper proposes Geometry-aware Neural Interpolation (GNI), the first framework to embed Neural Interpolation (NI) into the Depth-Image-Based Rendering (DIBR) pipeline. GNI constructs an interpretable reconstruction cost volume driven by depth hypotheses and employs a lightweight, low-dimensional network to efficiently model high-dimensional cost features, enabling geometry-guided synthesis of high angular-resolution light fields. Its core innovations are: (1) paradigmatic integration of NI and DIBR; (2) depth-dependent, interpretable cost volume modeling; and (3) compact representation of high-dimensional cost volumes. Extensive evaluations on multiple benchmarks demonstrate that GNI significantly outperforms both pure NI and conventional DIBR methods, substantially improving angular resolution and visual fidelity—particularly in large-baseline scenarios.
📝 Abstract
We present a novel Geometry-aware Neural Interpolation (Geo-NI) framework for light field rendering. Previous learning-based approaches either perform direct interpolation via neural networks, which we dubbed Neural Interpolation (NI), or explore scene geometry for novel view synthesis, also known as Depth Image-Based Rendering (DIBR). Both kinds of approaches have their own strengths and weaknesses in addressing non-Lambert effect and large disparity problems. In this paper, we incorporate the ideas behind these two kinds of approaches by launching the NI within a specific DIBR pipeline. Specifically, a DIBR network in the proposed Geo-NI serves to construct a novel reconstruction cost volume for neural interpolated light fields sheared by different depth hypotheses. The reconstruction cost can be interpreted as an indicator reflecting the reconstruction quality under a certain depth hypothesis, and is further applied to guide the rendering of the final high angular resolution light field. To implement the Geo-NI framework more practically, we further propose an efficient modeling strategy to encode high-dimensional cost volumes using a lower-dimension network. By combining the superiorities of NI and DIBR, the proposed Geo-NI is able to render views with large disparities with the help of scene geometry while also reconstructing the non-Lambertian effect when depth is prone to be ambiguous. Extensive experiments on various datasets demonstrate the superior performance of the proposed geometry-aware light field rendering framework.
