End-to-end differentiable design of geometric waveguide displays

2026-01-07

AI Summary

This work addresses the challenge in geometric waveguide-based augmented reality displays, where non-sequential light transport and polarization-sensitive multilayer thin-film coatings are difficult to co-optimize. We present the first end-to-end differentiable optimization framework that couples a differentiable Monte Carlo polarized ray tracer with a differentiable transfer-matrix thin-film solver, enabling joint optimization over system-level high-dimensional parameters—including thousands of film layer thicknesses and billions of non-sequential rays. An automatic layer-pruning mechanism is introduced to discover efficient optical topologies. In a representative design, the proposed method improves optical efficiency from 4.1% to 33.5%, while enhancing eyebox and field-of-view uniformity by approximately 17× and 11×, respectively.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Optimization
• Computer Vision: Learning & Optimization for CV

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Geometric waveguides are a promising architecture for optical see-through augmented reality displays, but their performance is severely bottlenecked by the difficulty of jointly optimizing non-sequential light transport and polarization-dependent multilayer thin-film coatings. Here we present the first end-to-end differentiable optimization framework for geometric waveguide that couples non-sequential Monte Carlo polarization ray tracing with a differentiable transfer-matrix thin-film solver. A differentiable Monte Carlo ray tracer avoids the exponential growth of deterministic ray splitting while enabling gradients backpropagation from eyebox metrics to design parameters. With memory-saving strategies, we optimize more than one thousand layer-thickness parameters and billions of non-sequential ray-surface intersections on a single multi-GPU workstation. Automated layer pruning is achieved by starting from over-parameterized stacks and driving redundant layers to zero thickness under discrete manufacturability constraints, effectively performing topology optimization to discover optimal coating structures. On a representative design, starting from random initialization within thickness bounds, our method increases light efficiency from 4.1\% to 33.5\% and improves eyebox and FoV uniformity by $\sim$17$\times$ and $\sim$11$\times$, respectively. Furthermore, we jointly optimize the waveguide and an image preprocessing network to improve perceived image quality. Our framework not only enables system-level, high-dimensional coating optimization inside the waveguide, but also expands the scope of differentiable optics for next-generation optical design.

Problem

• geometric waveguide
• non-sequential light transport
• polarization-dependent thin-film coatings
• joint optimization
• augmented reality displays

Innovation

• differentiable optics
• geometric waveguide
• polarization ray tracing
• thin-film optimization
• topology optimization