🤖 AI Summary
To address the stringent requirements of eye tracking in VR/AR devices—namely ultra-compact form factor, ultra-low power consumption, and real-time performance—this paper introduces i-FlatCam, the world’s first intelligent lensless camera. Methodologically, we propose a novel two-stage “predict-then-focus” tracking paradigm; design an intra-channel reused deep convolutional architecture to maximize hardware utilization; and develop a unified on-chip compression encoding scheme that jointly optimizes lensless optical imaging, depthwise separable convolution, and low-power edge computing. Experimental results demonstrate that the system achieves 3.16° angular accuracy, 253 FPS frame rate, and 91.49 µJ/frame energy efficiency within an ultra-miniaturized volume of 6.7 × 8.9 × 1.2 mm³—substantially outperforming state-of-the-art solutions in all key metrics.
📝 Abstract
We present a first-of-its-kind ultra-compact intelligent camera system, dubbed i-FlatCam, including a lensless camera with a computational (Comp.) chip. It highlights (1) a predict-then-focus eye tracking pipeline for boosted efficiency without compromising the accuracy, (2) a unified compression scheme for single-chip processing and improved frame rate per second (FPS), and (3) dedicated intra-channel reuse design for depth-wise convolutional layers (DW-CONV) to increase utilization. i-FlatCam demonstrates the first eye tracking pipeline with a lensless camera and achieves 3.16 degrees of accuracy, 253 FPS, 91.49 µJ/Frame, and 6.7mm×8.9mm×1.2mm camera form factor, paving the way for next-generation Augmented Reality (AR) and Virtual Reality (VR) devices.
