Amulet: Frame Extrapolation Through Sparse Layered Scene Representation and Adaptive Shading

2026-08-10

AI Summary

This work addresses the frame-rate bottleneck in high-refresh-rate displays caused by rendering latency, particularly the challenge of shading newly exposed regions (disocclusions) in dynamic scenes. To this end, the authors propose a sparse, hierarchical image-space caching architecture that decouples rasterization and shading from the display refresh rate. By leveraging predictable hole-filling, front-to-back cache traversal, and adaptive local shading updates, the method enables non-neural, multi-frame high-quality frame extrapolation while explicitly handling occlusion changes to avoid content hallucination. Experimental results demonstrate that the system achieves up to 250 Hz extrapolated frame rates at 4K resolution, generating multiple output frames per single shading pass and matching or surpassing state-of-the-art approaches such as DLSS across multiple quality metrics.

Abstract

We introduce Amulet, a rendering method that transforms a scene into a sparse, tiled and layered intermediate scene representation (cache) for high-frequency frame extrapolation. In contrast to reprojection-based techniques, Amulet explicitly rasterizes and stores potentially visible geometry in its layered image-space cache, allowing accurate shading and inpainting of newly disoccluded regions without hallucination. Our key contribution is a cache that is predictively filled with shading information for future views, amortized over multiple current frames. Novel views are synthesized by hierarchically traversing the cache front to back and refining stale or missing shading on the fly. Using a predictive, gradient-based scheduler that assigns lifetimes for each tile, we enable adaptive shading updates under motion and dynamic lighting. Amulet decouples the rasterization and shading rate from the refresh rate of the display. In many scenarios, our cache can use a single shaded frame to synthesize multiple extrapolated frames with only a few localized updates. In a typical application, we extrapolate a 60 Hz shading rate to a 240 Hz display. Amulet achieves up to 250 Hz at 4K resolution and is competitive with state-of-the-art frame generation methods, including DLSS and neural-flow approaches, in multiple metrics. Amulet explores the design space of sparse layered image-space representation. It enables accurate, non-neural multi frame extrapolation with explicit handling of disocclusions. Our findings show that Amulet can extrapolate many more frames than contemporary methods with high quality, rivaling latency-bound frame interpolation methods with similar quality in many scenes.

Problem

• frame extrapolation
• disocclusion handling
• high refresh rate rendering
• sparse layered representation
• adaptive shading

Innovation

• frame extrapolation
• sparse layered representation
• adaptive shading
• disocclusion handling
• image-space caching