GaussAnything: Semantic Intent-Driven Refinement of Evolving Gaussian Scenes for Standalone VR

2026-09-12

AI Summary

为解决独立VR头显计算和内存限制问题，GaussAnything通过基于用户意图的动态资源分配与渐进式场景发布方法优化3D环境渲染。

Abstract

Deploying reconstructed 3D environments on standalone VR headsets are constrained by limited compute and memory, and conventional level-of-detail policies optimize for visibility without accounting for the user's explicit inspection intent. We present GaussAnything, a native OpenXR system for intent-conditioned reallocation and progressive publication of evolving semantic Gaussian+SDF scenes. GaussAnything resolves class- or instance-level queries to persistent 3D objects and reallocates a fixed Gaussian resident budget toward the selected object while retaining global context, applying incremental, stable-identity updates coordinated with the TSDF-derived mesh through a source-epoch mechanism. Across eight scenes, an object query concentrates 88-90% of the fixed client budget onto the queried object without enlarging it, on-device rendering reproduces the host render to within a small margin (up to 36.7 dB), and the standalone client renders each stereo frame at a steady-state GPU cost of roughly 10 ms within the frame budget of standard standalone panels.

Problem

• standalone VR
• limited compute and memory
• level-of-detail policies
• user's explicit inspection intent

Innovation

• intent-driven
• Gaussian+SDF scenes
• reallocation
• stable-identity updates
• standalone VR