EPSilon: Efficient Point Sampling for Lightening of Hybrid-based 3D Avatar Generation

2025-07-18

AI Summary

To address the inefficiency in hybrid SMPL-NeRF-based virtual human generation—specifically, excessive spatial sampling redundancy leading to costly deformation computation and slow inference—this paper proposes a two-stage point-sampling optimization: Empty Ray Omission (ERO) and Empty Interval Omission (EIO). Integrated within a single-stage NeRF framework, these techniques dynamically prune invalid sampling points during volumetric rendering. Leveraging SMPL mesh priors and skinning weight-based deformation, the method enables geometry-aware sparse sampling while preserving high-fidelity rendering quality. Experiments demonstrate that the approach achieves a 20× inference speedup and 4× faster training convergence using only 3.9% of the original sampling points. This work establishes an efficient, practical paradigm for real-time, photorealistic 3D avatar generation.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Computer Vision: 3D Computer Vision

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

The rapid advancement of neural radiance fields (NeRF) has paved the way to generate animatable human avatars from a monocular video. However, the sole usage of NeRF suffers from a lack of details, which results in the emergence of hybrid representation that utilizes SMPL-based mesh together with NeRF representation. While hybrid-based models show photo-realistic human avatar generation qualities, they suffer from extremely slow inference due to their deformation scheme: to be aligned with the mesh, hybrid-based models use the deformation based on SMPL skinning weights, which needs high computational costs on each sampled point. We observe that since most of the sampled points are located in empty space, they do not affect the generation quality but result in inference latency with deformation. In light of this observation, we propose EPSilon, a hybrid-based 3D avatar generation scheme with novel efficient point sampling strategies that boost both training and inference. In EPSilon, we propose two methods to omit empty points at rendering; empty ray omission (ERO) and empty interval omission (EIO). In ERO, we wipe out rays that progress through the empty space. Then, EIO narrows down the sampling interval on the ray, which wipes out the region not occupied by either clothes or mesh. The delicate sampling scheme of EPSilon enables not only great computational cost reduction during deformation but also the designation of the important regions to be sampled, which enables a single-stage NeRF structure without hierarchical sampling. Compared to existing methods, EPSilon maintains the generation quality while using only 3.9% of sampled points and achieves around 20 times faster inference, together with 4 times faster training convergence. We provide video results on https://github.com/seungjun-moon/epsilon.

Problem

• Reduces computational cost in hybrid 3D avatar generation
• Improves inference speed by omitting empty space sampling
• Maintains quality while accelerating training and rendering

Innovation

• Efficient point sampling for hybrid 3D avatars
• Empty ray omission to skip empty space
• Empty interval omission to focus sampling