Helix4D: Complex 4D Mesh Generation

2026-05-25

AI Summary

Existing video-to-4D methods struggle with complex topological changes, transparent materials, thin structures, and internal surfaces. This work extends the Trellis2 framework from image-to-3D to video-driven 4D dynamic mesh generation by introducing a sliding-window cross-frame attention mechanism that anchors to the first frame to inherit pre-trained reconstruction quality. It further proposes a parameter-free 4D temporal positional encoding that reuses low-frequency spatial RoPE bands for temporal modeling. Evaluated on ActionBench and a newly curated dataset of complex dynamic scenes, the method achieves high-fidelity reconstructions and significantly improves modeling capabilities for challenging cases such as transparent objects and internal surfaces.

Abstract

Current video-to-4D methods struggle with complex topology changes, transparent materials, thin structures, and inner surfaces. We present Helix4D, a dynamic mesh generation framework by inheriting the expressive representation of Trellis2, adapting it from image-to-3D to video-conditioned 4D generation. Our design arises from two key questions: (a) how to enable Trellis2's frame-local attention to share information across frames while preserving its pretrained quality on rare cases such as transparent objects and inner surfaces, and (b) how to inject temporal information into a purely 3D positional encoding without breaking pretrained capabilities. We address (a) with a sliding-window cross-frame attention and anchor on the first frame. The first frame is generated by the base Trellis2 model and injected into our model, letting it inherit Trellis2's quality in rare cases through cross-frame attention. We address (b) with a 4D temporal encoding that repurposes redundant low-frequency spatial RoPE bands for time, extending the encoding from 3D with no additional parameters. Extensive experiments show the effectiveness of Helix4D for high-quality dynamic mesh generation on ActionBench and our own challenging complex dynamics set.

Problem

• 4D mesh generation
• topology changes
• transparent materials
• thin structures
• inner surfaces

Innovation

• 4D mesh generation
• cross-frame attention
• temporal encoding
• sliding-window attention
• video-conditioned generation