Selected work

Representative Papers

Exploring 6D Object Pose Estimation with Deformation

Apr 08, 2026

This work addresses the limitation of existing 6D object pose estimation methods, which typically assume objects are rigid or articulated and thus struggle with real-world deformations caused by wear, impact, or other factors. To bridge this gap, we introduce DeSOPE, the first large-scale dataset specifically designed for deformable object pose estimation, encompassing high-fidelity 3D models of 26 common object categories under standard and three realistic deformation states, along with 133,000 RGB-D frames. Leveraging a semi-automatic pipeline that integrates 2D instance segmentation, initial pose estimation, object-level SLAM refinement, and manual verification, we provide 665,000 high-accuracy pose annotations. Experiments demonstrate a significant performance drop in current methods as deformation intensifies, underscoring DeSOPE’s critical role as the first benchmark for evaluating 6D pose estimation under realistic object deformations.

0 citationsRead paper

Efficient Egocentric Action Recognition with Multimodal Data

Jun 02, 2025

On portable XR devices, egocentric action recognition (EAR) faces a fundamental trade-off among computational overhead, power consumption, and accuracy. Method: This paper proposes an asynchronous multimodal sampling and lightweight fusion framework. It first identifies the complementary temporal sampling characteristics between RGB video and high-frequency 3D hand pose sequences, then designs a cross-modal joint sampling-rate optimization strategy that synergistically leverages RGB frame-rate reduction and high-frequency hand-pose capture. Further, it introduces a low-complexity multimodal temporal modeling and feature fusion network. Results: Experiments demonstrate that the method maintains near-lossless recognition accuracy (degradation <0.5%), reduces CPU utilization by 3×, and significantly improves edge-side real-time performance and energy efficiency—establishing a novel paradigm for efficient EAR on wearable XR platforms.

0 citationsRead paper

SCFlow2: Plug-and-Play Object Pose Refiner with Shape-Constraint Scene Flow

Apr 12, 2025

Existing fine-tuning methods for 6D pose estimation suffer from noise sensitivity or require retraining for novel objects. This paper introduces SCFlow2—a plug-and-play, training-free RGB-D pose refinement framework. Methodologically, it pioneers the joint modeling of rigid-motion-embedded 3D scene flow and object-specific 3D shape priors, establishing a cyclic matching network constrained by both geometric consistency and shape fidelity, optimized robustly via iterative regularization. Crucially, SCFlow2 enables zero-shot generalization to unseen objects in the BOP benchmark, trained only once on multi-source 3D datasets (Objaverse, GSO, ShapeNet). As a post-processing module, it significantly boosts state-of-the-art pose estimators on BOP benchmarks, achieving over 15% average ADD(-S) improvement. The approach thus delivers strong generalizability, robustness to sensor noise and occlusion, and practical deployability without object-specific adaptation.

0 citationsRead paper

Hierarchical Flow Diffusion for Efficient Frame Interpolation

Apr 01, 2025

Existing diffusion-based video frame interpolation methods suffer from low accuracy and slow inference due to excessively large denoising ranges in latent space. To address this, we propose Hierarchical Optical Flow Diffusion Modeling (HLFM), the first framework to explicitly formulate bilateral optical flow as a hierarchical diffusion process—thereby decoupling motion estimation from content synthesis. We further design a flow-guided image synthesizer that enables end-to-end generation of high-fidelity intermediate frames. By hierarchically constraining the denoising search space via optical flow priors, HLFM achieves superior modeling precision without sacrificing computational efficiency. On multiple standard benchmarks, HLFM establishes new state-of-the-art performance in frame interpolation. Notably, it accelerates inference by over 10× compared to existing diffusion-based approaches while preserving strong temporal consistency and visual fidelity.

0 citationsRead paper

Lifting by Gaussians: A Simple, Fast and Flexible Method for 3D Instance Segmentation

Jan 31, 2025

To address the lack of efficient, general-purpose, and training-free semantic segmentation methods for 3D Gaussian Splatting (3DGS), this paper introduces the first zero-shot instance segmentation framework: it directly lifts and fuses 2D masks from SAM/FastSAM and visual features from CLIP/DINOv2 into the 3DGS Gaussian voxel space—without scene-specific fine-tuning. Its core innovation is a modular, training-free Gaussian feature lifting mechanism, fully compatible with arbitrary existing 3DGS reconstructions. We further establish the first benchmark for 3D radiance field segmentation targeting single-asset extraction. Experiments demonstrate state-of-the-art performance on both 2D novel-view semantic synthesis and 3D asset extraction, with over 10× inference speedup. The method naturally generalizes to other Gaussian representations, including 2D Gaussian Splatting.

0 citationsRead paper

Recent publications

Latest Papers

Exploring 6D Object Pose Estimation with Deformation

Apr 08, 2026

This work addresses the limitation of existing 6D object pose estimation methods, which typically assume objects are rigid or articulated and thus struggle with real-world deformations caused by wear, impact, or other factors. To bridge this gap, we introduce DeSOPE, the first large-scale dataset specifically designed for deformable object pose estimation, encompassing high-fidelity 3D models of 26 common object categories under standard and three realistic deformation states, along with 133,000 RGB-D frames. Leveraging a semi-automatic pipeline that integrates 2D instance segmentation, initial pose estimation, object-level SLAM refinement, and manual verification, we provide 665,000 high-accuracy pose annotations. Experiments demonstrate a significant performance drop in current methods as deformation intensifies, underscoring DeSOPE’s critical role as the first benchmark for evaluating 6D pose estimation under realistic object deformations.

0 citationsRead paper

Efficient Egocentric Action Recognition with Multimodal Data

Jun 02, 2025

On portable XR devices, egocentric action recognition (EAR) faces a fundamental trade-off among computational overhead, power consumption, and accuracy. Method: This paper proposes an asynchronous multimodal sampling and lightweight fusion framework. It first identifies the complementary temporal sampling characteristics between RGB video and high-frequency 3D hand pose sequences, then designs a cross-modal joint sampling-rate optimization strategy that synergistically leverages RGB frame-rate reduction and high-frequency hand-pose capture. Further, it introduces a low-complexity multimodal temporal modeling and feature fusion network. Results: Experiments demonstrate that the method maintains near-lossless recognition accuracy (degradation <0.5%), reduces CPU utilization by 3×, and significantly improves edge-side real-time performance and energy efficiency—establishing a novel paradigm for efficient EAR on wearable XR platforms.

0 citationsRead paper

SCFlow2: Plug-and-Play Object Pose Refiner with Shape-Constraint Scene Flow

Apr 12, 2025

Existing fine-tuning methods for 6D pose estimation suffer from noise sensitivity or require retraining for novel objects. This paper introduces SCFlow2—a plug-and-play, training-free RGB-D pose refinement framework. Methodologically, it pioneers the joint modeling of rigid-motion-embedded 3D scene flow and object-specific 3D shape priors, establishing a cyclic matching network constrained by both geometric consistency and shape fidelity, optimized robustly via iterative regularization. Crucially, SCFlow2 enables zero-shot generalization to unseen objects in the BOP benchmark, trained only once on multi-source 3D datasets (Objaverse, GSO, ShapeNet). As a post-processing module, it significantly boosts state-of-the-art pose estimators on BOP benchmarks, achieving over 15% average ADD(-S) improvement. The approach thus delivers strong generalizability, robustness to sensor noise and occlusion, and practical deployability without object-specific adaptation.

0 citationsRead paper

Hierarchical Flow Diffusion for Efficient Frame Interpolation

Apr 01, 2025

Existing diffusion-based video frame interpolation methods suffer from low accuracy and slow inference due to excessively large denoising ranges in latent space. To address this, we propose Hierarchical Optical Flow Diffusion Modeling (HLFM), the first framework to explicitly formulate bilateral optical flow as a hierarchical diffusion process—thereby decoupling motion estimation from content synthesis. We further design a flow-guided image synthesizer that enables end-to-end generation of high-fidelity intermediate frames. By hierarchically constraining the denoising search space via optical flow priors, HLFM achieves superior modeling precision without sacrificing computational efficiency. On multiple standard benchmarks, HLFM establishes new state-of-the-art performance in frame interpolation. Notably, it accelerates inference by over 10× compared to existing diffusion-based approaches while preserving strong temporal consistency and visual fidelity.

0 citationsRead paper

Lifting by Gaussians: A Simple, Fast and Flexible Method for 3D Instance Segmentation

Jan 31, 2025

To address the lack of efficient, general-purpose, and training-free semantic segmentation methods for 3D Gaussian Splatting (3DGS), this paper introduces the first zero-shot instance segmentation framework: it directly lifts and fuses 2D masks from SAM/FastSAM and visual features from CLIP/DINOv2 into the 3DGS Gaussian voxel space—without scene-specific fine-tuning. Its core innovation is a modular, training-free Gaussian feature lifting mechanism, fully compatible with arbitrary existing 3DGS reconstructions. We further establish the first benchmark for 3D radiance field segmentation targeting single-asset extraction. Experiments demonstrate state-of-the-art performance on both 2D novel-view semantic synthesis and 3D asset extraction, with over 10× inference speedup. The method naturally generalizes to other Gaussian representations, including 2D Gaussian Splatting.

0 citationsRead paper