Joint-Conditioned Stereo Surface Reasoning for Interaction Field Estimation

2026-09-07

AI Summary

该研究通过联合条件立体表面推理方法，解决手-物体交互场预测问题，特别是在小且部分遮挡的图像区域定位每个手关节最近的物体表面点。

Abstract

Predicting hand--object interaction fields requires locating the nearest object-surface point for each hand joint, often from small and partially occluded image regions. We view this task as joint-conditioned surface-endpoint estimation: each joint has its own nearest endpoint, while endpoints from the same hand can draw on shared local surface evidence. This structure motivates Joint-Conditioned Stereo Surface Reasoning (JSSR). A temporal-stereo network jointly predicts 3D joints, a direct interaction field, and per-view endpoint evidence. Calibrated candidate search evaluates endpoint hypotheses using joint-specific image compatibility and cross-view correspondence. A hand-shared candidate support lets joints draw on common surface evidence, and a learned residual gate controls the geometric correction when observations are ambiguous. Our system built on this method ranked third on the SHOW3D Interaction Field Challenge leaderboard.

Problem

• hand-object interaction
• nearest object-surface point
• partially occluded image regions
• joint-conditioned

Innovation

• Joint-Conditioned Stereo Surface Reasoning
• temporal-stereo network
• calibrated candidate search
• hand-shared candidate support
• learned residual gate