Selected work

Representative Papers

Foresight: Iterative Reasoning About Clues that Matter for Navigation

Jun 10, 2026

This work addresses the challenges of sparse language instructions, ambiguous goals, and difficulty in assessing the relevance of environmental cues in open-world, map-free navigation. To tackle these issues, the authors propose Foresight, a novel framework that integrates human-in-the-loop reinforcement learning into the test-time inference loop for the first time. Foresight employs a fine-tuned vision-language model to iteratively generate motion plans in image space and critically refines them by jointly reasoning over linguistic objectives and visual context. This enables dynamic focus on task-relevant, open-set environmental cues without reliance on predefined navigational priors. Experimental results demonstrate that Foresight improves task success rates by 37% in offline evaluation and across six real-world environments, reduces human intervention by 52%, and operates in real time on a Jetson AGX Orin platform.

0 citationsRead paper

PhyMotion: Structured 3D Motion Reward for Physics-Grounded Human Video Generation

May 13, 2026

Existing video generation methods often produce floating or physically implausible human motions due to the absence of reliable motion realism reward signals. This work proposes PhyMotion, which introduces a multidimensional physical plausibility assessment into motion reward design for the first time. By retargeting SMPL human meshes recovered from generated videos into the MuJoCo physics simulator, PhyMotion provides continuous and interpretable reward signals based on kinematic plausibility, contact and balance consistency, and dynamic feasibility. Experiments demonstrate that PhyMotion significantly outperforms existing approaches both in automatic metrics and human blind evaluations, achieving a +68 Elo score improvement while maintaining high visual quality and enabling efficient training.

0 citationsRead paper

Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation

Mar 16, 2026

Simulation-to-real transfer remains a central challenge in robotics, as mismatches between simulated and real-world dynamics often lead to failures. While reinforcement learning offers a principled mechanism for adaptation, existing sim-to-real finetuning methods struggle with exploration and long-horizon credit assignment in the low-data regimes typical of real-world robotics. We introduce Simulation Distillation (SimDist), a sim-to-real framework that distills structural priors from a simulator into a latent world model and enables rapid real-world adaptation via online planning and supervised dynamics finetuning. By transferring reward and value models directly from simulation, SimDist provides dense planning signals from raw perception without requiring value learning during deployment. As a result, real-world adaptation reduces to short-horizon system identification, avoiding long-horizon credit assignment and enabling fast, stable improvement. Across precise manipulation and quadruped locomotion tasks, SimDist substantially outperforms prior methods in data efficiency, stability, and final performance. Project website and code: https://sim-dist.github.io/

0 citationsRead paper

World Model Failure Classification and Anomaly Detection for Autonomous Inspection

Feb 17, 2026

This work addresses the challenge of unreliable sensor readings in industrial inspection robots caused by occlusions, limited viewpoints, or environmental anomalies, which hinder real-time task status assessment. The authors propose a hybrid framework that integrates supervised fault classification with unsupervised anomaly detection, uniquely combining conformal prediction and world models to enable policy-agnostic, distribution-free early discrimination among three states—success, known faults, and out-of-distribution anomalies—using compressed video inputs. The approach facilitates training data quality evaluation and model feedback, achieving over 90% recognition accuracy on both office and industrial instrument inspection datasets. It outperforms human observers in decision speed and has been successfully deployed on a Boston Dynamics Spot robot for real-time operation.

0 citationsRead paper

Delay-Aware Diffusion Policy: Bridging the Observation-Execution Gap in Dynamic Tasks

Dec 08, 2025

In dynamic robotic tasks, perception–actuation latency (tens to hundreds of milliseconds) induces state inconsistency, severely degrading control accuracy and task success rates. To address this, we propose the first policy learning framework that explicitly models empirically measured inference latency: it incorporates measured latency as an explicit conditional input into a diffusion-based policy, jointly optimizing latency-compensated trajectory generation and latency-aware action selection; latency-conditioned augmentation further enables natural generalization from the zero-latency assumption. Our approach is architecture-agnostic, requires no modification to underlying controllers, and integrates seamlessly with diverse imitation learning paradigms. Experiments across multi-task and multi-latency settings demonstrate significantly higher task success rates and improved robustness over baselines. Moreover, our work advances standardized evaluation grounded in empirically measured latency—establishing it as a principled benchmark for real-world robotic learning.

0 citationsRead paper

Recent publications

Latest Papers

Foresight: Iterative Reasoning About Clues that Matter for Navigation

Jun 10, 2026

This work addresses the challenges of sparse language instructions, ambiguous goals, and difficulty in assessing the relevance of environmental cues in open-world, map-free navigation. To tackle these issues, the authors propose Foresight, a novel framework that integrates human-in-the-loop reinforcement learning into the test-time inference loop for the first time. Foresight employs a fine-tuned vision-language model to iteratively generate motion plans in image space and critically refines them by jointly reasoning over linguistic objectives and visual context. This enables dynamic focus on task-relevant, open-set environmental cues without reliance on predefined navigational priors. Experimental results demonstrate that Foresight improves task success rates by 37% in offline evaluation and across six real-world environments, reduces human intervention by 52%, and operates in real time on a Jetson AGX Orin platform.

0 citationsRead paper

PhyMotion: Structured 3D Motion Reward for Physics-Grounded Human Video Generation

May 13, 2026

Existing video generation methods often produce floating or physically implausible human motions due to the absence of reliable motion realism reward signals. This work proposes PhyMotion, which introduces a multidimensional physical plausibility assessment into motion reward design for the first time. By retargeting SMPL human meshes recovered from generated videos into the MuJoCo physics simulator, PhyMotion provides continuous and interpretable reward signals based on kinematic plausibility, contact and balance consistency, and dynamic feasibility. Experiments demonstrate that PhyMotion significantly outperforms existing approaches both in automatic metrics and human blind evaluations, achieving a +68 Elo score improvement while maintaining high visual quality and enabling efficient training.

0 citationsRead paper

Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation

Mar 16, 2026

Simulation-to-real transfer remains a central challenge in robotics, as mismatches between simulated and real-world dynamics often lead to failures. While reinforcement learning offers a principled mechanism for adaptation, existing sim-to-real finetuning methods struggle with exploration and long-horizon credit assignment in the low-data regimes typical of real-world robotics. We introduce Simulation Distillation (SimDist), a sim-to-real framework that distills structural priors from a simulator into a latent world model and enables rapid real-world adaptation via online planning and supervised dynamics finetuning. By transferring reward and value models directly from simulation, SimDist provides dense planning signals from raw perception without requiring value learning during deployment. As a result, real-world adaptation reduces to short-horizon system identification, avoiding long-horizon credit assignment and enabling fast, stable improvement. Across precise manipulation and quadruped locomotion tasks, SimDist substantially outperforms prior methods in data efficiency, stability, and final performance. Project website and code: https://sim-dist.github.io/

0 citationsRead paper

World Model Failure Classification and Anomaly Detection for Autonomous Inspection

Feb 17, 2026

This work addresses the challenge of unreliable sensor readings in industrial inspection robots caused by occlusions, limited viewpoints, or environmental anomalies, which hinder real-time task status assessment. The authors propose a hybrid framework that integrates supervised fault classification with unsupervised anomaly detection, uniquely combining conformal prediction and world models to enable policy-agnostic, distribution-free early discrimination among three states—success, known faults, and out-of-distribution anomalies—using compressed video inputs. The approach facilitates training data quality evaluation and model feedback, achieving over 90% recognition accuracy on both office and industrial instrument inspection datasets. It outperforms human observers in decision speed and has been successfully deployed on a Boston Dynamics Spot robot for real-time operation.

0 citationsRead paper

Delay-Aware Diffusion Policy: Bridging the Observation-Execution Gap in Dynamic Tasks

Dec 08, 2025

In dynamic robotic tasks, perception–actuation latency (tens to hundreds of milliseconds) induces state inconsistency, severely degrading control accuracy and task success rates. To address this, we propose the first policy learning framework that explicitly models empirically measured inference latency: it incorporates measured latency as an explicit conditional input into a diffusion-based policy, jointly optimizing latency-compensated trajectory generation and latency-aware action selection; latency-conditioned augmentation further enables natural generalization from the zero-latency assumption. Our approach is architecture-agnostic, requires no modification to underlying controllers, and integrates seamlessly with diverse imitation learning paradigms. Experiments across multi-task and multi-latency settings demonstrate significantly higher task success rates and improved robustness over baselines. Moreover, our work advances standardized evaluation grounded in empirically measured latency—establishing it as a principled benchmark for real-world robotic learning.

0 citationsRead paper