Sample-Efficient Post-Training for LEGO Spatial-Physics Reasoning

2026-05-29

arXiv.org

AI Summary

本文针对LEGO组装中物理和语义不一致的问题，提出了一种基于模型的数据选择方法和一种结合物理有效性和几何奖励的强化学习方法PVPO，提高了组装效果。

Abstract

LLM-based LEGO assembly generation requires both semantic grounding and physical feasibility. We identify a data-induced failure mode, PhysHack, in which the assemblies satisfy physical-validity constraints while producing structures that are geometrically misaligned, semantically inconsistent, or poorly calibrated. To address this challenge, we propose a model-based data selection approach that uses only a small fraction of the training data while improving physically grounded LEGO assembly generation. Building on the selected trajectories, we introduce PVPO, a sample-efficient reinforcement learning method that couples physical feasibility with voxel-space geometric rewards. Our results show that physical validity alone is an insufficient proxy for reliable physical reasoning: models can learn to generate valid structures without preserving semantic or geometric fidelity. Experiments across model backbones and test-time scaling settings demonstrate that PVPO improves structural and semantic alignment, physical validity, structural stability, and calibration, while reducing reliance on extensive post-hoc rejection sampling. In particular, results on calibration show that PVPO mitigates PhysHack by making test-time selection more predictive of semantic and structural quality.

Problem

• LEGO assembly
• data-induced failure mode
• physhack
• physical-validity constraints
• geometric misalignment

Innovation

• model-based data selection
• PVPO
• reinforcement learning