Understanding Multimodal Failure in Action-Chunking Behavioral Cloning

2026-05-21

AI Summary

Behavior cloning often fails when a single observation corresponds to multiple valid action modes, a limitation particularly pronounced in action chunking strategies. This work systematically uncovers the underlying failure mechanism, revealing a fundamental trade-off between posterior-prior regularization and mode preservation. We further demonstrate that Lipschitz smoothness of the action-space transport map inherently constrains multimodal expressivity. Through a comparative analysis of latent-variable models and action-space generative models—supported by both theoretical reasoning and empirical validation—we precisely characterize the conditions under which each approach breaks down on synthetic multimodal tasks and robotic simulation benchmarks. Our findings provide principled theoretical insights and practical guidance for designing effective multimodal behavior cloning algorithms.

Abstract

Behavioral cloning becomes difficult when the same observation admits several valid actions. We study this problem for action-chunking policies and show that different multimodal parameterizations fail in different ways. For latent-variable policies, posterior-prior regularization makes deployment-time sampling more reliable, but excessive regularization removes the action-conditioned information needed to distinguish demonstrated modes. Reducing this regularization can preserve mode information, but then success depends on whether the prior covers the relevant latent regions. For action-space generative policies, multimodality is constrained by the smoothness of the base-to-action transport: a map with small Lipschitz constant cannot assign substantial probability to many well-separated modes. Covering many modes therefore requires either sharp transitions in base space or off-support bridge regions in action space. Experiments on synthetic multimodal tasks and robotic simulation benchmarks support these mechanisms.

Problem

• multimodal failure
• action-chunking
• behavioral cloning
• latent-variable policies
• generative policies

Innovation

• multimodal failure
• action-chunking
• behavioral cloning
• latent-variable policies
• Lipschitz constraint