TacStyle: Personalizing Tactile Robot Policies using Structured Behavior Representations

2026-06-12

AI Summary

Current robotic systems struggle to accurately interpret user preferences regarding operational styles—such as applied force—expressed through natural language. To address this challenge, this work proposes a novel approach that constructs a structured, continuous, and interpretable latent action space. Instead of directly generating behaviors from language, foundation models map linguistic preference prompts into this latent space, enabling the system to reason about and generate tactile manipulation policies aligned with user expectations. By integrating structured representation learning with language-guided preference inference, the method achieves fine-grained, high-fidelity personalization of robot behavior. Experiments in both simulation and real-world settings demonstrate its effectiveness, showing that only a small number of preference labels are sufficient to adapt behaviors precisely to individual user preferences.

Abstract

Robotic systems that assist humans should be capable of adapting their behaviors to individual user preferences. For instance, users may want a robot arm to adjust the amount of force it applies while folding their laundry or cleaning furniture. Natural language provides an intuitive way for humans to communicate such preferences. Recent progress in language-conditioned robot policies has shown that robots can successfully use language prompts to determine what task to perform. However, extending the same approach to realize how the task should be performed requires detailed labels describing the preferences or styles of trajectories in the task data. Not only is collecting such annotations challenging, but conditioning directly on these labels may also fail to provide fine-grained control over a continuous range of behaviors. For example, it can be difficult to convey the exact force that a robot must apply through abstract instructions like "apply a bit more pressure than before". Therefore, in this work, we propose using language to reason over preferred behaviors instead of directly generating them. We first learn a structured latent representation that organizes user preferences according to differences in the corresponding trajectories. Then, given a preference prompt, we use a foundation model to interpret this latent space and choose a value that produces the desired behavior. Through both simulation and real-world experiments, we show that selecting robot behaviors from an intuitively structured latent space enables more precise adaptation to user preferences while requiring significantly fewer preference labels than language-conditioned policies.

Problem

• personalization
• tactile robot policies
• user preferences
• language-conditioned control
• behavior adaptation

Innovation

• structured latent representation
• language-conditioned policy
• tactile robot policies
• preference adaptation
• foundation model