U-Define: Designing User Workflows for Hard and Soft Constraints in LLM-Based Planning

2026-05-04

AI Summary

This work addresses the limited transparency and controllability of large language models (LLMs) in task planning, which often hinder effective incorporation of user intent and real-world constraints. The authors propose an interactive planning framework that enables users to specify constraints in natural language as either hard rules or soft preferences. Hard rules are verified through formal model checking, while soft preferences are evaluated using an LLM-as-judge mechanism. By abstracting constraints into high-level types and applying differentiated validation strategies, the approach significantly enhances the reliability of generated plans and user control over the planning process. User studies demonstrate that the system maintains strong usability while substantially improving user ratings of usefulness, performance, and overall satisfaction.

Abstract

LLMs are increasingly used for end-user task planning, yet their black-box nature limits users' ability to ensure reliability and control. While recent systems incorporate verification techniques, it remains unclear how users can effectively apply such rigid constraints to represent intent or adapt to real-world variability. For example, prior work finds that hard-only constraints are too rigid, and numeric flexibility weights confuse users. We investigate how interaction workflows can better support users in applying constraints to guide LLM-generated plans, examining whether abstracting strictness into high-level types (i.e., hard and soft) paired with distinct verification mechanisms helps users more reliably express and align intent. We present U-Define, a system that lets users define constraints in natural language and categorize them as either hard rules that must not be violated or soft preferences that allow flexibility. U-Define verifies these types through complementary methods: formal model checking for hard constraints and LLM-as-judge evaluation for soft ones. Through a technical evaluation and user studies with general and expert participants, we find that user-defined constraint types improve perceived usefulness, performance, and satisfaction while maintaining usability. These findings provide insights for designing flexible yet reliable constraint-based workflows.

Problem

• LLM-based planning
• hard constraints
• soft constraints
• user intent
• workflow design

Innovation

• hard and soft constraints
• LLM-based planning
• formal model checking
• LLM-as-judge
• user-defined workflows