From Words to Widgets for Controllable LLM Generation

2026-04-12

AI Summary

This work addresses the challenge that users struggle to precisely control subjective preferences—such as tone and style—in large language model (LLM) generation through natural language prompts alone. To overcome this limitation, the authors propose Malleable Prompting, a novel approach that automatically parses natural language expressions of preference and maps them to intuitive graphical user interface (GUI) controls, such as sliders and dropdown menus, augmented with real-time visual feedback. During decoding, the method dynamically modulates the token probability distribution to enable transparent and fine-grained control over generation outcomes. User studies demonstrate that, compared to conventional textual prompting, Malleable Prompting significantly improves alignment with user preferences and is consistently perceived as more controllable and interpretable.

Technology Category

• Natural Language Processing: Prompt Engineering / Prompting
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Learning Human Values and Preferences

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Natural language remains the predominant way people interact with large language models (LLMs). However, users often struggle to precisely express and control subjective preferences (e.g., tone, style, and emphasis) through prompting. We propose Malleable Prompting, a new interactive prompting technique for controllable LLM generation. It reifies preference expressions in natural language prompts into GUI widgets (e.g., sliders, dropdowns, and toggles) that users can directly configure to steer generation, while visualizing each control's influence on the output to support attribution and comparison across iterations. To enable this interaction, we introduce an LLM decoding algorithm that modulates the token probability distribution during generation based on preference expressions and their widget values. Through a user study, we show that Malleable Prompting helps participants achieve target preferences more precisely and is perceived as more controllable and transparent than natural language prompting alone.

Problem

• controllable generation
• subjective preferences
• natural language prompting
• large language models
• user control

Innovation

• Malleable Prompting
• controllable generation
• GUI widgets
• preference modulation
• interactive LLM