From Prompting to Partnering: Personalization Features for Human-LLM Interactions

2025-03-02

AI Summary

Large language model (LLM) applications suffer from key interaction bottlenecks: labor-intensive prompt engineering, unreliable outputs, lack of personalization, and opaque decision-making. To address these, we propose five interface-level innovations—Reflective Prompting, Section Regeneration, Input–Output Mapping, Confidence Indicators, and Customizable Panels—that shift human–LLM interaction from unidirectional prompting toward bidirectional collaboration. Our approach integrates human–computer interaction (HCI), explainable AI (XAI), and UX design principles, validated through a two-phase qualitative study: in-depth interviews with eight users followed by high-fidelity prototype usability testing. Results demonstrate significant reductions in cognitive load, alongside improved output transparency, user controllability, and personalization capability. The framework empirically validates a viable pathway to enhancing user agency, system trustworthiness, and co-creation efficacy in LLM interfaces.

Technology Category

• Humans and AI: Intelligent User Interfaces
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large Language Models (LLMs), such as ChatGPT, exhibit advanced capabilities in generating text, images, and videos. However, their effective use remains constrained by challenges in prompt formulation, personalization, and opaque decision-making processes. To investigate these challenges and identify design opportunities, we conducted a two-phase qualitative study. In Phase 1, we performed in-depth interviews with eight everyday LLM users after they engaged in structured tasks using ChatGPT across both familiar and unfamiliar domains. Our findings revealed key user difficulties in constructing effective prompts, iteratively refining AI-generated responses, and assessing response reliability especially in domains beyond users' expertise. Informed by these insights, we designed a high-fidelity prototype incorporating Reflective Prompting, Section Regeneration, Input-Output Mapping, Confidence Indicators, and a Customization Panel. In Phase 2, user testing of the prototype indicated that these interface-level improvements may prove useful for reducing cognitive load, increasing transparency, and fostering more intuitive and collaborative human-AI interactions. Our study contributes to the growing discourse on human-centred AI, advocating for human-LLM interactions that enhance user agency, transparency, and co-creative interaction, ultimately supporting more intuitive, accessible, and trustworthy generative AI systems.

Problem

• Challenges in prompt formulation and personalization with LLMs
• Opaque decision-making processes in AI-generated responses
• User difficulties in assessing response reliability in unfamiliar domains

Innovation

• Reflective Prompting enhances user interaction clarity.
• Section Regeneration allows iterative response refinement.
• Confidence Indicators increase AI decision transparency.