AugGen: Augmenting Task-Based Learning in Professional Creative Software with LLM-Generated Scaffolded UIs

2025-11-28

AI Summary

Professional creative software poses high learning barriers due to complex interfaces, domain-specific jargon, and insufficient task-oriented guidance. This paper introduces AugGen, a scaffolding-based UI generation framework powered by large language models (LLMs), which parses user task specifications to automatically extract domain concepts and operational workflows, then dynamically synthesizes structured, task-aligned interfaces. Its core innovations include: (1) stage-wise tool organization, (2) salient highlighting of critical functions, (3) explicit conceptual association mapping, and (4) progressive disclosure of advanced features. AugGen is integrated into Blender and instantiated with two task-specific interface variants. A controlled user study demonstrates statistically significant reductions in cognitive load (*p* < 0.01), a 32% improvement in task completion rate, and a 28% gain in conceptual mastery—empirically validating LLM-driven dynamic UI restructuring as an effective approach for task-centered learning support.

Technology Category

• Humans and AI: Intelligent User Interfaces
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Professional creative software often presents steep learning curves due to complex interfaces, lack of structured task-aware guidance, and unfamiliar domain terminology. To address these challenges and augment user learning experience, we introduce AugGen, a method for generating scaffolded user interfaces that simplify interface complexity and support task-based learning. With the user's task, our method surfaces task-relevant tools to reduce distracting features, organizes the tools around task workflow stages to offer execution guidance, connects tools with domain concepts to foster learning engagement, and progressively discloses advanced features to manage learning progress. To evaluate the method, we used our LLM-assisted pipeline to generate two task-specific scaffolded UIs and deployed them in Blender, our professional 3D modeling testbed. We invited both beginner (N=32) and expert (N=8) users to evaluate our implemented interfaces. Results show that the scaffolded interfaces significantly reduced user-perceived task load, enhanced task performance via embedded guidance, and augmented concept learning during task execution.

Problem

• Simplifying complex interfaces in professional creative software
• Providing structured task-aware guidance for learning
• Connecting tools with domain concepts to enhance engagement

Innovation

• LLM-generated scaffolded UIs simplify interface complexity
• Task-relevant tools organized around workflow stages
• Progressive disclosure of advanced features manages learning