prototyping

Designers often struggle to effectively apply insights from human-computer interaction (HCI) research due to challenges in retrieval, disciplinary terminology barriers, and a lack of contextualized, actionable guidance. To address this gap, this work proposes ReFinE—the first Figma plugin that automatically synthesizes HCI research findings and integrates them directly into the UI design workflow. Leveraging natural language processing and context-aware recommendation techniques, ReFinE delivers real-time, visual, and actionable design suggestions tailored to the designer’s current task. Findings from a user study demonstrate that ReFinE significantly reduces cognitive load, enhances designers’ ability to incorporate empirical research evidence into their practice, improves design quality, and accelerates iterative prototyping cycles.

Automatically translating design mockups into high-quality, maintainable, and responsive front-end code remains challenging, as existing approaches relying solely on images struggle to recover intricate UI details. This work introduces Figma2Code, a novel task that establishes the first end-to-end automation pipeline from design to code within the realistic, multimodal environment of Figma, leveraging its rich metadata and asset information. To support this task, we construct a benchmark dataset comprising 213 high-quality, manually curated samples, processed through a combination of rule-based filtering, human annotation, multimodal large language model (MLLM) selection, and metadata refinement applied to community-collected design-code pairs. Systematic evaluation of ten state-of-the-art MLLMs reveals that while closed-source models achieve high visual fidelity, they exhibit significant shortcomings in layout responsiveness and code maintainability.

Existing UI prototyping tools provide weak support for integrating design artifacts such as screenshots and sketches, hindering component reuse, semantic integration, and cross-role collaboration. This paper proposes a novel UI prototyping paradigm grounded in Conceptual Blending Theory, the first to concretize cognitive-science-based blending mechanisms into an interactive tool. It enables semantic-level element mixing across heterogeneous design examples through example-driven component extraction and semantic alignment, lightweight vision–semantics mapping, and real-time blended preview—facilitating staged intent articulation by developers. An empirical study with 14 frontend developers demonstrates that the approach significantly reduces prototype initiation time (average improvement of 42%), stimulates highly unexpected creative combinations (68% novel composition rate), and enhances design–development collaboration efficiency.

This work addresses the protracted development cycles in traditional visual analytics (VA) prototyping that hinder rapid validation of novel ideas. The authors propose a scaffolded, AI-assisted development paradigm centered on the Artifact–Transform Workflow Language (ATWL) as a structured framework, integrating large language model–driven AI assistants with targeted expert interventions to efficiently construct high-quality VA prototypes within hours. The approach successfully instantiated innovative visual designs such as “soft Pareto fronts” and “constellation” groupings. Controlled experiments further revealed the critical influence of scaffolding design, timing of human-AI collaboration, and methods of knowledge injection on prototype quality, leading the authors to advocate for a taxonomy of knowledge expression in human-AI collaborative systems.

Existing evaluation benchmarks struggle to effectively assess the capability of large language models (LLMs) to generate dynamic HTML-based MiniApps with authentic interactive logic. To address this gap, this work introduces MiniAppBench, the first comprehensive benchmark focused on principle-driven, interactive MiniApp generation, along with MiniAppEval—a browser automation–based agent evaluation framework that systematically measures performance across three dimensions: intent consistency, static structure, and dynamic behavior. By leveraging a task set constructed from tens of millions of real-world generated samples and incorporating human-like exploratory testing, the approach overcomes the challenge of evaluation in scenarios lacking definitive ground-truth answers. Experimental results reveal significant shortcomings in current LLMs’ ability to produce high-quality MiniApps, while demonstrating that MiniAppEval aligns closely with human judgment, thereby establishing a reliable standard for research in interactive application generation.

This work proposes an end-to-end agent framework that addresses the limitations of traditional web usability evaluation, which relies on time-consuming user studies and expert reviews ill-suited for rapid iterative development. The framework uniquely integrates multimodal GUI perception with simulated user behavior profiling to interact directly with live web pages without requiring DOM parsing. It incorporates structured usability protocols—including the System Usability Scale (SUS), Single Ease Question (SEQ), and think-aloud methods—to automatically generate standardized user experience reports. Built upon the Avenir-Web architecture, the approach leverages joint visual-semantic modeling and multimodal action grounding to significantly enhance the automation and scalability of usability testing, thereby empowering developers to efficiently create highly usable web interfaces.

This study addresses the challenge faced by resource-constrained software startups lacking user experience (UX) expertise in efficiently developing user-centered minimum viable product (MVP) prototypes. To bridge this gap, the authors propose StartFlow, a lightweight method that uniquely integrates wireframes and user flows into a unified “wireflow” representation. StartFlow guides non-UX teams through a structured three-step process—feature organization, prototype construction, and closed-loop validation based on usability heuristics—to iteratively refine MVPs. Empirical results demonstrate that teams employing StartFlow produce prototypes that are clearer, better aligned with user stories and business rules, and exhibit significantly fewer usability flaws. Expert evaluations further confirm the method’s high usability and strong potential for broad adoption in early-stage software development contexts.

Existing design-to-code approaches lack a unified benchmark and exhibit insufficient robustness to errors in intermediate representations. This work proposes 1D-Bench, the first benchmark grounded in real-world e-commerce workflows, which requires models to iteratively generate executable React code compatible with a fixed toolchain, guided by potentially flawed intermediate representations and reference renderings. The study introduces a novel iterative evaluation paradigm that combines visual feedback with defective intermediate representations, prioritizing robustness under structural noise over literal consistency. The proposed method integrates multimodal large language models, component-level editing, execution-feedback-driven iterative refinement, and post-training with reinforcement learning based on synthetically generated repair trajectories. Experiments demonstrate that iterative editing substantially improves rendering success rates and visual similarity, whereas reinforcement learning yields only marginal gains due to sparse rewards and high variance.

This work addresses the inefficiency and heavy reliance on manual effort in generating method illustrations (MIs) for scientific papers. To overcome this limitation, we propose FigAgent—a multi-agent framework that, for the first time, abstracts human drawing expertise into reusable, evolvable, and composable illustration middleware. Inspired by human-like trial-and-error behavior, FigAgent employs an “explore-and-select” strategy to autonomously produce high-quality, structurally complex method illustrations. The framework integrates multi-agent collaboration, middleware evolution mechanisms, and experience distillation based on visually similar components. Experimental results demonstrate that FigAgent significantly enhances both the quality and automation level of method illustration generation compared to existing approaches.

Existing 3D generation methods struggle to simultaneously achieve high visual fidelity, real-time performance, and mobile deployment. This work proposes the first single-image 3D generation framework that balances deployment efficiency and interactive speed, producing high-quality meshes with baked normals, colored textures, and controllable face counts within 30 seconds; its Flash variant delivers preview-quality results in just 14 seconds. The approach integrates coarse-to-fine VecSet-based geometry generation, multi-view texture synthesis, and 3D back-projection inpainting, while performing mesh simplification, cleanup, normal baking, and parallel UV unwrapping directly on the GPU. Combined with model distillation and pipeline parallelism, the system minimizes end-to-end latency. Experiments demonstrate that the generated assets match the visual quality of commercial solutions, with both automated metrics and blind human evaluations confirming the method’s efficiency and practicality.