From SERPs to Agents: A Platform for Comparative Studies of Information Interaction

2026-01-14

AI Summary

This work addresses the lack of an efficient experimental platform for comparative user studies across diverse information access systems—such as retrieval-augmented generation (RAG) and autonomous agents—which currently suffer from high deployment and management costs. To bridge this gap, we present UXLab, an open-source system featuring a no-code, visual web dashboard that supports the entire research workflow, from participant recruitment and experiment configuration to backend integration with traditional search engines, vector databases, and large language models, along with comprehensive behavioral data collection. UXLab substantially lowers the technical barrier for conducting multi-system comparative studies and offers an extensible framework for future multimodal interaction research. A micro case study successfully uncovered distinct user behaviors between RAG and autonomous agent conditions, demonstrating the platform’s effectiveness in enhancing experimental efficiency and streamlining research processes.

Technology Category

• Humans and AI: User Experience and Usability
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Application Domains: Humanities & Computational Social Science

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

The diversification of information access systems, from RAG to autonomous agents, creates a critical need for comparative user studies. However, the technical overhead to deploy and manage these distinct systems is a major barrier. We present UXLab, an open-source system for web-based user studies that addresses this challenge. Its core is a web-based dashboard enabling the complete, no-code configuration of complex experimental designs. Researchers can visually manage the full study, from recruitment to comparing backends like traditional search, vector databases, and LLMs. We demonstrate UXLab's value via a micro case study comparing user behavior with RAG versus an autonomous agent. UXLab allows researchers to focus on experimental design and analysis, supporting future multi-modal interaction research.

Problem

• information access systems
• comparative user studies
• technical overhead
• RAG
• autonomous agents

Innovation

• UXLab
• no-code platform
• comparative user studies
• information interaction
• autonomous agents