Search or Chat? Comparing How We Learn About Debated Topics

2026-08-14

AI Summary

This study investigates the differential effects of conversational and search-based tools on informal learning regarding controversial topics. Through a crowdsourced comparative experiment, we analyzed how user characteristics and interaction patterns influence learning outcomes and critical reflection. Results indicate no significant differences between the two tools in terms of learning gains or critical reflection. Instead, individual traits such as attitude strength and intellectual humility were found to be stronger determinants of immediate learning efficacy than the specific information retrieval modality employed. These findings suggest that in the context of controversial subject matter, individual cognitive attributes exert a more pivotal influence than technological form factors. Consequently, this work offers novel insights into learning mechanisms within human-computer interaction by highlighting the primacy of learner dispositions over tool design in shaping educational outcomes for contentious issues.

Abstract

As large language models (LLMs) become more integrated into everyday information platforms, chat-based systems are emerging as a popular alternative to traditional web searches, especially for informational search and informal learning tasks. Despite this shift, little is known about how different tools affect learning outcomes. Our work aims to improve the understanding of how chat-based information access supports and impacts learning performance in informal learning settings. In this paper, we present the results of a crowdsourcing user study (N = 194) that compares learning about debated topics using a traditional search interface versus an LLM-powered chat interface. Through our analysis of learning outcomes, user characteristics, and interaction patterns, we found no significant differences in user learning gain or critical reflection on our study tasks. Our observations from the analysis of further exploratory variables suggest that, in the context of longstanding debated topics, user characteristics such as their attitude strength and level of intellectual humility might be more important in shaping immediate learning outcomes than the information access tool.

Problem

• Large Language Models
• Information Seeking
• Informal Learning
• Debated Topics
• Learning Outcomes

Innovation

• LLM-powered chat interface
• informal learning
• debated topics
• intellectual humility
• user study