Conversational AI increases political knowledge as effectively as self-directed internet search

2025-09-05

AI Summary

This study investigates the role of conversational AI in political information acquisition and its effects on citizens’ political knowledge and trust in misinformation, contextualized around the 2024 UK general election. Method: We employed a nationally representative online survey (N > 2,000) coupled with a series of double-blind randomized controlled trials—including multi-model chatbot comparisons and systematic prompt engineering—to rigorously assess generative AI’s efficacy relative to conventional web search. Contribution/Results: We find that 13% of UK voters used chatbots for election-related information in the week prior to voting. Crucially, AI-assisted search yielded equivalent gains in political knowledge compared to self-directed web search, without significantly increasing trust in misinformation. This work pioneers an integrated methodology—combining large-scale empirical survey data with high-fidelity experimental design—to establish a methodological benchmark for evaluating generative AI’s real-world impact on democratic information ecosystems.

Technology Category

• Humans and AI: Voting
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Conversational AI systems are increasingly being used in place of traditional search engines to help users complete information-seeking tasks. This has raised concerns in the political domain, where biased or hallucinated outputs could misinform voters or distort public opinion. However, in spite of these concerns, the extent to which conversational AI is used for political information-seeking, as well the potential impact of this use on users' political knowledge, remains uncertain. Here, we address these questions: First, in a representative national survey of the UK public (N = 2,499), we find that in the week before the 2024 election as many as 32% of chatbot users - and 13% of eligible UK voters - have used conversational AI to seek political information relevant to their electoral choice. Second, in a series of randomised controlled trials (N = 2,858 total) we find that across issues, models, and prompting strategies, conversations with AI increase political knowledge (increase belief in true information and decrease belief in misinformation) to the same extent as self-directed internet search. Taken together, our results suggest that although people in the UK are increasingly turning to conversational AI for information about politics, this shift may not lead to increased public belief in political misinformation.

Problem

• Assessing conversational AI's impact on political knowledge
• Comparing AI effectiveness with internet search for information
• Evaluating AI's role in political misinformation belief

Innovation

• Conversational AI for political information-seeking
• Randomized controlled trials comparing AI and search
• AI increases knowledge and reduces misinformation belief