From Experts to the Public: Governing Multimodal Language Models in Politically Sensitive Video Analysis

2024-09-15

arXiv.org

AI Summary

This study addresses the challenge of aligning multimodal large language models (MLLMs) with public values in politically sensitive video analysis, while mitigating structural biases introduced by centralized governance mechanisms into user decision-making. Method: We employ a two-phase experimental design—expert interviews followed by large-scale public deliberation—integrating quadratic voting, dynamic power allocation, and the Inclusive.AI platform to construct a participatory, DAO-based AI behavioral decision framework. Contribution/Results: Empirical findings reveal systematic value divergences between experts and the public in semantic interpretation of political videos; quadratic voting significantly enhances participants’ perceived commitment to liberal democracy and political equality; and the proposed democratic deliberative paradigm demonstrates both theoretical rigor and practical scalability. This work provides the first empirically grounded pathway integrating multimodal AI analysis with decentralized, value-sensitive governance—advancing participatory AI alignment in high-stakes sociopolitical domains.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Machine Learning: Large Multimodal Models (LMMs)
• Philosophy and Ethics of AI: Morality & Value-based AI

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

This paper examines the governance of multimodal large language models (MM-LLMs) through individual and collective deliberation, focusing on analyses of politically sensitive videos. We conducted a two-step study: first, interviews with 10 journalists established a baseline understanding of expert video interpretation; second, 114 individuals from the general public engaged in deliberation using Inclusive.AI, a platform that facilitates democratic decision-making through decentralized autonomous organization (DAO) mechanisms. Our findings show that while experts emphasized emotion and narrative, the general public prioritized factual clarity, objectivity of the situation, and emotional neutrality. Additionally, we explored the impact of different governance mechanisms: quadratic vs. weighted ranking voting and equal vs. 20-80 power distributions on users decision-making on how AI should behave. Specifically, quadratic voting enhanced perceptions of liberal democracy and political equality, and participants who were more optimistic about AI perceived the voting process to have a higher level of participatory democracy. Our results suggest the potential of applying DAO mechanisms to help democratize AI governance.

Problem

• Governance of multimodal LLMs in political video analysis
• Differences in interpretative priorities between experts and public
• Impact of voting mechanisms on AI behavior decision-making

Innovation

• Uses DAO mechanisms for democratic decision-making
• Compares quadratic vs weighted voting governance
• Focuses on multimodal LLMs in political videos