Toward Collective-Centric Evaluation of Preference Inference for Participatory Democracy

2026-09-02

AI Summary

研究针对参与式民主中偏好推断问题，提出以集体为中心的评估框架，并通过大规模多语言数据集测试现有方法，确保推断结果保持集体偏好特征。

Abstract

To scale up collective decision-making, participatory democracy platforms such as Polis and Remesh enable online deliberation among thousands of participants. However, at this scale, participants cannot review every opinion submitted by others, producing highly sparse voting data that misrepresent patterns of consensus, conflict, and minority support. Platforms therefore increasingly rely on Preference Inference (PI) models to predict missing votes. Yet this automation is not neutral: inferred preferences can artificially amplify, suppress, or reorder existing patterns of support, ultimately reshaping how the outcomes of a deliberation are interpreted. More generally, we lack a systematic understanding of how existing PI methods affect the collective preference landscape. To address this gap, we benchmark several existing PI approaches in this context. Moving beyond conventional user-centric evaluations centered on the accuracy of individual predictions, we introduce a collective-centric evaluation framework that measures whether inferred votes preserve salient properties of the broader preference landscape. We further contribute the largest multilingual dataset of its kind: four consultations spanning over 90k participants, 1M votes, and 22 languages. Our experiments show that models with comparable predictive accuracy can differ substantially in the degree to which they preserve the collective structure. These results demonstrate that accuracy alone is insufficient for evaluating \emph{PI} in democratic settings. By contributing a novel comprehensive and collective-centric evaluation benchmark for the task of PI, this work aims to support the development of AI systems that scale deliberation without compromising the integrity of its democratic outcomes.

Problem

• Preference Inference
• Participatory Democracy
• Collective Decision-making
• Sparse Voting Data

Innovation

• Collective-Centric Evaluation
• Preference Inference
• Participatory Democracy
• Multilingual Dataset