Botender: Supporting Communities in Collaboratively Designing AI Agents through Case-Based Provocations

2025-09-29

AI Summary

AI agents in online communities are typically designed by external or small technical teams, resulting in misalignment with the diverse needs of community members. Method: We propose a no-code, case-driven collaborative design methodology enabling non-technical users to co-define the behavior of large language model (LLM)-powered community bots directly within Discord. The approach employs interactive generation of representative scenario cases to stimulate collective reflection, surface behavioral improvement opportunities and points of disagreement, and support consensus-oriented iterative refinement. Contribution/Results: Deployed over five days across six Discord servers, the method significantly increased participation breadth and decision-making efficiency. Empirical results validate the effectiveness of case-based scaffolding in lowering design barriers, enhancing community agency, and facilitating consensus formation in participatory AI system design.

Technology Category

• Multiagent Systems: Mechanism Design
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

AI agents, or bots, serve important roles in online communities. However, they are often designed by outsiders or a few tech-savvy members, leading to bots that may not align with the broader community's needs. How might communities collectively shape the behavior of community bots? We present Botender, a system that enables communities to collaboratively design LLM-powered bots without coding. With Botender, community members can directly propose, iterate on, and deploy custom bot behaviors tailored to community needs. Botender facilitates testing and iteration on bot behavior through case-based provocations: interaction scenarios generated to spark user reflection and discussion around desirable bot behavior. A validation study found these provocations more useful than standard test cases for revealing improvement opportunities and surfacing disagreements. During a five-day deployment across six Discord servers, Botender supported communities in tailoring bot behavior to their specific needs, showcasing the usefulness of case-based provocations in facilitating collaborative bot design.

Problem

• Enabling communities to collaboratively design AI agents
• Addressing misalignment between bot design and community needs
• Facilitating bot behavior testing through case-based provocations

Innovation

• Collaborative bot design without coding
• Case-based provocations spark user reflection
• Tailors bot behavior to community needs