Great Power Brings Great Responsibility: Personalizing Conversational AI for Diverse Problem-Solvers

2025-02-11

AI Summary

This study addresses the neglect of cognitive diversity among newcomers in open-source communities by mainstream large language models (LLMs). We propose the first AI-powered, personalized dialogue support paradigm grounded in problem-solving style (PSS). Our method comprises three core components: (1) a cognitive style modeling framework; (2) empirical analysis of newcomer interaction behaviors in open-source settings; and (3) a persona-based prompting engineering approach that dynamically adapts AI responses to users’ distinct reasoning patterns. By explicitly modeling PSS, our framework mitigates implicit stylistic biases inherent in generic LLMs—thereby enhancing fairness, accessibility, and pedagogical effectiveness of AI guidance. Key contributions include: (1) formal definition and empirical validation of PSS as a critical dimension for personalization; (2) a style-aware dialogue support framework integrating cognitive modeling and adaptive prompting; and (3) a scalable, interpretable AI empowerment pathway tailored to marginalized newcomer cohorts.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: AI for Accessibility
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Newcomers onboarding to Open Source Software (OSS) projects face many challenges. Large Language Models (LLMs), like ChatGPT, have emerged as potential resources for answering questions and providing guidance, with many developers now turning to ChatGPT over traditional Q&A sites like Stack Overflow. Nonetheless, LLMs may carry biases in presenting information, which can be especially impactful for newcomers whose problem-solving styles may not be broadly represented. This raises important questions about the accessibility of AI-driven support for newcomers to OSS projects. This vision paper outlines the potential of adapting AI responses to various problem-solving styles to avoid privileging a particular subgroup. We discuss the potential of AI persona-based prompt engineering as a strategy for interacting with AI. This study invites further research to refine AI-based tools to better support contributions to OSS projects.

Problem

• Addressing biases in AI responses for newcomers
• Personalizing AI to match diverse problem-solving styles
• Enhancing AI tools for OSS project contributions

Innovation

• Personalizing AI with personas
• Adapting responses to styles
• Mitigating bias in LLMs