Personality-Guided Code Generation Using Large Language Models

2024-10-16

arXiv.org

AI Summary

This work investigates the impact of personality-guided prompting on large language models’ (LLMs) code generation performance. To address diverse programming tasks—such as algorithmic implementation and creative coding—we propose a lightweight, plug-and-play personality-guided paradigm that integrates instruction fine-tuning and context-aware prompt engineering to model personality traits (e.g., conscientiousness, creativity). We conduct systematic evaluation across seven mainstream LLMs—including CodeLlama and GPT-4—and four major benchmarks (HumanEval, MBPP, DS-1000, and CodeContests), covering 28 model–dataset combinations. Results show performance improvements in 23 configurations, with 11 exceeding +5% absolute pass@1 gain, five surpassing +10%, and a maximum improvement of +12.9%. This study provides the first empirical evidence that aligning personality traits with task requirements significantly enhances LLM code generation capability; moreover, our paradigm exhibits positive synergy with existing prompting strategies. All code and data are publicly released.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Code generation, the automatic creation of source code from natural language descriptions, has garnered significant attention due to its potential to streamline software development. Inspired by research that links task-personality alignment with improved development outcomes, we conduct an empirical study on personality-guided code generation using large language models (LLMs). Specifically, we investigate how emulating personality traits appropriate to the coding tasks affects LLM performance. We extensively evaluate this approach using seven widely adopted LLMs across four representative datasets. Our results show that personality guidance significantly enhances code generation accuracy, with improved pass rates in 23 out of 28 LLM-dataset combinations. Notably, in 11 cases, the improvement exceeds 5%, and in 5 instances, it surpasses 10%, with the highest gain reaching 12.9%. Additionally, personality guidance can be easily integrated with other prompting strategies to further boost performance. We open-source our code and data at https://github.com/IanWalls/Persona-Code.

Problem

• Investigates personality-guided code generation using LLMs
• Evaluates impact of personality traits on coding performance
• Demonstrates accuracy improvement in 23 out of 28 cases

Innovation

• Personality-guided code generation using LLMs
• Emulating task-appropriate personality traits
• Integration with other prompting strategies