FACTER: Fairness-Aware Conformal Thresholding and Prompt Engineering for Enabling Fair LLM-Based Recommender Systems

2025-02-05

AI Summary

This paper addresses pervasive demographic bias in large language model (LLM)-based recommender systems, proposing a dynamic fairness regulation framework that requires no model retraining. Methodologically, it integrates conformal prediction with dynamic prompt engineering to introduce a novel violation-driven fairness constraint mechanism: a semantic variance-based proxy metric quantifies bias; adaptive thresholds and violation-triggered mechanisms dynamically tighten fairness constraints in real time; and an adversarial prompt generator actively suppresses biased outputs by leveraging historical bias patterns. Experiments on MovieLens and Amazon datasets demonstrate a 95.5% reduction in fairness violations while preserving high recommendation accuracy, empirically validating semantic variance as a robust bias proxy.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Computer Vision: Bias, Fairness & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We propose FACTER, a fairness-aware framework for LLM-based recommendation systems that integrates conformal prediction with dynamic prompt engineering. By introducing an adaptive semantic variance threshold and a violation-triggered mechanism, FACTER automatically tightens fairness constraints whenever biased patterns emerge. We further develop an adversarial prompt generator that leverages historical violations to reduce repeated demographic biases without retraining the LLM. Empirical results on MovieLens and Amazon show that FACTER substantially reduces fairness violations (up to 95.5%) while maintaining strong recommendation accuracy, revealing semantic variance as a potent proxy of bias.

Problem

• Ensures fairness in LLM-based recommendations
• Reduces demographic biases without retraining
• Maintains accuracy while improving fairness

Innovation

• Fairness-aware conformal prediction
• Dynamic prompt engineering
• Adversarial prompt generator