User Feedback Provides a Unique Signal that LLMs Can not Detect

2026-09-02

AI Summary

研究通过合成和自然数据证明用户反馈是改进大型语言模型的有效信号，解决了现有评估范式中对反馈效用低估的问题。

Abstract

Harnessing naturally occurring feedback from user interactions offers a promising learning signal for Large Language Models (LLMs). However, recent studies suggest this feedback is inherently noisy and difficult to leverage effectively. We challenge this conception by demonstrating that user feedback is a highly actionable signal for improvement, and that its perceived ineffectiveness stems from a systematic bias in current evaluation paradigms. To isolate the usefulness of feedback, we construct synthetic data with a definitive ground truth, alongside naturalistic data to validate that our findings hold in real-world scenarios. By comparing model revisions generated with and without access to feedback across both settings, we show that feedback-informed revisions resolve targeted issues at significantly higher rates than baseline revisions. Finally, we expose the root of the evaluation bias: when a model successfully fixes an issue exclusively due to feedback, LLM judges frequently fail to identify the genuinely corrected response, systematically preferring inferior baseline outputs instead.

Problem

• User Feedback
• Large Language Models
• Evaluation Bias

Innovation

• user feedback
• large language models
• evaluation bias
• synthetic data