Your Mouse and Eyes Secretly Leak Your Preference: LLM Alignment using Implicit Feedback from Users

2026-06-18

AI Summary

This work addresses the limitations of current large language model (LLM) alignment methods, which rely on sparse and costly explicit human feedback and struggle to efficiently capture user preferences. To overcome this, the study systematically leverages real-world implicit user behavioral signals—such as eye-tracking fixations and mouse trajectories—and introduces IFLLM, a novel dataset comprising multi-turn dialogues paired with fine-grained behavioral data. A multimodal reward model is trained by fusing textual and behavioral signals, integrating both reward modeling and Direct Preference Optimization (DPO). Evaluated across eight mainstream LLMs, the approach achieves nearly a threefold relative improvement in response quality and boosts the accuracy of implicit-feedback-driven reward prediction from 55% to 64%, substantially enhancing both the efficiency and effectiveness of LLM alignment.

Abstract

To align a Large Language Model (LLM), most existing methods collect explicit human feedback and train a reward model to predict the human preference based on the response text. These existing methods have two key limitations. First, the users rarely provide explicit feedback for LLM responses, which makes the high-quality preference annotation expensive to collect. Second, the methods do not leverage implicit human feedback, which has proven vital to the economic moats of Internet giants. To quantify the value of implicit feedback, we build a new dataset called IFLLM, which collects 1336 multi-turn questions from the 59 Mechanical Turk workers, their mouse trajectories, and eye gazing points to the LLMs' responses from their webcams. IFLLM shows that the users have very diverse types of gazing behavior and mouse trajectories. Our reward model based on the implicit user feedback boosts the accuracy of the text-based reward model from 55% to 64% and nearly triples the relative response quality improvements after applying the DPO to eight LLMs, demonstrating the value of implicit feedback in the wild. Our data collection website, dataset, and codes can be found at https://github.com/themehulpatwari/llm-implicit-feedback/.

Problem

• LLM alignment
• implicit feedback
• human preference
• reward modeling
• user behavior

Innovation

• implicit feedback
• LLM alignment
• mouse trajectory
• eye tracking
• reward modeling