Different Feedback, Different Updates: Selective Self-Learning from User Interactions for Large Language Models

2026-08-10

AI Summary

This work addresses the challenge that existing methods struggle to effectively leverage the mixed improvement signals in user feedback, often resulting in ineffective or conflicting model updates. To overcome this, the authors propose SLIFT, a novel framework that interprets feedback semantics from a task-relative perspective, decomposing it into atomic components and categorizing them as Fix (task-validity corrections), Spec (conditional refinements), or Null (unreliable signals). SLIFT employs a frozen backbone combined with a dual LoRA adapter architecture—comprising Generalist and Specialist modules—and integrates feedback-conditioned self-distillation with residual-guided updating to enable hierarchical, selective, and fine-grained parameter adaptation. Evaluated on MemoryBench and WildFB benchmarks, SLIFT significantly outperforms baseline approaches, demonstrating its ability to precisely incorporate valid user feedback while preserving pre-existing knowledge.

Abstract

User feedback offers natural supervision for persistent LLM improvement, but a single message may support multiple behavioral changes with different scopes of generalization. We introduce SLIFT, a selective self-learning framework built on a task-relative view of user feedback. SLIFT decomposes each feedback message into atomic components and interprets each component relative to the original task as Fix, Spec, or Null: requirements for task validity, compatible condition-specific refinements, or content with no reliable positive update direction. To incorporate each change at the appropriate scope, SLIFT trains two complementary LoRA adapters on a shared frozen backbone: a Generalist that consolidates Fix requirements into default behavior through feedback-conditioned self-distillation, and a Specialist that observes only the task and Generalist response to supply residual guidance for applicable, unmet Spec refinements. Null components induce no positive update. Across backbones, SLIFT achieves strong performance on both MemoryBench and WildFB, with targeted analyses further examining its underlying mechanisms. We release our code at https://anonymous.4open.science/r/SLIFT.

Problem

• user feedback
• self-learning
• large language models
• selective update
• behavioral generalization

Innovation

• Selective Self-Learning
• User Feedback Decomposition
• Task-Relative Feedback Interpretation
• Dual LoRA Adapters
• Feedback-Conditioned Self-Distillation