SeDT: Sentence-Transformer Decision-Transformer Conditioning for Multi-Turn Conversation Reliability

2026-05-26

AI Summary

This work addresses the degradation in task performance and reliability of large language models during multi-turn dialogues, stemming from their difficulty in distinguishing critical historical constraints from irrelevant context. To mitigate this, the authors propose SeDT, a novel approach that adapts the return-to-go mechanism from offline reinforcement learning to dialogue systems. SeDT operates at inference time without requiring additional training, discarding context, or modifying model weights; instead, it dynamically annotates historical utterances with cumulative relevance scores derived from semantic, lexical, and positional signals, which are then used as conditional inputs to the model. Experimental results demonstrate that SeDT consistently outperforms baselines across all nine model–task combinations in the Lost-in-Conversation benchmark, achieving an average performance gain of 37.7% and simultaneously reducing unreliability in seven of these settings.

Abstract

Large language models (LLMs) achieve impressive performance when a task is fully specified in a single turn, yet the same models lose up to 39% of that performance when the identical task is revealed incrementally across multiple turns, a phenomenon documented at scale as Lost in Conversation. Crucially, this collapse is almost entirely a reliability failure; the best case, the aptitude only falls 16%, while the unreliability more than doubles (+112%). We argue that the root cause is structural, a flat conversation history assigns equal implicit weight to every prior turn, giving the model no signal to distinguish a critical constraint from incidental dialog. We present SeDT Sentence-transformer Decision-Transformer, a training-free inference-time method that resolves this by importing return-to-go conditioning from offline reinforcement learning. SeDT annotates each conversation shard with a cumulative relevance score derived from three complementary semantic, lexical, and positional signals and presents the full annotated history to the model at the final turn, without weight changes, without training data, and without discarding context. Evaluated on the Lost-in-Conversation benchmark in three LLMs and three generation tasks, SeDT outperforms the sharded baseline in all nine model-task combinations, with gains up to +37.7% in mean performance P and simultaneous reductions in unreliability in seven of the nine combinations. In short, telling the model which past turns matter is sufficient to substantially recover the performance lost in conversation.

Problem

• multi-turn conversation
• reliability
• large language models
• Lost in Conversation
• conversation history

Innovation

• SeDT
• multi-turn conversation reliability
• return-to-go conditioning
• training-free inference
• conversation history weighting