ICPO: Illocution-Calibrated Policy Optimization for Multi-Turn Conversation
This work addresses the challenge of "dialogue drift" in large language models during multi-turn conversations, where ambiguous initial instructions lead to persistent erroneous assumptions that are difficult to correct. To mitigate this, the paper proposes a novel reinforcement learning framework that, for the first time, incorporates pragmatic awareness into the reward mechanism. By detecting instruction ambiguity, the framework dynamically modulates reward signals to encourage the model to express epistemic humility or proactively seek clarification under uncertainty. This approach combines verifiable rewards with data augmentation using ambiguous prompts to enable fine-grained control over response style. Experimental results demonstrate that the proposed method improves average performance by 75% on multi-turn dialogue tasks while maintaining robustness on single-turn benchmarks, significantly enhancing both the cooperativeness and robustness of conversational agents.