Pathways of Thoughts: Multi-Directional Thinking for Long-form Personalized Question Answering

2025-09-23

AI Summary

Personalized question answering faces two key challenges: implicitly inferring user preferences from lengthy, noisy contexts, and generating responses that are factually accurate, contextually coherent, and aligned with the user’s cognitive background. To address these, we propose Path-of-Thought (PoT), a reasoning framework that models inference as a multi-path dynamic decision process. PoT enables plug-and-play orchestration of complementary cognitive operations—including reasoning, revision, personalization, and clarification—without task-specific fine-tuning, thereby supporting long-context personalized QA. By generating diverse answer candidates across multiple reasoning trajectories and aggregating them via preference-aware reweighting, PoT significantly improves response quality. On the LaMP-QA benchmark, it achieves up to a 13.1% relative improvement; human evaluation shows superiority in 66% of cases, parity in only 15%, and clear gains across diverse domains—demonstrating both effectiveness and strong generalization.

Technology Category

• Natural Language Processing: Question Answering
• Knowledge Representation and Reasoning: Preferences
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Personalization is essential for adapting question answering (QA) systems to user-specific information needs, thereby improving both accuracy and user satisfaction. However, personalized QA remains relatively underexplored due to challenges such as inferring preferences from long, noisy, and implicit contexts, and generating responses that are simultaneously correct, contextually appropriate, and aligned with user expectations and background knowledge. To address these challenges, we propose Pathways of Thoughts (PoT), an inference-stage method that applies to any large language model (LLM) without requiring task-specific fine-tuning. The approach models the reasoning of an LLM as an iterative decision process, where the model dynamically selects among cognitive operations such as reasoning, revision, personalization, and clarification. This enables exploration of multiple reasoning trajectories, producing diverse candidate responses that capture different perspectives. PoT then aggregates and reweights these candidates according to inferred user preferences, yielding a final personalized response that benefits from the complementary strengths of diverse reasoning paths. Experiments on the LaMP-QA benchmark for personalized QA show that PoT consistently outperforms competitive baselines, achieving up to a 13.1% relative improvement. Human evaluation corroborates these results, with annotators preferring outputs from PoT in 66% of cases and reporting ties in only 15% of cases.

Problem

• Adapting QA systems to user-specific information needs for better accuracy
• Inferring preferences from long, noisy, and implicit user contexts
• Generating responses aligned with user expectations and background knowledge

Innovation

• Dynamic cognitive operation selection during inference
• Multi-trajectory reasoning for diverse candidate generation
• Preference-based aggregation of complementary reasoning paths