The Analyst in the Prompt: Role, Retrieval, and Memory Biases in LLM Financial Analysis

2026-09-02

AI Summary

研究通过3575份SEC文件测试了12个LLM在不同用户情境下的金融分析偏差，并尝试了两种缓解策略，以减少角色和记忆对证据解释的影响。

Abstract

Large Language Models (LLMs) increasingly use user context such as memory, profiles, and role prompts to personalize their responses. This personalization can affect evidence-based judgment: the same evidence may lead to different conclusions under different user contexts. Finance provides a high-stakes setting to study this problem because decisions often depend on interpreting long and complex documents. We test this using 3,575 SEC filings across twelve LLMs. We compare persona-conditioned retrieval, neutral retrieval, and memory-framed context to separate the effect of evidence selection from the effect of interpretation. We find that most user-context spillover comes from how models interpret the same evidence under different roles, rather than from retrieving different evidence. We then test two simple mitigation strategies: expressing the same investor mindset as a user profile instead of an assistant role, and separating evidence-based and personalized outputs. Both reduce spillover, but neither removes it completely, and their effectiveness varies substantially across models.

Problem

• Large Language Models
• user context
• financial analysis
• evidence-based judgment
• SEC filings

Innovation

• persona-conditioned retrieval
• evidence interpretation
• user context spillover