MemoPhishAgent: Memory-Augmented Multi-Modal LLM Agent for Phishing URL Detection

2026-02-24

AI Summary

This work addresses the limitations of traditional phishing detection methods, which rely on static rules or blacklists and struggle to keep pace with rapidly evolving attacks, as well as existing large language model (LLM)-based approaches that employ deterministic prompting and fail to fully leverage LLMs’ reasoning capabilities. The paper proposes the first multimodal LLM agent integrated with an episodic memory mechanism, enabling dynamic invocation of specialized detection tools and reuse of historical reasoning trajectories to effectively identify both novel and known phishing threats. Without increasing computational overhead, the method substantially improves recall—by 13.6% on public benchmarks and by 20% in real-world social platform evaluations. Deployed in production, the system processes 60,000 high-risk URLs weekly with a recall rate of 91.44%, providing proactive protection for millions of users.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Multi-modal Vision

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Traditional phishing website detection relies on static heuristics or reference lists, which lag behind rapidly evolving attacks. While recent systems incorporate large language models (LLMs), they are still prompt-based, deterministic pipelines that underutilize reasoning capability. We present MemoPhishAgent (MPA), a memory-augmented multi-modal LLM agent that dynamically orchestrates phishing-specific tools and leverages episodic memories of past reasoning trajectories to guide decisions on recurring and novel threats. On two public datasets, MPA outperforms three state-of-the-art (SOTA) baselines, improving recall by 13.6%. To better reflect realistic, user-facing phishing detection performance, we further evaluate MPA on a benchmark of real-world suspicious URLs actively crawled from five social media platforms, where it improves recall by 20%. Detailed analysis shows episodic memory contributes up to 27% recall gain without introducing additional computational overhead. The ablation study confirms the necessity of the agent-based approach compared to prompt-based baselines and validates the effectiveness of our tool design. Finally, MPA is deployed in production, processing 60K targeted high-risk URLs weekly, and achieving 91.44% recall, providing proactive protection for millions of customers. Together, our results show that combining multi-modal reasoning with episodic memory yields robust phishing detection in realistic user-exposure settings.

Problem

• phishing URL detection
• large language models
• episodic memory
• multi-modal reasoning
• dynamic threat detection

Innovation

• memory-augmented agent
• multi-modal LLM
• episodic memory
• phishing detection
• tool orchestration