PhishIntel: Toward Practical Deployment of Reference-based Phishing Detection

2024-12-12

arXiv.org

AI Summary

This paper addresses the high latency and inefficient URL analysis of reference-based phishing detectors (RBPDs) in production deployment. We propose PhishIntel—the first end-to-end, low-latency phishing detection system designed for real-world operational environments. Its core is a fast-slow dual-task coordination architecture: the fast path delivers millisecond-scale responses via local blacklist lookups and caching; the slow path dynamically triggers RBPDs only when needed, performing online blacklist validation, web crawling, and content analysis. We introduce a novel dynamic task scheduling mechanism that maintains high zero-day phishing detection rates while significantly reducing average response latency. PhishIntel has been deployed as both an operational phishing intelligence platform and an Outlook add-in, marking the first robust, low-latency production deployment of RBPDs in real-world scenarios.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Data Mining & Knowledge Management: Intelligent Query Processing
• Machine Learning: Hardware-aware ML

Application Category

• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Phishing is a critical cyber threat, exploiting deceptive tactics to compromise victims and cause significant financial losses. While reference-based phishing detectors (RBPDs) have achieved notable advancements in detection accuracy, their real-world deployment is hindered by challenges such as high latency and inefficiency in URL analysis. To address these limitations, we present PhishIntel, an end-to-end phishing detection system for real-world deployment. PhishIntel intelligently determines whether a URL can be processed immediately or not, segmenting the detection process into two distinct tasks: a fast task that checks against local blacklists and result cache, and a slow task that conducts online blacklist verification, URL crawling, and webpage analysis using an RBPD. This fast-slow task system architecture ensures low response latency while retaining the robust detection capabilities of RBPDs for zero-day phishing threats. Furthermore, we develop two downstream applications based on PhishIntel: a phishing intelligence platform and a phishing email detection plugin for Microsoft Outlook, demonstrating its practical efficacy and utility.

Problem

• Enhance phishing detection efficiency
• Reduce URL analysis latency
• Facilitate real-world deployment

Innovation

• Fast-slow task system architecture
• Local blacklists and result cache
• Online blacklist verification and URL crawling