Deep Linear Hawkes Processes

2024-12-27

AI Summary

Existing point process models suffer from low computational efficiency and limited modeling capacity in large-scale settings, hindering their deployment in real-time applications such as healthcare and finance. To address this, we propose the Deep Linear Hawkes Process (DLHP), the first point process framework that integrates deep state-space modeling into temporal point processes. DLHP extends linear differential equations to stochastic jump-diffusion dynamics and employs a parallel scan algorithm for efficient, fully parallelizable discrete-time recursion—achieving linear time complexity (O(N)). This design overcomes fundamental bottlenecks of RNNs (inherent sequentiality) and attention-based models ((O(N^2)) complexity). Evaluated on eight real-world datasets, DLHP consistently outperforms state-of-the-art methods across multiple metrics: log-likelihood, event prediction accuracy, and training/inference efficiency.

Technology Category

• Machine Learning: Hardware-aware ML
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Marked temporal point processes (MTPPs) are used to model sequences of different types of events with irregular arrival times, with broad applications ranging from healthcare and social networks to finance. We address shortcomings in existing point process models by drawing connections between modern deep state-space models (SSMs) and linear Hawkes processes (LHPs), culminating in an MTPP that we call the deep linear Hawkes process (DLHP). The DLHP modifies the linear differential equations in deep SSMs to be stochastic jump differential equations, akin to LHPs. After discretizing, the resulting recurrence can be implemented efficiently using a parallel scan. This brings parallelism and linear scaling to MTPP models. This contrasts with attention-based MTPPs, which scale quadratically, and RNN-based MTPPs, which do not parallelize across the sequence length. We show empirically that DLHPs match or outperform existing models across a broad range of metrics on eight real-world datasets. Our proposed DLHP model is the first instance of the unique architectural capabilities of SSMs being leveraged to construct a new class of MTPP models.

Problem

• Time Point Process Models
• Efficiency Issues
• Mathematical Formulation Limitations

Innovation

• Deep Learning
• Linear Hawkes Process
• Efficient Computation