Categorical Distributions are Effective Neural Network Outputs for Event Prediction

2025-07-29

AI Summary

Neural temporal point process (NTPP) models have long favored continuous-time output distributions (e.g., exponential, Weibull), while simple categorical outputs remain underexplored—raising questions about whether existing benchmarks adequately expose underlying event-generation mechanisms. Method: We systematically compare categorical and continuous output distributions across diverse NTPP architectures and introduce a new, information-rich synthetic dataset designed to better discriminate modeling capabilities. Contribution/Results: (1) Categorical outputs match or exceed state-of-the-art continuous-distribution models in predictive accuracy on multiple real-world and synthetic datasets; (2) Standard evaluation is often confounded by data information bottlenecks and excessive regularization, obscuring true model capacity; (3) Categorical modeling inherently mitigates time-scale sensitivity and tail-distribution bias. This work challenges the prevailing design paradigm for NTPP output layers and establishes categorical modeling as a promising route toward lightweight, robust, and interpretable event-time prediction.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Calibration & Uncertainty Quantification
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

We demonstrate the effectiveness of using a simple neural network output, a categorical probability distribution, for the task of next spike prediction. This case study motivates an investigation into why this simple output structure is not commonly used with neural temporal point process models. We find evidence that many existing datasets for evaluating temporal point process models do not reveal much information about the underlying event generating processes, and many existing models perform well due to regularization effects of model size and constraints on output structure. We extend existing datasets and create new ones in order to explore outside of this information limited regime and find that outputting a simple categorical distribution is competitive across a wide range of datasets.

Problem

• Why categorical distributions are underused in neural temporal point processes
• Assessing dataset limitations in revealing event generation processes
• Evaluating categorical distribution effectiveness across diverse datasets

Innovation

• Uses categorical probability distribution for prediction
• Investigates limited datasets for temporal processes
• Extends datasets to validate categorical distribution effectiveness