Categorical Distributions are Effective Neural Network Outputs for Event Prediction

Abstract

We demonstrate the effectiveness of using a simple neural network output, a categorical probability distribution, for the task of next spike prediction. This case study motivates an investigation into why this simple output structure is not commonly used with neural temporal point process models. We find evidence that many existing datasets for evaluating temporal point process models do not reveal much information about the underlying event generating processes, and many existing models perform well due to regularization effects of model size and constraints on output structure. We extend existing datasets and create new ones in order to explore outside of this information limited regime and find that outputting a simple categorical distribution is competitive across a wide range of datasets.