Deep Continuous-Time State-Space Models for Marked Event Sequences

Area of Science:

• Machine Learning
• Time Series Analysis
• Stochastic Processes

Background:

• Marked temporal point processes (MTPPs) are crucial for modeling event sequences in various fields.
• Existing MTPP models face limitations in capturing continuous-time dynamics and expressivity.
• Deep state-space models (SSMs) offer advanced techniques for sequence modeling.

Purpose of the Study:

• Propose a novel and performant MTPP model, the state-space point process (S2P2).
• Address limitations of current MTPP models by leveraging deep SSM techniques.
• Imbue inductive biases for continuous-time event sequences not captured by discrete models.

Main Methods:

• Developed the S2P2 model, integrating stochastic jump differential equations with nonlinearities.
• Built upon classical linear Hawkes processes for an intensity-based MTPP.
• Utilized a parallel scan for efficient training and inference with linear complexity.

Main Results:

• S2P2 achieves state-of-the-art predictive likelihoods across eight real-world datasets.
• Demonstrated an average improvement of 33% over existing MTPP approaches.
• Showcased efficient training and inference with sublinear scaling.

Conclusions:

• The S2P2 model offers a highly expressive and performant solution for MTPPs.
• S2P2 effectively models continuous-time event sequences with strong inductive biases.
• This novel approach advances MTPP modeling in healthcare, finance, and social networks.