RIVA: Efficient relational inference with variate attention.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Complex Systems

Background:

• Interactive systems are prevalent across diverse fields, necessitating methods to understand component interactions.
• Current neural relational inference models face computational inefficiencies due to fully connected graphs.
• Transformer models, while effective for time series forecasting, struggle with time-invariant relational inference due to their attention mechanisms.

Purpose of the Study:

• To develop a novel relational inference model, RIVA, addressing limitations of existing methods.
• To improve the accuracy and efficiency of inferring interactions in dynamic systems.
• To explore explicit interaction graph extraction from attention mechanisms.

Main Methods:

• Proposed RIVA, a relational inference model featuring a variate attention mechanism.
• RIVA encodes entire dynamics, unlike vanilla Transformer's contextual attention.
• Incorporated inferred graph structure as a mask in causal attention for neighbor feature aggregation.

Main Results:

• RIVA demonstrated superior performance in time-invariant continuous interaction inference.
• The model achieved highly accurate future state predictions in dynamic environments.
• RIVA outperformed existing methods in capturing complex interactions.

Conclusions:

• RIVA offers an effective approach to relational inference in interactive systems.
• The variate attention mechanism and graph masking enhance interaction modeling.
• RIVA advances the field of dynamic systems analysis and prediction.