Multistructure Contrastive Learning for Pretraining Event Representation.

Area of Science:

• Natural Language Processing
• Artificial Intelligence
• Machine Learning

Background:

• Traditional event representation models struggle with disconnected yet semantically linked event segments.
• Data sparsity and limited labeled data in event chains lead to overfitting and undertraining in current models.

Purpose of the Study:

• To propose a heterogeneous event graph model (HeterEvent) for explicit representation of event segments.
• To develop a multistructure contrastive learning framework (MulCL) to address data sparsity and improve training robustness.
• To enhance downstream applications like similarity differentiation and missing event prediction.

Main Methods:

• Developed HeterEvent, a heterogeneous event graph model, to capture complex event relationships.
• Integrated MulCL, a multistructure contrastive learning framework, with sequential-view (SeqCL) and graph-view (GraCL) components.
• SeqCL captures sequential event characteristics, while GraCL enhances graph training robustness through corrupted graph comparisons.

Main Results:

• The proposed MulCL model significantly outperforms state-of-the-art baselines in event representation tasks.
• MulCL achieved an average accuracy improvement of 5.3% for inference-ability-based tasks compared to HeterEvent.
• For representation-ability tasks, MulCL showed a 2.7% accuracy improvement in hard similarity and a 4.1% improvement in Spearman's correlation for transitive sentence similarity.

Conclusions:

• The MulCL framework effectively addresses limitations in traditional event representation models.
• The model demonstrates superior performance in capturing both sequential and graph-based event structures.
• This approach offers a robust solution for event representation, improving accuracy in various downstream NLP tasks.