Enhance graph alignment for large language models.

Area of Science:

• Artificial Intelligence
• Graph Machine Learning
• Natural Language Processing

Background:

• Graph-structured data is common, and Large Language Models (LLMs) show potential for graph modeling.
• Graph-to-token methods convert graph data into sequences for LLMs, using instruction tuning for knowledge acquisition and task specialization.
• Existing methods suffer from task misalignment between self-supervised and supervised tuning, leading to negative transfer.

Purpose of the Study:

• To address the negative transfer issue in LLMs applied to graphs.
• To propose a novel approach, Graph Alignment Large Language Models (GALLM), that aligns self-supervised and supervised tasks.
• To enhance LLM performance on graph-related tasks, particularly improving generalizability and zero-shot capabilities.

Main Methods:

• Introduced a novel text matching task with aligned templates during self-supervised tuning.
• Proposed two category prompt methods with aligned templates for task-specific tuning.
• Utilized aligned task templates to bridge the gap between self-supervised and supervised learning stages.

Main Results:

• GALLM demonstrated substantial improvements in supervised learning performance.
• The model showed enhanced multi-dataset generalizability.
• Significant improvements were observed in zero-shot learning capabilities.

Conclusions:

• GALLM effectively mitigates negative transfer by aligning task templates.
• The proposed method enhances LLM performance on graph data across various tasks.
• GALLM shows promise as a foundational model for graph machine learning applications.