Automatic Graph Topology-Aware Transformer

Area of Science:

• Artificial Intelligence
• Machine Learning
• Graph Neural Networks

Background:

• Graph Transformers offer powerful representation capabilities for graph data.
• Manual design of Graph Transformer architectures requires significant expertise and time.
• Automating architecture search is crucial for efficient development.

Purpose of the Study:

• To propose an automated framework for constructing high-performance Graph Transformers.
• To develop an evolutionary Graph Transformer architecture search (EGTAS) method.
• To reduce the cost and complexity of designing effective Graph Transformer models.

Main Methods:

• Developed a comprehensive search space for Graph Transformers, including micro-level (strategies) and macro-level (topologies) designs.
• Implemented an evolutionary algorithm to search for optimal Graph Transformer architectures.
• Introduced a surrogate model for efficient performance prediction, reducing evaluation costs.

Main Results:

• The EGTAS framework successfully automates the construction of strong Graph Transformer architectures.
• EGTAS demonstrated efficacy across diverse graph-level and node-level tasks on various dataset scales.
• Constructed architectures by EGTAS rival state-of-the-art manual and automated methods.

Conclusions:

• EGTAS provides an effective and efficient solution for automating Graph Transformer architecture design.
• The proposed framework significantly lowers the barrier to entry for utilizing advanced Graph Transformers.
• Automated architecture search is a promising direction for advancing Graph Transformer research and application.