BiFormer: A Bipartite-stream Information Fusion framework for large-scale graph representation learning.

Area of Science:

• Graph Neural Networks
• Graph Transformers
• Machine Learning

Background:

• Graph Neural Networks (GNNs) capture local graph details, while Graph Transformers (GTs) capture global information.
• Both GNNs and GTs face scalability challenges on large graphs.
• Existing methods struggle to balance local and global information processing for large-scale graph tasks.

Purpose of the Study:

• To propose BiFormer, a novel framework that fuses GNN and GT strengths for large-scale graph processing.
• To address the scalability limitations of current GNN and GT models.
• To develop an efficient method for integrating local and global graph features.

Main Methods:

• BiFormer employs a three-module architecture: global feature extraction via a Transformer encoder on a pooled graph.
• Local feature extraction uses three parameter-free graph convolution kernels.
• A feature fusion module uses a Transformer encoder to integrate local and global node features without message passing.

Main Results:

• BiFormer enables mini-batch training by requiring only the pooled graph and mini-batched local features in memory.
• The framework demonstrates efficient processing of large-scale graphs.
• Experimental results show BiFormer outperforms mainstream GNNs and GTs.

Conclusions:

• BiFormer effectively integrates local and global information for large-scale graph representation learning.
• The proposed method offers a scalable and efficient solution compared to existing GNN and GT approaches.
• BiFormer achieves superior performance, highlighting its potential for complex graph-based tasks.