Binary Graph Convolutional Network With Capacity Exploration.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Graph Neural Networks

Background:

• Graph Neural Networks (GNNs) require substantial memory for large attributed graphs.
• Existing GNNs face limitations with memory constraints due to full-precision processing.

Purpose of the Study:

• To develop a memory-efficient and accelerated GNN model.
• To address the challenges of processing large attributed graphs with limited resources.

Main Methods:

• Proposed Binary Graph Convolutional Network (Bi-GCN) binarizing network parameters and node attributes.
• Utilized binary operations instead of floating-point matrix multiplications.
• Introduced a novel gradient approximation for back-propagation in Bi-GCNs.
• Developed an Entropy Cover Hypothesis to address potential capacity issues in binarized GNNs.

Main Results:

• Achieved an average of ~31x reduction in memory consumption for parameters and data.
• Demonstrated an average of ~51x acceleration in inference speed on citation networks (Cora, PubMed, CiteSeer).
• Extended binarization to other GNN variants, achieving similar efficiency gains.
• Confirmed comparable performance to full-precision baselines on seven node classification datasets.
• Validated the Entropy Cover Hypothesis for managing capacity in compressed GNNs.

Conclusions:

• Bi-GCN offers significant compression and acceleration for GNNs.
• The proposed methods effectively handle large graph data under memory constraints.
• The Entropy Cover Hypothesis provides a viable solution for capacity limitations in binarized GNNs.