Cross-Modal Graph Contrastive Learning with Cellular Images

Area of Science:

• Computational chemistry
• Bioinformatics
• Machine learning in drug discovery

Background:

• Learning molecular representations is crucial for drug discovery, chemistry, and medicine.
• Current graph neural network and self-supervised learning methods primarily use molecular structures, limiting their effectiveness in complex biological processes.
• There is a need for methods that integrate molecular data with biological context.

Purpose of the Study:

• To develop a unified framework for cross-modality pre-training by integrating molecular structures with phenotypic cell microscopy images.
• To improve the learning of molecular representations by incorporating biological context from cell imaging data.
• To enable mutual retrieval of molecules and corresponding cell images and infer functional molecules from cellular phenotypes.

Main Methods:

• A unified framework was constructed for cross-modality pre-training using graph neural networks and self-supervised learning.
• Multiple contrastive loss functions were employed to align molecular structures with high-content cell microscopy images.
• The model was evaluated on tasks including mutual retrieval of molecules and images, inference of functional molecules from cellular images, and molecular property/clinical outcome predictions.

Main Results:

• The proposed framework effectively aligns molecular structures with phenotypic cell images through contrastive learning.
• The model demonstrated success in mutual retrieval tasks between molecules and their corresponding cellular images.
• Significant improvements were observed in predicting molecular properties and clinical outcomes, outperforming existing methods.

Conclusions:

• Cross-modality learning by integrating molecular structures with cell imaging data enhances molecular representation learning.
• This approach bridges the gap between molecular information and biological phenotype, offering significant potential for drug discovery.
• The model's ability to infer functional molecules and predict clinical outcomes highlights its utility in pharmaceutical research.