CLOOME: contrastive learning unlocks bioimaging databases for queries with chemical structures.

Area of Science:

• Bioimage analysis
• Artificial intelligence
• Drug discovery

Background:

• Bioimage analysis is undergoing transformation due to advanced imaging and AI.
• Multi-modal AI systems offer potential for integrating diverse data modalities.
• Current bioimaging databases have limitations in knowledge extraction.

Purpose of the Study:

• To develop a retrieval system for querying bioimaging databases using chemical structures.
• To leverage multi-modal contrastive learning for unified bioimage and chemical structure embedding.
• To demonstrate the utility of this approach in drug discovery applications.

Main Methods:

• Utilized multi-modal contrastive learning paradigm.
• Developed bioimage and molecular structure encoders for unified embedding.
• Created a retrieval system to match chemical structures with corresponding bioimages.

Main Results:

• Achieved >70 times higher top-1 accuracy than random baseline in identifying correct bioimages for chemical structures.
• Demonstrated remarkable transferability of the bioimage encoder to drug discovery tasks.
• Successfully queried a database of ~2000 bioimages with chemical structures.

Conclusions:

• The developed multi-modal system effectively addresses limitations in bioimaging databases.
• This approach enables querying bioimages with chemical structures based on phenotypic effects.
• Paved the way for foundation models in microscopy image analysis and drug discovery.