An augmented reality microscope with real-time artificial intelligence integration for cancer diagnosis

Area of Science:

• Digital pathology
• Medical imaging
• Artificial intelligence in oncology

Background:

• Microscopic tissue assessment is crucial for cancer diagnosis and staging, but suffers from variability and pathologist shortages.
• Artificial intelligence (AI) offers potential improvements but faces barriers like digitization costs and deployment challenges.

Purpose of the Study:

• To propose a cost-effective solution, the augmented reality microscope (ARM), for integrating AI into pathology.
• To demonstrate the utility of ARM in real-time cancer detection and identification.

Main Methods:

• Development and implementation of an augmented reality microscope (ARM).
• Overlaying AI-based information onto the microscopic view of tissue samples in real time.
• Testing ARM for detection of metastatic breast cancer and identification of prostate cancer.

Main Results:

• The ARM successfully demonstrated utility in detecting metastatic breast cancer.
• The ARM showed effectiveness in identifying prostate cancer.
• The system operated with latency compatible with real-time clinical use.

Conclusions:

• The augmented reality microscope (ARM) offers a cost-effective method for integrating AI into pathology.
• ARM has the potential to overcome barriers to AI adoption in cancer diagnosis.
• This technology can improve the accuracy and efficiency of cancer diagnosis and staging.