An automated approach for real-time informative frames classification in laryngeal endoscopy using deep learning

Chiara Baldini ^1,2, Muhammad Adeel Azam ^1,2, Claudio Sampieri ^3,4,5

¹Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy.
²Departement of Informatics, Bioengineering, Robotics and Systems Engineering, University of Genoa, Genoa, Italy.
³Department of Experimental Medicine (DIMES), University of Genoa, Genoa, Italy. claudio.sampieri@outlook.com.
⁴Department of Otolaryngology, Hospital Clínic, C. de Villarroel, 170, 08029, Barcelona, Spain. claudio.sampieri@outlook.com.
⁵Unit of Head and Neck Tumors, Hospital Clínic, Barcelona, Spain. claudio.sampieri@outlook.com.
⁶Unit of Otolaryngology, Trento, Italy.
⁷Otorhinolaryngology Head-Neck Surgery Department, Hospital Universitari Joan XXIII de Tarragona, Tarragona, Spain.
⁸Department of Otolaryngology, Hospital Clínic, C. de Villarroel, 170, 08029, Barcelona, Spain.
⁹Unit of Head and Neck Tumors, Hospital Clínic, Barcelona, Spain.
¹⁰Translational Genomics and Target Therapies in Solid Tumors Group, Institut d́Investigacions Biomèdiques August Pi i Sunyer, IDIBAPS, Barcelona, Spain.
¹¹Faculty of Medicine, University of Barcelona, Barcelona, Spain.
¹²Unit of Otorhinolaryngology-Head and Neck Surgery, IRCCS Ospedale Policlinico San Martino, Genoa, Italy.
¹³Department of Surgical Sciences and Integrated Diagnostics (DISC), University of Genoa, Genoa, Italy.
¹⁴The BioRobotics Institute and Department of Excellence in Robotics and AI, Scuola Superiore Sant'Anna, Pisa, Italy.

⁰Department of Advanced Robotics, Istituto Italiano di Tecnologia, Genoa, Italy.

European Archives of Oto-Rhino-Laryngology : Official Journal of the European Federation of Oto-Rhino-Laryngological Societies (eufos) : Affiliated with the German Society for Oto-Rhino-Laryngology - Head and Neck Surgery +

|

May 2, 2024

View abstract on PubMed

Summary

Artificial intelligence (AI) can automatically select informative frames from laryngoscopy videos. This AI system shows high accuracy and real-time capabilities, aiding in diagnosis and data management.

Area Of Science

Otolaryngology
Medical Imaging
Artificial Intelligence

Background

Laryngoscopy generates large video datasets, posing challenges for manual review and data extraction.
Automated informative image selection can improve data management and diagnostic accuracy in laryngoscopy.

Purpose Of The Study

To demonstrate the feasibility of an AI system for automatic, real-time selection of informative frames during laryngoscopy.
To provide visual feedback to otolaryngologists during examinations.

Main Methods

Deep learning models, including ResNet-50, were trained and tested on internal and external datasets of laryngoscopy images (white light and narrow band).
The best-performing model's real-time performance was assessed using four testing videos.

Main Results

The ResNet-50 model achieved high precision (95% internal, 97% external) and F1-scores (96% internal, 93% external).
The model demonstrated excellent performance in identifying diagnostically relevant frames.

Conclusions

The AI model shows excellent performance for identifying key frames in laryngoscopic videos.
The system's accuracy and real-time capabilities make it promising for clinical use in quality control and AI-assisted tumor detection.

Keywords:

Artificial intelligence Deep learning Laryngeal cancer Laryngoscopy Larynx