Accuracy of Cloud-Based Speech Recognition Open Application Programming Interface for Medical Terms of Korean

Area of Science:

• Medical Informatics
• Natural Language Processing
• Speech Recognition Technology

Background:

• Limited data exists on the accuracy of cloud-based speech recognition (SR) open application programming interfaces (APIs) for medical terminology.
• Evaluating the performance of these APIs in Korean is crucial for clinical applications.

Purpose of the Study:

• To assess the medical term recognition accuracy of currently available cloud-based SR open APIs in Korean.
• To compare the performance of Naver Clova SR, Google Speech-to-Text, and Amazon Transcribe for medical terms.

Main Methods:

• Analysis of SR accuracy using doctor-patient conversation recordings from a tertiary medical center in Korea.
• Comparison of transcriptions from three cloud-based SR open APIs: Naver Clova SR, Google Speech-to-Text, and Amazon Transcribe.
• Calculation of accuracy rate based on the number of correctly recognized medical terms.

Main Results:

• Naver Clova SR achieved the highest medical term recognition accuracy (75.1%), significantly outperforming Google Speech-to-Text (50.9%) and Amazon Transcribe (57.9%).
• Amazon Transcribe showed higher accuracy than Google Speech-to-Text.
• Naver Clova SR excelled in accuracy across word classes, but performance differences diminished for words longer than five characters.

Conclusions:

• Naver Clova SR, a Korean-developed API, offers superior accuracy for recognizing Korean medical terms compared to Google and Amazon alternatives.
• Despite current limitations in medical terminology recognition, cloud-based SR APIs present a promising technology with potential for improvement.
• Combining the strengths of different SR engines could enhance overall performance for medical applications.