Evaluating automatic creaky voice detection methods

There is growing interest in research on the non-modal voice quality, creaky voice; however, its identification often relies on time-consuming manual annotation, leading to a recent focus on automatic creak detection methods. Various automatic methods have been proposed, which rely on varying types and combinations of acoustic cues for creak detection. In this paper, we compare the performance of three automatic tools, the AntiMode method, the Creak Detector algorithm, and the Roughness algorithm, against manual annotation of creak using data from 80 Australian English speakers. We explore the possibility that tools used in combination may yield more accurate creak detection than individual tools used alone. Based on method comparisons, we present options for researchers, including an "out-of-the-box" approach, which supports combining automatic tools, and propose additional steps to further improve creak detection. We found restricting analysis to sonorant segments significantly improves automatic creak detection, and tools performed consistently better on female speech than male speech. Findings support previous work showing detection may be optimised by performing a creak probability threshold sweep on a subset of data prior to applying the Creak Detector algorithm on new datasets. Results provide promising solutions for advancing efficient large-scale research on creaky voice.