The conceptual similarity of intonational tones and its effects on intertranscriber reliability

Tests of intertranscriber agreement in prosodically-labeled corpora have been used as an objective performance measure of reliability. Reasonably high agreements among labelers have been found, but systematic disagreements exist, indicating that some intonational patterns are more difficult for transcribers to label while others are easier. This may be due to differences in the way transcribers distinguish between tonal labels for pitch events. We developed a method to map the subjective similarity space for the categories in an intonational transcription system. From this map, we derived a conceptual tone similarity similarity index indicating the distance between tone categories. This subjective similarity index is used to predict the intertranscriber reliability. It is found that tones which are conceptually similar result in higher transcriptional disagreements while tones which are conceptually dissimilar result in lower disagreement.