Use of the magnitude estimation technique for assessing the performance of text-to-speech synthesis systems

As text-to-speech systems develop, it becomes necessary to compare various solutions and to evaluate whether a change in the synthesis procedure has an effect on the listener's attitude to the system. The possibility of directly scaling intelligibility, naturalness, and user's satisfaction (i.e., acceptability) with the magnitude estimation technique is investigated. A magnitude estimation protocol suitable for this purpose is described. In general, within the limits of the methodological constraints discussed in this paper, the procedure appears to be reliable and valid for quantifying the perceived attributes of synthesized speech.