The Impact of Data Dependence on Speaker Recognition Evaluation.

Area of Science:

• Speech processing
• Statistical modeling
• Machine learning evaluation

Background:

• Speaker recognition systems rely on the detection cost function (DCF) for performance evaluation.
• Data dependency, arising from repeated subject usage, complicates accurate standard error (SE) estimation for the DCF.
• Existing bootstrap methods may not adequately account for this data dependency.

Purpose of the Study:

• To investigate the impact of data dependency on the SE of the DCF in speaker recognition.
• To compare the effectiveness of different bootstrap methods (i.i.d., one-layer, two-layer) in handling data dependency.
• To analyze the influence of data dependency on both the magnitude and variability of SE.

Main Methods:

• A two-layer data structure was implemented to group dependent target and non-target scores.
• Nonparametric one-layer and two-layer bootstrap methods were applied, alongside the standard i.i.d. bootstrap.
• Hypothesis testing was used to compare the SE distributions estimated by the different bootstrap methods.

Main Results:

• Data dependency was found to increase both the SE and its variation.
• The two-layer bootstrap method provided more conservative SE estimates than the one-layer bootstrap.
• The study explored the underlying reasons for the differing impacts of the bootstrap methods.

Conclusions:

• Data dependency is a critical factor affecting the reliability of DCF-based speaker recognition evaluation.
• The two-layer bootstrap method is recommended for more robust SE estimation in the presence of data dependency.
• Further investigation into the rationale behind bootstrap method performance is warranted.