A comparison of automatic and manual measures of turn-taking in monolingual and bilingual contexts

The Language ENvironment Analysis system (LENA) records children's language environment and provides an automatic estimate of adult-child conversational turn count (CTC) by automatically identifying adult and child speech in close temporal proximity. To assess the reliability of this measure, we examine correlation and agreement between LENA's CTC estimates and manual measurement of adult-child turn-taking in two corpora collected in the USA: a bilingual corpus of Spanish-English-speaking families with infants between 4 and 22 months (n = 37), and a corpus of monolingual families with English-speaking 5-year-olds (n = 56). In each corpus for each child, 100 30-second segments were extracted from daylong recordings in two ways, yielding a total of 9300 minutes of manually annotated audio. LENA's CTC estimate for the same segments was obtained through the LENA software. The two measures of CTC had low correlations for the segments from the monolingual 5-year-olds sampled in both ways, and somewhat higher correlations for the bilingual samples. LENA substantially overestimated CTC on average, relative to manual measurement, for three out of four analysis conditions, and limits of agreement were wide in all cases. Segment-level analyses demonstrated that accidental contiguity had the largest individual impact on LENA's average CTC error, affecting 12-17% of analyzed segments. Other factors significantly contributing to CTC error were speech from other children, presence of multiple adults, and presence of electronic media. These results indicate wide discrepancies between LENA's CTC estimates and manual CTCs, and call into question the comparability of LENA's CTC measure across participants, conditions, and developmental time points.