Performance and Biases of the LENA and ACLEW Algorithms in Analyzing Language Environments in Down, Fragile X,...

Wearable recorders are used in research and clinical practice to collect and measure children's vocalizations and the language environment in which they occur. Recordings generate vast amounts of audio, making manual analysis impractical and requiring automated processing. Two automated algorithms have emerged: the proprietary LENA (Language ENvironment Analysis) and the open-source ACLEW (Analyzing Child Language Experiences around the World) systems; yet, systematic performance comparisons remain scarce. Here, we validate and compare the performance of these two algorithms across key measures: audio segmentation into speaker categories, conversational turn count (CTC), adult word count (AWC), and child vocalization count (CVC). This analysis is based on 25 h of manually annotated audio recordings from 50 age-matched U.S. children with diverse neurodevelopmental profiles: children with Down syndrome, Fragile X syndrome, and Angelman syndrome, children at elevated likelihood of autism, and low-risk controls. We hypothesized that the algorithms might be less accurate for children with neurodevelopmental conditions, since these children often show different patterns of volubility and vocal maturity compared to the typically developing children used to train the algorithms. Thus, we assessed the performance of algorithms across diagnostic groups, a crucial validation step for both cross-population research and the evaluation of language interventions. Results reveal that while algorithms achieve similar performance across groups, they show different patterns: LENA makes fewer segmentation mistakes but misses many segments (identification error rate = 81.3%, percent correct = 45.3%), while ACLEW shows the opposite pattern (identification error rate = 129.4%, percent correct = 69.4%). Both LENA and ACLEW achieve reasonable levels of accuracy in their automatic counts (Pearson's r ranging from 0.78 to 0.92) and maintain stable performance across diagnostic groups. We conclude with recommendations for the validation and potential use of these algorithms in research and clinical practice. SUMMARY: Our comparison of the LENA and ACLEW algorithms in analyzing children's language environment and vocal production across five neurodevelopmental profiles reveals similar performance but distinct error patterns. LENA makes fewer errors but misses speech (81.3% error rate, 45.3% correct); ACLEW makes more errors but captures more speech (129.4% error rate, 69.4% correct). Both algorithms maintain consistent performance across diagnostic groups, supporting their reliability for research with these 2-year-old populations with diverse neurodevelopmental profiles. Variations in algorithm performance are primarily driven by the total speaking time of surrounding speakers (other children and adults), rather than the diagnostic group.