Automated integration of spatial location and eye gaze tracking: An extensible multimodal system for developmental...

Advances in computer vision and mobile sensing enable the study of development in naturalistic environments, yet existing motion tracking systems lack the integration of automated analysis and accessibility needed in developmental research. Current approaches force trade-offs between precision and ecological validity, requiring structured protocols that constrain spontaneous interactions or manual coding that cannot scale for large-scale studies of individual differences. We present an integrated system combining high-density data collection with machine learning-supported analysis using off-the-shelf components. The architecture integrates ceiling-mounted cameras, Pupil Labs eye trackers, and recording hardware, synchronized via Lab Streaming Layer (LSL) protocol with millisecond precision. The modular architecture supports deployment in freely designed environments and can be extended with additional LSL-compatible sensors. Ceiling-mounted cameras operate unobtrusively without requiring wearables, addressing key feasibility challenges in infant and toddler research. The automated pipeline employs off-the-shelf computer vision models for person detection and tracking, with vision-language models distinguishing children from caregivers without manual annotation. The system integrates spatial positioning with eye tracking data, and prototype-based object detection identifies researcher-specific stimuli through reference photographs rather than custom model training. We demonstrate deployment feasibility with toddler-caregiver dyads (from 24 months of age) in free play and validate system accuracy across an extended age range (24 to 63 months). Children tolerated the equipment well, and the pipeline extracted high-density behavioral metrics without manual coding. This system enables scalable measurement of unconstrained early behavior, addressing a methodological gap constraining developmental research.