Screen Detection from Egocentric Image Streams Leveraging Multi-View Vision Language Model

Area of Science:

• Child development
• Human-computer interaction
• Wearable technology

Background:

• Accurate monitoring of children's screen time is crucial for research on health and social behaviors.
• Current methods like self-reports and bulky sensors lack accuracy and efficiency.
• Existing technologies struggle to capture quantitative screen exposure data effectively.

Purpose of the Study:

• To develop an efficient and accurate framework for monitoring children's screen exposure.
• To introduce a novel approach combining wearable sensors and advanced AI for screen time tracking.
• To improve the methodologies used in studying screen use and its impact on child development.

Main Methods:

• Developed a novel screen detection framework using egocentric images from a wearable sensor called the screen time tracker (STT).
• Utilized a multi-view vision language model (VLM) to dynamically interpret screen exposure from multiple image streams.
• Validated the framework using a dataset of children's free-living activities.

Main Results:

• The proposed multi-view VLM framework demonstrated significant improvements over conventional vision language models and object detection models.
• The system achieved higher accuracy and efficiency in capturing quantitative screen exposure data.
• The lightweight hardware design of the STT combined with the VLM offers a practical solution.

Conclusions:

• The novel screen detection framework provides an accurate and efficient method for monitoring children's screen time.
• This technology has significant potential to advance research in child behavior, screen use, and related health outcomes.
• The combination of wearable sensors and VLM offers a promising direction for future child behavior studies.