EgoTrigger: Toward Audio-Driven Image Capture for Human Memory Enhancement in All-Day Energy-Efficient Smart Glasses

Area of Science:

• Human-Computer Interaction
• Artificial Intelligence
• Wearable Technology

Background:

• All-day smart glasses promise continuous contextual sensing for daily assistance.
• Integrating AI for memory enhancement faces significant energy efficiency challenges for continuous sensing.

Purpose of the Study:

• To develop an energy-efficient sensor management strategy for all-day smart glasses.
• To enable human memory enhancement applications through intelligent, context-aware sensing.

Main Methods:

• Introduced EgoTrigger, a system leveraging audio cues to selectively activate power-intensive cameras.
• Utilized a lightweight audio model (YAMNet) and custom classification head for hand-object interaction (HOI) audio cue detection.
• Evaluated performance on the QA-Ego4D and a newly introduced Human Memory Enhancement Question-Answer (HME-QA) dataset.

Main Results:

• EgoTrigger reduced frame capture by 54% on average, saving energy in sensing and transmission.
• Achieved comparable performance to continuous sensing for episodic memory tasks.
• Demonstrated effective triggering of image capture from specific HOI audio cues.

Conclusions:

• Context-aware triggering is a promising strategy for energy-efficient smart glasses.
• Enables functional, all-day smart glasses for memory recall and routine activity support.
• Addresses the critical energy efficiency challenge for continuous sensing in wearable AI.