From Sensing to Sense-Making: A Framework for On-Person Intelligence with Wearable Biosensors and Edge LLMs

Area of Science:

• Human-Computer Interaction
• Artificial Intelligence
• Biomedical Engineering

Background:

• Wearable biosensors generate vast physiological and behavioral data, often overwhelming users with raw information.
• Current systems typically rely on dashboards or simple alarms, requiring significant user cognitive effort for interpretation and decision-making in high-stakes fields.

Purpose of the Study:

• To propose on-person cognitive co-pilots that leverage multimodal sensing and local large language models (LLMs) for real-time decision support.
• To outline a technical framework and research agenda for developing trustworthy AI augmentation in isolated and extreme environments.

Main Methods:

• Integrating multimodal wearable sensing with edge AI accelerators and TinyML pipelines.
• Utilizing locally hosted LLMs for synthesizing sensor data with contextual information.
• Developing attention-appropriate cues for delivering recommendations while preserving user autonomy.

Main Results:

• Identified enabling conditions including mature wearable sensing, edge AI, TinyML, privacy-preserving learning, and deployable open-weight LLMs.
• Highlighted critical research gaps in sensor validity, uncertainty calibration, LLM reasoning verification, interaction design, and governance models.

Conclusions:

• Local, uncertainty-aware reasoning is essential for trustworthy, low-latency AI augmentation in demanding operational settings.
• A layered technical framework and research agenda are proposed to address the identified gaps in human-automation interaction for cognitive co-pilots.