Related Experiment Video
Updated: Jan 10, 2026
Author Spotlight: Capturing Infant-Caregiver Interactions Through Synchronized Multimodal Data Collection
Published on: May 31, 2024
Multimodal journaling as engagement tool for parents of premature babies
1Department of Information & Communications Engineering, Aalto University, Espoo, Finland.
Insights
Parents using a new journaling tool for premature babies in the Neonatal Intensive Care Unit (NICU) preferred speech over combined speech and gesture. Further design is needed for effective multimodal engagement.
Area of Science:
- Digital Health
- Human-Computer Interaction
- Parental Support
Background:
- Premature birth presents significant psychological challenges for parents.
- Parental engagement tools can reduce stress and improve understanding of infant care.
- Multimodal interaction holds potential for enhanced digital health tools.
Purpose of the Study:
- To test the feasibility and applicability of a multimodal journaling prototype for parents of premature infants.
- To evaluate user interaction methods (technical performance and usability) of the prototype.
- To identify optimal interaction designs for parental engagement in the Neonatal Intensive Care Unit (NICU).
Main Methods:
- Development of a multimodal journaling prototype integrating speech and gesture input.
- Technical performance evaluation of the multimodal integration module.
- Usability evaluation of user interaction patterns through observation.
Main Results:
- The prototype demonstrated high technical performance with over 95% accuracy.
- Usability testing revealed a preference for speech-only interaction over combined speech and gesture.
- Observed interaction patterns significantly differed from the intended multimodal use.
Conclusions:
- While technically sound, the current multimodal journaling prototype requires design refinement for better usability.
- Future research should focus on aligning interaction design with parental experiences for natural multimodal usage.
- Optimizing multimodal approaches is crucial for effective parental and patient engagement in digital health.
Abstract:
Parents of premature babies face extreme psychological challenges when a child is born too early - a situation for which no parents can be prepared. Parental engagement solutions, such as logging the baby's progress, can alleviate this stress by fostering a comforting feeling of involvement in the care and a clearer understanding of the overall situation. This paper presents a multimodal journaling prototype designed specifically for parents of premature babies in the Neonatal Intensive Care Unit (NICU). The general goal was to test its feasibility and applicability as an engagement tool, while the specific goal was to evaluate user interaction methods from both technical performance and usability perspectives. The implementation, built around a multimodal integration module, demonstrated high technical performance, yielding greater than 95% accuracy. However, usability evaluation revealed a significant finding: observed interaction patterns showed a substantially larger proportion of speech-only interaction compared to the intended multimodal combination of speech and gesture. To fully realize the benefits of a multimodal approach for parental engagement, and more broadly for patient engagement in digital health, further research is imperative. This work must focus on refining the interaction design to better align with the lived parental experience and seamlessly support natural multimodal usage.
More Related Videos
11:50Clinical Practice Protocol of Creative Music Therapy for Preterm Infants and Their Parents in the Neonatal Intensive Care Unit
Published on: January 7, 2020
11:14A Novel Experimental and Analytical Approach to the Multimodal Neural Decoding of Intent During Social Interaction in Freely-behaving Human Infants
Published on: October 4, 2015
