A methodology for the capture and analysis of hybrid data: a case study of program debugging

Area of Science:

• Cognitive Science
• Human-Computer Interaction
• Computer Science Education

Background:

• Analyzing complex user interactions requires integrating diverse data types.
• Understanding cognitive processes in computer programming is challenging with traditional methods.

Purpose of the Study:

• To introduce a novel methodology for capturing and analyzing hybrid data.
• To demonstrate the application of this methodology in computer programming.
• To provide insights into user reasoning through multimodal data analysis.

Main Methods:

• Developed a software tool to capture visual attention focus by selectively blurring screen stimuli.
• Collected hybrid data including computer interaction logs, audio recordings, and visual attention data.
• Employed a mixed-methods approach combining qualitative and quantitative analyses.

Main Results:

• Successfully captured and analyzed hybrid data from a computer programming task.
• Demonstrated the feasibility of integrating computer interaction, audio, and visual attention data.
• Identified patterns in user behavior and cognitive processes.

Conclusions:

• The proposed methodology offers a robust framework for multimodal data analysis in HCI and cognitive science.
• This approach provides deeper insights into user reasoning and learning in complex domains.
• Further research should explore limitations and expand applications of the hybrid data analysis technique.