MURMR: AMultimodal Sensing Pipeline for Automated Group Behavior Analysis in Mixed Reality

When teams collaborate in immersive environments, coordination breakdowns can go undetected without automated analysis, and this has direct consequences for task performance. Yet existing methods rely on external observation and manual annotation, and none offers an annotation-free approach for analyzing temporal collaboration dynamics from headset-native data. We introduce MURMR, a passive sensing pipeline that captures and analyzes multimodal interaction data from commodity MR headsets without external instrumentation. Two complementary modules address different levels of analysis: a structural module that generates automated multimodal sociograms and network metrics at both session and intra-session granularities, and a temporal module that applies unsupervised deep clustering to identify moment-to-moment dyadic behavioral patterns without predefined taxonomies. An exploratory deployment with 48 participants in a co-located object-sorting task reveals that intra-session structural analysis captures significant within-session variability that can be lost in session-level aggregation, with gaze, audio, and position modalities each contributing non-redundantly. The temporal module identifies five emergent behavioral patterns. Against egocentric video, a single coder matched 83% of a 100-window sample drawn from the 5,334 total windows. Cross-tabulation shows that behavioral transitions consistently occur within structurally stable states, demonstrating that the two modules capture complementary dynamics. These results establish that passive headset sensing provides meaningful signal for automated, multi-level team collaboration analysis in immersive environments.