Video summarization for home-based egocentric footage in spinal cord injury rehabilitation

Wearable cameras provide a means to assess hand function in individuals with spinal cord injury (SCI) beyond clinical settings. Previous studies have found that clinicians acknowledge the potential of egocentric video to monitor and inform rehabilitation. Nonetheless, the need for time-intensive manual review of the footage remains a challenge to its integration into clinical practice. To address this barrier, we investigated the utility of video summarization for egocentric videos of hand use after SCI.

A dataset comprising 316 egocentric videos from 20 individuals with cervical SCI was used. Individuals wore head-mounted cameras to record daily activities in their home. Three unsupervised video summarization algorithms were applied: DR-DSN (reinforcement learning), CTVSUM (contrastive learning), and CA-SUM (attention-based learning). The resulting summaries were manually evaluated on a subset of five videos (each summarized by all three algorithms) by 15 participants using five criteria rated on a 5-point Likert scale: (C1) inclusion of hand movements, (C2) visibility of difficulties and compensation, (C3) contextual clarity, (C4) depiction of hand function, and (C5) preservation of key information. Additionally, summaries were assessed using computational metrics: coverage, temporal distribution, diversity, and representativeness.

An average manual rating of 3.7 ± 1.2 was observed. Ratings differed significantly across both evaluation criteria (F = 13.69, p < 0.001, ${\eta }^2$  = 0.167) and algorithms (F = 24.00, p < 0.001, ${\eta }^2$  = 0.103). In particular, summaries were rated higher for C3 and lower for C2, while CA-SUM consistently received the highest scores. Among the computational metrics, diversity showed a strong negative association with manual ratings (b = -4.8, p = 0.032, R² = 0.827), while representativeness was positively associated (b = 17.8, p = 0.047, R² = 0.779).

All three algorithms produced adequate video summaries that captured essential content. However, enhancing the depiction of aspects such as functional difficulties and compensatory strategies could further improve the clinical value of the summaries. Moreover, discrepancies between computational and manual evaluations highlight the need to train algorithms on more human-centered criteria. Overall, this work demonstrates the potential of automatic video summarization to support the integration of wearable cameras into outpatient SCI rehabilitation.