Explaining cultural emotion in Chinese pop music with multimodal AI: educational and socio-emotional implications

This study investigates the evolutionary patterns of emotions in Chinese pop songs and examines how mainstream emotional characteristics align with shifting socio-cultural values, highlighting the implications for music education and culturally responsive pedagogy.

We construct a corpus of Chinese pop songs from 1980 onward. Building on an improved Thayer emotion model, we introduce external affective resources to semantically enhance labels and apply semantic mapping to optimize manual annotations. We propose a multimodal emotion recognition model (M2ER) that integrates lyric and audio features to achieve fine-grained, segment-level annotation. Emotional evolution across periods is analyzed via distributional profiles, intensity dynamics, and association patterns.

The label optimization procedure improves the discriminability and representativeness of emotion categories. M2ER attains recognition accuracies of 84.49% (complete songs) and 78.65% (segments). We identify a historical trajectory from "lively and passionate" to "lyrical and gentle," and subsequently to "diverse and delicate," closely linked to broader socio-cultural transformations. Educational implications. By explaining which lyrical and acoustic cues underpin specific affective states and how these cues shift across eras, our findings offer actionable evidence for curriculum design (e.g., periodized listening modules), language-through-lyrics activities, and socio-emotional learning (SEL) in music classrooms. The corpus and interpretable analyses support culturally responsive teaching by situating repertoire choices within community histories and value change, and by scaffolding discussions of identity, emotion regulation, and aesthetic preference among diverse learners.

Multimodal machine learning provides a new lens to understand public affect as expressed in popular music and to translate these insights into educational practice. The study deepens the cultural interpretation of emotion in music while informing school-based and community music education that aims to foster emotional well-being and inclusive engagement.