一种基于精细的情感和特征融合的视听相关性匹配方法
Zhibin Su1,2,3, Yiming Feng2,3, Jinyu Liu2,3
1State Key Laboratory of Media Convergence and Communication, Beijing 100024, China.
Sensors (Basel, Switzerland)
|September 14, 2024
概括
这项研究引入了一种新的混合模型,用于根据情感相似性匹配音乐和视频. 该模型提高了复杂的视听编辑任务的准确性和观众体验.
科学领域:
- 人工智能的人工智能
- 计算机视觉 计算机视觉
- 情感计算是一种情感计算.
背景情况:
- 现有的智能编辑工具使用基于情感一致性或特征相似性的交叉模式匹配.
- 这些方法与复杂的视听场景作斗争,导致准确性低,观众感知不佳.
研究的目的:
- 为艺术视听作品提出混合匹配模型,重点关注情感分布相似性和整合性.
- 提高音乐和视频编辑中视听匹配的准确性和主观体验.
主要方法:
- 开发了一种混合模型,结合了正规相关性分析 (CCA) 和细粒度的情感相似性.
- 精细的KCCA融合功能使用匹配和不匹配的音乐视频对.
- 使用XGBoost进行相关性预测,包括情感语义和因子距离.
主要成果:
- 拟议的模型在视听匹配中实现了高预测准确度.
- 实验结果显示,对视听协会有更好的主观体验.
- 该模型有效地平衡了特征参数和情感语义认知.
结论:
- 新型情感匹配模型为音乐视频检索和编辑提供了一种新的技术方法.
- 这项研究从感官角度探讨了视听情感关联机制.
- 这些发现有助于改进用于创建视听内容的智能工具.
相关概念视频
Association Areas of the Cortex
5.2K
Association areas are regions of the cerebral cortex that do not have a specific sensory or motor function. Instead, they integrate and interpret information from various sources to enable higher cognitive processes such as memory, learning, and decision-making. Some key association areas include the following:
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
5.2K
Perceiving Loudness, Pitch, and Location
202
The human brain perceives pitch through two primary mechanisms reflected in place theory and frequency theory. Each mechanism describes how sound waves are interpreted as specific pitches by the brain, offering insights into the intricate processes of auditory perception.
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
Place theory, or place coding, suggests that different pitches are heard because various sound waves activate specific locations along the cochlea's basilar membrane. The brain determines the pitch of a sound by...
202
Classification of Signals
420
In signal processing, signals are classified based on various characteristics: continuous-time versus discrete-time, periodic versus aperiodic, analog versus digital, and causal versus noncausal. Each category highlights distinct properties crucial for understanding and manipulating signals.
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
420


