卡图像标题基于语义概念提示符和多模式功能优化.
Wenjin Hu1, Lang Qiao2, Wendong Kang2
1School of Mathematics and Computer Science, Northwest Minzu Univsersity, Lanzhou 730030, China.
Journal of imaging
|August 25, 2023
概括
这项研究介绍了SCAMF-Net,以改善各种Thangka艺术品的中文图片标题. 新的语义概念提示 (SCP) 和多式特征优化 (MFO) 模块显著提高了标题的准确性和丰富性.
科学领域:
- 人工智能的人工智能
- 计算机视觉 计算机视觉
- 数字人文学科 数字人文学科
背景情况:
- 卡图像具有丰富的多样性,这给自动标题带来了挑战.
- 现有的深度学习模型产生了对Thangka艺术的不准确和简单的中文标题.
研究的目的:
- 为Thangka图像开发一个先进的图像标题网络.
- 为了提高thangka艺术的中文标题的准确性和语义丰富性.
主要方法:
- 提出了语义概念提示符和多模式特征优化网络 (SCAMF-Net).
- 引入了一个语义概念提示符 (SCP) 模块,用于更丰富的文本编码.
- 开发了一种多模式特征优化 (MFO) 模块,以改善图像-文本特征相关性.
主要成果:
- 在多个指标 (BLEU-4,METEOR,ROUGE,CIDEr,SPICE) 上,SCAMF-Net 在Thangka数据集上的基线模型表现明显优于基线模型.
- 取得了实质性的改进,特别是CIDEr得分增加了76.6%.
- 在MSCOCO数据集上,与最先进的方法相比,表现出优越的性能.
结论:
- SCAMF-Net有效地解决了当前用于Thangka图像标题的方法的局限性.
- 拟议的SCP和MFO模块增强语义理解和视觉文本相关性.
- 这种方法为为文化丰富的图像生成高质量,描述性的中文标题提供了有希望的解决方案.
相关概念视频
Natural and Artificial Concepts
191
In psychology, concepts can be divided into two categories: natural and artificial. Natural concepts are formed through direct or indirect experiences. For example, consider the concept of snow. If you live in a place with regular snowfall, such as Essex Junction, Vermont, you know snow through direct experiences. You’ve seen it fall, touched it, shoveled it, and played in it. You recognize its texture, appearance, and even its smell. In contrast, if you live on an island like Saint...
191
Encoding
202
Information enters the brain through encoding, which is the input of information into the memory system. Once sensory information is received from the environment, the brain labels or codes it. The information is then organized with similar information and connected to existing concepts. Encoding occurs through automatic processing and effortful processing.
Automatic processing involves the encoding of details like time, space, frequency, and the meaning of words, usually done without conscious...
Automatic processing involves the encoding of details like time, space, frequency, and the meaning of words, usually done without conscious...
202
Stereotype Content Model
14.8K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.8K
Nonconscious Mimicry
4.6K
Nonconscious mimicry occurs when individuals alter their mannerisms to match the behaviors and expressions of those nearby, without intention.
4.6K
Normal and Tangetial Components: Problem Solving
202
Consider a man with a mass of 70 kg seated in a chair connected to a pin support through a member BC. If the man maintains an upright position, the task is to determine the horizontal and vertical reactions of the chair on the man when the member makes a 45° angle with the horizontal. At this moment, the man has a speed of 5 m/s, increasing at a rate of 1 m/s².
202
Upsampling
262
Managing signal sampling rates is essential in digital signal processing to maintain signal integrity. A decimated signal, characterized by a reduced frequency range due to its lower sampling rate, can be upsampled by inserting zeros between each sample. This upsampling process expands the original spectrum and introduces repeated spectral replicas at intervals dictated by the new Nyquist frequency. To refine this zero-inserted sequence, it is passed through a lowpass filter with a cutoff...
262


