没有词汇的图像分类和语义细分
IEEE transactions on pattern analysis and machine intelligence
|January 28, 2026
概括
本研究介绍了无词汇图像分类 (VIC) 并建议从外部数据库 (CaSED) 进行类别搜索. 在没有预定义的词汇库的情况下,CaSED可以进行图像分类和细分,优于现有的模型.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 大视觉语言模型 (VLMs) 在图像分类和细分方面表现出色,但需要预定义的类别词汇.
- 这种限制对于现实世界中未知或不断演变的语义上下文的场景是不切实际的.
- 现有的方法难以应对无限制的语义空间的广,包括细粒度的类别.
研究的目的:
- 引入无词汇图像分类 (VIC) 任务,使分类能够进入不受约束的语义空间.
- 建议从外部数据库 (CaSED) 进行类别搜索,这是VIC的无培训方法.
- 扩展CaSED用于无词汇的语义细分.
主要方法:
- CaSED利用预先训练有素的VLM和外部数据库,从语义上类似的标题中识别候选类别.
- 图像通过将其匹配到使用VLM的最佳候选类别来进行分类.
- 通过对图像区域进行分类,CaSED在本地应用用于无词汇的语义细分.
主要成果:
- 在没有预定义的词汇库的情况下,CaSED有效地执行图像分类.
- 在分类和语义细分基准上,CaSED变体实现了最先进的性能.
- 提出的方法在使用较少参数的同时显著优于更复杂的VLM.
结论:
- 没有词汇的图像分类和细分是可行的和有效的.
- CaSED提供了一种强大且参数效率高的解决方案,用于不受约束的语义理解.
- 这项工作为在动态和开放式环境中应用VLM开辟了新的途径.
相关概念视频
Force Classification
2.4K
Forces play a crucial role in the study of physics and engineering. They are essential in describing the motion, behavior, and equilibrium of objects in the physical world. Forces can be classified based on their origin, type, and direction of action.
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
Contact and non-contact forces are two of the most widely used categories of forces. As the name suggests, contact forces require physical contact between two objects to act upon each other. Examples of contact forces include frictional,...
2.4K
Classification of Neurotransmitters
5.3K
Neurotransmitters play a crucial role in the communication between neurons in the autonomic nervous system. Neurons in the autonomic nervous system can be cholinergic or adrenergic depending on the neurotransmitters synthesized. Cholinergic neurons use acetylcholine as their primary neurotransmitter. This includes all the preganglionic fibers of the sympathetic and pre- and postganglionic fibers of the parasympathetic nervous systems. In addition, neurons of the somatic nervous system also use...
5.3K
Classification of Leukocytes
5.7K
Leukocytes are classified into two groups based on the presence or absence of cytoplasmic granules. Granular leukocytes, which contain granules, belong to the myeloid lineage and are divided into three subtypes: neutrophils, eosinophils, and basophils. These cells are roughly spherical and characterized by the granules in their cytoplasm.
Neutrophils are the most abundant type of granular leukocytes, comprising 50-70% of all leukocytes. They feature small, evenly distributed granules and a...
Neutrophils are the most abundant type of granular leukocytes, comprising 50-70% of all leukocytes. They feature small, evenly distributed granules and a...
5.7K
Classification of Illness
8.7K
The meaning of illness is individualized to each person who experiences an alteration in health. In contrast, disease is a medical term indicating a pathological change in the structure and function of the body or mind. It is a condition that has specific symptoms and boundaries.
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
An illness is a response to a disease in which the person's level of functioning is changed compared with a previous level. The general classification of illness includes acute and chronic.
Acute illness is severe...
8.7K
Classification of Bones
9.9K
The bones of the human skeletal system are of varied shapes, sizes, and functions. They can be classified based on their shape and function into four major classes: long bones, short bones, flat bones, and irregular bones. Some classifications include a fifth type, the sesamoid bones, as a separate class, whereas others categorize them under short bones.
Long and Short Bones
The appendicular skeleton, particularly the upper and lower limbs, is primarily made of long and short bones. The...
Long and Short Bones
The appendicular skeleton, particularly the upper and lower limbs, is primarily made of long and short bones. The...
9.9K
Seizures: Classification
1.6K
Epilepsy is primarily characterized by unpredictable seizures, either provoked by an identifiable factor, such as injury or illness, or unprovoked, occurring spontaneously without apparent cause.
Seizures are typically classified into two main categories: focal and generalized seizures.
Focal Seizures
Focal seizures originate from specific regions of the brain. These seizures are further sub-classified into two types:
Seizures are typically classified into two main categories: focal and generalized seizures.
Focal Seizures
Focal seizures originate from specific regions of the brain. These seizures are further sub-classified into two types:
1.6K


