StudyTypeTeller-Large语言模型自动分类研究研究类型进行系统审查
Simona Emilova Doneva1, Shirin de Viragh1, Hanna Hubarava1
1Center for Reproducible Science, https://ror.org/02crff812University of Zurich, Zurich, Switzerland.
Research synthesis methods
|February 2, 2026
概括
从变压器 (BERT) 模型中微调的双向编码器表示表现在系统审查的科学研究类型的分类中优于生成预训练变压器 (GPT) 模型. 自动分类有助于管理大量出版物.
科学领域:
- 生物医学信息学 生物医学信息学
- 自然语言处理自然语言处理.
- 系统审查方法论 系统审查方法论
背景情况:
- 系统审查面临挑战,因为出版量不断增加,手工选需要大量的劳动力.
- 像GPT这样的生成大型语言模型 (LLM) 显示了自动化科学研究类型分类的潜力.
- 在生物医学研究分类和与BERT模型的比较中,GPT模型的性能尚未得到充分证实.
研究的目的:
- 评估和比较GPT-3.5和GPT-4的性能与BERT模型对多类科学研究类型的分类.
- 在这项任务中,开发和使用人类注释的语料库来培训和评估NLP模型.
- 评估高级提示策略对GPT模型性能的影响.
主要方法:
- 创建了一个由人类注释的2645个PubMed标题和摘要集,涵盖了14种不同的研究类型.
- GPT-3.5,GPT-4,以及已建立的BERT模型都在这个集体上进行了微调和评估.
- 使用F1分数测量性能,比较模型在分类研究类型中的准确性.
主要成果:
- 微调的BERT模型始终优于GPT模型,F1得分高于0.8.
- 在GPT模型中,F1得分约为0.6,而高级提示策略的改进最小.
- 这项研究证明了BERT在这个特定的多类分类任务中的卓越表现.
结论:
- 较小,精心调整的BERT模型在生物医学文献中对科学研究类型进行分类时,比较大的GPT模型更有效.
- 尽管GPT目前的局限性,但自动分类方法对减少系统审查中的选工作量有希望.
- 开发的注释体是推动NLP在证据合成中的应用的宝贵资源.
更多相关视频
相关概念视频
Classifying Matter by Composition
90.4K
Matter: Pure Substances and Mixtures
According to its composition, the matter can be classified into two broad categories — pure substances and mixtures.
A pure substance is a form of matter that has a constant composition throughout with uniform properties. For example, any sample of sucrose has the same composition and same physical properties, such as melting point, color, and sweetness, regardless of the source from which it is isolated.
A mixture is composed of two or...
According to its composition, the matter can be classified into two broad categories — pure substances and mixtures.
A pure substance is a form of matter that has a constant composition throughout with uniform properties. For example, any sample of sucrose has the same composition and same physical properties, such as melting point, color, and sweetness, regardless of the source from which it is isolated.
A mixture is composed of two or...
90.4K
Classifying Matter by State
103.4K
Chemistry is the study of matter and the changes it undergoes. Matter is anything that has mass and occupies space. Matter is all around us; the air, water, soil, mountains, even our bodies are all examples of matter. Matter is divided into three states — solid, liquid, and gas — that are commonly found on earth. The fourth state of matter, plasma, occurs naturally in the interiors of stars.
103.4K
Automatic Processing and Automatic Social Behavior
253
Automatic processing refers to the cognitive operations that occur without conscious intent or awareness, playing a fundamental role in shaping social cognition and behavior. These processes enable individuals to navigate complex social environments efficiently by relying on mental shortcuts and pre-existing knowledge structures known as schemas. One of the most influential mechanisms underlying automatic processing is priming, which subtly activates mental representations through exposure to...
253
Language
913
Language is a unique communication system that uses words and systematic rules to organize and transmit information. Unlike other forms of communication, which may involve postures, movements, odors, or vocalizations, language relies on symbols and grammar. This makes human communication distinct from that of other species, who also communicate but do not use language in the same way humans do.
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
Corballis and Suddendorf (2007) and Tomasello and Rakoczy (2003) highlight the role of language in...
913
Review and Preview
8.4K
In statistics, several tools are used to interpret the data. Measures of central tendency represent the characteristics of the data, such as mean, median, and mode. Additionally, measures of variance like standard deviation and range are used to find the spread of data from the mean. Relative standing measures the distance between data locations. Commonly used measures of relative standings are percentile, z score, and quartiles.
Percentiles are a type of fractile that partition data into...
Percentiles are a type of fractile that partition data into...
8.4K
Review and Preview
11.3K
Data are individual items of information obtained from a population or sample. Data may be classified as qualitative (categorical), quantitative continuous, or quantitative discrete. Because it is not practical to measure the entire population in a study, researchers use samples to represent the population. A random sample is a representative group from the population chosen by using a method that gives each individual in the population an equal chance of being included in the sample. Random...
11.3K


