文本注释的开源LLM:模型设置和微调的实用指南
Meysam Alizadeh1, Maël Kubli1, Zeynab Samei2
1Department of Political Science, University of Zurich, 8050 Zurich, Switzerland.
概括
微调的开源大型语言模型 (LLM) 显著提高了政治学文本分类任务的性能,优于零射击模型,并为少数射击培训提供了切实可行的替代方案.
科学领域:
- 政治科学 政治科学是指政治学.
- 计算社会科学 计算社会科学
- 自然语言处理自然语言处理.
背景情况:
- 开源大型语言模型 (LLM) 越来越多地用于文本分析.
- 学者需要指导LLM表现特定的政治学任务.
- 在社会科学研究中建立LLM有效性的基准至关重要.
研究的目的:
- 评估开源LLMs在政治学文本分类中的表现.
- 为了比较零射击和微调的LLM能力在任务,如立场,主题和相关性.
- 为LLM有效性提供一个基准,并为学术决策提供信息.
主要方法:
- 使用零射击和微调方法评估开源LLM.
- 使用新闻文章和推特数据集进行文本注释任务.
- 与有限的注释数据相比,微调与少数射击训练进行比较.
主要成果:
- 微调增强了开源LLM性能,与零射击GPT-3.5和GPT-4相匹配或超过.
- 微调的开源LLM仍然落后于微调的GPT-3.5.5.
- 微调更有效,而不是使用适度数据进行几次射击训练.
结论:
- 精心调整的开源LLM适用于政治学中的各种文本注释应用.
- 该研究为社会科学研究中的LLM绩效提供了实际的基准.
- 有一个Python笔记本可用于帮助研究人员应用LLM进行文本注释.
相关概念视频
Light Acquisition
In order to produce glucose, plants need to capture sufficient light energy. Many modern plants have evolved leaves specialized for light acquisition. Leaves can be only millimeters in width or tens of meters wide, depending on the environment. Due to competition for sunlight, evolution has driven the evolution of increasingly larger leaves and taller plants, to avoid shading by their neighbors with contaminant elaboration of root architecture and mechanisms to transport water and nutrients.
Improving Translational Accuracy
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
Improving Translational Accuracy
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...


