实体和关系提取的硬提示的LLM中的一致性-准确性相关性:从植物健康数据的经验发现
Xinzhi Yao1,2, Claire Nédellec3, Jingbo Xia2
1National Research Institute for Agriculture, Food and Environment, University Paris-Saclay, MaIAGE, Jouy-en-Josas, France.
Genomics & informatics
|February 11, 2026
概括
大型语言模型 (LLM) 在信息提取 (IE) 中显示了不同的准确性和一致性. 一致性,测量输出稳定性,是关键可靠性指标,特别是在考虑语义意义而不是表面变化时.
科学领域:
- 自然语言处理自然语言处理.
- 人工智能的人工智能
- 提取信息 提取信息
背景情况:
- 大型语言模型 (LLM) 越来越多地用于信息提取 (IE).
- 对LLM输出的稳定性和可靠性存在担忧.
- 一致性是一种拟议的指标,用于补充评估可靠性的准确性.
研究的目的:
- 检查实体和关系提取生成LLM的准确性和一致性之间的关系.
- 在语义层面评估一致性,区分可恢复和关键输出变化.
- 评估不同任务复杂度的准确性-一致性相关性的模型依赖变化.
主要方法:
- 在EPOP植物健康库中对四个LLM (GPT,DeepSeek,Qwen,Kimi) 的系统评估.
- 硬提示生成LLM的应用用于实体和关系提取.
- 区分可回收和关键输出变化以改进一致性解释.
主要成果:
- 观察到准确性和一致性之间的模型依赖的正相关性.
- 相关性因信息提取任务的复杂性而异.
- 语义层面的一致性测量对于结构化预测任务至关重要,不考虑表面的变化.
结论:
- 一致性是一个有价值的,虽然并不总是完全相关的,在IE中LLM可靠性的指标.
- 测量语义层面的一致性对于准确的评估至关重要.
- 这些发现有助于使用自我一致性作为信心过器,以及在专业领域设计可靠的生成IE管道.
更多相关视频
相关概念视频
Correlation of Experimental Data
491
Dimensional analysis simplifies complex physical problems and guides experimental investigations, but it does not provide complete solutions. It identifies the dimensionless groups that influence a phenomenon, but experimental data is needed to establish the specific relationships and validate theoretical predictions.
For example, a spherical particle moving through a viscous fluid experiences drag. Dimensional analysis shows that the drag force depends on the particle's diameter, velocity,...
For example, a spherical particle moving through a viscous fluid experiences drag. Dimensional analysis shows that the drag force depends on the particle's diameter, velocity,...
491
Correlations
36.6K
Correlation means that there is a relationship between two or more variables (such as ice cream consumption and crime), but this relationship does not necessarily imply cause and effect. When two variables are correlated, it simply means that as one variable changes, so does the other. We can measure correlation by calculating a statistic known as a correlation coefficient. A correlation coefficient is a number from -1 to +1 that indicates the strength and direction of the relationship between...
36.6K
Plant Tissue Culture
40.8K
Plant tissue culture is widely used in both primary and applied science. Applications range from plant development studies to functional gene studies, crop improvement, commercial micropropagation, virus elimination, and conservation of rare species.
40.8K
Uncertainty in Measurement: Accuracy and Precision
104.7K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.
104.7K
Improving Translational Accuracy
15.0K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
15.0K
Correlation and Causation
42.9K
Statistical tests can calculate whether there is a relationship, or correlation, between independent and dependent variables. An indirect relationship of the variables signifies a correlation, while a direct relationship shows causation. If it is determined that no connection exists between the variables, then the correlation is a coincidence.
Correlation versus Causation
If the dependent variable increases or decreases when the independent variable increases, there is a positive or negative...
Correlation versus Causation
If the dependent variable increases or decreases when the independent variable increases, there is a positive or negative...
42.9K


