一种用于长文本语义文本相似性的方法
Omar Zatarain1, Juan Carlos González-Castolo2, Silvia Ramos-Cabral1
1Department of Computer Science and Engineering, University of Guadalajara, Ameca, Jalisco, Mexico.
PeerJ. Computer science
|September 24, 2025
概括
本研究提出了一种新的方法,用于使用句子转换器和大型语言模型评估长文件中的语义相似性. 较小的句子转换器为分析长文提供了经济有效的替代方案.
科学领域:
- 自然语言处理自然语言处理.
- 人工智能的人工智能
- 信息检索 信息检索
背景情况:
- 在长文件中评估语义相似性在计算上具有挑战性.
- 现有的方法往往需要大量的数据集或与文本截断作斗争.
研究的目的:
- 引入一种用于在长文件中检测语义相似性的新方法.
- 利用句子转换器和大型语言模型进行有效的信息检索.
- 为选择性代检索开发一种分析模糊策略.
主要方法:
- 文本预处理包括将文档分成句子.
- 使用预先训练的句子转换器和任何符号容量的大型语言模型 (LLM).
- 采用分析模糊策略与参数调整,将相似性分为四个类别:相同,相同的主题,概念相关和非相关.
- 通过处理一对句子来避免文本的截断.
主要成果:
- 该方法有效地检测长文件中的语义相似性,而无需事先培训.
- 较小的句子转换器证明了长文本分析的可靠性和成本效益.
- 模糊策略允许选择性检索,即使在噪音条件下.
结论:
- 拟议的方法为长文档相似性提供了大型语言模型的经济替代方案.
- 句子转换器是广泛文本语义相似性分析的可行工具.
- 该方法促进了准确的信息提取和文档比较.
相关概念视频
Causes of Similarity-Dissimilarity Effect
254
The similarity-dissimilarity effect, a fundamental concept in social psychology, explains how interpersonal similarities and differences influence attraction and social interactions. This effect is supported by three key psychological perspectives: balance theory, social comparison theory, and consensual validation.Balance Theory and Cognitive ConsistencyBalance theory, developed by Fritz Heider, posits that individuals seek cognitive consistency in their relationships. When two people share...
254
Improving Translational Accuracy
3.5K
3.5K
Improving Translational Accuracy
14.1K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
14.1K
Longitudinal Research
13.1K
Sometimes we want to see how people change over time, as in studies of human development and lifespan. When we test the same group of individuals repeatedly over an extended period of time, we are conducting longitudinal research. Longitudinal research is a research design in which data-gathering is administered repeatedly over an extended period of time. For example, we may survey a group of individuals about their dietary habits at age 20, retest them a decade later at age 30, and then again...
13.1K
Comparing Copy Number Variations and SNPs
18.6K
Sequencing of the human genome has opened up several best-kept secrets of the genome. Scientists have identified thousands of genome variations that exist within a population. These variations can be a single nucleotide or a larger chromosomal variation.
Copy number variations or CNVs are the structural variations that cover more than 1kb of DNA sequence. The single nucleotide polymorphism (SNP), on the other hand, is a single nucleotide change or a point mutation that is found in more than 1%...
Copy number variations or CNVs are the structural variations that cover more than 1kb of DNA sequence. The single nucleotide polymorphism (SNP), on the other hand, is a single nucleotide change or a point mutation that is found in more than 1%...
18.6K
Multi-species Conserved Sequences
4.6K
Next-generation sequencing technologies have created large genomic databases of a variety of animals and plants. Ever since the human genome project was completed, scientists studied the genome of primates, mammals, and other phylogenetically distant living beings. Such large-scale studies have provided new insights into the evolutionary relationship between organisms.
Although the genome of each species varies greatly from each other, a few sequences are highly conserved. Such conserved...
Although the genome of each species varies greatly from each other, a few sequences are highly conserved. Such conserved...
4.6K
