作为法官的LLM:使用大型语言模型对搜索查询解析的自动评估
Mehmet Selman Baysan1, Serkan Uysal2, İrem İşlek1
1sahibinden.com, Istanbul, Türkiye.
Frontiers in big data
|August 5, 2025
概括
大型语言模型 (LLM) 提供了一种评估搜索查询解析的新方法. 基于LLM的评估显示了与人类判断的高度一致,对现实世界的系统有效.
科学领域:
- 自然语言处理自然语言处理.
- 信息检索 信息检索
- 机器学习评估 机器学习评估
背景情况:
- 对于大型语言模型 (LLM),搜索系统的传统评估方法是不够的.
- 需要新的方法来评估LLM在搜索应用程序中产生的结构化输出.
研究的目的:
- 建议使用LLMs评估结构化输出的一般框架.
- 专注于在线分类平台中的搜索查询解析作为一个用例.
- 为了提高可靠性,引入新的基于LLM的评估技术.
主要方法:
- 利用LLM的上下文推理进行评估.
- 实施三种方法:点向,对向和通过/失败评估.
- 引入一个上下文评估快速路由策略来缓解幻觉.
主要成果:
- 基于LLM的评估,与人类判断达成约90%的一致性.
- 在小型和大型数据集上进行了实验.
- 拟议的框架在查询解析中表现出有效性.
结论:
- 基于LLM的评估是一种可扩展,可解释和有效的替代传统方法.
- 该框架为现实世界搜索系统提供了强大的查询解析.
- 这种方法提高了在搜索中LLM应用程序的可靠性.
相关概念视频
01:00Detection of Gross Error: The Q Test
6.4K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
6.4K
01:27Language and Cognition
442
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
442
02:07Improving Translational Accuracy
2.7K
2.7K
01:18Parseval's Theorem
653
Parseval's theorem is a fundamental concept in signal processing and harmonic analysis. It asserts that for a periodic function, the average power of the signal over one period equals the sum of the squared magnitudes of all its complex Fourier coefficients. This theorem, named after Marc-Antoine Parseval, provides a powerful tool for analyzing the energy distribution in signals.
Interestingly, Parseval's theorem also holds for the trigonometric form of the Fourier series, which...
Interestingly, Parseval's theorem also holds for the trigonometric form of the Fourier series, which...
653
01:22Language Development
450
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
450
01:10Higher Mental Functions of the Brain: Language
1.0K
Language is a system of communication that allows the expression of thoughts, ideas, and feelings. The brain processes language in both hemispheres.
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
Language formation and comprehension take place in the dominant hemisphere. The dominant hemisphere is responsible for understanding the meaning of spoken, written, or sign language, as well as the ability to communicate. For most people, the left hemisphere is the dominant one. The right hemisphere, then, gives tone and emotional context to the...
1.0K