大型语言模型对在线调查研究的潜在生存威胁
1Department of Government, Dartmouth College, Hanover, NH 03755.
概括
大型语言模型现在可以生成逃避检测的合成调查受访者,威胁到在线研究的完整性. 迫切需要新的验证方法来确保数据可靠性并防止错误信息.
科学领域:
- 社会科学 社会科学 社会科学
- 计算机科学 计算机科学
- 数据科学数据科学数据科学
背景情况:
- 在线调查研究是跨科学学科数据收集的基石.
- 人工智能的日益复杂化,特别是大型语言模型 (LLM),挑战了传统的数据收集假设.
研究的目的:
- 调查LLM对在线调查研究完整性的威胁.
- 设计和测试一种能够模仿人类调查反应的自主合成受访者.
- 评估当前数据质量检查对复杂的人工智能代理的脆弱性.
主要方法:
- 使用LLM技术开发一个自主合成响应器.
- 测试代理人逃避标准数据质量检查的能力,包括注意力检查,遵循指令和逻辑题.
- 对合成数据的连贯性,可信性和一致性与人类响应基准的评估.
主要成果:
- 合成受访者在6,000次注意力检查试验中取得了99.8%的合格率,逃避检测.
- 该代理产生了内部一致和可信的数据,模仿人类人格和记忆.
- 合成的受访者产生了对指定的教育水平进行校准的语言上复杂的开放式回应.
结论:
- 连贯的调查答案是人类的基本假设不再有效.
- 目前的数据验证方法不足以检测先进的AI驱动调查受访者.
- 迫切需要制定新的数据验证标准,并重新评估对低障碍在线数据收集的依赖.
更多相关视频
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
1.3K
08:05Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
Published on: June 30, 2020
8.0K
相关概念视频
Surveys
16.6K
Often, psychologists develop surveys as a means of gathering data. Surveys are lists of questions to be answered by research participants, and can be delivered as paper-and-pencil questionnaires, administered electronically, or conducted verbally. Generally, the survey itself can be completed in a short time, and the ease of administering a survey makes it easy to collect data from a large number of people.
16.6K
Language and Cognition
696
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
696
Survey Safety
363
Surveying near highways, rough terrain, or power lines involves significant risks. Working along highways is particularly dangerous and requires the use of warning signs and flagmen. It is safest to avoid working directly on roads and use offsets whenever possible. When highway work is unavoidable, it must follow all safety guidelines. Surveyors should wear bright clothing, such as orange reflective vests, to ensure visibility to motorists, coworkers, and hunters. In construction zones, wearing...
363
Language Development
810
Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...
810
Longitudinal Research
13.0K
Sometimes we want to see how people change over time, as in studies of human development and lifespan. When we test the same group of individuals repeatedly over an extended period of time, we are conducting longitudinal research. Longitudinal research is a research design in which data-gathering is administered repeatedly over an extended period of time. For example, we may survey a group of individuals about their dietary habits at age 20, retest them a decade later at age 30, and then again...
13.0K
Types of Surveys
334
Surveys are essential for marking property boundaries near water bodies. Different types of surveys are defined, each with its own function. Land surveys mark the property boundaries, while route surveys determine the position of properties on nearby highways. Topographic surveys create maps by capturing the three-dimensional features of the land. Hydrographic surveys focus on the shapes of underwater areas and the movement of streams through the properties. Mine surveys determine the relative...
334
