模拟推理和自我验证在精神病诊断表现的通用大型语言模型中:横截面研究
medRxiv : the preprint server for health sciences
|September 18, 2025
概括
模拟推理和自我验证显著提高了用于精神病诊断的大型语言模型 (LLM) 的积极预测值 (PPV),而不会影响灵敏度. 将这些方法结合起来,可以提高LLM在行为健康应用中的诊断准确性.
科学领域:
- 人工智能在医学中的应用
- 计算精神病学是一种计算精神病学.
- 自然语言处理自然语言处理.
背景情况:
- 大型语言模型 (LLM) 对精神病学有希望,但在诊断准确性方面存在局限性.
- 潜在的风险和缺陷使基于LLM的系统在精神病诊断中的应用变得复杂.
- 模拟推理和自我验证是提高LLM绩效的新兴技术.
研究的目的:
- 评估模拟推理 (使用大型推理模型,LRMs) 和自我验证对LLM精神病诊断性能的影响.
- 为了比较不同模型和推断方法之间的诊断准确度指标 (灵敏度和积极的预测值).
主要方法:
- 提取了106个DSM-5-TR病例简介和诊断.
- 利用来自两个供应商 (OpenAI,Google) 的LLM和LRM,使用基本和自验证推断方法.
- 使用灵敏度和正预测值 (PPV) 评估诊断性能,使用线性混合效应模型进行分析.
主要成果:
- 灵敏度在0.732到0.817之间;PPV在0.534到0.779.之间.
- 自验和LRM模型显著改善了PPV (p<0.003).
- 在PPV的模型类型和供应商之间发现了相互作用效应 (LRM:OpenAI,p=0.021).
结论:
- 模拟推理和自我验证都显著改善了精神病诊断的PPV,但敏感度没有显著变化.
- 添加自我验证提示增强了PPV,即使使用模拟推理.
- 将手工制作的提示与自动模拟推理相结合,可以优化LLM在行为健康方面的应用.
更多相关视频
07:31Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack
Published on: May 15, 2020
7.5K
05:47Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
Published on: June 13, 2025
1.3K
相关概念视频
Diagnostic and Statistical Manual of Mental Disorders (DSM)
1.1K
The Diagnostic and Statistical Manual of Mental Disorders (DSM) serves as the primary classification system for mental health disorders, providing standardized diagnostic criteria for clinicians and researchers. First published by the American Psychiatric Association (APA) in 1952, the DSM has undergone several revisions to reflect evolving psychiatric understanding. The fifth edition, DSM-5, released in 2013, introduced key updates that expanded diagnostic categories and modified diagnostic...
1.1K
Modeling in Therapy
382
Modeling, a key technique in therapy, uses observational learning to help clients acquire and practice new skills by watching therapists demonstrate desired behaviors. This approach, rooted in Albert Bandura's concept of vicarious learning, plays a significant role in therapeutic interventions for various psychological conditions, including social anxiety, ADHD, and depression.
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
Participant Modeling
Participant modeling involves therapists demonstrating calm and effective behaviors in...
382
Language and Cognition
716
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
716
Self-Report Tests of Personality
776
Self-report inventories are objective personality assessments that use multiple-choice items or numbered scales, typically ranging from 1 (strongly disagree) to 5 (strongly agree). They are often called Likert scales after Rensis Likert. These inventories are widely used due to their ease of administration and cost-effectiveness. One of the most prominent examples is the Minnesota Multiphasic Personality Inventory (MMPI), initially developed in the 1940s to assess abnormal personality traits.
776
Self-Schemas
35.4K
In general, a schema is a mental construct consisting of a cluster or collection of related concepts (Bartlett, 1932). There are many different types of schemata, and they all have one thing in common: schemata are a method of organizing information that allows the brain to work more efficiently. When a schema is activated, the brain makes immediate assumptions about the person or object being observed.
35.4K
Self-Discrepancy Theory
18.9K
One influential perspective on what motivates people's behavior is detailed in Tory Higgin's self-discrepancy theory (Higgins, 1987). He proposed that people hold disagreeing internal representations of themselves that lead to different emotional states.
18.9K
