沃尔多:从非结构化的自我报告中自动发现不良事件
Karan S Desai1, Vijay M Tiyyala2, Pranav Tiyyala3
1University of Michigan Medical School, University of Michigan, Ann Arbor, Michigan, United States of America.
PLOS digital health
|September 30, 2025
概括
使用基于RoBERTa的机器学习工具Waldo的自动不良事件 (AE) 检测,在识别来自消费者健康产品的社交媒体数据的安全信号时,实现了99.7%的准确性.
科学领域:
- 药监和药物安全 药监和药物安全
- 在医疗保健中的自然语言处理 (NLP)
- 社交媒体分析用于公共卫生
背景情况:
- 检测不良事件 (AE) 对消费者健康产品安全至关重要,但传统上是劳动密集型和昂贵的.
- 不受监管的产品,如大麻衍生产品 (CDP),往往缺乏强大的市场后监管,需要新的检测方法.
- 需要自动化解决方案来提高效率,降低成本,并从非结构化数据中识别罕见的安全信号.
研究的目的:
- 开发和评估一个名为Waldo的自动机器学习工具,用于从非结构化社交媒体文本中检测AE.
- 为了比较不同机器学习模型 (N-gram,BERT,RoBERTa) 对于AE检测的性能.
- 将表现最好的模型与AI聊天机器人进行基准测试,并将其应用于识别CDP用户叙述中的AI.
主要方法:
- 训练并评估了三种模型:N-gram,BERT和RoBERTa,这些模型是基于1万份人类注释的与CDP相关的AE报告.
- 选择了表现最好的模型 (RoBERTa),并将其命名为Waldo.
- 他们将Waldo与ChatGPT (gpt-3.5-turbo-0613) 进行了比较,并将其应用于分析来自20个子网址的437,132条帖子.
主要成果:
- 沃尔多 (RoBERTa) 实现了99.7%的准确性,F1得分为95.1%的积极类,明显优于ChatGPT (94.4%的准确性,38%的F1得分).
- 沃尔多从437,132个分析的帖子中确定了28832个潜在的AE.
- 在各个分支机构中,AE率各不相同,其中r/Marijuana显示最高 (12.7%),r/weedstocks显示最低 (0.1%).
结论:
- 沃尔多通过自动化从社交媒体上检测AE有效地解决了未受监管的消费者健康产品安全监测中的关键缺口.
- 该工具以高精度在规模上处理非正式的用户叙述,这是传统行业系统缺乏的能力.
- 沃尔多已经开源,以促进卫生界立即应用.
更多相关视频
07:50A Metadata Extraction Approach for Clinical Case Reports to Enable Advanced Understanding of Biomedical Concepts
Published on: September 20, 2018
16.4K
11:21Methodology for Establishing a Community-Wide Life Laboratory for Capturing Unobtrusive and Continuous Remote Activity and Health Data
Published on: July 27, 2018
8.6K
相关概念视频
Pharmacovigilance
1.6K
Post-marketing surveillance is a critical component of pharmaceutical regulation, often uncovering unanticipated adverse drug reactions (ADRs) once a drug is widely used over an extended period.
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
This process, termed pharmacovigilance, aims to detect, evaluate, and minimize harmful effects related to medication use. The data collection for pharmacovigilance depends on spontaneous reporting systems, where healthcare professionals or patients voluntarily report suspected ADRs.
In some cases, there...
1.6K
Types of Reports II: Incident or Occurrence Report
1.2K
An Incident or Occurrence Report in a healthcare setting is a crucial document used to record any unexpected occurrence that may or may not have affected a patient, employee, or visitor. Such reports are critical to improving patient safety and include all details leading up to and including the event.
Purposes:
In the healthcare industry, reports play a crucial role in documenting incidents within an agency. The primary objective of these reports is to ensure patient safety, uphold the...
Purposes:
In the healthcare industry, reports play a crucial role in documenting incidents within an agency. The primary objective of these reports is to ensure patient safety, uphold the...
1.2K
Documentation of Nursing Diagnosis
1.6K
The nurse documents nursing diagnoses and enters them into the patient record. The identified patient's nursing diagnosis is either written out with a plan of care or entered into the electronic health record.
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
In some settings, data-driven computerized decision support systems are in place, allowing for more accurate nursing diagnoses. The database within one of these systems includes diagnostic labels defining characteristics, activities, and indicators for nursing. A nurse enters...
1.6K
Data Collection I
7.8K
Data collection gathers information needed to make accurate judgments about a patient's present condition. During a health history interview, subjective data is collected from the patient, their caregivers, or family members, and objective data is collected through observations and physical assessment. Patients are the primary source of subjective data. Thus information gathered from patients through interviews, observations, and physical examination is primary data. Secondary sources of...
7.8K
