在系统审查中验证Loon Lens 1.0用于自主抽象选和以信心为导向的人在循环工作流程
Ghayath Janoudi1, Mara Uzun1, Tim Disher2
1Loon Inc, Ottawa, Ontario, Canada.
概括
一个AI平台的Loon Lens 1.0准确地选了标题和摘要,以便进行系统的审查. 信任度得分指导人类监督,提高精度和准确性,同时减少工作量.
科学领域:
- 研究中的人工智能.
- 系统的文学评论 系统的文学评论
- 信息科学 信息科学 信息科学
背景情况:
- 标题和摘要选是系统文献审查 (SLR) 的关键但耗时的阶段.
- 手动选需要大量的人力资源,并且可能是审查过程中的瓶.
研究的目的:
- 评估Loon Lens 1.0的性能,这是一个AI平台,用于SLR中自主标题和抽象选.
- 为了确定Loon Lens 1.0的信任分数是否可以有效地针对最小的人类监督.
主要方法:
- 使用Loon Lens 1.0重新选了8个单反相机 (3796个引用),并将结果与双人审查员进行了比较.
- 测量准确度,灵敏度,精度和特异性,使用启动的95%置信区间.
- 使用后勤回归以基于信心得分和决策的模型错误,告知模拟的人在循环中的策略.
主要成果:
- 卢恩镜头1.0实现了高精度 (95.5%),灵敏度 (98.9%) 和特异性 (95.2%),精度为63.0%.
- 错误集中在低至中等信心的"包括"决策中.
- 对低至中等信心"包括"记录的模拟人体审查 (3.8%的引用) 将精度提高到81.4%,准确度提高到98.2%.
结论:
- Loon Lens 1.0有效地复制了单反相机的人体查性能,达到98.9%的灵敏度和95.2%的特异性.
- 以信任为指导的人类监督,专注于一小部分记录 (≤5.8%),显著提高精度 (89.9%) 和整体准确性 (99.0%),同时保持高灵敏度.
- 人工智能驱动的信心评分提供了一种可行的策略,以优化SLR标题和摘要选中审查员的努力.
相关概念视频
01:15Data Validation
576
Method validation is a crucial process in analytical chemistry designed to confirm that a given method consistently produces reliable and high-quality results. This process is essential when a method is applied to different sample matrices or when procedural modifications are made, ensuring that the results meet acceptable standards across various applications.
Key parameters for method validation include:
Key parameters for method validation include:
576
01:03Data Validation
6.3K
Data validation is an essential part of a comprehensive assessment. Validation is confirming or verifying and opening the door to gathering more assessment data as it clarifies vague or unclear data. The process of checking and verifying the collected information is called data validation. The primary purpose of data validation is to ensure data is as free from error, bias, and misinterpretation as possible.
Nursing assessment guides are generally based on holistic models rather than medical...
Nursing assessment guides are generally based on holistic models rather than medical...
6.3K