评估大型语言模型在澳大利亚医疗保健系统的导航能力:比较研究
Joshua Simmich1,2, Megan Heather Ross1,2, Trevor Glen Russell1,2
1RECOVER Injury Research Centre, Faculty of Health, Medicine and Behavioural Sciences, The University of Queensland, 288 Herston Rd, Queensland, Brisbane, 4029, Australia, +61 7 3365 5560.
JMIR AI
|October 8, 2025
概括
生成型人工智能搜索工具的准确性与健康信息导航的传统搜索引擎相提并论. 虽然有效,但用户体验没有显著改善,这表明需要进一步开发人工智能.
科学领域:
- 医疗信息学 医疗信息学
- 医疗保健中的人工智能
- 人与计算机的交互
背景情况:
- 导航澳大利亚的医疗保健系统,特别是在农村地区,提出了重大挑战.
- 使用大型语言模型 (LLM) 的生成性搜索工具为改善健康信息检索提供了潜在的潜力.
- 与医疗保健中的传统搜索引擎相比,人们对生成人工智能的准确性和可靠性存在担忧.
研究的目的:
- 为了比较生成人工智能 (AI) 搜索 (微软Copilot) 与传统搜索引擎 (谷歌网络搜索) 对健康信息导航的有效性.
- 评估人工智能驱动的健康信息检索工具的准确性和用户体验.
主要方法:
- 一项基于网络的调查,涉及97名澳大利亚昆士兰州的成年人.
- 参与者使用微软Copilot或谷歌网络搜索来寻找基于场景的医疗保健导航问题.
- 通过正确度评分和数值得分来衡量准确度;使用技术评分问卷 (TRQ) 评估用户体验.
主要成果:
- 与谷歌网络搜索相比,微软Copilot在2个特定的医疗护理导航任务中表现出卓越的性能.
- 对于剩余的6个任务,没有观察到显著的性能差异.
- 用户对谷歌网络搜索的收养意愿和生活质量影响的评价较高,但对学习努力的评价较低. 两种工具在感知价值,信心和隐私问题上都获得了相似的评分.
结论:
- 生成型人工智能工具可以达到与医疗保健导航传统搜索引擎可比的准确性.
- 尽管精度相似,但在本研究中,生成人工智能并没有提高整体用户体验.
- 需要进一步的研究来评估人工智能的进步和用户对这些技术的适应.
相关概念视频
Documentation in Long-Term and Home Healthcare Setting
1.4K
Documentation in long-term care facilities and home healthcare settings is crucial for ensuring continuous, coordinated, and comprehensive care for patients. Each setting has its specific documentation processes and tools:
Long-Term Care Facilities
Long-Term Care Facilities
1.4K
Methods of Documentation VI: Case Management Model
857
The case management model is a multidisciplinary approach that involves healthcare professionals from diverse disciplines, such as physicians, nurses, therapists, social workers, and pharmacists, working collaboratively to address the various needs of patients. Each healthcare professional brings unique expertise and perspectives, contributing to a more comprehensive understanding of the patient's condition and tailoring treatment plans accordingly.
For example, a patient with a chronic...
For example, a patient with a chronic...
857
Methods Of Healthcare Delivery System
4.0K
At the different levels of the healthcare system, we see varying methods of healthcare used. These methods include managed care systems, case management, and primary healthcare.
Managed Care System:
The managed care system is designed to control the cost while maintaining the quality of care. The patient's care from admission to discharge is planned by the primary care provider or the case manager, also known as the gatekeeper. In a managed care system, the number of care providers is...
Managed Care System:
The managed care system is designed to control the cost while maintaining the quality of care. The patient's care from admission to discharge is planned by the primary care provider or the case manager, also known as the gatekeeper. In a managed care system, the number of care providers is...
4.0K
Secondary Healthcare System
2.0K
Secondary healthcare is offered by a specialist, generally in hospitals or clinics for patients referred by primary healthcare providers. It occurs when a person has an illness or injury that requires specific medical care. Secondary care is often referred to as acute care. Secondary care can range from uncomplicated care to repair a minor laceration or treat a strep throat infection to more complicated emergent care, such as treating a head injury sustained in an automobile accident. Whatever...
2.0K
Introduction To Health Care Delivery System
3.8K
The healthcare system is constantly changing and complex. Various services are available from different healthcare providers, but gaining access to these services has become challenging for people with limited healthcare insurance. Uninsured people present a challenge to healthcare because they frequently postpone or forego treatment.
The Institute of Medicine (IOM) advocates for a patient-centered, effective, safe, timely, equitable, and effective healthcare system. The National Priorities...
The Institute of Medicine (IOM) advocates for a patient-centered, effective, safe, timely, equitable, and effective healthcare system. The National Priorities...
3.8K


