构建一个模块化的健康信息爬虫:利用Apache Nutch为Tala-Med搜索引擎提供服务
Nanxing Deng1, Martin Boeker2, Raphael Scheible2
1School of Computation, Information and Technology, Technical University of Munich, Germany.
Studies in health technology and informatics
|May 17, 2025
概括
tala-med的健康搜索引擎使用Apache Nutch进行了升级,以改善网络抓取和索引. 这提高了可扩展性和定制性,以更好地基于证据的健康信息检索.
科学领域:
- 医疗信息学 医疗信息学
- 计算机科学 计算机科学
背景情况:
- 2020年推出的tala-med搜索引擎提供来自德国来源的可靠健康信息.
- 由于技术限制,需要进行升级以提高其性能和可扩展性.
研究的目的:
- 用Apache Nutch取代现有的Fess爬虫,以改善网络爬虫和索引功能.
- 为了提高tala-med搜索引擎的可扩展性和定制性.
主要方法:
- 使用Docker实现了一个新的系统,集成PostgreSQL,Apache Nutch和ElasticSearch.
- 使用管理器通过扩展的Nutch-REST接口与自定义配置来编排流程.
- 嵌入域特定的URL过器,用于在抓取过程中选择有针对性的内容.
主要成果:
- 该系统在65小时的测试中成功处理了大约23,000个网站.
- 在健康信息环境中展示了Apache Nutch在大规模网络爬虫和索引方面的有效性.
结论:
- 集成Apache Nutch显著提高了tala-med搜索引擎的可扩展性和灵活性.
- 未来的工作将集中在长期部署和生成一个全面的搜索索引进行进一步分析.
相关概念视频
ER Retrieval Pathway
3.8K
In the secretory pathway, vesicles transport proteins from one cellular compartment to another in forward transport to deliver the protein to its correct location. Occasionally, misfolded proteins and incorrect proteins escape their original compartments, and a retrieval pathway is used to return the escaped proteins to their original compartment.
The ER uses many checkpoints to prevent the entry of incorrectly folded or a resident protein as cargo onto a transport vesicle. These mechanisms...
The ER uses many checkpoints to prevent the entry of incorrectly folded or a resident protein as cargo onto a transport vesicle. These mechanisms...
3.8K
Tertiary Healthcare System
2.2K
Specialized care provided over an extended period is called tertiary care. Usually, a primary or secondary care physician will refer a patient to tertiary care. A patient's maximum physical and mental function is restored in tertiary care, which is caused due to the impact of a chronic illness or condition. Tertiary care aims to achieve the highest level of functioning possible while managing chronic illness. For example, a patient who falls and fractures their hip will need secondary care...
2.2K
Integrated Healthcare System
2.4K
An integrated healthcare system (IHS) is a set of organizations that provides for or arranges to provide coordinated and continuous service to a defined population. The IHS takes responsibility for that particular population's health status and outcome, both clinically and fiscally. An integrated healthcare system is a well-organized, well-coordinated, and collaborative network. The integrated delivery system is a network that connects different healthcare providers to deliver organized,...
2.4K
Healthcare Agencies II
960
There are various healthcare agencies in the United States—some of which are managed by religious institutions and others by different government branches.
Parish nursing is a growing specialty nursing profession that focuses on holistic healthcare, health promotion, and illness prevention. It blends professional nursing practice with a health ministry, focusing on health and healing within the context of a Christian community. Parish nurses serve as health educators, referral sources,...
Parish nursing is a growing specialty nursing profession that focuses on holistic healthcare, health promotion, and illness prevention. It blends professional nursing practice with a health ministry, focusing on health and healing within the context of a Christian community. Parish nurses serve as health educators, referral sources,...
960
Health Information Technology and Healthcare Information System
1.7K
Health Information Technology (HIT)
Health Information Technology, commonly called HIT, integrates advanced information systems and technology in healthcare settings. Its primary functions include:
Health Information Technology, commonly called HIT, integrates advanced information systems and technology in healthcare settings. Its primary functions include:
1.7K
Nursing Clinical Information System
1.4K
Nursing Clinical Information System (NCIS)
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
A Nursing Clinical Information System (NCIS) is a specialized type of healthcare information system tailored to meet the unique needs of nursing practice. It incorporates the principles of nursing informatics to streamline information management and improve the quality of care delivery.
Critical attributes of NCIS include:
1.4K


