Online Health Search Via Multidimensional Information Quality Assessment Based on Deep Language Models: Algorithm...

Area of Science:

• Information Retrieval
• Health Informatics
• Artificial Intelligence

Background:

• Online health information is rife with misinformation, posing risks to individuals seeking medical advice.
• Current information retrieval systems primarily rank results based on query-document relevance, neglecting information quality.

Purpose of the Study:

• To investigate a multidimensional information quality retrieval model using deep learning.
• To enhance the effectiveness and safety of online health information search results.

Main Methods:

• Simulated online health search scenarios using 32 health inquiries and 1 billion web documents.
• Assessed document quality (usefulness, supportiveness, credibility) using pre-trained language models on 6030 human-annotated pairs.
• Evaluated performance via transfer learning and domain adaptation techniques.

Main Results:

• The usefulness model distinguished best between helpful and harmful documents (+5.6%) in transfer learning.
• Combined models (usefulness, supportiveness, credibility) showed the largest distinction on helpful topics (+16.9%).
• Domain adaptation demonstrated robust performance, with help-harm compatibility above +4.4% across dimensions.

Conclusions:

• Integrating automatic ranking models for specific information quality dimensions enhances health information retrieval.
• The proposed approach can improve the quality and safety of online health searches for the public.