Crowdsourcing with Enhanced Data Quality Assurance: An Efficient Approach to Mitigate Resource Scarcity Challenges in...

Area of Science:

• Artificial Intelligence
• Healthcare Informatics
• Natural Language Processing

Background:

• Large Language Models (LLMs) show promise in healthcare but require high-quality labeled data.
• Data acquisition is challenging and costly, especially in low-resource healthcare settings.
• Existing methods struggle with data quality for specialized AI applications.

Purpose of the Study:

• To develop and evaluate a crowdsourcing framework with integrated quality control for healthcare data.
• To assess the impact of enhanced data quality on LLM performance for autism symptom prediction.
• To address data scarcity and quality issues in resource-constrained healthcare domains.

Main Methods:

• Implementation of a crowdsourcing framework with pre-, real-time-, and post-data gathering quality control.
• Evaluation of data quality improvements using quantitative metrics.
• Fine-tuning of a healthcare-specific LLM (Bio-BERT) on crowdsourced data for autism symptom prediction.
• Comparison of LLM performance against a baseline model.

Main Results:

• Real-time quality control demonstrated a 19% improvement in data quality over pre-quality control measures.
• Fine-tuning Bio-BERT with crowdsourced data led to increased recall but decreased precision compared to the baseline.
• The crowdsourcing approach showed potential for improving LLM performance in data-limited scenarios.

Conclusions:

• Crowdsourcing, enhanced by robust quality control, is a viable strategy for acquiring high-quality healthcare data.
• Optimized data acquisition can improve the efficacy of LLMs in healthcare, particularly for tasks like symptom prediction.
• Findings offer insights for developing more effective and resource-efficient healthcare AI solutions.