PICO element detection in medical text without metadata: are first sentences enough?

Area of Science:

• Medical Informatics
• Natural Language Processing
• Evidence-Based Medicine

Background:

• Efficient identification of patient, intervention, comparison, and outcome (PICO) components is crucial for evidence-based medicine.
• Automated PICO element detection aids in synthesizing medical literature.
• The effectiveness of using only initial sentences for training classifiers is underexplored.

Purpose of the Study:

• To evaluate if first sentences of PICO components are sufficient for training naive Bayes classifiers for sentence-level PICO element detection.
• To compare the performance of classifiers trained on first sentences versus all sentences within PICO components.

Main Methods:

• Extracted 19,854 structured abstracts from PubMed for randomized controlled trials with P/I/O labels.
• Trained naive Bayes classifiers using either the first sentences (CF) or all sentences (CA) of PICO elements.
• Compared classifier performance using ten-fold cross-validation, measuring recall, precision, and F-measures.

Main Results:

• No significant performance difference between CF and CA for outcome (O) element detection (F-measure ≈ 0.73).
• CA significantly outperformed CF for intervention (I) element detection (recall: 0.752 vs. 0.620; F-measure: 0.728 vs. 0.662).
• CF showed higher precision for patient (P) elements (0.714 vs. 0.665) but lower recall (0.766 vs. 0.811) compared to CA.

Conclusions:

• Using only the first sentences (CF) is not consistently superior to using all sentences (CA) for PICO element detection.
• Classifier performance for PICO element identification varies depending on the specific element (P, I, or O).
• Full sentence data (CA) offers advantages for detecting intervention elements, while first sentences (CF) may offer higher precision for patient elements.