Narrative Search Engine for Case Series Assessment Supported by Artificial Intelligence Query Suggestions

Area of Science:

• Pharmacovigilance and Drug Safety
• Artificial Intelligence in Healthcare
• Information Retrieval

Background:

• Manual review of large adverse event report (case series) datasets is challenging.
• Developing effective search queries for these datasets is often a manual and time-consuming process.
• AI-driven query suggestions can assist assessors in identifying relevant case narratives.

Purpose of the Study:

• To evaluate the feasibility of an AI-supported narrative search engine for identifying specific case narratives.
• To assess the effectiveness of artificial intelligence (AI) in suggesting relevant search terms for adverse event reports.

Main Methods:

• A narrative search engine utilizing Best Match 25 (BM25) was developed.
• The engine incorporated AI-generated query suggestions from English and biomedical word embedding models.
• Performance was evaluated on VigiBase data, comparing recall and precision against exact-match and RM3 searches.

Main Results:

• The AI-supported engine achieved an average recall of 56.4%, significantly higher than exact-match (21.8%) and RM3 (34.4%) searches.
• Precision remained comparable (54.5% vs. 43.1%) despite the increased recall.
• The system successfully retrieved a greater percentage of relevant narratives from the VigiBase dataset.

Conclusions:

• AI-query suggestions enhance narrative search engines for adverse event report analysis.
• This approach offers a viable alternative to traditional search methods, improving signal detection.
• The developed system facilitates the retrieval of crucial information within large case series.