External validation of a knowledge-based clinical toxicology consultant for diagnosing single exposures

Clinical prediction models assist healthcare providers in estimating disease likelihood using multiple patient-specific factors. While these models are increasingly used, few undergo external validation, which limits their clinical utility. In toxicology, timely diagnosis is crucial and expertise scarce, making knowledge-based systems an excellent choice to support medical decision-making. A knowledge-based system which was developed using data from the Florida Poison Information Center Network has shown promise but lacks external validation. This study aims to evaluate its performance using data from the Lyon Poison Information Center to assess its generalizability.

To externally validate the Florida diagnostic model, we used anonymized clinical data from the Lyon Poison Information Center. French symptom and substance descriptions were standardized and translated into English using expert-curated dictionaries. These were then mapped to the clinical effects and substances used in the original model. For each case, the model generated likelihood ratios for all possible substances, producing a ranked diagnostic list. Model performance was assessed using Top-K accuracy.

External validation was conducted on 602 moderate to major severity poisoning cases. The overall Top-10% accuracy of the model was 75.4%, with the highest accuracy in major severity cases (87.0%) and the lowest in fatal cases (57.6%). These results were comparable to internal validation metrics reported in a prior study. In major cases, external validation even outperformed internal validation.

Despite challenges related to language and database compatibility, the system showed robust accuracy, which supports its clinical applicability. Limitations include its current restriction to single-substance poisonings and the need for further validations across diverse populations.

The system demonstrates promising accuracy and highlights the importance of continued international validation to support broader clinical adoption. With further refinement and integration of additional patient-specific data, such models could play a key role in enhancing rapid, data-driven toxicological diagnoses worldwide.