A Critical Evaluation of Safety Signal Analysis Using Algorithmic Standardised MedDRA Queries

Algorithmic Standardised MedDRA^® Queries (aSMQs) are increasingly used to enhance the efficiency of safety signal detection. The manner that aSMQs affect capture of potential safety cases is unclear.

Our objective was to characterise the performance of aSMQs with respect to their potential for double counting, the likelihood of events in aSMQ positive cases being clinically related, how frequently terms are used for algorithmically positive cases, and the face validity of positive cases based on the drug inducing events. We were also interested in what effect requiring symptoms to overlap temporally would have on performance.

We reviewed adverse event (AE) datasets of New Drug Applications and Biological License Applications and compiled a database including preferred terms and corresponding SMQs, SMQ term categories, AE start day, AE duration, drug name, and Anatomical Therapeutic Chemical class. Two reviewers independently determined if the algorithm was met and, if so, whether the broad terms overlapped temporally.

A total of 107 marketing applications were reviewed, including 103,928 patients and 277,430 AEs. Use of algorithms condensed the number of AEs to between 5 and 8% and the incidence to about 1.5% relative to when the SMQs are used without the algorithm. Certain aSMQs exhibited a potential for overcounting. Requiring symptoms to temporally overlap helped to eliminate irrelevant cases.

Our findings demonstrate that algorithmic and temporal assessment increased specificity of case retrieval, though the reduction in the number of terms or incidence seemed excessive for certain aSMQs. Evaluating the day of AE onset and duration improve specificity through identification of outlying events. Identification of drug classes known to cause the aSMQ's clinical condition provides face validity for this tool, yet detection of cases associated with novel classes may provide new understanding of these disorders. Improvements in some of the SMQ term lists may improve the performance of SMQs in general.