Related Experiment Video
Updated: Aug 24, 2025
An R-Based Landscape Validation of a Competing Risk Model
Published on: September 16, 2022
Comparison of the MOdified NARanjo Causality Scale (MONARCSi) for Individual Case Safety Reports vs. a Reference
Shaun M Comfort1, Bruce Donzanti2, Darren Dorrell3
1Roche-Genentech, Inc., A Member of the Roche Group, 1 DNA Way, South San Francisco, CA, 94080, USA. comforts@gene.com.
Introduction:
In 2018, we published the MONARCSi algorithmic decision support tool showing high inter-rater agreement, moderate sensitivity, and high specificity compared with drug-event pairs (DEPs) previously reviewed using current, industry-established approaches. Following publication, MONARCSi was implemented as a prototype system to facilitate medical review of individual case safety reports (ICSRs). This paper presents subsequent evaluation of MONARCSi-supported causality assessments against an independent, best achievable reference standard.
Objective:
This paper describes the development of an independent reference standard (i.e., reference comparator) using a sample of DEPs evaluated by Roche subject matter experts (SMEs) and subsequent performance analysis for both the reference standard and MONARCSi.
Methods:
Roche collected a random sample of 131 DEPs evaluated by an external vendor using the MONARCSi prototype during 2020, and collectively referred to as the VMON (Vendor using the MONARCSi system for medical review) dataset. An internal group of causality SMEs (aka CAUSMET) were recruited and trained to assess the same DEPs independently using the MONARCSi structure with Global Introspection to determine their individual assessments of causality. The CAUSMET final causality was determined using a majority voting rule.
Results:
Binary comparison of the aggregate results showed substantial agreement (Gwet kappa = 0.81) between the VMON and reference standard CAUSMET assessments. Bayesian latent class modeling showed that both the reference standard and VMON assessments exhibited similar high posterior mean sensitivity and specificity (CAUSMET: 89 and 93%, respectively; VMON: 87 and 94%, respectively). Finally, comparison of the sensitivity and specificity suggested no obvious difference across groups.
Conclusion:
Analysis of causality results from the assessments by independent internal SMEs using MONARCSi shows there is no obvious difference in performance between the aggregate CAUSMET and VMON assessments based on the comparison of specificity and sensitivity. These results further support use of MONARCSi as a decision support tool for evaluating drug-event causality in a consistent and documentable manner.
Related Concept Videos
01:23Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
01:12Relative Risk
01:14SBAR II: Application of SBAR
SBAR Report from a Nurse to a Health Care Provider
S: "Hello, Dr. Smith. This is Jane, RN, from the Med Surg unit. I am calling to tell you about Ms. White in Room 210, who is experiencing increased pain and redness at her incision site. Her recent...
01:21Types of Reports II: Incident or Occurrence Report
Purposes:
In the healthcare industry, reports play a crucial role in documenting incidents within an agency. The primary objective of these reports is to ensure patient safety, uphold the...
01:12Hazard Ratio
For example, in a clinical trial...
01:20Comparing the Survival Analysis of Two or More Groups