A comparison between clinical decision support system and clinicians in breast cancer

We are building a clinical decision support system (CSCO AI) for breast cancer patients to improve the efficiency of clinical decision-making. We aimed to assess cancer treatment regimens given by CSCO AI and different levels of clinicians.

400 breast cancer patients were screened from the CSCO database. Clinicians with similar levels were randomly assigned one of the volumes (200 cases). CSCO AI was asked to assess all cases. Three reviewers were independently asked to evaluate the regimens from clinicians and CSCO AI. Regimens were masked before evaluation. The primary outcome was the proportion of high-level conformity (HLC).

The overall concordance between clinicians and CSCO AI was 73.9% (3621/4900). It was 78.8% (2757/3500) in the early-stage, higher than that in the metastatic stage (61.7% [864/1400], p < 0.001). The concordance was 90.7% (635/700) and 56.4% (395/700) in adjuvant radiotherapy and second-line therapy respectively. HLC in CSCO AI was 95.8% (95%CI:94.0%-97.6%), significantly higher than that in clinicians (90.8%, 95%CI:89.8%-91.8%). Considering professions, the HLC of surgeons was 85.9%, lower than that of CSCO AI (OR = 0.25,95%CI: 0.16-0.41). The most significant difference in HLC was in first-line therapy (OR = 0.06, 95%CI:0.01-0.41). When clinicians were divided according to their levels, there was no statistical significance between CSCO AI and higher level clinicians.

Decision from CSCO AI for breast cancer was superior than most clinicians did except in second-line therapy. The improvements in process outcomes suggest that CSCO AI can be widely used in clinical practice.