Generating Reward Functions Using IRL Towards Individualized Cancer Screening.

Area of Science:

• Oncology
• Artificial Intelligence
• Decision Science

Background:

• Cancer screening personalization is crucial to reduce overdiagnosis and accommodate participant heterogeneity.
• Partially observable Markov decision processes (POMDPs) offer a framework for individualized screening policies but require accurate reward functions.
• Defining appropriate reward functions for POMDPs in cancer screening remains a significant challenge.

Purpose of the Study:

• To propose and evaluate the use of inverse reinforcement learning (IRL) for deriving reward functions in POMDP models for lung and breast cancer screening.
• To develop personalized screening decision-making tools that align with expert clinical judgment.

Main Methods:

• Employed maximum entropy (MaxEnt) IRL with an adaptive step size to learn reward functions efficiently.
• Integrated a multiplicative model to learn state-action pair rewards for POMDPs.
• Developed and evaluated two POMDP models for lung and breast cancer screening using retrospective physician decisions.

Main Results:

• The IRL-derived reward functions, when integrated with POMDP models, yielded personalized screening recommendations comparable to expert physicians.
• High agreement (Cohen's Kappa) was observed between POMDP recommendations and physician predictions for breast cancer screening.
• A decreasing trend in agreement was noted for lung cancer screening, indicating areas for further refinement.

Conclusions:

• Inverse reinforcement learning provides a viable method for creating effective reward functions for POMDP-based personalized cancer screening.
• The developed POMDP models demonstrate the potential to support individualized screening decisions, achieving expert-level performance in certain contexts.
• Further research may be needed to optimize IRL-POMDP models for lung cancer screening to fully match expert performance.