Operational safety risk modeling in a naval organization

Following numerous mishaps and near-misses, the U.S. Naval Surface Force established the Operational Surface Risk Indicators (OSRI) project to explore a robust proactive risk analysis and reduction capability. The OSRI model leverages multisource enterprise-wide data to forecast potential risks for individual warships to help leaders make informed operational decisions and ultimately improve safety outcomes.

Machine learning was used to predict risk scores based on various input features that describe a ship's current state, including crew training, staffing levels, experience, and turnover. Machine learning techniques like stacking and bagging were employed in novel ways to enhance model interpretability and fairness. To make the model's output more tangible, predicted risk scores were converted into calibrated probabilities of mishaps; additionally, nearest neighbor techniques were integrated to provide insights on how current high-risk ships may be similar to past ships that experienced mishaps.

The model highlights risk indicators as potential areas for improvement and incorporates additional methods to derive insights from risk modeling.

From an operational perspective, the OSRI model aims to enhance leaders' intuition about risk by identifying indicators that are drivers of risk.