An IOHMM-Based Framework to Investigate Drift in Effectiveness of IoT-Based Systems

IoT-based systems, when interacting with the physical environment through actuators, are complex systems difficult to model. Formal verification techniques carried out at design-time being often ineffective in this context, these systems have to be quantitatively evaluated for effectiveness at run-time, i.e., for the extent to which they behave as expected. This evaluation is achieved by confronting a model of the effects they should legitimately produce in different contexts to those observed in the field. However, this quantitative evaluation is not informative on the drifts in effectiveness, it does not help designers investigate their possible causes, increasing the time needed to resolve them. To address this problem, and assuming that models of legitimate behavior can be described by means of Input-Output Hidden Markov Models (IOHMMs), a novel generic unsupervised clustering-based IOHMM structure and parameters learning algorithm is developed. This algorithm is first used to learn a model of legitimate behavior. Then, a model of the observed behavior is learned from observations gathered in the field. A second algorithm builds a dissimilarity graph that makes clear structural and parametric differences between both models, thus providing guidance to designers to help them investigate possible causes of drift in effectiveness. The approach is validated on a real world dataset collected in a smart home.