Towards accurate models for predicting smartphone applications' QoE with data from a living lab study

Progressively, smartphones have become the pocket Swiss army knife for everyone. They support their users needs to accomplish tasks in numerous contexts. However, the applications executing those tasks are regularly not performing as they should, and the user-perceived experience is altered. In this paper, we present our approach to model and predict the Quality of Experience (QoE) of mobile applications used over WiFi or cellular network. We aimed to create predictive QoE models and to derive recommendations for mobile application developers to create QoE aware applications. Previous works on smartphone applications' QoE prediction only focus on qualitative or quantitative data. We collected both qualitative and quantitative data "in the wild" through our living lab. We ran a 4-week-long study with 38 Android phone users. We focused on frequently used and highly interactive applications. The participants rated their mobile applications' expectation and QoE and in various contexts resulting in a total of 6086 ratings. Simultaneously, our smartphone logger (mQoL-Log) collected background information such as network information, user physical activity, battery statistics, and more. We apply various data aggregation approaches and features selection processes to train multiple predictive QoE models. We obtain better model performances using ratings acquired within 14.85 minutes after the application usage. Additionally, we boost our models' performance with the users expectation as a new feature. We create an on-device prediction model with on-smartphone only features. We compare its performance metrics against the previous model. The on-device model performs below the full features models. Surprisingly, among the following top three features: the intended task to accomplish with the app, application's name (e.g., WhatsApp, Spotify), and network Quality of Service (QoS), the user physical activity is the most important feature (e.g., if walking). Finally, we share our recommendations with the application developers, and we discuss the implications of QoE and expectations in mobile application design.