A User-Centered Evaluation of a COVID-19 Intelligent Query System (COVID-IQS)

As the fight against COVID-19 continues, it is critical to discover and accumulate knowledge in scientific literature to combat the pandemic. In this work, we shared the experience in developing an intelligent query system on COVID-19 literature. We conducted a user-centered evaluation with 12 researchers in our institution and identified usability issues in four categories: distinct user needs, functionality errors, suboptimal information display, and implementation errors. Furthermore, we shared two lessons for building such a COVID-19 literature search engine. We will deploy the system and continue refining it through multiple phases of evaluation to aid in redesigning the system to accommodate different user roles as well as enhancing repository features to support collaborative information seeking. The successful implementation of the COVID-IQS can support knowledge discovery and hypothesis generation in our institution and can be shared with other institutions to make a broader impact.