State of the Practice for Medical Imaging Software Based on Open Source Repositories

We review the state of the practice for the development of medical imaging (MI) software based on data available in open-source repositories. We selected 29 projects from 48 candidates and assessed nine software qualities by answering 108 questions for each. Using the analytic hierarchy process (AHP) on the quantitative data, we ranked the MI software. The top five are 3D Slicer, ImageJ, Fiji, OHIF Viewer, and ParaView. This is consistent with the community's view, with four of these also appearing in the top five using GitHub metrics (stars per year). The quality and quantity of documentation present in a project correlate quite well with its popularity. Generally, MI software is in a healthy state: in the repositories, we observed 88% of the documentation artifacts recommended by research software development guidelines, and 100% of MI projects use version control tools. However, the current state of the practice deviates from existing guidelines as some recommended artifacts are rarely present (such as a test plan, requirements' specification, and code style guidelines), low usage of continuous integration (17% of the projects), low use of unit testing (~ 50% of projects), and room for improvement with documentation. From developer interviews, we identified seven concerns: lack of development time, lack of funding, technology hurdles, correctness, usability, maintainability, and reproducibility. We recommend increasing effort on documentation, increasing testing by enriching datasets, increasing continuous integration, moving to web applications, employing linters, using peer reviews, and designing for change.