Seeking the wisdom of crowds through challenge-based competitions in biomedical research

“Challenge-based competitions” refers to a framework for addressing fundamental research questions in which the community is presented with a challenge, the data to address the challenge, and independent, unbiased assessment to rank submitted solutions. Although the typical result of such efforts is a robust performance evaluation of diverse methodologies, challenge-based competitions reach far beyond algorithm assessment. Here, we discuss the impact of challenge-based competitions in the areas of organizing and building communities and driving innovation.