Accurate wisdom of the crowd from unsupervised dimension reduction

Area of Science:

• Computational Biology
• Machine Learning
• Biomedical Informatics

Background:

• Wisdom of the crowd enhances collective intelligence and prediction accuracy.
• Current crowd wisdom models primarily focus on binary predictions, limiting applications in biomedical fields.
• Biomedical data analysis often involves complex datasets requiring advanced prediction methods.

Purpose of the Study:

• To demonstrate the analogy between crowd wisdom and unsupervised dimension reduction in machine learning.
• To extend crowd wisdom applications to continuous data beyond binary predictions.
• To improve prediction accuracy in biomedical disciplines using generalized crowd wisdom solutions.

Main Methods:

• Analogizing crowd wisdom to one-dimensional unsupervised dimension reduction (e.g., PCA, Isomap).
• Applying crowd wisdom solutions to real-world biomedical data (transcription factor target prediction, skin cancer diagnosis) and simulated data.
• Handling both binary and continuous responses, including confidence levels.

Main Results:

• Crowd wisdom is shown to be analogous to unsupervised dimension reduction techniques.
• Generalized crowd wisdom solutions accurately handle continuous data and outperform supervised methods.
• Effectiveness demonstrated in transcription factor target prediction and skin cancer diagnosis.

Conclusions:

• Unifies crowd wisdom and unsupervised dimension reduction, extending crowd wisdom to continuous data.
• Offers accurate and mature crowd wisdom solutions applicable to large-scale biomedical data.
• Highlights the potential of crowd wisdom for accurate predictions in high-throughput sequencing and imaging.