Collective intelligence in medical diagnosis systems: A case study

Area of Science:

• Medical Informatics
• Artificial Intelligence in Medicine
• Clinical Decision Support

Background:

• Accurate medical diagnosis is critical but challenging.
• Single expert knowledge may be limited.
• Collective intelligence offers a potential solution.

Purpose of the Study:

• To evaluate the application of collective intelligence via consensus methods in medical diagnosis.
• To compare diagnostic accuracy against single expert knowledge.
• To assess the impact on knowledge base item count and consensus levels.

Main Methods:

• Utilized ontological structures for ten diseases.
• Developed two knowledge bases with five diseases each.
• Conducted experiments with empty and populated knowledge bases.
• Five experts iteratively refined signs/symptoms and diagnostic tests.
• Evaluated consensus levels against a Gold Standard (GS).

Main Results:

• Knowledge bases generated through consensus had more items than the GS.
• Lower consensus levels correlated with higher item counts.
• The minimum agreement level (20%) surpassed GS sign counts.
• Full expert agreement led to a decrease in knowledge base items.

Conclusions:

• Collective intelligence, through consensus, can increase physician agreement.
• Consensus methods allow for dynamic information gathering and collaboration.
• Physicians can access broader, updated knowledge compared to literature-based systems.