Cleaning by clustering: methodology for addressing data quality issues in biomedical metadata.

Area of Science:

• Bioinformatics
• Data Science
• Genomics

Background:

• High-quality metadata is crucial for efficient searching and filtering of biomedical datasets.
• The Gene Expression Omnibus (GEO) contains over 44 million key-value pairs suffering from quality issues like redundancy and inconsistency.
• Existing metadata quality issues hinder researchers' ability to find relevant datasets.

Purpose of the Study:

• To develop a clustering-based approach to address data quality issues in gene expression metadata.
• To improve the accuracy, structure, and completeness of biomedical metadata descriptions.

Main Methods:

• Proposed a clustering-based approach using three similarity measures for metadata keys.
• Designed a scalable agglomerative clustering algorithm to group similar keys.

Main Results:

• The algorithm successfully clustered similar metadata keys based on name, core concept, and value.
• Evaluated using a gold standard, the algorithm generated 18 clusters with 355 keys.
• Achieved a superior average F-Score of 0.63 compared to four other methods.

Conclusions:

• The clustering algorithm effectively identifies and groups similar metadata keys, resolving scalability issues for data cleaning.
• This approach facilitates the identification of duplicate and erroneous metadata entries.
• The algorithm is adaptable for use with other biomedical data types.