Can Researchers Assess the Suitability of Datasets to Answer Their Research Questions, with Access to Metadata Only?

Area of Science:

• Health Informatics
• Data Science
• Biomedical Research

Background:

• Effective identification and assessment of research datasets are crucial for advancing health research.
• Metadata catalogues are essential tools for discovering and evaluating data resources.

Purpose of the Study:

• To evaluate the efficacy of a metadata catalogue in enabling researchers to determine dataset suitability for specific research questions.
• To identify challenges researchers face when using metadata to assess data.

Main Methods:

• Researchers assessed the answerability of research questions using descriptions of five datasets within a metadata catalogue.
• Thirteen researchers evaluated ten question-dataset pairings, with data from electronic health records (EHR) included.
• Assessments were analyzed after excluding instances of participant misunderstanding or lack of subject matter expertise.

Main Results:

• 80% of assessments (87 out of 109) were correct, indicating general usability of the metadata catalogue.
• Researchers encountered significant difficulties assessing a dataset comprising EHR data.
• The primary reason for incorrect assessments was the inability to locate pertinent information within the metadata catalogue.

Conclusions:

• Metadata catalogues show promise for aiding researchers in assessing dataset suitability.
• Improvements are needed to enhance the clarity and completeness of metadata, especially for complex data types like EHR.
• Optimizing metadata accessibility is key to improving data discoverability and research efficiency.