Evaluation of the extraction of methodological study characteristics with JATSdecoder

Area of Science:

• Computer Science
• Bioinformatics
• Scientific Publishing

Background:

• Scientific articles contain crucial methodological information.
• Extracting this information manually is time-consuming and prone to errors.
• Automated methods are needed to efficiently process large volumes of research.

Purpose of the Study:

• Introduce and evaluate the study.character module for extracting study characteristics from NISO-JATS coded articles.
• Assess the module's accuracy in identifying key methodological features.
• Demonstrate the module's utility for large-scale analysis and systematic reviews.

Main Methods:

• The study.character module processes text from method and results sections.
• It employs heuristic-driven extraction procedures.
• Performance is evaluated against a manually curated dataset of 288 PDF articles.

Main Results:

• study.character reliably extracts methodological study characteristics from psychological research articles.
• High accuracy and low false positive rates were observed for most extractions.
• Non-detections were primarily due to PDF conversion errors and complex text structures.

Conclusions:

• The study.character module is a valuable tool for automated extraction of methodological study characteristics.
• It facilitates the examination of methodological trends and aids in identifying study sets for meta-analyses and systematic reviews.
• Further development may address challenges related to PDF conversion and complex text structures.