Data-driven classification of the certainty of scholarly assertions

Area of Science:

• Linguistics
• Computer Science
• Scholarly Communication

Background:

• Scholarly writing uses grammatical structures to indicate certainty.
• Existing certainty categorization systems lack objective validation regarding reader interpretation.
• Automated text analysis often misses subtle linguistic cues of certainty.

Purpose of the Study:

• To objectively test the validity of different scholarly certainty classification systems.
• To determine how researchers categorize assertions based on certainty.
• To develop an automated method for detecting and representing certainty in scholarly text.

Main Methods:

• Administered questionnaires to researchers to classify scholarly assertions.
• Utilized three distinct certainty classification systems for analysis.
• Developed and validated a machine learning model for automated certainty detection.

Main Results:

• Identified three distinct categories of certainty along a spectrum from high to low.
• Achieved high accuracy in automated certainty detection: 89.2% (author-annotated) and 82.2% (publicly-annotated).
• Demonstrated the feasibility of embedding certainty as metadata in machine-accessible formats like Nanopublications.

Conclusions:

• Reader interpretation of scholarly certainty can be objectively categorized.
• Machine learning models can accurately detect and classify linguistic certainty cues.
• Integrating certainty metadata into text-mining enhances scholarly information extraction.