A large-scaled corpus for assessing text readability

Area of Science:

• Computational Linguistics
• Educational Technology
• Psycholinguistics

Background:

• Readability metrics are crucial for understanding text complexity.
• Existing corpora may lack sufficient size, scope, or relevant metadata for advanced research.
• Developing robust readability models requires diverse and well-annotated text datasets.

Purpose of the Study:

• Introduce the CommonLit Ease of Readability (CLEAR) corpus.
• Provide researchers with a comprehensive resource for studying text readability and discourse processing.
• Facilitate the development and validation of new readability metrics.

Main Methods:

• Compilation of approximately 5000 text excerpts spanning over 250 years and multiple genres.
• Generation of unique readability scores based on teacher ratings of text difficulty.
• Inclusion of metadata such as publication year and genre for each excerpt.

Main Results:

• The CLEAR corpus contains ~5000 text excerpts with associated readability scores and metadata.
• The corpus covers a wide range of historical periods and genres, offering broad applicability.
• Reliability metrics for the human readability ratings have been established and are presented.

Conclusions:

• The CLEAR corpus represents a significant advancement over previous readability resources.
• It provides a valuable foundation for research in reading comprehension, discourse analysis, and educational technology.
• The corpus enables the development and testing of more accurate and nuanced readability assessment tools.