The dataset for extending EMNIST evaluation.

Area of Science:

• Computer Science
• Machine Learning
• Pattern Recognition

Background:

• Handwritten character recognition is crucial for digitizing historical documents and improving user interfaces.
• Existing datasets like EMNIST-letters and NIST databases are widely used but may have limitations for comprehensive model evaluation.
• Current evaluation methods often rely on cross-validation, potentially overestimating model generalization.

Purpose of the Study:

• To introduce a novel dataset for a more rigorous evaluation of machine learning models in handwritten character recognition.
• To facilitate deeper analysis of models trained on EMNIST-letters and NIST data.
• To propose an independent evaluation framework to assess model robustness.

Main Methods:

• Development of a new, independently constructed dataset for handwritten character recognition.
• Compilation and summary of popular machine learning models trained on the EMNIST-letters dataset.
• Comparative analysis of model performance using traditional cross-validation and the new dataset.

Main Results:

• The new dataset provides a complementary resource for evaluating handwritten character recognition models.
• Performance evaluation highlights potential discrepancies between cross-validation results and performance on independent data.
• The study identifies areas for improvement in current model evaluation practices.

Conclusions:

• A new dataset is essential for a more thorough evaluation of handwritten character recognition models.
• Independent data evaluation is critical for understanding true model generalization capabilities.
• The dataset and source code are publicly available for further research.