Recognition of the script in Serbian documents using frequency occurrence and co-occurrence analysis.

Area of Science:

• Computational Linguistics
• Natural Language Processing
• Document Analysis

Background:

• Serbian language documents can utilize either Latin or Cyrillic scripts.
• While visually similar, these scripts exhibit distinct statistical properties.
• Accurate script identification is crucial for text processing and analysis.

Purpose of the Study:

• To develop and evaluate a method for automatically distinguishing between Serbian Latin and Cyrillic scripts.
• To leverage statistical measures of letter occurrence and co-occurrence for script recognition.

Main Methods:

• Modeling individual letters based on their position within the baseline area to determine script type.
• Performing frequency analysis on the occurrence of modeled script types.
• Computing a co-occurrence matrix for script elements.
• Utilizing the co-occurrence matrix as a criterion for script differentiation.

Main Results:

• Significant statistical dissimilarities were observed in the occurrence of modeled letters between Latin and Cyrillic scripts.
• The co-occurrence matrix analysis provided a strong basis for distinguishing between the two scripts.
• Experiments on a diverse database of printed and web documents yielded encouraging results.

Conclusions:

• The proposed method effectively differentiates Serbian Latin and Cyrillic scripts based on statistical script analysis.
• The approach demonstrates high accuracy and robustness across various document types.
• This technique offers a reliable solution for automatic script identification in Serbian texts.