Building a Korean morphological analyzer using two Korean BERT models.

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Artificial Intelligence

Background:

• Morphological analysis is crucial for understanding Korean word structure.
• Traditional analyzers face challenges with varying input/output lengths.
• Encoder-decoder architectures are suitable for sequence-to-sequence tasks like morphological analysis.

Purpose of the Study:

• To implement a Korean morphological analyzer using the Transformer architecture.
• To investigate the effectiveness of Bidirectional Encoder Representations from Transformers (BERT) for initialization.
• To address the limitations of insufficient training data and long training times.

Main Methods:

• Utilized a Transformer architecture, an encoder-decoder model with self-attention.
• Initialized both encoder and decoder with two types of Korean BERT models (raw corpus and morphologically analyzed dataset).
• Fine-tuned the Transformer model on a relatively small corpus.

Main Results:

• Parameter initialization with pretrained BERT models alleviated the lack of training data.
• Training time was significantly reduced compared to training from scratch.
• Determined optimal encoder and decoder layer configurations for performance enhancement.

Conclusions:

• Transformer-based models, initialized with BERT, offer an effective solution for Korean morphological analysis.
• Pretrained models are vital for overcoming data scarcity and improving training efficiency.
• Optimizing model architecture (layer count) further enhances analyzer performance.