Constructing Chinese taxonomy trees from understanding and generative pretrained language models.

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Knowledge Representation

Background:

• Hypernym taxonomic trees are crucial for understanding lexical relationships in NLP.
• Existing methods for constructing these trees, especially in Chinese, face challenges in accuracy and efficiency.
• Pre-trained models and generative large language models offer potential for improvement.

Purpose of the Study:

• To develop and evaluate a novel method, CHRRM, for constructing hypernym taxonomy trees in the Chinese language domain.
• To enhance the effectiveness of hypernym relationship extraction using pre-trained models and generative large language models.
• To analyze the impact of hyperparameter optimization and generative model annotation on taxonomy construction accuracy.

Main Methods:

• Utilizing pre-trained models (e.g., Roberta-wwm-ext-large) for initial hypernym relationship prediction.
• Forming a maximum spanning tree from predicted relationships to construct the taxonomy.
• Employing generative large language models (e.g., ChatGPT, ChatGLM) for word annotation to improve relationship identification.
• Optimizing hyperparameters for pre-trained models specific to taxonomy construction.

Main Results:

• The CHRRM model achieved a significant relative improvement of 15.67% in hypernym taxonomy construction.
• An F1 score of 67.9 was obtained on the Chinese WORDNET validation dataset, an increase from the previous 58.7.
• Roberta-wwm-ext-large demonstrated superior performance in taxonomic tree construction.
• Generative large language models showed promise in enhancing accuracy but have limitations in quality and resources.

Conclusions:

• The CHRRM model, leveraging optimized pre-trained models and generative LLMs, effectively constructs Chinese hypernym taxonomy trees.
• Generative large language models can improve downstream Natural Language Understanding (NLU) tasks, though resource constraints need consideration.
• The study validates the feasibility and benefits of integrating generative LLMs into NLP tasks like taxonomy construction.