HiGen: Hierarchy-Aware Sequence Generation for Hierarchical Text Classification.

Area of Science:

• Natural Language Processing
• Machine Learning
• Bioinformatics

Background:

• Hierarchical text classification (HTC) presents challenges due to complex label taxonomies and imbalanced datasets.
• Existing models often rely on static document representations, which may not capture the varying relevance of text sections across different hierarchy levels.

Purpose of the Study:

• To develop a novel text-generation-based framework, HiGen, for dynamic document representation in HTC.
• To improve the performance of HTC models, particularly for classes with limited examples, by adapting language models to in-domain knowledge.

Main Methods:

• Proposed HiGen framework utilizing language models for dynamic text representations.
• Introduced a level-guided loss function to align text and label semantics.
• Implemented a task-specific pretraining strategy for enhanced in-domain adaptation.
• Developed the ENZYME dataset for HTC, focusing on Enzyme Commission (EC) number prediction from PubMed articles.

Main Results:

• HiGen demonstrated superior performance over existing methods on the ENZYME, WOS, and NYT datasets.
• The framework effectively handled data imbalance and mitigated performance issues for classes with few examples.
• Task-specific pretraining significantly boosted performance for under-represented classes.

Conclusions:

• The proposed HiGen framework offers a significant advancement in hierarchical text classification by employing dynamic document representations.
• The ENZYME dataset provides a valuable resource for HTC research, particularly in the bioinformatics domain.
• HiGen's approach is effective in addressing data imbalance and improving model generalization in complex classification tasks.