Research on fine-tuning algorithms for Large Language Models integrating Uncertainty Modeling and External Memory...

Area of Science:

• Natural Language Processing
• Machine Learning
• Artificial Intelligence

Background:

• Fine-tuning large language models is computationally expensive.
• Existing methods often struggle with robustness and contextual completeness.
• Improving confidence calibration and reducing noise influence are critical.

Purpose of the Study:

• To propose a parameter-efficient fine-tuning framework.
• To enhance robustness, confidence calibration, and contextual completeness in NLP tasks.
• To provide a stable and efficient fine-tuning paradigm.

Main Methods:

• Integrating uncertainty modeling for feature-level estimation and cross-layer propagation.
• Employing external memory augmentation with key-value retrieval and gated fusion.
• Utilizing GPT-2 Small, GPT-2 Medium, and LLaMA3-8B as backbone models.

Main Results:

• The proposed framework consistently outperforms mainstream fine-tuning methods in accuracy and F1 score.
• Demonstrated improved robustness under learning-rate sensitivity and missing-information settings.
• Achieved stable performance across text classification and named entity recognition tasks.

Conclusions:

• The framework offers a novel, efficient, and interpretable approach to fine-tuning.
• It achieves a favorable balance between performance, parameter efficiency, and deployment feasibility.
• Provides a practical basis for future extensions to complex NLP scenarios.