ResVaRe: Parameter-efficient fine-tuning for large language models via cross-layer residual vector adaptation and...

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Machine Learning

Background:

• Full fine-tuning of large language models (LLMs) is costly due to updating all parameters.
• Existing parameter-efficient fine-tuning methods often focus on either attention or feed-forward layers, potentially causing instability.
• Optimizing adaptation and alignment objectives together can lead to instability in current methods.

Purpose of the Study:

• To introduce ResVaRe, a novel parameter-efficient fine-tuning framework.
• To couple residual-enhanced vector adaptation (ResVaRA) in self-attention with residual-enhanced representation editing (ResRED) in feed-forward layers.
• To enable stable and controllable LLM adaptation with reduced computational cost.

Main Methods:

• ResVaRe integrates ResVaRA for capturing task-specific structure in attention layers and ResRED for fine-grained representation editing in feed-forward layers.
• The framework employs cross-layer residual aggregation to coordinate these two adaptation branches.
• A progressive two-phase schedule prioritizes learning attention-side structure before activating feed-forward editing.

Main Results:

• ResVaRe achieved consistent or competitive gains over strong baselines on natural language understanding, instruction-following, and reasoning benchmarks using Llama- and Qwen-family models.
• The method demonstrated significant improvements in alignment on the Chinese Values Alignment Dataset for Llama3-1B and Qwen2.5-0.5B.
• ResVaRe maintained a favorable safety-helpfulness balance while enhancing model alignment.

Conclusions:

• Combining structural adaptation in attention with residual-enhanced representation editing in feed-forward layers offers a practical approach to LLM adaptation.
• ResVaRe provides a stable and controllable method for fine-tuning LLMs with a significantly reduced trainable budget.
• The framework's effectiveness across diverse benchmarks highlights its potential for advancing LLM customization.