Unraveling LoRA Interference: Orthogonal Subspaces for Robust Model Merging

Area of Science:

• Artificial Intelligence
• Machine Learning
• Natural Language Processing

Background:

• Fine-tuning large language models (LMs) for specific tasks improves performance but incurs high deployment and storage costs.
• Model merging aims to combine multiple task-specific models into a single multi-task model without retraining.
• Existing merging techniques struggle with models fine-tuned using low-rank adaptation (LoRA), often leading to performance degradation.

Purpose of the Study:

• To address the performance degradation issues when merging LoRA-fine-tuned models.
• To propose a novel method that enables robust merging of LoRA models by considering the interplay between model parameters and data distributions.
• To enhance the efficiency and effectiveness of creating multi-task models from individual task-specific models.

Main Methods:

• Proposed Orthogonal Subspaces for Robust model Merging (OSRM) to constrain the LoRA subspace before fine-tuning.
• Ensured that task-specific updates do not negatively impact other tasks.
• Integrated OSRM with existing merging algorithms to minimize interference between tasks.

Main Results:

• OSRM significantly boosts the performance of merged models compared to existing methods.
• The proposed method successfully preserves single-task accuracy after merging.
• Experiments across various datasets and LMs demonstrated OSRM's effectiveness and robustness to merging hyperparameters.
• OSRM showed improved robustness to merging hyperparameters.

Conclusions:

• The interplay between data and parameters is crucial for effective model merging, especially with LoRA.
• OSRM offers a plug-and-play solution for merging LoRA-fine-tuned models, enhancing multi-task learning capabilities.
• The method provides a practical approach to reduce deployment costs while maintaining high performance across multiple tasks.