WLR: Well-conditioned linear reconstruction for retraining-free pruning of LLMs

Structured pruning is widely recognized as an effective method for reducing the size and computational demands of large language models (LLMs). Traditionally, structured pruning follows a pretrain-prune-retrain framework. Given the high cost of retraining LLMs, recent research has focused on efficient compensation methods to replace the retraining phase. However, many compensation methods introduce additional parameters into the pruned model, which affect deployment and inference. Additionally, many compensation techniques overlook the issue of ill-conditioning caused by outliers in LLMs during the process of solving optimization problems, leading to numerical instability and inefficiency in solutions. To overcome these challenges, we propose Well-conditioned Linear Reconstruction (WLR), a structured pruning compensation method that avoids introducing extra parameters. WLR reconstructs pruned layers using a linear combination of preserved channels while addressing the issue of ill-conditioning during the process. We evaluated our method on LLaMA-V1/V2/V3 and OPT families across multiple language tasks, achieving performance that surpasses the state-of-the-art methods.