U-SplitDoRA: an improved privacy-preserved U-shaped split parameter-efficient fine-tuning framework through weight...

As large language models (LLMs) are getting bigger with respect to the parameter count, ranging from a few million to billions, methods like parameter-efficient fine-tuning (PEFT) have emerged as a crucial approach for adapting these LLMs, such as GPT, Llama, and DeepSeek, to resource-constrained and privacy-sensitive environments. The robustness of large language models (LLMs) while operating on complex tasks and with large datasets makes them feasible for various application domains. This also demands the availability of more public datasets to train LLMs in the future. The federated learning (FL) technique, where several entities collaboratively train a machine learning model without sharing their data, is a widely adopted decentralized training framework. This is followed by a central server, which aggregates the models to create a global model. FL LLM fine-tuning has gained attention recently to overcome the aforementioned training data scarcity issue. LLMs are collaboratively fine-tuned by several data owners without disclosing their private data. The large number of trainable parameters has a direct effect on training such complex models on the client side. The split learning technique, through model partitioning, solves the training overhead by offloading certain training tasks to the server side. Previous research based on the split learning approach for FL LLM fine-tuning, namely SplitLoRA and HSpliLoRA, sets the foundation for further research in this direction. Frameworks like SplitLoRA have already enabled collaborative fine-tuning through model partitioning, but privacy preservation and adaptation quality remain open research challenges. U-SplitDoRA-an improved privacy preserved U-shaped split parameter-efficient fine-tuning framework through weight decomposition for large language models-is proposed. U-SplitDoRA harnesses the parallelization power of FL through the split learning approach, using weight-decomposed low-rank adaptation (DoRA) as the PEFT technique. To further address privacy concerns, the U-shaped paradigm is adapted while splitting the model. By partitioning the model into three parts (head, body, and tail), with the head and tail remaining on the client side while the body is on the server side, it ensures that neither raw data nor labels are exposed to the server, thus providing strong privacy. Additionally, replacing low-rank adaptation (LoRA) with DoRA as the PEFT method further enhances adaptation, as it updates both the magnitude and direction of weights, resulting in superior expressiveness and reducing the gap between PEFT fine-tuning and full parameter fine-tuning to a minimal margin. Experiments are conducted using GPT-2-S and GPT-2-M trained on the E2E benchmark dataset. The simulation results confirm that U-SplitDoRA attains better accuracy scores and convergence speed than other SOTA LLM fine-tuning frameworks. Thus, the proposed method addresses key gaps in privacy and adaptation quality, paving the way for efficient, robust, and privacy-preserving fine-tuning of LLM models in a distributed setting.