Research on Fine-Tuning Optimization Strategies for Large Language Models in Tabular Data Processing.

Area of Science:

• Natural Language Processing (NLP)
• Machine Learning
• Artificial Intelligence

Background:

• Large language models (LLMs) have advanced NLP but struggle with tabular data.
• Effective processing of structured data is crucial for broader LLM applications.

Purpose of the Study:

• To investigate fine-tuning strategies for optimizing LLMs on tabular data.
• To analyze the impact of decimal truncation, multi-dataset mixing, and JSON key-value pair ordering.

Main Methods:

• Fine-tuning LLMs using specific data preprocessing techniques.
• Evaluating performance based on decimal truncation, multi-dataset mixing, and key-value pair shuffling.

Main Results:

• Decimal truncation reduces noise, improving learning efficiency.
• Multi-dataset mixing enhances generalization and stability.
• Randomizing JSON key-value pair order boosts adaptability to data structure variations.

Conclusions:

• Fine-tuning strategies significantly impact LLM performance and robustness on tabular data.
• Provides practical methods for improving LLM effectiveness in structured data processing.
• Establishes theoretical foundations for future LLM optimization across diverse applications.