A fine-tuning enhanced RAG system with quantized influence measure as AI judge

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Information Retrieval

Background:

• Retrieval-augmented generation (RAG) systems combine information retrieval with generative models.
• Large language models (LLMs) offer advanced comprehension but require efficient fine-tuning.
• Vector databases enable efficient storage and retrieval of large datasets.

Purpose of the Study:

• To enhance RAG systems by integrating fine-tuned LLMs with vector databases.
• To improve LLM performance and applicability through parameter-efficient fine-tuning and user feedback integration.
• To increase the precision of result selection in RAG systems using a novel AI Judge mechanism.

Main Methods:

• Integration of fine-tuned LLMs (using LoRA and QLoRA methodologies) with vector databases.
• Incorporation of user feedback into the model training process for continuous adaptation.
• Development and application of a Quantized Influence Measure (QIM) as an AI Judge for result selection.

Main Results:

• Demonstrated a framework for advanced RAG systems combining LLMs and vector databases.
• Showcased the effectiveness of LoRA and QLoRA for efficient LLM fine-tuning.
• Validated the utility of user feedback and QIM for enhancing system accuracy and user-centricity.

Conclusions:

• The integrated approach offers a significant advancement in chatbot technology and retrieval systems.
• This research provides a foundation for more sophisticated, precise, and user-centric conversational AI.
• Publicly released dataset, processing package, model, and app to foster community development.