On-device query intent prediction with lightweight LLMs to support ubiquitous conversations

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Human-Computer Interaction

Background:

• Current Conversational Agents (CAs) rely on rigid, rule-based dialogue models, limiting flexibility and scalability.
• Large Language Models (LLMs) offer advanced capabilities but often require cloud deployment, raising privacy concerns for end-users.
• On-device LLMs are needed for personalized, ubiquitous, and privacy-preserving conversational AI.

Purpose of the Study:

• To investigate the fine-tuning of lightweight pre-trained LLMs for on-device intent prediction in CAs.
• To evaluate LLM performance, memory footprint, and privacy implications for mobile and edge applications.
• To identify optimal LLM architectures balancing performance and resource constraints.

Main Methods:

• Leveraging transfer learning to fine-tune lightweight pre-trained LLMs, specifically RoBERTa and XLNet.
• Focusing on user query intent prediction for CA dialogue management.
• Conducting experiments to assess model performance against benchmarks like ChatGPT and evaluate memory usage.

Main Results:

• Fine-tuned RoBERTa and XLNet models demonstrate strong intent prediction capabilities for on-device CAs.
• These models achieve performance on par with ChatGPT, despite their significantly smaller size.
• The selected LLMs offer a favorable trade-off between performance, memory footprint, and suitability for on-device deployment.

Conclusions:

• Lightweight LLMs, when fine-tuned, are highly suitable for on-device Conversational Agents, addressing privacy and scalability limitations.
• RoBERTa and XLNet present a practical solution for developing personalized, privacy-preserving CAs.
• This research provides valuable insights for stakeholders on balancing LLM capabilities with deployment constraints.