A human-centered automated machine learning agent with large language models for multimodal data management and...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Natural Language Processing

Background:

• Automated Machine Learning (AutoML) aims to simplify machine learning model development.
• Current AutoML systems often require technical expertise and structured data, limiting accessibility.
• Large Language Models (LLMs) show promise in understanding and generating code, but their integration into AutoML is nascent.

Purpose of the Study:

• To develop an LLM-driven AI agent for accessible, end-to-end AutoML.
• To enable natural language interaction throughout the entire machine learning workflow.
• To reduce reliance on predefined rules and technical expertise in AutoML.

Main Methods:

• An end-to-end ML pipeline was implemented, including automated data loading, pre-processing, task identification, and model training.
• A novel LLM-based data processing approach was proposed to interpret diverse data formats.
• An adaptive hyperparameter optimization strategy combining LLM knowledge and performance feedback was developed.

Main Results:

• The LLM-driven agent successfully automated the entire ML pipeline via natural language interaction.
• The proposed data processing method handled diverse data formats without manual intervention.
• The adaptive hyperparameter optimization improved search efficiency and model performance.
• Evaluations on 10 diverse datasets showed superior performance compared to traditional AutoML frameworks.

Conclusions:

• The LLM-driven AI agent significantly enhances AutoML accessibility by enabling natural language interaction.
• This approach bridges the gap between user intent and ML implementation, lowering technical barriers.
• The developed methods contribute to a more intuitive and powerful AutoML framework.