Tool retrieval bridge: Aligning vague instructions with retriever preferences via bridge model

Area of Science:

• Artificial Intelligence
• Natural Language Processing

Background:

• Tool learning enables large language models (LLMs) to tackle real-world problems.
• Efficient tool retrieval is crucial due to the vast and dynamic nature of available tools.
• Existing retrieval methods fail with vague, real-world instructions, unlike academic benchmarks.

Purpose of the Study:

• To address the performance degradation of tool retrieval caused by vague instructions.
• To introduce a novel benchmark (VGToolBench) simulating realistic, ambiguous user queries.
• To propose an effective method for enhancing tool retrieval with vague instructions.

Main Methods:

• Constructed VGToolBench, a benchmark featuring human-like vague instructions.
• Analyzed the impact of vague instructions on current tool retrieval techniques.
• Developed the Tool Retrieval Bridge (TRB), a model to rewrite vague instructions into specific ones.

Main Results:

• Vague instructions significantly impair tool retrieval performance.
• TRB effectively reduces ambiguity in instructions, bridging the gap for retrievers.
• TRB consistently and substantially improves performance across various retrieval methods, e.g., BM25 saw an 111.51% improvement.

Conclusions:

• The proposed TRB approach is effective in handling vague instructions for tool retrieval.
• TRB enhances the practical applicability of LLM tool learning in real-world scenarios.
• The method offers a significant performance boost, making LLM tool use more robust.