Distilling Reasoning Ability From Large Language Models With Adaptive Thinking.

Area of Science:

• Artificial Intelligence
• Natural Language Processing
• Machine Learning

Background:

• Chain-of-thought distillation (CoT-distillation) enhances small language models (SLMs) by mimicking large language model (LLM) reasoning.
• Existing methods often use pre-thinking (rationale before answer), which can make answer correctness sensitive to rationale errors.

Purpose of the Study:

• To develop a more robust CoT-distillation method for SLMs.
• To improve SLM performance and efficiency by addressing limitations of pre-thinking mechanisms.

Main Methods:

• Proposed a post-thinking mechanism where SLMs generate answers before rationales.
• Introduced an adaptive-thinking mechanism with a perception module (soft prompt tuning) to dynamically switch between pre-thinking and post-thinking based on question complexity.

Main Results:

• The post-thinking approach mitigates answer sensitivity to rationale errors.
• Rationale generation acts as an error amplifier, focusing SLM learning on challenging samples.
• The adaptive-thinking mechanism integrates the benefits of both pre-thinking and post-thinking, improving overall performance.

Conclusions:

• The proposed post-thinking and adaptive-thinking mechanisms offer a robust and efficient approach to CoT-distillation for SLMs.
• Experimental results across 12 datasets validate the effectiveness of the novel methods.