Template-Based Contrastive Distillation Pretraining for Math Word Problem Solving.

Area of Science:

• Natural Language Processing
• Artificial Intelligence
• Computational Linguistics

Background:

• Math word problem (MWP) solving requires understanding narrative, identifying quantifiers/variables, and mapping them to solution equations.
• Existing deep learning models for MWPs often overlook the crucial grounding equation logic.
• Pretrained language models (PLMs) possess rich knowledge and semantic representations beneficial for MWP solving but remain underexplored in this domain.

Purpose of the Study:

• To develop a novel approach for MWP solving that incorporates both mathematical logic and real-world knowledge.
• To leverage the power of PLMs for MWP solving by addressing their limitations in capturing equation logic.
• To introduce a pretrained encoder, MathEncoder, designed to enhance MWP solver performance.

Main Methods:

• Proposed a template-based contrastive distillation pretraining (TCDP) approach using a PLM-based encoder.
• Incorporated mathematical logic knowledge via multiview contrastive learning on symbolic solution templates.
• Retained rich real-world knowledge and semantic representations through knowledge distillation from a teacher PLM.

Main Results:

• Developed MathEncoder, a pretrained PLM-based encoder specifically for MWP solving.
• Constructed MathSolver by integrating MathEncoder into a state-of-the-art MWP solver (GTS).
• Achieved superior performance over existing methods on the Math23K and CM17K benchmarks, demonstrating enhanced MWP understanding.

Conclusions:

• The TCDP approach effectively integrates mathematical logic and real-world knowledge into PLMs for MWP solving.
• MathEncoder represents a significant advancement in MWP solver capabilities.
• The proposed method sets a new benchmark for MWP solving performance.