AF-CuRL: Stable Reinforcement Learning for Resource-Constrained Long-Form Reasoning in Edge-Intelligent Systems

Area of Science:

• Artificial Intelligence
• Machine Learning
• Intelligent Systems

Background:

• Intelligent systems require long-form reasoning under computational constraints.
• Reinforcement learning (RL) for decision generation is unstable in low-resource settings due to reward sparsity and credit assignment issues.
• Existing methods often lead to non-convergent or verbose generation.

Purpose of the Study:

• To propose AF-CuRL (Answer-Focused Curriculum Reinforcement Learning), a lightweight RL framework.
• To stabilize long-form generation for resource-constrained intelligent systems without increasing model size or computational cost.
• To improve optimization learnability in RL for decision generation tasks.

Main Methods:

• Developed AF-CuRL with answer-focused token reweighting to concentrate policy updates on reward-critical sequence regions.
• Implemented a two-phase curriculum reward schedule prioritizing stable termination and output regularity before correctness.
• Evaluated AF-CuRL on a 1.5B-parameter language model using mathematical reasoning tasks under constrained training settings.

Main Results:

• AF-CuRL demonstrated consistent improvements in decision accuracy and generation regularity compared to standard RL baselines.
• Observed higher termination reliability and reduced generation length in experiments.
• Showcased effectiveness in stabilizing long-form generation for resource-limited intelligent systems.

Conclusions:

• Structured objective design in RL is more effective than model scaling for stable, efficient long-form reasoning in resource-limited systems.
• AF-CuRL provides a practical RL solution for intelligent systems operating under real-world constraints.
• The proposed framework addresses challenges of reward sparsity and credit assignment in edge and embedded environments.