AF-CuRL:在边缘智能系统中用于资源受限的长形式推理的稳定增强学习
Ziqin Yan1,2, Yurong Wang2,3, Qingsheng Yue1
1School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China.
Sensors (Basel, Switzerland)
|March 14, 2026
概括
我们介绍了以答案为中心的课程强化学习 (AF-CuRL),这是对资源有限的智能系统的稳定框架. AF-CuRL通过专注于关键奖励和使用课程时间表来增强长型推理,提高决策准确性和生成规律性.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 智能系统 智能系统
背景情况:
- 智能系统在计算约束下需要长形式的推理.
- 决策生成的强化学习 (RL) 由于奖励稀疏性和信用分配问题,在资源较少的环境中不稳定.
- 现有的方法往往导致非融合或多词生成.
研究的目的:
- 提出AF-CuRL (以答案为中心的课程强化学习),一个轻量级的RL框架.
- 为了稳定资源有限的智能系统的长形式生成,而不会增加模型大小或计算成本.
- 改进RL中的优化学习能力,用于决策生成任务.
主要方法:
- 开发了AF-CuRL,以响应为重点的代币重权,以将政策更新集中在奖励关键序列区域上.
- 实施了两阶段的课程奖励计划,优先考虑稳定的终止和正确性之前的输出规律性.
- 在1.5B参数语言模型上评估AF-CuRL,在受约束的培训环境下使用数学推理任务.
主要成果:
- 与标准RL基线相比,AF-CuRL在决策准确性和生成规律性方面取得了持续的改进.
- 在实验中观察到更高的终结可靠性和更短的生成长度.
- 在稳定资源有限的智能系统的长形式生成方面表现出有效性.
结论:
- 在RL中结构化的目标设计比模型缩放更有效,在资源有限的系统中实现稳定,高效的长形式推理.
- AF-CuRL为在现实世界限制下运行的智能系统提供了实用的RL解决方案.
- 拟议的框架解决了边缘和嵌入式环境中奖励稀疏性和信用分配的挑战.
相关概念视频
Reasoning
494
Reasoning is the action of thinking about something in a logical, sensible way. It is integral to problem-solving, decision-making, and critical thinking. Reasoning can be inductive or deductive. Reasoning involves transforming information into conclusions, which is essential for problem-solving, decision-making, and critical thinking.
Inductive reasoning involves deriving generalizations from specific observations. This type of reasoning helps form beliefs about the world. For example,...
Inductive reasoning involves deriving generalizations from specific observations. This type of reasoning helps form beliefs about the world. For example,...
494
Stability of Equilibrium Configuration: Problem Solving
1.1K
The stability of equilibrium configurations is an important concept in physics, engineering, and other related fields. In simple terms, it refers to the tendency of an object or system to return to its equilibrium position after being disturbed. The stability of an equilibrium configuration can be analyzed by considering the potential energy function of the system and examining its behavior near the equilibrium point.
Problem-solving in the context of the stability of equilibrium configuration...
Problem-solving in the context of the stability of equilibrium configuration...
1.1K
Reason and Intuition
7.7K
The human brain processes information for decision-making using one of two routes: an intuitive system and a rational system (Epstein, 1994; popularized by Kahneman, 2011 as System 1 and System 2, respectively). The intuitive system is quick, impulsive, and operates with minimal effort, relying on emotions or habits to provide cues for what to do next, while the rational system is logical, analytical, deliberate, and methodical. Research in neuropsychology suggests that the...
7.7K
Statically Indeterminate Problem Solving
809
Statically indeterminate problems are those where statics alone can not determine the internal forces or reactions. Consider a structure comprising two cylindrical rods made of steel and brass. These rods are joined at point B and restrained by rigid supports at points A and C. Now, the reactions at points A and C and the deflection at point B are to be determined. This rod structure is classified as statically indeterminate as the structure has more supports than are necessary for maintaining...
809
Inductive Reasoning
69.0K
Inductive reasoning is a form of logical thinking that uses related observations to arrive at a general conclusion. It is uncertain and operates in degrees to which the conclusions are credible. As such, inductive arguments can be weak or strong, rather than valid or invalid, and conclusions can be used to formulate testable, falsifiable hypotheses.
Inductive reasoning is common in descriptive science. A life scientist makes observations and records them. This data can be qualitative or...
Inductive reasoning is common in descriptive science. A life scientist makes observations and records them. This data can be qualitative or...
69.0K
Observational Learning
1.2K
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
1.2K
