通过强化学习来增强代码生成的查询
Dawei Yuan1, Guojun Liang2, Tingting Li3
1School of Computer Science, Guangdong University of Science and Technology, Dongguan, 523083, China. yuandawei@gdust.edu.cn.
Scientific reports
|October 24, 2025
概括
我们开发了一个强化学习框架 (RL4QE) 来增强自然语言查询,以改进DeepSeek代码生成. 这种方法通过使用文本和执行奖励,将代码相似性提高了34.3%.
科学领域:
- 人工智能的人工智能
- 自然语言处理自然语言处理.
- 软件工程 软件工程 软件工程
背景情况:
- 代码生成的深度学习模型经常与细微的自然语言查询作斗争.
- 提高提示符的语义理解对于生成准确和功能性的代码至关重要.
研究的目的:
- 引入一个强化学习框架 (RL4QE) 来增强自然语言查询,以改善代码生成.
- 在DS1000基准上评估RL4QE的有效性,并分析奖励组件.
主要方法:
- 使用REINFORCE算法训练了一个参数精炼器 (使用LoRA的Qwen).
- 一个标量奖励函数将文本相似度指标 (BLEU-4,ROUGE-L,F1,重叠) 与执行信号 (单元测试,语法/时限处罚) 结合起来.
- 在训练期间,发电机模型 (DeepSeek) 保持不变.
主要成果:
- 在DS1000基准上,RL4QE在代码相似性方面取得了34.3%的改进.
- BLEU-4被认为是最可靠的文本奖励指标.
- 与完全微调相比,低级调整 (LoRA) 显示出更高的性能和参数效率.
- 该框架显示了不同基础模型的可转移性,突出了建筑的重要性.
结论:
- 强化学习有效地提高了代码生成任务的自然语言查询.
- RL4QE提供了一个参数高效和可重复的方法来提高代码生成质量.
- 该框架的灵活性允许与各种基础模型进行集成.
相关概念视频
Reinforcement
826
Positive and negative reinforcement are key concepts in operant conditioning, a learning process where the consequences of a behavior affect the likelihood of that behavior being repeated.
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
Positive reinforcement occurs when a behavior is followed by the presentation of a rewarding stimulus, increasing the frequency of that behavior. For example:
826
Reinforcement Schedules
453
Positive reinforcement is a powerful method for teaching new behaviors to both animals and humans. B.F. Skinner demonstrated this with his experiments using rats in a Skinner box. When a rat pressed a lever, it received a food pellet. This immediate reward encouraged the rat to repeat the behavior. This method, where a reward follows every instance of the behavior, is known as continuous reinforcement. It is highly effective for establishing new behaviors quickly.
Once a behavior is learned,...
Once a behavior is learned,...
453
Observational Learning
824
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
824
Associative Learning
1.2K
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
1.2K
Woodward–Hoffmann Selection Rules and Microscopic Reversibility
3.8K
Electrocyclic reactions, cycloadditions, and sigmatropic rearrangements are concerted pericyclic reactions that proceed via a cyclic transition state. These reactions are stereospecific and regioselective. The stereochemistry of the products depends on the symmetry characteristics of the interacting orbitals and the reaction conditions. Accordingly, pericyclic reactions are classified as either symmetry-allowed or symmetry-forbidden. Woodward and Hoffmann presented the selection criteria for...
3.8K
Heuristics
644
Heuristics are problem-solving strategies that use mental shortcuts to simplify decision-making. Unlike algorithms, which must be followed precisely to achieve a correct result, heuristics offer a general problem-solving framework. They save time and energy but can sometimes lead to less rational decisions.
People often rely on heuristics when faced with an overload of information, limited time, low importance of the decision, limited information, or when a heuristic readily comes to mind. For...
People often rely on heuristics when faced with an overload of information, limited time, low importance of the decision, limited information, or when a heuristic readily comes to mind. For...
644
