通过快速工程和强大的评估,使用大型语言模型生成可靠的软件项目任务流
Mohammed Sarim1, Faraz Masood1, Manas Maheshwari1
1Department of Computer Science, Aligarh Muslim University, Aligarh, Uttar Pradesh, 202002, India.
Scientific reports
|October 8, 2025
概括
大型语言模型 (LLM) 可以将软件文档转换为任务流. 一个新的指标表明,即使是基本的提示也能为人工智能驱动的软件规划带来可靠的结果.
科学领域:
- 人工智能的人工智能
- 软件工程 软件工程 软件工程
- 自然语言处理自然语言处理.
背景情况:
- 大型语言模型 (LLM) 显示了将非结构化的软件文档转化为结构化的任务流的潜力.
- 然而,LLM产生的输出往往缺乏软件工程任务所必需的程序可靠性.
研究的目的:
- 通过使用各种提示策略,对领先的LLM (Gemini 2.5 Pro,Grok 3,GPT-Omni,DeepSeek-R1,LLaMA-3) 进行基准评估.
- 引入和验证一种新的评估指标,即混合语义相似度指标 (HSSM),用于评估程序可靠性.
主要方法:
- 从"构建你自己的X"库中利用现实世界的软件教程进行基准测试.
- 实施了五种提示策略:零射击,思想链和ISO 21502指导.
- 开发了HSSM,结合了句子转换器嵌入和上下文意识的关键术语重叠,用于语义和程序评估.
主要成果:
- 与传统指标 (BERTScore,SBERT,USE) 相比,HSSM表现优越,差异较低 (1.5-2.9% CV),与人类判断的相关性更高.
- 即使是Zero-Shot提示也实现了高对齐 (96.33% HSSM) 任务流生成,当与HSSM评估时.
- 根据提示策略和模型架构,LLM表现不同.
结论:
- 该研究提供了一个可扩展的框架,用于评估软件工程中LLM生成的任务流.
- HSSM提供了一种强大的方法来评估程序一致性,这对于可靠的AI辅助软件规划至关重要.
- 调查结果表明,在人工智能驱动的项目管理,快速工程和程序生成工具中,LLM的潜力很大.
相关概念视频
Improving Translational Accuracy
3.5K
3.5K
Improving Translational Accuracy
14.0K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
14.0K
Distribution Reliability and Automation
469
Distribution reliability in electrical power systems is critical for ensuring an uninterrupted power supply to consumers at minimal cost. According to IEEE Standard Terms, reliability is the probability that a device will function without failure over a specified time period or amount of usage. For electric power distribution, this translates to maintaining continuous power supply and addressing customer concerns over power outages. Several indices, as defined by IEEE Standard 1366-2012, are...
469
Quantifying Work
23.8K
As a system undergoes a change, its internal energy can change, and energy can be transferred from the system to the surroundings, or from the surroundings to the system.
23.8K
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
253
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
253
Laminar Flow: Problem Solving
476
Laminar flow occurs when a fluid moves smoothly in parallel layers with minimal mixing and turbulence. In fluid mechanics, ensuring laminar flow within a pipe is essential for precise control of flow characteristics, especially in engineering applications. The key factor in determining whether flow remains laminar is the Reynolds number, a dimensionless quantity that depends on the fluid's velocity, density, viscosity, and the pipe's diameter. A Reynolds number of 2100 or lower...
476

