通过小型模型改进大型模型:降低成本和更好的性能
Dong Chen1, Fei Gao1, Shuo Zhang2
1School of Computer and Artificial Intelligence, Zhengzhou University, Zhengzhou, 450000, China.
概括
数据切换+ (DS+) 允许小型和大型模型协作,降低成本和提高性能. 这种范式有效地路由查询,提高预训练的大型模型 (PLM) 的效率.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 自然语言处理自然语言处理.
背景情况:
- 预训练的大型模型 (PLM) 提供了高性能,但会带来大量的计算成本.
- 对PLM的付费API的依赖加剧了产品团队的经济负担.
- 小模型可以在特定的数据分布或更简单的子任务中有效.
研究的目的:
- 引入数据切换+ (DS+),这是一个新的模式,用于小型和大型人工智能模型之间的有效协作.
- 展示DS+如何降低与使用PLM相关的经济负担.
- 为了证明DS+可以通过优化任务分配来提高大型模型的性能.
主要方法:
- 为各种任务培训小型模型,并评估他们的信心水平.
- 开发基于小模型信任分数的查询路由机制.
- 实施一个协作框架,小模型处理更简单的查询,大模型处理复杂的查询.
主要成果:
- DS+显著降低了使用大型模型的成本,情绪分析成本降低了31.18%.
- DS+提高了准确性,在亚马逊产品情绪分析中达到95.64%,而ChatGPT的94.43%.
- 在注入特定任务知识的过程中,DS+的协作方法比传统的微调更有效.
结论:
- DS+提供了一种具有成本效益和性能提升的解决方案,用于利用PLM.
- 该范式展示了人工智能中混合型小型大型模型系统的潜力.
- 与微调相比,DS+提供了一种优越的方法来将任务特定的知识集成到PLM中.
相关概念视频
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
271
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
271
Typical Model Studies
610
Fluid mechanics model studies often utilize scaled-down systems to predict fluid behavior in full-scale environments, such as river flows, dam spillways, and structures interacting with open surfaces. Maintaining Froude number similarity in river models is crucial, as it replicates surface flow features like wave patterns and velocities.
610
Improving Translational Accuracy
14.0K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
14.0K
Improving Translational Accuracy
3.5K
3.5K
Reducing Line Loss
351
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss in...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss in...
351
Modeling and Similitude
598
Scaled modeling is a fundamental technique in engineering, enabling the study of large and complex systems by creating smaller, manageable replicas that recreate critical characteristics of the original. In hydrology and civil infrastructure, for example, scaled models of dams help analyze water flow, turbulence, and pressure. This method allows for accurate predictions of real-world behavior within a controlled environment, significantly reducing the cost and time involved in full-scale...
598


