Search research articles

关于 JoVE

概览领导团队博客 JoVE 帮助中心

作者

出版流程编辑委员会范围与政策同行评审常见问题投稿

图书馆员

用户评价订阅访问资源图书馆顾问委员会常见问题

研究

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments 存档

教育

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual 教师资源中心教师网站

使用条款与条件

相关概念视频

Quantifying and Rejecting Outliers: The Grubbs Test

Quantifying and Rejecting Outliers: The Grubbs Test

Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...

Quadratic Models

Quadratic Models

Quadratic models are mathematical representations used to describe relationships in which the rate of change changes at a constant rate. These models appear in a wide variety of natural and engineered systems, especially those involving motion, forces, and optimization. One common application is analyzing the vertical motion of objects influenced by gravity, such as a ball thrown into the air.In such scenarios, the object's height changes over time in a curved pattern, rising to a maximum point...

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...

Language Development

Language Development

Children master language quickly and with relative ease, supported by both biological predisposition and reinforcement. B. F. Skinner (1957) proposed that language is learned through reinforcement, while Noam Chomsky (1965) argued that language acquisition mechanisms are biologically determined.
The critical period for language acquisition suggests that the ability to acquire language is at its peak early in life. As people age, this proficiency decreases. Language development begins very...

Truncation in Survival Analysis

Truncation in Survival Analysis

Truncation in survival analysis refers to the exclusion of individuals or events from the dataset based on specific criteria related to the time of the event. This exclusion can happen in two primary forms: left truncation and right truncation.
Left truncation occurs when individuals who experienced the event of interest before a certain time are not included in the study. This is often due to a "delayed entry" into the study where only those who survive until a certain entry point are...

Outliers and Influential Points

Outliers and Influential Points

An outlier is an observation of data that does not fit the rest of the data. It is sometimes called an extreme value. When you graph an outlier, it will appear not to fit the pattern of the graph. Some outliers are due to mistakes (for example, writing down 50 instead of 500), while others may indicate that something unusual is happening. Outliers are present far from the least squares line in the vertical direction. They have large "errors," where the "error" or residual is the...

您也可能阅读

相关文章

通过共同作者、期刊和引用图与本文相关的文章。

排序

Same author

WLR: Well-conditioned linear reconstruction for retraining-free pruning of LLMs.

Neural networks : the official journal of the International Neural Network Society·2026

Same author

Hyperbolic Binary Neural Network.

IEEE transactions on neural networks and learning systems·2024

Same author

RGP: Neural Network Pruning Through Regular Graph With Edges Swapping.

IEEE transactions on neural networks and learning systems·2023

Same author

[Distribution and differentiation of marrow mesenchymal cells in tumor tissue: experimental with rabbits].

Zhonghua yi xue za zhi·2007

Same author

[The clinical value of measuring plasma level of very long chain fatty acids in Addison disease].

Zhonghua nei ke za zhi·2007

Same author

A readily-prepared, convergent, oxygen reduction electrocatalyst.

Chemical communications (Cambridge, England)·2007

Same journal

A boundary-regularization-enhanced video anomaly detection network based on context-adaptive spatio-temporal conditional diffusion.

Neural networks : the official journal of the International Neural Network Society·2026

Same journal

MT<sup>2</sup>-CSD and LLM-CRAN: A new dataset and an LLM-based multi-semantic knowledge fusion model for conversational stance detection.

Neural networks : the official journal of the International Neural Network Society·2026

Same journal

TriAlignNet: A triple-path cross-modality alignment framework for multimodal time series forecasting.

Neural networks : the official journal of the International Neural Network Society·2026

Same journal

Anchor-based disentanglement framework for incremental multi-view clustering.

Neural networks : the official journal of the International Neural Network Society·2026

Same journal

Complex-valued amplitude-phase interference modeling for adversarially robust classification.

Neural networks : the official journal of the International Neural Network Society·2026

Same journal

TraNce: Type-aware hypergraph neural network with biological mediators for drug repositioning.

Neural networks : the official journal of the International Neural Network Society·2026

查看所有相关文章

Search research articles

相关实验视频

Updated: Jan 9, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

OOPS:用于大型语言模型的基于异常意识和二次编程的结构化修剪.

Jiateng Wei¹, Siqi Li¹, Jingyang Xiang¹

¹Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou, 310027, China.

Neural networks : the official journal of the International Neural Network Society

|December 3, 2025

概括

此摘要是机器生成的。

本研究介绍了OOPS,这是一种用于大型语言模型 (LLM) 的新型结构化修剪框架. OOPS有效地减少模型大小和资源需求,而不会造成性能损失,优于现有方法.

关键词:

大型语言模型.模型的压缩压缩.网络修剪是为了修剪网络.

更多相关视频

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

相关实验视频

Last Updated: Jan 9, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

科学领域:

人工智能的人工智能
机器学习机器学习
自然语言处理自然语言处理.

背景情况:

大型语言模型 (LLM) 面临着部署的挑战,因为它们的大小和资源需求.
结构化的修剪方法旨在减轻这些问题,分为无再培训和基于再培训的方法.
现有的没有再培训的方法往往导致性能下降,而基于再培训的方法是计算密集的.

研究的目的:

为大型语言模型 (LLM) 开发一个高效的结构化修剪框架,尽量减少资源消耗并保持性能.
解决现有的无再培训和基于再培训的修剪技术的局限性.
提出一个新的框架,OOPS (基于Outlier-aware和二进制编程的结构化修剪),可以平衡效率和有效性.

主要方法:

OOPS采用一个由三个组成部分构成的框架:偏差值意识的修剪单元选择,基于二次编程的重建和分层蒸.
偏差值意识的单元选择和二次编程重建使无需重新训练的修剪成为可能.
可选地加入分层蒸来进一步提高性能,特别是在基于再培训的场景中.

主要成果:

与现有的无再培训方法相比,OOPS在不需要再培训的情况下显示出更高的性能.
在结合分层蒸时,OOPS优于基于再培训的方法,并降低了计算成本.
来自4个LLM家族的11个模型和多个任务的评估证实了OOPS的有效性.

结论:

OOPS为大型语言模型的结构化修剪提供了有效和高效的解决方案.
该框架成功地减少了模型大小和资源消耗,同时保持或提高了性能.
OOPS提供了一种多功能的方法,在无再培训和基于再培训的修剪范式中表现出色.