对于图形神经网络的多重解释集合蒸
Kang Liu1, Yuqi Zhang1, Shunzhi Yang2
1School of Computer Science, South China Normal University, Guangzhou, 510000, China.
概括
多重解释集团蒸 (MIED) 通过使用多解释器学生模型和新型采样策略来增强图形知识蒸. 这种方法提高了学习效率和概括性,在节点分类任务中超过了现有的方法.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 图形神经网络的神经网络
背景情况:
- 现有的图形知识蒸方法因简单的逻辑对齐而难以吸收有限的"暗知识",导致过度匹配和不完整的模式捕获.
- 单一的学生视角限制了基于图表的任务的学习效率和概括能力.
研究的目的:
- 为改进图形知识蒸引入一种新的多重解释合并蒸 (MIED) 方法.
- 通过实现多样化的知识解释和增强学生模型的稳定性和概括性来解决现有方法的局限性.
主要方法:
- 开发了学生解释 (SI) 组件,一个使用多个单层MLP的多解释器,以从多样化的学生输出中解释知识,减轻表示偏见.
- 引入了混合采样,用于教师 (百分比随机) 和学生/SI组件 (正负) 输出的不同策略,以协调样本选择.
- 实现了层次更新,以提高稳定性和概括性,通过使用指数级移动平均值来对学生的最后一个层参数进行基于SI组件的融合.
主要成果:
- 在七个现实数据集上的节点分类任务中,MIED显著优于现有方法,比图形卷积网络 (GCN) 平均提高5.56%,比多层感知子 (MLP) 平均提高27.43%.
- 与使用多个单个学生相比,MIED实现了可比或更好的准确性,效率明显提高 (6.00%更快,空间减少了50.00%).
结论:
- MIED为图形知识蒸提供了一个可扩展,可泛化和强大的解决方案,在复杂样本上特别有效.
- 提出的方法成功地提高了教师"黑暗知识"的吸收,并提高了学生的模型表现,超出了传统方法.
相关概念视频
01:07Multiple Bar Graph
10.2K
As the name suggests, a multiple bar graph is the same as a bar graph but has multiple bars to depict relationships between different data values. One can include as many parameters as possible. However, each parameter must have the same unit of measurement.
Each bar or column in the multiple bar graph represents a data value. These graphs are used primarily in interrelating two or more sets of data. The categories of different kinds of data are listed along the horizontal or x-axis, whereas...
Each bar or column in the multiple bar graph represents a data value. These graphs are used primarily in interrelating two or more sets of data. The categories of different kinds of data are listed along the horizontal or x-axis, whereas...
10.2K
02:26Protein Networks
4.6K
An organism can have thousands of different proteins, and these proteins must cooperate to ensure the health of an organism. Proteins bind to other proteins and form complexes to carry out their functions. Many proteins interact with multiple other proteins creating a complex network of protein interactions.
These interactions can be represented through maps depicting protein-protein interaction networks, represented as nodes and edges. Nodes are circles that are representative of a protein,...
These interactions can be represented through maps depicting protein-protein interaction networks, represented as nodes and edges. Nodes are circles that are representative of a protein,...
4.6K
01:01Distillation: Vapor–Liquid Equilibria
4.7K
Distillation is a separation technique that takes advantage of the boiling point properties of disparate elements in a mixture. To perform distillation, we begin by heating a miscible mixture of two liquids with a significant difference in boiling points (at least 20°C). As the solution heats up and reaches the bubble point of the more volatile component, some molecules of the more volatile component transition into the gas phase and travel upward into the condenser, which is a glass tube...
4.7K
01:07Ogive Graph
6.9K
An ogive graph is sometimes called a cumulative frequency polygon. It is one type of frequency polygon that shows cumulative frequency. In other words, the cumulative percentages are added to the graph from left to right. An ogive graph plots cumulative frequency on the vertical y-axis and class boundaries along the horizontal x-axis. It’s very similar to a histogram; only instead of rectangles, an ogive displays a single point where the top right of the rectangle would be. Creating this...
6.9K
01:30Graphing Antiderivatives
77
The concept of an antiderivative is fundamental in calculus, describing how a function's values accumulate over time. This process is closely related to physical motion, such as the movement of a rolling ball. As the ball progresses, its position changes in response to variations in velocity, just as an antiderivative graph reflects the cumulative effect of the original function's values.Graphing an antiderivative requires interpreting how a function's values influence the shape of its...
77
01:30Graphs of Functions
360
Graphs of functions provide a visual representation of how output values change in response to varying inputs. Each point on the graph corresponds to an ordered pair, where the x-coordinate (independent variable) determines the horizontal position and the y-coordinate (dependent variable) determines the vertical position. Linear functions like y = x give a straight line, indicating a constant rate of change.Nonlinear functions display more complex behaviors. Even power functions generate...
360