関連する実験動画
Updated: Sep 9, 2025

13:19
Deep Neural Networks for Image-Based Dietary Assessment
Published on: March 13, 2021
9.3K
増量学習におけるソフトマックスの再考
Zheng Zhai1, Jiali Zhang2, Haiyu Wang3
1Department of Statistics, Faculty of Arts and Sciences, Beijing Normal University, Zhuhai, Guangdong, China.
まとめ
この研究は,新しい蒸留損失を導入することによって,漸進的な学習における壊滅的な忘却に対処します. 機械学習モデルの 精度を向上させ 忘却を軽減します
科学分野:
- 機械学習
- 人工知能
- 深層学習
背景:
- 忘却は,新しいデータで訓練されたときに,以前学習した情報をモデルが忘れるようにする,インクリメンタル・ラーニングの主要な障害です.
- 標準的なソフトマックスクロスエントロピー蒸留損失は識別不能であり,効果的なインクリメンタル学習を妨げています.
研究 の 目的:
- 漸進的な学習における壊滅的な忘却を緩和するための新しい戦略を提案する.
- ソフトマックスクロスエントロピー蒸留損失の特定できない問題に対処する.
主な方法:
- 蒸留中の不均衡の体重を相殺するために不均衡不変蒸留損失を導入した.
- 定期的な予測/蒸留損失で,シフトに敏感な代替手段で問題を特定する.
- LWF,LWM,LUCIRなどの既存のフレームワークに統合した5つの新しいアプローチを開発しました.
主要な成果:
- 複数のインクリメンタル・ラーニング・フレームワークで予測精度が一貫して向上しています.
- 大規模な数値実験で 忘却率の大幅な減少を達成しました
- CIFAR-100では平均精度が11%以上向上し,LWF,LWM,LUCIRでは忘却が16%以上減少しました.
結論:
- 提案された戦略は 漸進的な学習における 壊滅的な忘却を効果的に緩和します
- 新しいアプローチは,蒸留ベースのインクリメンタル・ラーニングのパフォーマンスを高めます.
- この研究は,より堅固なインクリメンタル・ラーニング・システムを構築するための実用的な解決策を提供します.
関連する概念動画
Residuals and Least-Squares Property
7.8K
The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
The process of fitting the best-fit...
7.8K
Multi-input and Multi-variable systems
149
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
149
Associative Learning
569
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
569
Observational Learning
310
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
310
Improving Translational Accuracy
11.8K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.8K
Regression Toward the Mean
6.5K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.5K
