不要害怕奇怪的激活功能: EUAF及其他
Qianchao Wang1, Shijun Zhang2, Dong Zeng3
1Center of Mathematical Artificial Intelligence, Department of Mathematics, The Chinese University of Hong Kong, Hong Kong Special Administrative Region of China.
概括
我们介绍了参数基本通用激活函数 (PEUAF),这是一个新的超表达式激活函数. PEUAF在工业数据集上表现出卓越的性能,在图像数据集上表现出具有竞争力的准确性,推动了深度学习激活功能的研究.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
背景情况:
- 超表达式激活函数在近似复杂函数方面提供了理论上的优势.
- 现有的超表达功能在识别,可扩展性和实际应用方面面临挑战.
- 对它们的广泛适用性和特殊形式的有限理解阻碍了它们在现实世界中被采用.
研究的目的:
- 提出一种新的超表达式激活函数,即参数基本通用激活函数 (PEUAF).
- 通过全面的实验来证明PEUAF的有效性和通用性.
- 解决超表达激活功能的开发和应用中的关键瓶问题.
主要方法:
- 参数基本通用激活函数 (PEUAF) 的开发.
- 对工业和图像数据集进行系统和全面的实验评估 (CIFAR-10,Tiny-ImageNet,ImageNet).
- 分析使用PEUAF的模型,包括混合激活功能配置.
主要成果:
- 基于PEUAF的模型在几个基线工业数据集上实现了最高性能.
- 在混合激活设置中结合PEUAF的模型显示在图像数据集上具有竞争力的测试准确性.
- 超表达式激活函数的概括,证明任何连续函数都可以近似.
结论:
- PEUAF是一种有效且实用的超表达激活功能.
- 这项研究克服了识别和应用超表达函数的局限性.
- PEUAF有助于推进深度学习中的超表达式激活函数的开发和现实应用.
相关概念视频
Association Areas of the Cortex
4.9K
Association areas are regions of the cerebral cortex that do not have a specific sensory or motor function. Instead, they integrate and interpret information from various sources to enable higher cognitive processes such as memory, learning, and decision-making. Some key association areas include the following:
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
Prefrontal Association Area: This area is located in the frontal lobe and is involved in planning, decision-making, and moderating social behavior. It connects with primary motor areas,...
4.9K
Basic Continuous Time Signals
181
Basic continuous-time signals include the unit step function, unit impulse function, and unit ramp function, collectively referred to as singularity functions. Singularity functions are characterized by discontinuities or discontinuous derivatives.
The unit step function, denoted u(t), is zero for negative time values and one for positive time values, exhibiting a discontinuity at t=0. This function often represents abrupt changes, such as the step voltage introduced when turning a car's...
The unit step function, denoted u(t), is zero for negative time values and one for positive time values, exhibiting a discontinuity at t=0. This function often represents abrupt changes, such as the step voltage introduced when turning a car's...
181
Atomic Force Microscopy
3.3K
Atomic force microscopy (AFM) is a type of scanning probe microscopy that can analyze topographic details of various specimens like ceramics, glass, polymers, and biological samples. AFM offers over 1000 times more resolution than the optical imaging system. Images generated from AFM are three-dimensional surface profiles, offering an advantage over the flat, two-dimensional images from other imaging techniques.
The AFM Probe
The probe is regarded as the heart of any AFM setup and comprises the...
The AFM Probe
The probe is regarded as the heart of any AFM setup and comprises the...
3.3K
Basic signals of Fourier Transform
465
The Fourier Transform is a pivotal mathematical tool in signal processing, enabling the transformation of time-domain signals into their frequency-domain representations. Among the numerous elements within this domain, certain functions like the sinc function, delta function, and exponential signals hold significant importance due to their unique properties and implications.
The sinc function, defined as sinc(x) = sin(πx)/(πx), is particularly notable for its symmetry and behavior at...
The sinc function, defined as sinc(x) = sin(πx)/(πx), is particularly notable for its symmetry and behavior at...
465
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
38
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
38
Atomic Fluorescence Spectroscopy
230
Atomic fluorescence spectroscopy (AFS) is an analytical technique that involves the electronic transitions of atoms in a flame, furnace, or plasma being excited by electromagnetic (EM) radiation. When these atoms absorb energy, they become excited and subsequently release energy as they return to their original state. This emitted light, or "fluorescence," is observed at a right angle to the incident beam. Both absorption and emission processes transpire at distinct wavelengths, which...
230


