审查学习功能和语义信息措施的演变,以了解深度学习
Chenguang Lu1,2
1Intelligence Engineering and Mathematics Institute, Liaoning Technical University, Fuxin 123000, China.
Entropy (Basel, Switzerland)
|May 27, 2023
概括
数十年前提出的语义互联信息 (SeMI) 相当于现代的深度学习指标,如互联信息神经估计 (MINE). 这种连接为深度学习模型提供了新的见解和潜在的简化.
科学领域:
- 信息理论 信息理论
- 深度学习 (Deep Learning) 是一种深度学习.
- 机器学习 机器学习
背景情况:
- 深度学习的新趋势是使用相似性函数和估计相互信息 (EMI) 作为学习目标.
- 基本上,EMI与语义互联信息 (SeMI) 是相同的,这是30年前引入的概念.
研究的目的:
- 审查语义信息测量和学习函数的历史.
- 介绍作者关于语义信息的G理论及其应用.
- 探索SeMI,香农的相互信息 (MI) 和各种机器学习概念之间的关系.
主要方法:
- 对语义信息和学习功能的进化历史的审查.
- 介绍G理论与速率-忠实性函数R (G) 的介绍.
- 将G理论应用于多标签学习,MI分类和混合模型.
主要成果:
- 证明混合模型和受限制的博尔兹曼机器由于SeMI最大化和Shannon的MI最小化而趋同.
- 在这些模型中,信息效率G/R接近1的识别.
- 探索SeMI作为加强学习的奖励函数.
结论:
- G理论为深度学习提供了一个解释框架.
- 将语义信息理论和深度学习结合起来,可以加速这两个领域的进步.
- 通过高斯频道混合模型来简化深度学习的潜力,用于预训练.
关键词:
有限制的博尔茨曼机器软马克斯功能 软马克斯功能深度学习是一种深度学习.估计的相互信息.一般化的.学习功能学习功能学习功能尽可能多地相互交流信息.语义信息衡量语义信息的措施.类似性函数是一个相似性函数.更多相关视频
09:47Author Spotlight: Advancing Alzheimer's Research – Exploring Early Detection and Multi-Omics Approaches
Published on: December 15, 2023
1.2K
08:05Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
Published on: June 30, 2020
7.6K
相关概念视频
Introduction to Learning
480
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
480
Observational Learning
246
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
246
Higher Mental Functions of Brain: Learning and Memory
900
Memory is one of the most vital higher mental functions of the brain. Memory is closely related to learning because it enables us to retain information and experiences from our past to use them in our present life. It also helps us to remember facts, events, and skills, such as riding a bike or swimming. There are two types of memory — declarative memory, which involves memorizing facts or events, and procedural memory, which enables us to remember how to do something like writing or...
900
Generalization, Discrimination, and Extinction
668
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
668
Cognitive Learning
455
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
455
Associative Learning
465
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
465
