Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Unifying cost and information in information-theoretic competitive learning.

Ryotaro Kamimura1

  • 1Information Science Laboratory, Tokai University, 1117, Kitakaname Hiratsuka Kanagawa 259-1292, Japan. ryo@cc.u-tokai.ac.jp

Neural Networks : the Official Journal of the International Neural Network Society
|August 23, 2005
PubMed
Summary

This study introduces cost into information maximization for better data representation. Minimizing cost ensures connection weights accurately reflect input patterns, improving learning efficiency and generalization.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Collective mutual information maximization to unify passive and positive approaches for improving interpretation and generalization.

Neural networks : the official journal of the International Neural Network Society·2017
Same author

Cooperation-controlled learning for explicit class structure in self-organizing maps.

TheScientificWorldJournal·2014
Same author

Self-enhancement learning: target-creating learning and its application to self-organizing maps.

Biological cybernetics·2011
Same author

Selective information enhancement learning for creating interpretable representations in competitive learning.

Neural networks : the official journal of the International Neural Network Society·2011
Same author

Cooperative information maximization with Gaussian activation functions for self-organizing maps.

IEEE transactions on neural networks·2006
Same author

Multi-layered greedy network-growing algorithm: extension of greedy network-growing algorithm to multi-layered networks.

International journal of neural systems·2004

Area of Science:

  • Machine Learning
  • Computational Neuroscience

Background:

  • Information maximization is a framework for understanding neural representations.
  • A limitation is that maximizing information alone may not yield faithful representations.

Purpose of the Study:

  • To introduce a cost function into information maximization to improve data representation fidelity.
  • To enhance competitive learning by balancing information gain with representation accuracy.

Main Methods:

  • Developed a novel approach by incorporating a cost term representing the average distance between input patterns and connection weights.
  • Modified the objective function to maximize the ratio of information to its associated cost.
  • Applied the enhanced method to political data analysis, a voting attitude problem, and the Wisconsin cancer dataset.

Related Experiment Videos

Main Results:

  • The introduction of cost led to connection weights that better reflect the input patterns.
  • Achieved representations that are more faithful to the original data compared to standard information maximization.
  • Demonstrated improved generalization performance on benchmark datasets.
  • Observed efficient learning within a reduced timeframe.

Conclusions:

  • Incorporating cost into information maximization is an effective strategy for obtaining faithful data representations.
  • The proposed method enhances competitive learning by ensuring representational accuracy.
  • This approach offers a promising direction for improving machine learning models and understanding neural computation.