Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Learning curves for mutual information maximization.

R Urbanczik1

  • 1Institut für Theoretische Physik, Universität Würzburg, Am Hubland, D-97074 Würzburg, Germany.

Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics
|August 26, 2003
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Laboratory tests focusing on sputum.

The international journal of tuberculosis and lung disease : the official journal of the International Union against Tuberculosis and Lung Disease·2010
Same author

Scaling up tuberculosis culture services: a precautionary note.

The international journal of tuberculosis and lung disease : the official journal of the International Union against Tuberculosis and Lung Disease·2009
Same author

Enumerating constrained elementary flux vectors of metabolic networks.

IET systems biology·2007
Same author

Yield of serial sputum specimen examinations in the diagnosis of pulmonary tuberculosis: a systematic review.

The international journal of tuberculosis and lung disease : the official journal of the International Union against Tuberculosis and Lung Disease·2007
Same author

Functional stoichiometric analysis of metabolic networks.

Bioinformatics (Oxford, England)·2005
Same author

An improved algorithm for stoichiometric network analysis: theory and applications.

Bioinformatics (Oxford, England)·2004
Same journal

Tension on dsDNA bound to ssDNA-RecA filaments may play an important role in driving efficient and accurate homology recognition and strand exchange.

Physical review. E, Statistical, nonlinear, and soft matter physics·2016
Same journal

Publisher's Note: Amplitude-phase coupling drives chimera states in globally coupled laser networks [Phys. Rev. E 91, 040901(R) (2015)].

Physical review. E, Statistical, nonlinear, and soft matter physics·2016
Same journal

Erratum: Shapes of sedimenting soft elastic capsules in a viscous fluid [Phys. Rev. E 92, 033003 (2015)].

Physical review. E, Statistical, nonlinear, and soft matter physics·2016
Same journal

Erratum: Attenuation of excitation decay rate due to collective effect [Phys. Rev. E 90, 022142 (2014)].

Physical review. E, Statistical, nonlinear, and soft matter physics·2016
Same journal

Publisher's Note: Role of connectivity and fluctuations in the nucleation of calcium waves in cardiac cells [Phys. Rev. E 92, 052715 (2015)].

Physical review. E, Statistical, nonlinear, and soft matter physics·2016
Same journal

Publisher's Note: Lattice Boltzmann approach for complex nonequilibrium flows [Phys. Rev. E 92, 043308 (2015)].

Physical review. E, Statistical, nonlinear, and soft matter physics·2016
See all related articles

This study analyzes unsupervised learning via mutual information maximization. It demonstrates that this method recognizes data structure in large samples but may require regularization for perceptron-like networks due to slow convergence.

Area of Science:

  • Machine Learning
  • Artificial Intelligence
  • Information Theory

Background:

  • Unsupervised learning aims to discover patterns in data without explicit labels.
  • Mutual information quantifies statistical dependence between variables, useful for feature extraction.
  • Previous work (Becker & Hinton, 1992) explored mutual information maximization for neural networks.

Purpose of the Study:

  • To analyze an unsupervised learning procedure maximizing mutual information between network outputs.
  • To investigate the recognition of data structure using mutual information maximization.
  • To evaluate learning curves and convergence properties for perceptron-like networks.

Main Methods:

  • Analysis of an unsupervised learning algorithm maximizing mutual information.

Related Experiment Videos

  • Theoretical analysis for a generic data model in the large sample limit.
  • Calculation of learning curves for zero-temperature Gibbs learning with perceptron-like networks.

    • Main Results:

    • Mutual information maximization effectively recognizes underlying data structure in large samples.
    • Learning curves for perceptron-like networks indicate potentially slow convergence.
    • Regularization techniques are considered to improve convergence speed.

    Conclusions:

    • Mutual information maximization is a viable unsupervised learning strategy for structure discovery.
    • Convergence speed is a key consideration, especially for restricted network models.
    • Further research into regularization methods can enhance the practical applicability of this approach.