Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Feature extraction through LOCOCODE.

S Hochreiter1, J Schmidhuber

  • 1Fakultät fur Informatik, Technische Universität München, München 80290, Germany. hochreit@informatik.tu-muenchen.de

Neural Computation
|March 23, 1999
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

African swine fever: a global factor affecting agricultural markets over the medium term.

Revue scientifique et technique (International Office of Epizootics)·2022
Same author

Optimising traceability in trade for live animals and animal products with digital technologies.

Revue scientifique et technique (International Office of Epizootics)·2020
Same author

Salvage therapy with everolimus reduces the severity of treatment-refractory chronic GVHD without impairing disease control: a dual center retrospective analysis.

Bone marrow transplantation·2014
Same author

Introductory lecture the epidemiology and determinants of obesity in developed and developing countries.

International journal for vitamin and nutrition research. Internationale Zeitschrift fur Vitamin- und Ernahrungsforschung. Journal international de vitaminologie et de nutrition·2007
Same author

Learning nonregular languages: a comparison of simple recurrent networks and LSTM.

Neural computation·2002
Same author

Learning to forget: continual prediction with LSTM.

Neural computation·2000

Low-complexity coding and decoding (LOCOCODE) offers a novel unsupervised learning approach. It discovers minimal, low-complexity features for data representation, outperforming standard methods in feature extraction and data source separation.

Area of Science:

  • Computational neuroscience
  • Machine learning
  • Information theory

Background:

  • Traditional unsupervised learning methods often overlook the inherent complexity of code generators.
  • Existing approaches like Independent Component Analysis (ICA) and Principal Component Analysis (PCA) have limitations in handling unknown numbers of data sources and feature complexity.

Purpose of the Study:

  • To introduce Low-complexity coding and decoding (LOCOCODE), a novel approach for sensory coding and unsupervised learning.
  • To demonstrate LOCOCODE's ability to extract minimal, low-complexity features for data representation and unmix independent data sources.
  • To explore the connection between regularization techniques and unsupervised learning.

Main Methods:

  • Implementation of LOCOCODE using autoassociators trained with flat minimum search.

Related Experiment Videos

  • Application of LOCOCODE to benchmark problems including the 'bars' dataset and vowel recognition.
  • Comparison of LOCOCODE performance against standard autoencoders, ICA, and PCA.

    • Main Results:

    • LOCOCODE extracts minimal, low-complexity features that are often sparse, factorial, or local, unlike standard autoencoders.
    • It successfully unmixes an unknown number of independent data sources, outperforming ICA and PCA on complex benchmark tasks.
    • LOCOCODE generates biologically plausible feature detectors for real-world images and achieves superior data compression (fewer bits per pixel) compared to ICA and PCA.

    Conclusions:

    • LOCOCODE provides an effective method for unsupervised learning and sensory coding by prioritizing information-theoretic complexity.
    • The approach establishes a link between regularization and Independent Component Analysis (ICA)-related research, suggesting a path toward unifying these fields.
    • LOCOCODE demonstrates potential as a powerful preprocessor for machine learning tasks, enhancing classification performance.