Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Estimating mutual information.

Alexander Kraskov1, Harald Stögbauer, Peter Grassberger

  • 1John-von-Neumann Institute for Computing, Forschungszentrum Jülich, D-52425 Juelich, Germany.

Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics
|July 13, 2004
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Electrophysiologic Characteristics of Epidural Spinal Signals in Preclinical Models of Spinal Cord Stimulation.

Neuromodulation : journal of the International Neuromodulation Society·2026
Same author

Electrophysiological Characteristics of Epidural Spinal Signals in Preclinical Models of Spinal Cord Stimulation.

bioRxiv : the preprint server for biology·2026
Same author

From static to dynamic: how object rotation influences grasp decisions in ambiguous settings.

Experimental brain research·2025
Same author

A novel method to assess motor planning deficits in patients with parkinson's disease and mild cognitive impairment.

Scientific reports·2025
Same author

Aftermath epidemics: Percolation on the sites visited by generalized random walks.

Physical review. E·2023
Same author

Many universality classes in an interface model restricted to non-negative heights.

Physical review. E·2023

We developed new methods to estimate mutual information (MI) using k-nearest neighbor distances. These data-efficient and adaptive estimators accurately detect independence between variables, improving signal processing tasks.

Area of Science:

  • Information Theory
  • Machine Learning
  • Statistical Analysis

Background:

  • Estimating mutual information (MI) is crucial for understanding dependencies between random variables.
  • Conventional methods like binning can be data-inefficient and lack adaptivity.
  • There is a need for improved estimators that are more accurate and robust.

Purpose of the Study:

  • To introduce two novel classes of improved estimators for mutual information (MI).
  • To develop estimators based on k-nearest neighbor (k-NN) distances for enhanced data efficiency and adaptivity.
  • To provide estimators for redundancies among multiple random variables and assess their utility in signal processing.

Main Methods:

  • Estimating mutual information using k-nearest neighbor distances.

Related Experiment Videos

  • Developing novel algorithms for entropy estimation from sample data.
  • Comparing proposed algorithms with existing methods and evaluating performance on independent component analysis (ICA) tasks.

    • Main Results:

    • The proposed estimators demonstrate high data efficiency and adaptivity, resolving structures at small scales.
    • The estimators accurately vanish for independent distributions (M(X,Y) ≈ 0 if μ(x,y)=μ(x)μ(y)) across various dimensions.
    • The methods show effectiveness in assessing independence, improving ICA, and estimating blind source separation reliability.

    Conclusions:

    • The k-NN based estimators offer significant improvements over traditional methods for mutual information estimation.
    • These novel estimators are robust, data-efficient, and accurate, particularly for detecting independence.
    • The developed techniques have practical applications in signal processing, including enhancing independent component analysis and source separation.