Nonnegative matrix factorization with the Itakura-Saito divergence: with application to music analysis
Cédric Févotte1, Nancy Bertin, Jean-Louis Durrieu
1CNRS-TELECOM ParisTech, 75014 Paris, France. fevotte@telecom-paristech.fr
Neural Computation
|September 13, 2008
Summary
Nonnegative matrix factorization (NMF) using Itakura-Saito (IS) divergence offers a statistically grounded approach for audio signal processing. IS-NMF demonstrates superior performance in music signal representation, denoising, and upmixing compared to Euclidean and KL divergences.
Area of Science:
- Signal Processing
- Machine Learning
- Statistical Modeling
Background:
- Nonnegative matrix factorization (NMF) is a widely used technique for dimensionality reduction and feature extraction.
- Existing NMF methods often employ Euclidean or Kullback-Leibler (KL) divergences, which may not optimally represent certain data types.
- The Itakura-Saito (IS) divergence offers an alternative cost function with potential advantages for specific applications.
Purpose of the Study:
- To theoretically, algorithmically, and experimentally investigate Nonnegative Matrix Factorization (NMF) with the Itakura-Saito (IS) divergence.
- To establish the statistical underpinnings of IS-NMF, linking it to Gaussian component models and maximum likelihood estimation.
- To explore the application of Bayesian priors for regularization within the IS-NMF framework.
Main Methods:
- Development of a space-alternating generalized expectation-maximization (SAGE) algorithm for IS-NMF, guaranteeing convergence to a stationary point.
- Investigation of a gradient multiplicative algorithm for IS-NMF, with observed practical convergence.
- Comparative experimental analysis of IS-NMF against Euclidean-NMF and KL-NMF using audio spectrogram data.
Main Results:
- IS-NMF is shown to be equivalent to maximum likelihood estimation of variance parameters in a superimposed Gaussian component model.
- The SAGE algorithm provides a novel, convergent method for IS-NMF.
- Experimental results indicate IS-NMF's superior performance in representing music signals, denoising, and mono-to-stereo upmixing compared to Euclidean and KL NMF.
Conclusions:
- IS-NMF provides a statistically robust and effective method for audio signal representation and processing.
- The Itakura-Saito divergence is better suited for capturing the semantics of music signals than traditional NMF cost functions.
- IS-NMF shows significant promise for practical applications in audio enhancement and analysis.
Related Concept Videos
01:14Extraction: Partition and Distribution Coefficients
The distribution law or Nernst's distribution law is the law that governs the distribution of a solute between two immiscible solvents. This law, also known as the partition law, states that if a solute is added to the mixture of two immiscible solvents at a constant temperature, the solute is distributed between the two solvents in such a way that the ratio of solute concentrations in the solvents remains constant at equilibrium.
For extracting a solute from an aqueous phase into an organic...
For extracting a solute from an aqueous phase into an organic...
01:08Vector Algebra: Method of Components
It is cumbersome to find the magnitudes of vectors using the parallelogram rule or using the graphical method to perform mathematical operations like addition, subtraction, and multiplication. There are two ways to circumvent this algebraic complexity. One way is to draw the vectors to scale, as in navigation, and read approximate vector lengths and angles (directions) from the graphs. The other way is to use the method of components.
In many applications, the magnitudes and directions of...
In many applications, the magnitudes and directions of...
01:26Linear Approximation in Frequency Domain
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear.
01:21Mass Spectrometry: Complex Analysis
Mass spectrometry is an important technique for the identification of pure compounds. However, it has some limitations for the analysis of complex mixtures, often due to excessive fragmentation making the spectrum too complicated to decipher. Mass spectrometry can be combined with suitable separation methods in sequence, forming hyphenated methods, which are useful in the analysis of complex mixtures.
GC–MS is a powerful hyphenated method commonly used in forensics and environmental...
GC–MS is a powerful hyphenated method commonly used in forensics and environmental...
01:20Inverse z-Transform by Partial Fraction Expansion
The inverse z-transform is a crucial technique for converting a function from its z-domain representation back to the time domain. One effective method for finding the inverse z-transform is the Partial Fraction Method, which involves decomposing a function into simpler fractions with distinct coefficients. These fractions correspond to known z-transform pairs, facilitating the inverse transformation process.
To begin the process, the poles of the function are identified and the function is...
To begin the process, the poles of the function are identified and the function is...
01:20Kendall's Coefficient of Concordance
Kendall's Coefficient of Concordance (W), also known as Kendall's W, is a non-parametric statistical measure used to assess the agreement or concordance between multiple raters or judges when they rank a set of items. It is often used when you have ordinal data (ranks) and you want to see if there is consistency or consensus among the raters. It is widely applied in research areas such as psychology, medicine, and social sciences, where multiple judges are asked to rank or rate subjects or...