Multivariate Density Estimation by Neural Networks

Area of Science:

• Computational Statistics
• Machine Learning
• Data Science

Background:

• Accurate estimation of the Probability Density Function (PDF) is crucial for understanding data generating processes (DGP).
• Existing parametric and nonparametric density estimators often rely on assumptions or suffer from numerical inaccuracies.
• Neural networks (NNs) have shown promise in estimating cumulative distribution functions (CDFs), but direct PDF estimation remains challenging.

Purpose of the Study:

• To propose novel nonparametric methods using neural networks (NNs) for accurate Probability Density Function (PDF) estimation.
• To overcome limitations of existing density estimation techniques by avoiding assumptions on the data generating process (DGP).
• To extend existing NN-based cumulative distribution function (CDF) estimation literature by providing analytical derivatives for precise PDF calculation.

Main Methods:

• Utilized neural networks (NNs) to estimate the cumulative distribution function (CDF) of the data.
• Derived analytical derivatives of the NN-obtained CDF, thereby eliminating numerical approximation errors for PDF estimation.
• Applied the method to various neural network architectures, including multilayer perceptrons (MLPs), for both continuous and discrete distributions, and in multivariate settings.

Main Results:

• The proposed NN-based method provides more accurate PDF estimates compared to traditional methods by avoiding numerical differentiation.
• The approach demonstrates effectiveness for both continuous and discrete distributions, as well as for correlated variables in multivariate scenarios.
• Performance was validated through extensive Monte Carlo simulations on complex distributions and a real-world application estimating vehicle counts.

Conclusions:

• The proposed NN method offers a robust and accurate nonparametric approach for PDF estimation, applicable across diverse data types and complexities.
• Analytical differentiation of NN-derived CDFs is a key innovation enabling superior PDF accuracy.
• The method shows significant potential for applications in statistical modeling, machine learning, and real-world data analysis, such as traffic flow analysis.