Small Stochastic Data Compactification Concept Justified in the Entropy Basis

Area of Science:

• Data Science
• Machine Learning
• Information Theory

Background:

• Data dimensionality is a significant challenge in machine learning, impacting tasks like classification and clustering.
• Data compactification aims to reduce dimensionality while minimizing information loss, a process complicated by stochastic parameters.

Purpose of the Study:

• To propose a new model for structured stochastic data collections using relative entropy.
• To develop an iterative procedure for compacting such data by maximizing relative entropy.
• To assess the compactification procedure's effectiveness and its relevance to data reliability.

Main Methods:

• Modeling structured stochastic data collections in terms of relative entropy.
• Formalizing compactification as an iterative procedure maximizing relative entropy of data projections.
• Developing an approximation for the relative entropy function to reduce computational complexity.
• Assessing compactification using information capacity and information loss metrics.

Main Results:

• A stable and efficient iterative procedure for data compactification of stochastic data.
• Demonstrated effectiveness of the proposed method compared to Principal Component Analysis and Random Projection.
• The proposed metrics are relevant for assessing data reliability and completeness.

Conclusions:

• The proposed relative entropy-based compactification method offers a robust solution for high-dimensional stochastic data.
• This approach enhances data management and reliability assessment in machine learning.
• The method shows superior stability and efficiency over existing techniques.