Related Experiment Video
Updated: Sep 10, 2025
07:35
Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
Published on: October 11, 2018
7.6K
Feature Ranking on Small Samples: A Bayes-Based Approach
Aleksandra Vatian1, Natalia Gusarova1, Ivan Tomilov1
1School of Translational Information Technologies, ITMO University, 197101 St. Petersburg, Russia.
Entropy (Basel, Switzerland)
|August 28, 2025
Summary
A new Bayesian approach ranks features effectively, even on small datasets. This model-free method shows superior stability and consistency compared to existing techniques for feature importance analysis.
Area of Science:
- Machine Learning
- Statistical Modeling
Background:
- Feature ranking is crucial for predicting target attributes, yet research often focuses on selection/extraction rather than ranking.
- Existing methods may lack robustness, especially with limited data.
Purpose of the Study:
- To introduce a novel, model-free Bayesian method for feature ranking on small datasets.
- To establish a framework for benchmarking feature ranking algorithms.
Main Methods:
- A Bayesian approach for feature ranking.
- Experimental comparison with classical frequency methods, logistic regression, and SHAP.
- Validation on synthetic and public medical datasets.
Main Results:
- The proposed method demonstrates high self-consistency (stability) even with 50 samples.
- Outperforms logistic regression and SHAP in stability and monotonicity.
- Shows comparable or superior performance to other methods as sample size increases.
Conclusions:
- The Bayesian feature ranking method is robust and reliable for small datasets.
- It offers a significant improvement in stability and consistency for influence factor analysis.
- Applicable across diverse fields like industry, forensics, and psychology.
Related Concept Videos
01:02Ranks
286
Unlike parametric methods, nonparametric statistics are ideal for nominal and ordinal data, requiring fewer assumptions about the population's nature or distribution. This makes nonparametric methods easier to apply and interpret, as they do not depend on parameters like mean or standard deviation. One common approach in nonparametric analysis is to sort data according to a specific criterion. For instance, we might arrange weather data from hottest to coldest days in a month or rank cities...
286
01:19Expected Frequencies in Goodness-of-Fit Tests
2.6K
A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n) to the number of categories (k).
2.6K
01:02Quantifying and Rejecting Outliers: The Grubbs Test
2.0K
Sometimes, a data set can have a recorded numerical observation that greatly deviates from the rest of the data. Assuming that the data is normally distributed, a statistical method called the Grubbs test can be used to determine whether the observation is truly an outlier. To perform a two-tailed Grubbs test, first, calculate the absolute difference between the outlier and the mean. Then, calculate the ratio between this difference and the standard deviation of the sample. This...
2.0K
01:19Survival Tree
159
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
159
01:21Friedman Two-way Analysis of Variance by Ranks
296
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
296
01:20Cluster Sampling Method
12.7K
Appropriate sampling methods ensure that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
12.7K