Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: Apr 30, 2026

Author Spotlight: Impact of Intergenic Interactions on Disease-Identifying Dark Biomarkers
03:37

Author Spotlight: Impact of Intergenic Interactions on Disease-Identifying Dark Biomarkers

Published on: March 1, 2024

1.6K

Fractional norm regularization: learning with very few relevant features.

Ata Kaban

    IEEE Transactions on Neural Networks and Learning Systems
    |May 9, 2014
    PubMed
    Summary
    This summary is machine-generated.

    Related Concept Videos

    Partial Fractions01:28

    Partial Fractions

    389
    A partial fraction is a component of a rational expression represented as the sum of simpler fractions. When a rational function is expressed as a ratio of two polynomials, it can often be decomposed into a sum of fractions whose denominators are simpler polynomials, typically linear or irreducible quadratic factors. This process is called partial fraction decomposition, and it is used to simplify complex expressions for integration, solving equations, or analysis.Partial fraction decomposition...
    389
    Linear Approximation in Frequency Domain01:26

    Linear Approximation in Frequency Domain

    502
    Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
    In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
    502
    Residuals and Least-Squares Property01:11

    Residuals and Least-Squares Property

    7.1K
    The vertical distance between the actual value of y and the estimated value of y. In other words, it measures the vertical distance between the actual data point and the predicted point on the line
    If the observed data point lies above the line, the residual is positive, and the line underestimates the actual data value for y. If the observed data point lies below the line, the residual is negative, and the line overestimates the actual data value for y.
    The process of fitting the best-fit...
    7.1K
    Application of Linearization and Approximation01:29

    Application of Linearization and Approximation

    193
    A drone flying through complex terrain often relies on more than one sensing method to estimate small changes in altitude. Along with direct measurements, air pressure provides a useful indirect indicator of vertical movement. Atmospheric pressure decreases as altitude increases, and this relationship is commonly described using an exponential model. Although accurate, converting pressure measurements into altitude values requires calculations that are too complex to perform repeatedly during...
    193
    Linearization and Approximation01:26

    Linearization and Approximation

    233
    Linearization is a mathematical technique used to approximate complex, nonlinear functions with simpler linear models in the vicinity of a chosen reference point. The method is based on the idea that, although a function may be difficult to evaluate exactly, its behavior near a specific input value can often be closely approximated by the tangent line at that point. This approach is particularly useful when small deviations from a known value are involved.Consider the square root function, for...
    233
    Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving01:29

    Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

    438
    Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
    In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
    438

    You might also read

    Related Articles

    Articles linked to this work by shared authors, journal, and citation graph.

    Sort by
    Same author

    PAC-Bayes Guarantees for Data-Adaptive Pairwise Learning.

    Entropy (Basel, Switzerland)·2025
    Same author

    Optimization and Learning With Randomly Compressed Gradient Updates.

    Neural computation·2023
    Same author

    Classification of mislabelled microarrays using robust sparse logistic regression.

    Bioinformatics (Oxford, England)·2013
    Same author

    A fast algorithm for robust mixtures in the presence of measurement errors.

    IEEE transactions on neural networks·2010
    Same journal

    Granular Ball-Based Noise-Resistant Fuzzy Multineighborhood Feature Selection via Label Enhancement and Feature Graph.

    IEEE transactions on neural networks and learning systems·2026
    Same journal

    Fighting Evolving Spam With ARTMAP Models: A Noise-Resilient Online Detection Framework.

    IEEE transactions on neural networks and learning systems·2026
    Same journal

    HyperSAT: Unsupervised Hypergraph Neural Networks for Weighted MaxSAT Problems.

    IEEE transactions on neural networks and learning systems·2026
    Same journal

    Negation of Basic Belief Assignment in Multisource Information Fusion on Dempster-Shafer Theory With Applications in Pattern Classification.

    IEEE transactions on neural networks and learning systems·2026
    Same journal

    Intervention Feasible Region and Driver Risk Capacity Aware Human-Machine Collaborative Safe Trajectory Planning.

    IEEE transactions on neural networks and learning systems·2026
    Same journal

    A Unified Differential Denoising Learning Framework With a Pre-Trained Model and Fuzzy Graph Networks for Drug-Drug Interaction Prediction.

    IEEE transactions on neural networks and learning systems·2026
    See all related articles

    Nonconvex regularization, particularly fractional norms, may outperform L1-regularization in high-dimensional logistic regression. The optimal choice depends on the feature relevance fraction, with fractional norms best for very few relevant features.

    Area of Science:

    • Machine Learning
    • Statistical Learning Theory

    Background:

    • High-dimensional learning tasks often involve numerous irrelevant features.
    • L1-norm regularization is effective, unlike L2-norm, for such scenarios.
    • Nonconvex regularization, like fractional semi-norms, shows promise but its classification performance is unclear.

    Purpose of the Study:

    • Analyze advantages and disadvantages of nonconvex regularization in Lq-regularized logistic regression.
    • Provide intuition for sparse estimation in very high dimensions.
    • Develop a data-dependent generalization error bound using PAC-Bayes methodology.

    Main Methods:

    • Analysis of norm concentration in high dimensions.
    • Application of Probably Approximately Correct (PAC)-Bayes methodology for generalization error bounding.

    Related Experiment Videos

    Last Updated: Apr 30, 2026

    Author Spotlight: Impact of Intergenic Interactions on Disease-Identifying Dark Biomarkers
    03:37

    Author Spotlight: Impact of Intergenic Interactions on Disease-Identifying Dark Biomarkers

    Published on: March 1, 2024

    1.6K
  • Experimental validation using PAC-Bayes bounds to select regularization parameters.

    • Main Results:

    • High-dimensional analysis provides insights into sparse estimation.
    • A PAC-Bayes bound for Lq-regularized logistic regression is derived.
    • Experimental results confirm the bound's utility in guiding regularization choice.

    Conclusions:

    • Optimal regularization choice is contingent on the proportion of relevant features.
    • Fractional norms with small exponents are most effective when relevant features are scarce.
    • The study elucidates the role of nonconvex regularization in high-dimensional classification.