Jove
Visualize
Contact Us

Related Experiment Videos

Novel maximum-margin training algorithms for supervised neural networks.

Oswaldo Ludwig1, Urbano Nunes

  • 1ISR-Institute of Systems and Robotics, Department of Electrical and Computer Engineering, University of Coimbra Polo II, 3030-290 Coimbra, Portugal. oludwig@isr.uc.pt

IEEE Transactions on Neural Networks
|April 23, 2010
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Control of Brain Activity in hMT+/V5 at Three Response Levels Using fMRI-Based Neurofeedback/BCI.

PloS one·2016
Same author

Toward a reliable gaze-independent hybrid BCI combining visual and natural auditory stimuli.

Journal of neuroscience methods·2015
Same author

ISRUC-Sleep: A comprehensive public dataset for sleep researchers.

Computer methods and programs in biomedicine·2015
Same author

Probabilistic Social Behavior Analysis by Exploring Body Motion-Based Patterns.

IEEE transactions on pattern analysis and machine intelligence·2015
Same author

A two-step automatic sleep stage classification method with dubious range detection.

Computers in biology and medicine·2015
Same author

Improving the generalization capacity of cascade classifiers.

IEEE transactions on cybernetics·2013
Same journal

Universal perceptron and DNA-like learning algorithm for binary neural networks: LSBF and PBF implementations.

IEEE transactions on neural networks·2013
Same journal

Guest editorial: special section on white box nonlinear prediction models.

IEEE transactions on neural networks·2011
Same journal

Data-based fault-tolerant control of high-speed trains with traction/braking notch nonlinearities and actuator failures.

IEEE transactions on neural networks·2011
Same journal

Guest editorial: special section on data-based control, modeling, and optimization.

IEEE transactions on neural networks·2011
Same journal

Neural network-based multiple robot simultaneous localization and mapping.

IEEE transactions on neural networks·2011
Same journal

Data-driven model-free adaptive control for a class of MIMO nonlinear discrete-time systems.

IEEE transactions on neural networks·2011
See all related articles
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

This study introduces three novel multilayer perceptron (MLP) training methods, including maximum-margin gradient descent with adaptive learning rate (MMGDX) and minimization of interclass interference (MICI). These methods offer efficient training with linear complexity, outperforming traditional support vector machine (SVM) approaches.

Area of Science:

  • Machine Learning
  • Artificial Intelligence
  • Computational Neuroscience

Background:

  • Multilayer perceptrons (MLPs) are fundamental in deep learning but require efficient training methods.
  • Traditional support vector machine (SVM) training methods can be computationally intensive with high time and space complexity.
  • Novel approaches are needed to improve MLP training efficiency and performance, particularly for binary classification tasks.

Purpose of the Study:

  • To propose and evaluate three novel training methods for multilayer perceptron (MLP) binary classifiers.
  • To develop methods that are computationally efficient, with linear time and space complexity.
  • To enhance classification performance in terms of accuracy, area under the ROC curve (AUC), and balanced error rate.

Main Methods:

Related Experiment Videos

  • Maximum-margin gradient descent with adaptive learning rate (MMGDX): Directly optimizes the margin of the MLP output-layer hyperplane, jointly optimizing MLP layers.
  • Minimization of interclass interference (MICI): Utilizes an objective function inspired by Fisher discriminant analysis to create a desirable statistical distribution in the MLP hidden layer.
  • Assembled neural network (ASNN): A hybrid approach combining neurons from networks trained by MICI, MMGDX, and Levenberg-Marquardt (LM).
  • Maximum area under the ROC curve (AUC) is used as a stopping criterion for MMGDX and MICI.

Main Results:

  • The proposed MMGDX and MICI methods achieve linear time and space complexity (O(N)), significantly outperforming SVMs (O(N^3) time, O(N^2) space).
  • The ASNN model demonstrates robust performance by leveraging the strengths of individual training methods.
  • Experimental results on benchmark datasets show that the proposed methods achieve competitive or superior performance compared to state-of-the-art classifiers in accuracy, AUC, and balanced error rate.

Conclusions:

  • The novel training methods (MMGDX, MICI, ASNN) offer significant improvements in computational efficiency and classification performance for MLPs.
  • These methods provide effective alternatives to traditional SVM training, especially for large datasets.
  • The study highlights the potential of maximal-margin principles and information theory in developing advanced neural network training algorithms.