Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Input-output HMMs for sequence processing.

Y Bengio1, P Frasconi

  • 1Dept. of Comput. Sci. and Oper. Res., Montreal Univ., Que.

IEEE Transactions on Neural Networks
|January 1, 1996
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Universal autofocus for quantitative volumetric microscopy of whole mouse brains.

Nature methods·2021
Same author

Adaptive importance sampling to accelerate training of a neural probabilistic language model.

IEEE transactions on neural networks·2008
Same author

A general framework for adaptive processing of data structures.

IEEE transactions on neural networks·2008
Same author

Taking on the curse of dimensionality in joint distributions using neural networks.

IEEE transactions on neural networks·2008
Same author

Cost functions and model combination for VaR-based asset allocation using neural networks.

IEEE transactions on neural networks·2008
Same author

Experiments on the application of IOHMMs to model financial returns series.

IEEE transactions on neural networks·2008
Same journal

Universal perceptron and DNA-like learning algorithm for binary neural networks: LSBF and PBF implementations.

IEEE transactions on neural networks·2013
Same journal

Guest editorial: special section on white box nonlinear prediction models.

IEEE transactions on neural networks·2011
Same journal

Data-based fault-tolerant control of high-speed trains with traction/braking notch nonlinearities and actuator failures.

IEEE transactions on neural networks·2011
Same journal

Guest editorial: special section on data-based control, modeling, and optimization.

IEEE transactions on neural networks·2011
Same journal

Neural network-based multiple robot simultaneous localization and mapping.

IEEE transactions on neural networks·2011
Same journal

Data-driven model-free adaptive control for a class of MIMO nonlinear discrete-time systems.

IEEE transactions on neural networks·2011
See all related articles

We introduce a novel Input-Output Hidden Markov Model (IOHMM) for sequence processing. This recurrent neural network-like model excels at grammatical inference tasks, demonstrating strong generalization capabilities.

Area of Science:

  • Artificial Intelligence
  • Machine Learning
  • Computational Linguistics

Background:

  • Sequence processing is a fundamental challenge in AI and machine learning.
  • Traditional models like Hidden Markov Models (HMMs) have limitations in mapping input to output sequences.
  • Recurrent neural networks offer powerful sequence processing but can be complex to train.

Purpose of the Study:

  • To propose a novel discrete-state model for sequence processing that represents past context.
  • To introduce the Input-Output Hidden Markov Model (IOHMM) with a modular, recurrent connectionist architecture.
  • To demonstrate the effectiveness of IOHMMs in grammatical inference tasks.

Main Methods:

  • Developed a recurrent connectionist architecture with state-associated subnetworks.

Related Experiment Videos

  • Interpreted the model statistically as an Input-Output Hidden Markov Model (IOHMM).
  • Employed Estimation-Maximization (EM) or Generalized EM (GEM) algorithms for training, treating state trajectories as missing data.

    • Main Results:

    • IOHMMs enable mapping input sequences to output sequences, similar to recurrent neural networks.
    • The model utilizes a more discriminant learning paradigm compared to HMMs.
    • Experimental results on the seven Tomita grammars show excellent generalization capabilities for IOHMMs.

    Conclusions:

    • IOHMMs provide a robust framework for sequence processing and grammatical inference.
    • The modular architecture and EM-based training facilitate efficient learning and adaptation.
    • IOHMMs represent a promising advancement for tasks requiring sequence-to-sequence mapping and pattern recognition.