Parallel, self-organizing, hierarchical neural networks with continuous inputs and outputs

Area of Science:

• Artificial Intelligence
• Machine Learning
• Signal Processing

Background:

• Traditional parallel, self-organizing, hierarchical neural networks (PSHNNs) typically use quantized outputs.
• Existing PSHNN architectures limit their applicability in scenarios requiring nuanced output representation.

Purpose of the Study:

• To introduce and evaluate a novel PSHNN architecture capable of handling continuous-valued outputs.
• To assess the performance of continuous-valued PSHNNs in the domain of speech signal prediction.

Main Methods:

• Developed a new PSHNN model allowing continuous-valued outputs.
• Trained network stages using the delta rule, sequential least-squares, and backpropagation (BP) algorithms.
• Investigated a revised BP algorithm for learning input nonlinearities and compared single BP networks with PSHNNs composed of smaller BP networks.

Main Results:

• The novel continuous-valued PSHNNs demonstrated superior performance compared to traditional linear prediction methods for speech sample prediction.
• Networks trained with delta rule, sequential least-squares, and BP algorithms all showed significant improvements.
• Replacing a single BP network with a PSHNN composed of smaller BP stages of equivalent complexity yielded enhanced performance.

Conclusions:

• Continuous-valued PSHNNs represent a significant advancement over quantized-output networks, particularly for complex prediction tasks like speech signal processing.
• The PSHNN architecture, especially when utilizing BP networks in its stages, offers a more effective approach than single, large BP networks for achieving higher prediction accuracy.
• This research opens avenues for more sophisticated and accurate neural network applications in time-series prediction and signal processing.