Generalization and capacity of extensively large two-layered perceptrons

Michal Rosen-Zvi¹, Andreas Engel, Ido Kanter

¹Minerva Center and Department of Physics, Bar-Ilan University, Ramat-Gan, 52900 Israel.

Physical Review. E, Statistical, Nonlinear, and Soft Matter Physics

|October 9, 2002

Abstract:

The generalization ability and storage capacity of a treelike two-layered neural network with a number of hidden units scaling as the input dimension is examined. The mapping from the input to the hidden layer is via Boolean functions; the mapping from the hidden layer to the output is done by a perceptron. The analysis is within the replica framework where an order parameter characterizing the overlap between two networks in the combined space of Boolean functions and hidden-to-output couplings is introduced. The maximal capacity of such networks is found to scale linearly with the logarithm of the number of Boolean functions per hidden unit. The generalization process exhibits a first-order phase transition from poor to perfect learning for the case of discrete hidden-to-output couplings. The critical number of examples per input dimension, alpha(c), at which the transition occurs, again scales linearly with the logarithm of the number of Boolean functions. In the case of continuous hidden-to-output couplings, the generalization error decreases according to the same power law as for the perceptron, with the prefactor being different.

Related Experiment Videos

Generalization and capacity of extensively large two-layered perceptrons

Related Concept Videos

MAMMAL - Molecular Aligned Multi-Modal Architecture and Language for biomedical discovery.

Generation of Automated Nephrometry Scores Through Direct Prediction of Each Component.

Network-based prediction and real-world patient data observation identify doxycycline as a repurposable drug in Alzheimer's disease.

Critical evaluation of real-world evidence of repurposable medicines in the Alzheimer's disease drug development pipeline using a target trial emulation.

Drug Development.

Characterization of German SF6 Emissions.

Tension on dsDNA bound to ssDNA-RecA filaments may play an important role in driving efficient and accurate homology recognition and strand exchange.

Publisher's Note: Amplitude-phase coupling drives chimera states in globally coupled laser networks [Phys. Rev. E 91, 040901(R) (2015)].

Erratum: Shapes of sedimenting soft elastic capsules in a viscous fluid [Phys. Rev. E 92, 033003 (2015)].

Erratum: Attenuation of excitation decay rate due to collective effect [Phys. Rev. E 90, 022142 (2014)].

Publisher's Note: Role of connectivity and fluctuations in the nucleation of calcium waves in cardiac cells [Phys. Rev. E 92, 052715 (2015)].

Publisher's Note: Lattice Boltzmann approach for complex nonequilibrium flows [Phys. Rev. E 92, 043308 (2015)].

LncPNdeep: A long non-coding RNA classifier based on large language model with peptide and nucleotide embedding.

A biologically structured hierarchical vision transformer-CNN framework for robust tomato leaf disease classification.

Explainable and reliable kidney CT image classification using self-supervised DeiT-Tiny transformer.

Geometric constraints and cognitive inputs jointly shape emergent brain dynamics and topology.

Benchmarking Generalizability in Deep Learning-Based White Matter Tract Segmentation.

Automated Neuron Tracing with Imitation Learning.

Related Experiment Videos

Generalization and capacity of extensively large two-layered perceptrons

Abstract:

Related Concept Videos

Related Articles