Learning and development in neural networks--the importance of prior experience

Infants can discriminate between familiar and unfamiliar grammatical patterns expressed in a vocabulary that is distinct from that used earlier during familiarization (Cognition 70(2) (1999) 109; Science 283 (1999) 77). Various models have captured the data, although each required that discrimination be distinct, in terms of the computational process, from familiarization. This article describes a simple recurrent network (SRN), equipped only with the assumption that it should predict what comes next, which models the data without distinguishing between familiarization and discrimination. To accomplish this, the SRN requires pre-training on a range of sequences instantiating different structures and different vocabulary items to those used subsequently during familiarization and test. Pre-training enables the network to avoid replacing structure acquired during familiarization with structure experienced at test. An equivalent enabling condition may underpin infants' resistance to catastrophic interference between the different structures and vocabulary items to which they are exposed.